Vehicle screen control method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a vehicle screen control method and device, electronic equipment and storage medium, relating to the technical field of vehicle screen control, the method comprising: obtaining a screen control instruction; determining a target window and a focus window based on the screen control instruction; and controlling the focus window in the target window based on the screen control instruction. Thus, the present scheme converts ambiguous natural language or remote instructions into precise control behaviors with clear spatial context and operation objects, improving control accuracy and safety, providing a technical foundation for multi-user collaborative display and barrier-free human-computer interaction, and effectively solving the problem of "seeing but not pointing, speaking but not controlling" in existing systems.

Description

Technical Field

[0001] This disclosure relates to the field of vehicle screen control technology, and in particular to a vehicle screen control method, device, electronic device and storage medium. Background Technology

[0002] With the rapid development of smart cockpit technology, in-vehicle displays have been widely used in vehicle human-machine interaction systems, undertaking multiple functions such as navigation, entertainment, vehicle settings, and air conditioning control. To improve driving safety and ease of operation, voice interaction technology has been widely integrated into in-vehicle systems, allowing users to control screen functions through natural language commands, such as "turn on the air conditioning," "play music," or "navigate to the nearest gas station."

[0003] However, current in-vehicle display software architectures and interaction logic generally suffer from a globalized and non-regionalized design flaw. Specifically, the display screen is typically treated as a unified interaction plane, and the system does not establish clear logical divisions or independent control contexts for different functional areas on the screen (such as the driver's information area, the passenger's entertainment area, and the central control area). Therefore, in voice control scenarios, when a user issues commands involving spatial location, such as "click the button on the right" or "close the window above," the system lacks the semantic understanding and coordinate mapping capabilities of the screen area, making it unable to accurately identify the specific object the user is referring to. Summary of the Invention

[0004] This disclosure aims to at least partially address one of the technical problems in the related art.

[0005] Therefore, one objective of this disclosure is to propose a method for controlling an in-vehicle screen.

[0006] The second objective of this disclosure is to provide an in-vehicle screen control device.

[0007] The third objective of this disclosure is to propose an electronic device.

[0008] The fourth objective of this disclosure is to provide a non-transitory computer-readable storage medium.

[0009] The fifth objective of this disclosure is to propose a vehicle.

[0010] To achieve the above objectives, a first aspect of this disclosure provides a vehicle screen control method, comprising: acquiring a screen control command; and controlling a focus window within a target window based on the screen control command, wherein the target window is the window corresponding to the screen control command, and the focus window is the window to be controlled corresponding to the screen control command.

[0011] According to one embodiment of this disclosure, determining the target window includes: obtaining a window identifier in the screen control command; and determining the target window from a plurality of candidate windows based on the window identifier.

[0012] According to one embodiment of this disclosure, the method further includes: generating the window identifier, wherein generating the window identifier includes: obtaining key information in the screen control instructions; and generating the window identifier based on the key information.

[0013] According to one embodiment of this disclosure, the method further includes: in response to the absence of a window identifier in the screen control instruction, obtaining the location information of the screen control instruction; and determining a target window that matches the location information from a plurality of candidate windows.

[0014] According to one embodiment of this disclosure, determining the focus window based on the screen control command includes: obtaining a focus window identifier in the screen control command; and determining the focus window based on the focus window identifier.

[0015] According to one embodiment of this disclosure, the method further includes: generating the screen control instruction, wherein generating the screen control instruction includes: acquiring instruction data and location information input by the user, wherein the instruction data is one of voice data, eye-tracking data, or gesture data; generating the screen control instruction based on the instruction intent, instruction parameters, and the location information, wherein the instruction intent and the instruction parameters are generated based on the instruction data.

[0016] According to one embodiment of this disclosure, the method further includes: generating the screen control instructions, wherein generating the screen control instructions includes: acquiring instruction data input by the user; and generating the screen control instructions based on the instruction data.

[0017] According to one embodiment of this disclosure, controlling the focus window in the target window based on the screen control instructions includes: determining a logical operation module based on the screen control instructions; assigning the logical operation module to the focus window and displaying it in the target window.

[0018] According to one embodiment of this disclosure, when the screen control command includes adjusting the display range of at least one target application or desktop shortcut component, controlling the focused window within the target window based on the screen control command includes:

[0019] Adjust the display range of the target application or desktop shortcut in the target window according to the display priority of the target application or desktop shortcut.

[0020] According to one embodiment of this disclosure, in any two layers, the target window corresponding to the layer with higher display priority can be overlaid on the target window corresponding to the layer with lower display priority; or... In any two applications, the target window of the application with higher display priority can encroach on the display area of ​​the target window of the application with lower display priority.

[0021] According to one embodiment of this disclosure, where the screen control command is a display interaction area, controlling the focused window within the target window based on the screen control command includes: The interactive area is displayed at the target location of the target window; wherein the target window is the driver's display area and / or the passenger's display area.

[0022] To achieve the above objectives, a second aspect of this disclosure provides an in-vehicle screen control device, comprising: an acquisition module for acquiring screen control commands; and a control module for controlling a focus window in a target window based on the screen control commands, wherein the target window is the window corresponding to the screen control commands, and the focus window is the window that needs to be controlled corresponding to the screen control commands.

[0023] To achieve the above objectives, a third aspect of this disclosure provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to implement the vehicle screen control method as described in the first aspect of this disclosure.

[0024] To achieve the above objectives, a fourth aspect of this disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to implement the vehicle screen control method as described in the first aspect of this disclosure.

[0025] To achieve the above objectives, a fifth aspect of this disclosure provides a vehicle including a seatbelt control device as described in a second aspect of this disclosure; or an electronic device as described in a third aspect of this disclosure; or a computer-readable storage medium as described in a fourth aspect of this disclosure.

[0026] Beneficial effects: This solution transforms ambiguous natural language or remote commands into precise control behaviors with clear spatial context and operation objects, improving control accuracy and security, providing a technical foundation for multi-user collaborative display and barrier-free human-computer interaction, and effectively solving the problem of "seeing but not being able to click accurately, speaking clearly but not being able to control correctly" in existing systems. Attached Figure Description

[0027] Figure 1 This is a flowchart of an embodiment of an in-vehicle screen control method disclosed herein; Figure 2 This is a flowchart of another in-vehicle screen control method according to one embodiment of the present disclosure; Figure 3 This is a flowchart of another in-vehicle screen control method according to one embodiment of the present disclosure; Figure 4 This is a flowchart of another in-vehicle screen control method according to one embodiment of the present disclosure; Figure 5 This is a flowchart of another in-vehicle screen control method according to one embodiment of the present disclosure; Figure 6 This is a block diagram of an in-vehicle screen control device disclosed herein; Figure 7 This is a block diagram of an electronic device proposed in this disclosure. Detailed Implementation

[0028] Embodiments of this disclosure are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this disclosure, and should not be construed as limiting this disclosure.

[0029] Current technologies rely heavily on application-level intent recognition (such as "open settings"), which can only trigger predefined global functions and cannot dynamically parse spatial directional commands based on the current interface layout. Even if some systems introduce screen element traversal mechanisms (such as accessibility services), their response logic is still mainly based on linear focus switching, making it difficult to support multi-area parallel interaction or cross-area precise positioning.

[0030] These limitations significantly restrict the accuracy, flexibility, and user experience of voice control in multi-occupant, multi-tasking cockpit environments. Especially in vehicles where the driver and front passenger share a large, interconnected screen, the lack of regional differentiation makes voice commands prone to ambiguity and even misoperation, reducing interaction efficiency and potentially posing safety hazards.

[0031] Therefore, there is an urgent need for a technical solution that can logically divide the in-vehicle display screen and support voice-based spatial directional control in order to achieve a more accurate, safe, and personalized intelligent cockpit human-machine interaction experience.

[0032] To address the aforementioned problems, this disclosure proposes a method for controlling an in-vehicle screen. Figure 1 This is a schematic diagram illustrating an exemplary embodiment of an in-vehicle screen control method proposed in this disclosure, as shown below. Figure 1 As shown, the in-vehicle screen control method includes the following steps: S101, Obtain screen control commands.

[0033] This disclosure provides a precise window control method driven by screen control commands for in-vehicle, mobile terminal, or multi-screen collaborative scenarios. By parsing screen control commands input by the user or automatically triggered by intelligent devices (such as vehicles), the method dynamically identifies the target window and the focus window, and performs precise operations on the focus element in the target context, thereby solving the problem of ambiguity in voice or remote control caused by the lack of regional semantics in existing systems.

[0034] In this embodiment of the disclosure, screen control commands can originate from various interaction channels, such as manually inputted zoom-in / expand screen control commands, manually inputted open / close application screen control commands, or voice-generated screen control commands, or screen control commands generated by the vehicle itself. No limitations are imposed herein.

[0035] In one possible implementation, users can input screen control commands via voice input: such as "click the '+' button on the passenger side air conditioning panel", "close the driver's side navigation window", and "increase the display area of ​​the passenger side display screen". In another possible way, users can also use gesture / eye-tracking commands: recognizing directional movements through a camera or sensor; In another possible implementation, users can also issue screen control commands via remote control devices, such as structured control commands sent by mobile applications. It should be noted that screen control commands can be natural language text or structured commands.

[0036] The processor can perform semantic understanding of screen control commands and extract key elements. For example: Action types: tap, swipe, long press, close, maximize, expand, gestures, etc. Target areas / applications: such as "air conditioning panel", "navigation map", "music player", "voice card", "video application", "DOC shortcut bar", "driver's first display screen (key parameter screen)", "passenger's display screen", "driver's second display screen (center console screen)"; Spatial context: such as "driver's side", "passenger's area", "global window", "floating window", "full-screen area".

[0037] S102, control the focus window in the target window based on the screen control command, wherein the target window is the window corresponding to the screen control command, and the focus window is the window that needs to be controlled according to the screen control command.

[0038] It should be noted that a vehicle may include multiple physical screens, each of which may include a window or be divided into multiple windows.

[0039] For example, taking the main control screen located in the front cabin of the vehicle as an example, in some instances, the main control screen can be an ultra-wide screen display. The main control screen can be divided into multiple sub-screens for playback. For example, it can be divided into a screen facing the driver to provide navigation services for the driver, a sub-screen facing the rear seats to provide video services for the rear passengers, and a sub-screen facing the front passenger to provide video services for the front passenger.

[0040] In this embodiment of the disclosure, the processor can maintain a window metadata table for all currently active windows, such as window identification (ID), application, window type (primary / secondary / global), screen coordinate range (x1, y1, x2, y2), Z-axis level, visibility status, etc.

[0041] Upon receiving a screen control command, the target description (e.g., "passenger air conditioning panel") in the command can be combined with the spatial context (e.g., "passenger area") to match the window corresponding to the target description (i.e., the target window) in the following way: Semantic matching: Maps the "air conditioner panel" to the registered air conditioner control interface window; Region filtering: Search for windows that meet the criteria only within the passenger's physical area (such as the right half of the screen); Priority sorting: If there are multiple candidates, select the window that is foremost on the Z-axis and has the highest visibility (or display it as the top layer).

[0042] In some embodiments, the target window can also be the playback window corresponding to the screen control command, such as "display the video being played in full screen" where the screen control command is "display the video being played in full screen" and the target window is the playback window of the video content; the target window can also be the desktop display corresponding to the screen control command, such as "return to the driver's desktop" where the screen control command is "return to the driver's desktop". Specifically, the embodiments disclosed herein do not specifically limit the screen control commands and the target window.

[0043] Within the target window, further identify the specific interactive element that the user intends to operate, i.e., the focus window. The focus window can be the window of a specific application or the window of a specific operation component.

[0044] In this embodiment of the disclosure, the processor injects a corresponding focus window into the target window by calling a specific logic module. For example: If the operation is "click", then send a click operation event; If the operation is "swipe", calculate the start and end coordinates and send a scroll operation or simulate a touch trajectory.

[0045] In one possible implementation, security checks can be added to the system during control execution, for example as follows: Verify that the focused window is in an operable state; Check the current user permissions; for example, the passenger's commands cannot operate the driver's safety-related controls. To prevent high-frequency false triggering, an operation cooling time can be added in this embodiment to prevent high-frequency false triggering.

[0046] In another possible implementation, upon completion of the operation, optional operation feedback can be generated, such as generating a voice announcement: "The passenger seat temperature has been increased." Visual highlighting of focus elements can also be used, for example, to debug or assist visual users. The focus history can also be updated for reference in subsequent commands.

[0047] In this embodiment, screen control commands are first acquired, and then the focus window is controlled within a target window based on the screen control commands. The target window is the window corresponding to the screen control command, and the focus window is the window that needs to be controlled according to the screen control command. Thus, this solution transforms ambiguous natural language or remote commands into precise control behaviors with clear spatial context and operation objects, improving control accuracy and security. It provides a technical foundation for multi-user collaborative display and barrier-free human-computer interaction, effectively solving the problem in existing systems of "seeing but not being able to click accurately, speaking clearly but not being able to control correctly."

[0048] In the above embodiments, the target window can also be determined by... Figure 2 To further explain, the method includes: S201, retrieve the window identifier from the screen control command.

[0049] It should be noted that the window identifier is pre-designed and can take many forms; no restrictions are imposed here.

[0050] In one possible implementation, window identifiers can be semantic identifiers, such as natural language descriptions like "navigation window," "air conditioning panel screen," "music playback screen," "driver's first display screen (key parameter screen)," "passenger's display screen," and "driver's second display screen (central control screen)."

[0051] In another possible implementation, the window identifier can also be a spatial identifier, such as "driver's side window", "right pop-up", "top notification bar", or "DOC shortcut bar".

[0052] In another possible implementation, the window identifier can also be a function identifier, such as "the window that is playing a video" or "the recently opened settings page".

[0053] In another possible implementation, window identifiers can also be pre-designed numbers or codes: for example, "Window 1", "Window 2", "Window A", "Window B", etc.

[0054] There are several ways to obtain the window identifier in the screen control instructions. In one possible implementation, key information in the screen control instructions can be obtained, and then the window identifier can be generated based on the key information.

[0055] In multi-tasking, multi-window smart terminal systems, users trigger screen control commands through actions such as clicking, swiping, voice commands, and vehicle auto-triggers. For commands to be executed correctly, the system needs to clearly identify which window the command applies to (i.e., the target window). However, not all control commands explicitly carry a complete window identifier. Failure to accurately reconstruct the window identifier can lead to mis-issued commands, functional malfunctions, or security risks.

[0056] In this embodiment of the disclosure, the window identifier can be determined by extracting key information.

[0057] In one possible implementation, fields that can be used for window recognition can be parsed from screen control instructions, such as application package name, window title / label, UI element ID, and entity in voice intent. The key information is then used as query conditions to initiate a matching request to the processor or operating system, and a window identifier is generated based on the matching result.

[0058] S202, determine the target window from multiple candidate windows based on the window identifier.

[0059] In this embodiment, each candidate window may include one or more window identifiers, but there is no overlap between the window identifiers of any pair of candidate windows. Therefore, the target window can be determined by matching the window identifier with the window identifier of each candidate window. Through structured window identifier extraction and multi-dimensional candidate window matching, a high-precision and robust target window determination mechanism is constructed, providing key technical support for the implementation of new interaction methods such as voice, gesture, and remote control in a multi-window environment, and significantly improving the accuracy, security, and user experience of human-computer interaction.

[0060] In one possible implementation, in response to the absence of a window identifier in the screen control command, the location information or display hierarchy of the screen control command is obtained, and the target window matching the location information or display hierarchy is determined from multiple candidate windows. By introducing location information or display hierarchy as supplementary basis for window recognition, the target window can still be accurately and reliably determined even when the screen control command lacks a window identifier. This supports a more natural, flexible, and secure multi-window interaction experience, and is particularly suitable for complex scenarios such as split-screen, floating windows, system controls, multimodal input, and multi-level display.

[0061] It should be noted that the location information in this embodiment includes, but is not limited to: Absolute coordinates, such as "click coordinates (1200, 600)"; Relative location descriptions, such as "top right corner", "left third area", "bottom pop-up"; Directional indicators, such as "left window", "top notification bar", "bottom DOC shortcut bar"; Spatial relationship description, such as "buttons near the volume icon".

[0062] Based on location information, the system traverses multiple currently active candidate windows to obtain the layout attributes of each candidate window on the display screen, including the window's screen coordinate range (x1, y1, x2, y2), center point position, logical region to which it belongs (such as driver's area, passenger's area), and Z-axis level.

[0063] Subsequently, a spatial matching algorithm is used to determine the target window that best matches the location information. Specific matching strategies include: If the location information is absolute coordinates, then select the foremost (highest Z-order) visible window containing those coordinates; If the location information is a relative area description, such as "right side", then the candidate windows whose center horizontal coordinate is located in the right half of the screen are selected, and the one with the highest Z-axis level is selected first. If the location information is a directional indicator, such as "the left window", then calculate the horizontal distance between the center point of each window and the center of the screen, and select the valid window that is closest to the specified direction; In the case of multiple matching windows, priority is further sorted by window visibility, focus state and recent interaction timestamp to ensure that the target window matches the user's current operation context.

[0064] Finally, the candidate window with the highest matching degree is determined as the target window, and it is used as the operation context for subsequent focus element recognition and interactive control.

[0065] In the above embodiments, the focus window can be determined based on screen control commands, or it can be determined through... Figure 3 To further explain, the method includes: S301, retrieve the focus window identifier from the screen control command.

[0066] S302, determine the focus window based on the focus window identifier.

[0067] In multi-window, highly complex human-computer interaction interfaces (such as in-vehicle smart cockpits, ultra-widescreen displays, and multi-tasking operating systems), users often need to perform operations on specific controls (such as buttons, sliders, input boxes, and display borders) or specific applications (such as navigation software, music playback software, video software that can be displayed in full screen, and control icons in the DOC shortcut bar). To achieve precise control, this solution introduces a focus window identification mechanism. By parsing the focus identification information in the screen control commands, it accurately identifies the focus window that the user intends to operate on from multiple interactive elements within the target window, thereby supporting subsequent fine-grained operations such as clicking, swiping, inputting, and adjusting.

[0068] The focus window identifier can take many forms, and no specific limitations are imposed here. For example, it may include the following forms: Semantic descriptive identifiers, such as "volume plus button", "temperature slider", and "cancel button in confirmation dialog box"; Structured attribute identifiers, such as control ID, text label, control type, control icon, spatial location identifier, application (software) name, etc.

[0069] Functional role identifiers, such as "currently playing video application" or "main button in recently popped-up alert boxes".

[0070] In the above embodiments, screen control commands can also be generated through... Figure 4 To further explain, the method includes: S401, acquire user input command data and location information, wherein the command data is one of voice data, eye-tracking data or gesture data.

[0071] In this embodiment of the disclosure, the instruction data originates from at least one of the following natural interaction modalities.

[0072] Voice data: User voice can be collected through vehicle microphone arrays or mobile device pickup modules.

[0073] Eye-tracking data: This includes gaze coordinates, gaze duration, and saccade trajectory, which can be obtained through infrared cameras or dedicated eye-tracking sensors.

[0074] Gesture data: Gesture types (such as tap, swipe, clench fist), gesture trajectories, and hand spatial coordinates that can be recognized by devices such as millimeter-wave radar or cameras.

[0075] Vehicle driving status: can detect the driving status of the vehicle (such as parking, assisted driving, etc.).

[0076] Location information is used to describe the positioning of user interaction behavior in the screen coordinate system or physical space, and may include a variety of information, such as: absolute screen coordinates (x, y): such as the pixel position of the gaze point output by eye tracking; relative area identifiers: such as "driver's side", "lower right of passenger side", "top of center console", "DOC shortcut bar"; spatial pointing vectors: such as the direction vector formed by the starting and ending points of the gesture; multimodal fusion location: for example, "speaking 'that button' + looking at a certain area" will use the gaze area as the location context.

[0077] S402 generates screen control commands based on command intent, command parameters, and location information, wherein the command intent and command parameters are generated based on command data.

[0078] In this embodiment of the disclosure, the instruction intent and instruction parameters can be extracted using a corresponding parsing engine based on the modal type of the instruction data: Voice data parsing: Converting speech into text through automatic speech recognition; Extraction using natural language understanding models: Eye-tracking data analysis: In one possible approach, eye movements themselves typically do not carry semantic intent, but can assist in intent disambiguation or trigger gaze + fixation pause operations.

[0079] For example, if a button is stared at for more than a threshold time (e.g., 800ms), the intent is inferred to be a click, and the stared area is used as the location anchor point of the "target object" in combination with the historical context.

[0080] Gesture data parsing: This involves using gesture recognition algorithms, such as MediaPipe or custom convolutional neural network models, to output the data. For example, the intention and parameters of the command are determined based on start and end coordinates, swipe direction, zoom level, and gesture speed.

[0081] By combining location information (such as coordinates or area labels) with command parameters (such as "air conditioner" or "adjust display range (size)"), target windows are filtered from the currently active windows (see the aforementioned window identifier matching mechanism for details).

[0082] Within the target window, specific control operations are determined using location information (such as eye-tracking coordinates, passenger-side display) or parameter descriptions (such as "temperature slider" or "full-screen display"). The final output screen control commands can be directly executed by the underlying window management service, accessibility service, or automation framework, completing the closed loop from user intent to UI operation.

[0083] In this embodiment, user-input command data and location information are first acquired. The command data can be voice data, eye-tracking data, or gesture data. Then, screen control commands are generated based on the command intent, command parameters, and location information, wherein the command intent and command parameters are generated based on the command data. Therefore, this solution achieves high-precision, low-burden, and cross-scenario natural human-computer interaction by integrating multimodal inputs such as voice, eye-tracking, gestures, and manual input, and combining intent, parameters, and location information to generate structured screen control commands. This significantly enhances the system's universality, accessibility, and intelligence level while improving operational accuracy and efficiency.

[0084] In another possible implementation, the user can directly input command data, and the processor can generate screen control commands based on this data. For example, the user can input command data via a touchscreen or terminal in the vehicle, and the processor will generate screen control commands upon receiving the data. This eliminates the need for multi-level menus or voice input; the user can directly input structured commands on the touchscreen interface. The system avoids complex processing steps such as voice recognition and natural language understanding. As a parallel solution for receiving screen control commands, this approach increases its practicality.

[0085] In the above embodiments, the focus window can be controlled within the target window based on screen control commands, and can also be controlled through... Figure 5 To further explain, the method includes: S501 is a logic operation module that determines logic based on screen control commands.

[0086] In practice, on smart terminals (such as smartphones, smart cockpits, and smart home control screens), users not only operate specific applications but also frequently use a range of system-level or device-level "non-application functions." These functions are typically not part of any single application but are provided directly by the operating system or device firmware. Although these functions lack independent application interfaces, they are crucial to the user experience.

[0087] In this embodiment, each non-application function is encapsulated as an independent logic operation module. It should be noted that a logic operation module is a lightweight, reusable functional unit that encapsulates specific interaction logic and UI components. It can include various types; for example, it may include: Control panel modules: such as volume / brightness sliders, temperature regulators; Quick action menus: such as the "Copy / Paste / Share" context menu; Input assistance modules: such as virtual keyboard, numeric dial pad, and voice input pop-up window; Status feedback module: such as progress bar, confirmation dialog box, operation success message; Adaptive toolbar: A collection of function buttons that are dynamically grouped according to the current application context.

[0088] In this embodiment of the disclosure, the logic operation module determined based on the screen control instructions can be determined by static matching. For example, if the instruction intent corresponds one-to-one with the pre-registered module, it can be loaded directly.

[0089] It can also be dynamically generated. For example, if the instruction contains compound parameters (such as "create a brightness adjustment bar from 10% to 90%), the module will be built in real time based on the template engine.

[0090] In another possible implementation, context-dependent adaptation is also possible: the style, language, and units of the module are automatically adjusted based on the target window's region (driver / passenger), user preferences, or system settings.

[0091] S502 assigns the logic operation module to the focus window and displays it in the target window.

[0092] After obtaining the logic operation module, logically associate the logic operation module with the focus window to establish a "trigger source - operation module" mapping relationship.

[0093] After binding the logic operation module to the focus window, the focus window gains extended interactive capabilities. For example, a text box that originally only displays the temperature value can now pop up an adjustment slider when clicked.

[0094] In this embodiment of the disclosure, a logic operation module can be dynamically assigned to an empty focus window, and the combined result can be displayed in the target window. The focus window itself does not contain any specific content; it serves only as a container and can be flexibly assembled and presented according to different logic operation modules.

[0095] It should be noted that the display position of the logic operation module is determined by the following factors: Focus window coordinates: The module appears near the focus window by default (such as directly below or offset from the upper right corner); Screen available space: Avoid obscuring key information and automatically avoid the status bar, navigation bar, or other windows; Regional affiliation constraint: Ensure that modules are displayed based on display priority. When a window with a high display priority is executed, the display range of a window with a low display priority is dynamically adjusted. For example, when the driver's screen and the passenger's screen share the same large display screen, when the driver's screen expands its display range, the passenger's screen automatically shrinks its display range. User history preference: If a user has dragged a similar module before, the last position will be used first.

[0096] The display methods for logic operation modules can be varied, and no limitations are set here. For example, they may include: Embedded: As part of the target window's user interface tree, it is managed throughout the window's lifecycle; Floating: Displayed as a system-level floating window, supporting visual effects such as semi-transparency, rounded corners, and shadows; Modal pop-ups: For operations that require user confirmation (such as deletion), a blocking dialog box is presented.

[0097] In another possible implementation, the processor can also invoke the graphics rendering engine to draw the logic operation module onto the target window's layer. Furthermore, controls within the module (such as sliders and buttons) register event listeners to support subsequent user interactions (such as dragging the slider or clicking confirmation). The operation results can be fed back to the original focused window in real time (e.g., temperature value updates) or trigger new screen control commands.

[0098] In this embodiment, a logical operation module is first determined based on screen control instructions, then the logical operation module is assigned to the focus window and displayed in the target window. This solution achieves a high degree of coordination between function, context, and interface by driving module selection through instructions, binding the focus window, and finally displaying the control in the target window, thereby improving the user experience and system efficiency in complex human-computer interaction scenarios.

[0099] This disclosure uses the example of screen control commands including adjusting the display range of at least one target application or desktop shortcut component to illustrate the following. When controlling the focus window in the target window based on screen control commands, the following methods can be used, but are not limited to: adjusting the display range of the target application or the desktop shortcut component in the target window according to the display priority of the target application or the desktop shortcut component.

[0100] In the embodiments of this disclosure, the display mode adjustment command is a control signal used to change the display size, display position, or display form of one or more target applications or desktop shortcut components on the display screen. The target application may be a navigation application, video playback application, music application, vehicle settings application, etc., and the specific application is not limited. A desktop shortcut component refers to a panel element located on the desktop interface used to quickly launch corresponding functions or display information, such as a desktop toolbar (e.g., a Dock bar), function cards (e.g., weather cards, media cards), etc., and the specific desktop shortcut component is not limited. The display range refers to the area occupied by the target application or desktop shortcut component on the screen, which can be expressed by width, height, area, or a ratio relative to the screen. It should be noted that the above examples are merely illustrative and do not limit the specific content.

[0101] The sources of display mode adjustment commands include, but are not limited to, the following: zoom-in commands, expand commands, application open commands, or application close commands manually input by the user via the touchscreen; control commands input by the user via voice; and adjustment commands automatically generated by the vehicle system based on vehicle status (such as gear changes, parking start, etc.). For example, if a user taps the icon of a target application on the touchscreen, the system generates a command to open that target application; this command is a display mode adjustment command used to adjust the display area of ​​the target application from zero to a preset size. If the user says "zoom in on navigation" via voice, the system generates a command to zoom in on the display area of ​​the navigation application. When the vehicle shifts from a non-P gear to P gear, the system automatically generates a command to switch the video playback target application to full-screen display. It should be noted that the above examples are for illustrative purposes only and do not limit the specific content.

[0102] As an optional approach, the display mode adjustment command may also include instructions to expand or collapse desktop shortcut components (such as the Dock or function cards); when the user clicks the expand button of the Dock, the system obtains the adjustment command to expand the Dock, which is used to increase the display area of ​​the desktop shortcut component or change its display form.

[0103] Through the above methods, the system can identify display range adjustment needs from different sources, providing a basis for adjusting the display range according to display priority.

[0104] In the embodiments of this disclosure, display priority refers to the display allocation level pre-set for a target application or desktop shortcut component; wherein, target control refers to a desktop shortcut component or an interactive element corresponding to a desktop shortcut component; when multiple target applications or desktop shortcut components simultaneously need to occupy display area (target window), or when changes in the display range of a target application or desktop shortcut component affect the display range of other components, the system determines the adjustment method of the display range of each component based on the display priority. The display priority can be preset by the system, or it can be dynamically adjusted according to user habits or vehicle status, and the specifics are not limited.

[0105] Adjusting the display area of ​​a target window can be done by: increasing or decreasing the display size of the target application or control's window; moving the display position of the target application or control; removing the target application or control from the target window (i.e., shrinking the target window's display area to zero); or switching the target application or control from a hidden state to a visible state (i.e., expanding the display area from zero to a preset size). The adjustment is based on display priority: target applications or desktop shortcuts with higher display priority have priority in display resource allocation, and changes to their display area will squeeze the display area of ​​target applications or controls with lower display priority; conversely, changes to the display area of ​​lower-priority target applications or controls will not affect higher-priority components.

[0106] In this way, the system can automatically coordinate the display area of ​​each target application or desktop shortcut component in the target window according to the display priority, thus avoiding display conflicts.

[0107] Optionally, based on any of the above embodiments or optional methods, the present disclosure embodiments may also employ the following method, but not limited thereto: the display priority includes layer display priority; in any two layers, the target window corresponding to the layer with higher display priority covers the target window corresponding to the layer with lower display priority.

[0108] In this embodiment of the disclosure, a layer refers to an independent plane on the display screen used to carry different types of display content; each layer may contain one or more target windows corresponding to target applications or desktop shortcuts. Layer display priority refers to the display hierarchy order pre-set for different layers, which is used to determine the occlusion relationship of each layer when the projection areas of multiple layers on the screen overlap.

[0109] In any two layers, the target window of the layer with higher display priority is visually positioned above the target window of the layer with lower display priority. In other words, the content of the high-priority layer will obscure the content of the corresponding overlapping area in the low-priority layer. The overlapping relationship between layers does not affect the independent operation and updating of the content within each layer, but only determines the final visual effect presented on the screen.

[0110] As an optional approach, the number of layers, their names, and the display priority of each layer can be preset by the system or dynamically adjusted based on vehicle status or user configuration. For example, in a parking scenario, the surround view image can be temporarily elevated to the highest display priority for the corresponding target window to ensure that the parking view is not obscured by other content. It should be noted that the above examples are for illustrative purposes only and do not limit the specific content.

[0111] In another embodiment of this disclosure, the screen control command is to display an interactive area. When controlling the focus window in the target window based on the screen control command, it can be implemented in the following ways, but is not limited to: displaying the interactive area at the target position of the target window; wherein the target window is the driver's display area and / or the passenger's display area.

[0112] As one possible approach, the interactive area is located at a target position within a target window, which is either the driver's side display area or the passenger side display area. The interactive area can be displayed only in the driver's side display area, or only in the passenger side display area. Alternatively, the interactive area can be displayed simultaneously in both the driver's side display area and the passenger side display area.

[0113] In another embodiment, the interactive area differs from the persistently displayed target application or desktop shortcut; it typically appears after the interaction begins and disappears after the interaction ends or the exit condition is met. The target position refers to the space reserved for the interactive area within the driver's or passenger's display area; this position can be fixed or dynamically calculated. It should be noted that the above examples are merely illustrative and do not limit the specific content.

[0114] Specifically, when the interactive information is triggered in the driver's area, the system displays the interactive area at the target location in the second sub-display area; when the interactive information is triggered in the passenger's area, the system displays the interactive area in the second display area through a floating window.

[0115] The distinction between the driver's and passenger's areas is determined by sound zones; sound zone recognition refers to using a microphone array to collect sound source signals, determine the direction of the voice command, and thus determine whether the voice command comes from the driver's or passenger's area. As an optional method, in addition to sound zone recognition, the trigger area can also be determined by the coordinate position of the touch point on the touchscreen; the specific recognition method is not limited.

[0116] Corresponding to the vehicle screen control methods provided in the above embodiments, one embodiment of this disclosure also provides a vehicle screen control device. Since the vehicle screen control device provided in this disclosure corresponds to the vehicle screen control methods provided in the above embodiments, the implementation methods of the above vehicle screen control methods are also applicable to the vehicle screen control device provided in this disclosure, and will not be described in detail in the following embodiments.

[0117] Figure 6 Figure 6 shows a schematic diagram of an in-vehicle screen control device proposed in this disclosure. The in-vehicle screen control device 600 includes: an acquisition module 610 and a control module 620.

[0118] The acquisition module 610 is used to acquire screen control commands.

[0119] The control module 620 is used to control the focus window in the target window based on screen control commands, wherein the target window is the window corresponding to the screen control command, and the focus window is the window that needs to be controlled according to the screen control command.

[0120] According to one embodiment of this disclosure, determining the target window includes: obtaining a window identifier in the screen control command; and determining the target window from a plurality of candidate windows based on the window identifier.

[0121] According to one embodiment of this disclosure, the method further includes: in response to the absence of a window identifier in the screen control instruction, obtaining the location information of the screen control instruction; and determining a target window that matches the location information from a plurality of candidate windows.

[0122] According to one embodiment of this disclosure, determining the focus window based on the screen control command includes: obtaining a focus window identifier in the screen control command; and determining the focus window based on the focus window identifier.

[0123] According to one embodiment of this disclosure, generating the screen control command includes: acquiring user-input command data and location information, wherein the command data is one of voice data, eye-tracking data, or gesture data; and generating the screen control command based on the command intent, command parameters, and the location information, wherein the command intent and the command parameters are generated based on the command data.

[0124] According to one embodiment of this disclosure, generating the screen control instructions includes: acquiring instruction data input by a user; and generating the screen control instructions based on the instruction data.

[0125] According to one embodiment of this disclosure, controlling the focus window in the target window based on the screen control instructions includes: determining a logical operation module based on the screen control instructions; assigning the logical operation module to the focus window and displaying it in the target window.

[0126] According to one embodiment of this disclosure, when the screen control command includes adjusting the display range of at least one target application or desktop shortcut component, controlling the focused window within the target window based on the screen control command includes: Adjust the display range of the target application or desktop shortcut in the target window according to the display priority of the target application or desktop shortcut.

[0127] According to one embodiment of this disclosure, in any two layers, the target window corresponding to the layer with higher display priority can be overlaid on the target window corresponding to the layer with lower display priority; or... In any two applications, the target window of the application with higher display priority can encroach on the display area of ​​the target window of the application with lower display priority.

[0128] According to one embodiment of this disclosure, the screen control command is a display interaction area, and the control of the focused window within the target window based on the screen control command includes: The interactive area is displayed at the target location of the target window; wherein the target window is the driver's display area and / or the passenger's display area.

[0129] Therefore, this solution transforms ambiguous natural language or remote commands into precise control behaviors with clear spatial context and operation objects, improving control accuracy and security, providing a technical foundation for multi-user collaborative display and barrier-free human-computer interaction, and effectively solving the problem of "seeing but not being able to click accurately, speaking clearly but not being able to control correctly" in existing systems.

[0130] To implement the above embodiments, this disclosure also proposes an electronic device 700, such as... Figure 7 As shown, the electronic device 700 includes a processor 701 and a memory 702 communicatively connected to the processor. The memory 702 stores instructions that can be executed by at least one processor. The instructions are executed by at least one processor 701 to implement the vehicle screen control method as described in the first aspect of this disclosure.

[0131] To implement the above embodiments, this disclosure also proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to implement the vehicle screen control method as described in the first aspect of this disclosure.

[0132] To implement the above embodiments, this disclosure also proposes a computer program product, including a computer program that, when executed by a processor, implements the vehicle screen control method as described in the first aspect of this disclosure.

[0133] To implement the above embodiments, this disclosure provides a vehicle including the above-described in-vehicle screen control device; or the above-described electronic device; or the above-described computer-readable storage medium.

[0134] In the description of this disclosure, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this disclosure and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this disclosure.

[0135] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise expressly specified.

[0136] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0137] Although embodiments of the present disclosure have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A method for controlling an in-vehicle screen, characterized in that, The method includes: Obtain screen control commands; The screen control command controls the focus window within the target window, where the target window is the window corresponding to the screen control command, and the focus window is the window that needs to be controlled according to the screen control command.

2. The method according to claim 1, characterized in that, Determining the target window includes: Obtain the window identifier from the screen control command; The target window is determined from multiple candidate windows based on the window identifier.

3. The method according to claim 2, characterized in that, The method further includes: Generating the window identifier, wherein generating the window identifier includes: Obtain key information from the screen control instructions; The window identifier is generated based on the key information.

4. The method according to claim 2 or 3, characterized in that, The method further includes: In response to the absence of a window identifier in the screen control command, the location information of the screen control command is obtained; The target window that matches the location information is determined from multiple candidate windows.

5. The method according to any one of claims 1-4, characterized in that, Determining the focus window based on the screen control instructions includes: Obtain the focus window identifier from the screen control command; The focus window is determined based on the focus window identifier.

6. The method according to any one of claims 1-5, characterized in that, The method further includes: Generating the screen control instructions, wherein generating the screen control instructions includes: The system acquires user-input command data and location information, wherein the command data is one of voice data, eye-tracking data, or gesture data. The screen control command is generated based on the command intent, command parameters, and the location information, wherein the command intent and the command parameters are generated based on the command data.

7. The method according to any one of claims 1-5, characterized in that, The method further includes: Generating the screen control instructions, wherein generating the screen control instructions includes: Obtain user input command data; The screen control instructions are generated based on the instruction data.

8. The method according to claim 1, characterized in that, The control of the focused window in the target window based on the screen control command includes: The logic operation module is determined based on the screen control instructions; The logic operation module is assigned to the focus window and displayed in the target window.

9. The method according to any one of claims 1-8, characterized in that, When the screen control commands include adjusting the display range of at least one target application or desktop shortcut component, the control of the focused window within the target window based on the screen control commands includes: Adjust the display range of the target application or desktop shortcut in the target window according to the display priority of the target application or desktop shortcut.

10. The method according to claim 9, characterized in that, In any two layers, the target window corresponding to the layer with higher display priority can be overlaid on the target window corresponding to the layer with lower display priority; or, In any two applications, the target window of the application with higher display priority can encroach on the display area of ​​the target window of the application with lower display priority.

11. The method according to any one of claims 1-10, characterized in that, The screen control command is for displaying an interactive area. The control of the focused window within the target window based on the screen control command includes: The interactive area is displayed at the target location of the target window; wherein the target window is the driver's display area and / or the passenger's display area.

12. A vehicle-mounted screen control device, characterized in that, The device includes: The acquisition module is used to acquire screen control commands; The control module is used to control the focus window within a target window based on the screen control command, wherein the target window is the window corresponding to the screen control command, and the focus window is the window that needs to be controlled according to the screen control command.

13. An electronic device, characterized in that, Including memory and processor; The processor reads executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement the method as described in any one of claims 1-11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-11.

15. A vehicle, characterized in that, include: The vehicle screen control device as described in claim 12; Or the electronic device as described in claim 13.

Images