A multitask window scheduling method, system and vehicle-mounted terminal

Abstract

This application discloses a multi-task window scheduling method, system, and vehicle terminal. The method includes: receiving a request to add an application to a multi-window display; binding the application window of the application to the seat area indicated by the target seat area identifier carried in the request, obtaining a window-level seat area binding table; and publishing the seat area binding status containing the table to a status bus; obtaining the status from the status bus; determining the application windows allowed to operate on each input source based on the binding table; determining the focus window and directing the input event to the focus window upon receiving an input event; and determining the audio output channel of each application window and outputting audio accordingly. This application establishes seat area affiliation on an application window basis, ensuring that input operations and audio output of each application window within a single screen are based on the same binding table, achieving cross-seat area input and audio isolation within a single screen, and avoiding inconsistencies caused by scattered affiliation information.

Description

Technical Field

[0001] This application relates to the field of vehicle electronic technology, and in particular to a multi-task window scheduling method, system and vehicle terminal. Background Technology

[0002] In-vehicle infotainment systems can now display and run multiple application windows in parallel on a single in-vehicle display screen. For example, a navigation application window, a music application window, and a video application window can be displayed simultaneously on the central control display screen for different passengers to use.

[0003] Current in-vehicle operating systems divide seating areas at the display screen level, with each display screen belonging to a single seating area. Whether an input source is allowed to operate a particular application window, and which audio output channel the audio of a particular application window is played from, are all derived from the seating area to which the display screen containing the application window belongs. Furthermore, the display layout, input focus, and audio output channel of the application window are determined by independent subsystems, each of which stores the information used for its determination and synchronizes with each other through mutual calls.

[0004] When a single in-vehicle display shows multiple application windows facing different occupants in parallel, the premise that one display corresponds to one seating area no longer holds true. The application windows themselves do not have seating area affiliation, so it is impossible to determine which application windows each input source is allowed to operate on, nor is it possible to determine the audio output channel of each application window. The input source corresponding to the driver's seat area can obtain focus on the application window facing other occupants, and the audio of that application window is also sent to the vehicle's common sound field. At the same time, the affiliation information on which the determination is based is stored in various subsystems, which is prone to inconsistencies, and any change requires multiple mutual calls to be transmitted to other subsystems, resulting in a large linkage delay. Summary of the Invention

[0005] This application aims to solve the problem that when multiple application windows are displayed in parallel on a single vehicle display screen, each application window does not have a seat area affiliation, resulting in the display, input operation and audio output being unable to be performed based on consistent affiliation information.

[0006] Firstly, this application provides a multi-task window scheduling method applied to an in-vehicle terminal. The in-vehicle terminal includes an in-vehicle display screen, multiple input sources, and multiple audio output channels. The in-vehicle display screen displays multiple application windows in parallel to form a multi-window display. Each application window is used to carry an application that requests to join the multi-window display. The vehicle where the in-vehicle terminal is located is divided into multiple seating areas. Each input source corresponds to one seating area, and each audio output channel corresponds to one seating area. The method is characterized by having a state bus that transmits states via a publish-subscribe mechanism. The method includes the following steps: Receive a request to add the application to be added to the multi-window display; the request carries a target seating area identifier, which is used to indicate one of the multiple seating areas; The application window of the application to be added is bound to the seat area indicated by the target seat area identifier to obtain a window-level seat area binding table; the window-level seat area binding table records the correspondence between each application window and the seat area bound to that application window; The seat area binding status, which includes the window-level seat area binding table, is published to the status bus; The seat area binding status is obtained from the state bus, and the first operation and the second operation are performed according to the window-level seat area binding table therein; The first operation includes: for each input source, determining whether the seating area bound to each application window matches the seating area corresponding to the input source, and determining the application window whose determination result matches as the application window allowed to operate on the input source; upon receiving an input event from a target input source among the plurality of input sources, determining one of the application windows allowed to operate on the target input source as the focus window, publishing the focus attribution status to the status bus, and directing the input event to the focus window according to the focus attribution status; the focus attribution status includes the identifier of the focus window; The second operation includes: for each application window, determining the audio output channel corresponding to the seating area bound to the application window as the audio output channel of the application window, publishing the audio authorization status to the status bus, and directing the audio of the application window to the audio output channel of the application window according to the audio authorization status; the audio authorization status includes the correspondence between each application window and the audio output channel of the application window.

[0007] Secondly, this application also provides a multi-task window scheduling system, the system being installed in an in-vehicle terminal, the in-vehicle terminal including an in-vehicle display screen, multiple input sources, and multiple audio output channels, the in-vehicle display screen displaying multiple application windows in parallel to form a multi-window display, each application window being used to carry an application that has been added to the multi-window display; the vehicle where the in-vehicle terminal is located is divided into multiple seating areas, each input source corresponding to one seating area, and each audio output channel corresponding to one seating area; the system includes: A state bus is used to transmit state in a publish-subscribe manner. A window registration module, connected to the status bus, is used to receive a request to add an application to the multi-window display, the request carrying a target seating area identifier, the target seating area identifier being used to indicate one of the multiple seating areas; The window registration module is also used to bind the application window of the application to be added to the seat area indicated by the target seat area identifier to obtain a window-level seat area binding table, and publish the seat area binding status containing the window-level seat area binding table to the status bus. The window-level seat area binding table records the correspondence between each application window and the seat area bound to that application window. A focus arbitrator, connected to the state bus, is used to obtain the seat area binding status from the state bus, and according to the window-level seat area binding table therein, for each input source, determine whether the seat area bound to each application window matches the seat area corresponding to the input source, and determine the application window whose determination result matches as the application window that is allowed to operate on the input source. The focus arbiter is further configured to, upon receiving an input event from a target input source among the plurality of input sources, determine a focus window within the application window where the target input source is allowed to operate, publish the focus attribution status to the status bus, and direct the input event to the focus window according to the focus attribution status; the focus attribution status includes an identifier of the focus window; An audio arbitrator, connected to the state bus, is used to obtain the seat area binding status from the state bus, and, based on the window-level seat area binding table therein, determine the audio output channel corresponding to the seat area bound to the application window as the audio output channel of the application window for each application window. The audio arbitrator is also used to publish the audio authorization status to the status bus, and to direct the audio of each application window to the audio output channel of the application window according to the audio authorization status; the audio authorization status includes the correspondence between each application window and the audio output channel of the application window.

[0008] Thirdly, this application also provides an in-vehicle terminal, including: At least one processor; A memory that is communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the multitasking window scheduling method as described in the first aspect.

[0009] Fourthly, this application also provides a non-volatile computer storage medium, characterized in that the non-volatile computer storage medium stores computer-executable instructions; The computer-executable instructions are executed by one or more processors, which may cause the one or more processors to perform the multitasking window scheduling method as described in the first aspect.

[0010] At least one advantage of the multi-task window scheduling method, system, and vehicle terminal provided in this application embodiment is that: each application window is bound to a seat area and recorded in a window-level seat area binding table, thus reducing seat area affiliation from the display screen granularity to the application window granularity, so that each application window has a clear seat area affiliation when multiple application windows are displayed in parallel on a single vehicle display screen; and the binding table is transmitted via a state bus in a publish-subscribe manner, and is determined by the application windows that are allowed to operate on each input source and the audio output channels of each application window according to the same binding table.

[0011] Therefore, input and audio isolation across seating areas can be achieved on the same in-vehicle display screen without the need for additional displays. Input sources whose corresponding seating areas do not match the seating areas bound to the application window will not gain focus from the application window, and the audio of the application window will only be output to the audio output channel corresponding to its bound seating area. At the same time, the determination of input and audio is based on the same binding table on the state bus, avoiding inconsistencies caused by the ownership information being stored in various subsystems. Furthermore, there is no need for mutual calls between the determinations, shortening the linkage path. Attached Figure Description

[0012] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0013] Figure 1 A schematic diagram of a multi-task window scheduling system provided for embodiments of this application; Figure 2 A flowchart illustrating a multi-task window scheduling method provided for an embodiment of this application; Figure 3 A schematic diagram illustrating the correspondence between the application window and the seating area, input source, and audio output channel provided for embodiments of this application; Figure 4 A flowchart illustrating the process of performing admission verification on a joining request and applying the window during driving state degradation, as provided in the embodiments of this application. Figure 5 A schematic diagram illustrating the state transition of a vehicle's driving state provided in an embodiment of this application; Figure 6 A timing diagram illustrating the dispatching of security alarm states via a state bus, as provided in the embodiments of this application. Figure 7 A schematic diagram of the state bus structure provided for an embodiment of this application; Figure 8 A schematic diagram illustrating audio level allocation and audio gain suppression provided for embodiments of this application; Figure 9 A schematic diagram illustrating the process of saving and restoring the seat area binding state as provided in the embodiments of this application; Figure 10 A schematic diagram of the structure of the vehicle-mounted terminal provided in the embodiments of this application. Detailed Implementation

[0014] To facilitate understanding of this application, a more detailed description is provided below with reference to the accompanying drawings and specific embodiments. It should be noted that when an element is described as being "fixed to" another element, it can be directly on the other element, or one or more intermediate elements may exist between them. When an element is described as being "connected" to another element, it can be directly connected to the other element, or one or more intermediate elements may exist between them. The terms "upper," "lower," "inner," "outer," "bottom," etc., used in this specification indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Furthermore, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0015] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items.

[0016] Furthermore, the technical features involved in the different embodiments of this application described below can be combined with each other as long as they do not conflict with each other.

[0017] Existing in-vehicle terminals can display and run multiple application windows in parallel on the central control display screen, such as simultaneously displaying navigation, music, and video application windows. Regarding the differentiation of occupant positions within the vehicle, current in-vehicle operating systems divide seating areas by display screen granularity; that is, a single display screen belongs to one seating area, the central control display screen belongs to the driver's seat area, and the rear display screens belong to the rear seat area. Whether an input source is allowed to operate a particular application window, and which audio output channel plays the audio from a particular application window, are all derived from the seating area to which the display screen containing that application window belongs.

[0018] In terms of processing, the determination of the application window display layout, the allocation of input focus, and the allocation of audio output channels are respectively completed by the window management subsystem, the input management subsystem, and the audio management subsystem. Each of the three subsystems saves a copy of the information used for judgment and synchronizes between the subsystems by calling each other.

[0019] When a single in-vehicle display shows multiple application windows facing different occupants in parallel, the premise that one display corresponds to one seating area no longer holds true. In this case, the application windows themselves do not have seating area affiliation, making it impossible to determine which application window each input source is allowed to operate on, or which audio output channel each application window should be directed to. As a result, input sources corresponding to the driver's seat area can obtain focus on the application window facing the front passenger, and the audio from that application window is also sent to the vehicle's common sound field. Multiple application windows on the same display cannot be isolated from each other in terms of input operation and audio output. Furthermore, because the information used for determination is stored across three subsystems, the affiliation information of the same application window is prone to inconsistencies across different subsystems, and any change requires multiple inter-system calls to be transmitted to the remaining subsystems, resulting in significant linkage delays.

[0020] Please see Figure 1 The vehicle-mounted terminal 100 described in this application includes a vehicle-mounted display screen 200, multiple input sources 300, and multiple audio output channels 400. The vehicle in which the vehicle-mounted terminal 100 is located is divided into multiple seating areas, with each input source 300 and each audio output channel 400 corresponding to one seating area.

[0021] A seating area refers to the area within a vehicle divided according to the seating positions of passengers. In this embodiment, the vehicle is divided into two seating areas: a driver's seat area and a front passenger seat area. Input sources 300 include input source 301 corresponding to the driver's seat area and input source 302 corresponding to the front passenger seat area. Input source 301 is a knob located on the center console for the driver to operate, and input source 302 is a touch input device located on the front passenger side. Audio output channels 400 include audio output channel 401 corresponding to the driver's seat area and audio output channel 402 corresponding to the front passenger seat area. Audio output channel 401 is connected to the in-vehicle speakers, and audio output channel 402 is connected to the headphone output on the front passenger side.

[0022] In other embodiments, the vehicle may be divided into more seating areas, such as the rear left seat area and the rear right seat area, and correspondingly more input sources and more audio output channels may be connected. The input sources may also be steering wheel buttons, voice input devices or touch input devices. This embodiment does not limit the number of seating areas or the specific form of input sources and audio output channels.

[0023] The vehicle-mounted display screen 200 displays multiple application windows in parallel to form a multi-window display. Multi-window display refers to a display method in which multiple application windows are displayed and run simultaneously on the same vehicle-mounted display screen. Each application window is used to carry an application that requests to join the multi-window display, and each application window occupies a non-overlapping display area on the vehicle-mounted display screen 200.

[0024] The vehicle-mounted terminal 100 is equipped with a scheduling system 1, which includes a state bus 10, a window registration module 20, a window scheduler 30, a focus arbiter 40, and an audio arbiter 50. It may also include a security protection module 60 and an application compatibility module 70. Each of the above modules is connected to the state bus 10, and there is no direct calling relationship between the modules.

[0025] State bus 10 refers to the component that transmits states in a publish-subscribe manner. The party that generates a state publishes the state to state bus 10, and the party that needs to use the state subscribes to the state from state bus 10 and obtains the state from state bus 10. The party that generates the state and the party that uses the state do not make direct calls to each other, nor do they need to know about each other's existence.

[0026] The state bus 10 can be implemented as a system-level service within the system service process of the vehicle terminal 100. In this case, modules within the same process publish and obtain status through in-process calls, and modules across processes access each other through inter-process communication interfaces. The state bus 10 can also be implemented as an independently running process, or it can be implemented using communication middleware for the vehicle field. In this case, the modules can be distributed on different processes or even different processors.

[0027] Please see Figure 2 Based on the aforementioned vehicle-mounted terminal 100, this embodiment provides a multi-task window scheduling method, which is executed by the vehicle-mounted terminal 100 and includes the following steps: S101, Receive a request to add the application to be added to the multi-window display, the request carrying the target seating area identifier.

[0028] The target seating area identifier is an identifier used to indicate one of the multiple seating areas divided in the vehicle. The seating area it indicates is the occupant position that the application window of the application to be added faces.

[0029] The request can be generated by the user clicking the application icon on the desktop of the in-vehicle terminal 100 and selecting the location to join, or by the user selecting via voice input or a knob. Taking the user clicking the video application and selecting to open it on the passenger side as an example, the received request carries a target seating area identifier indicating the passenger seat area.

[0030] S102. Bind the application window of the application to be added to the seat area indicated by the target seat area identifier to obtain the window-level seat area binding table.

[0031] The window-level seating area binding table 21 is a data table that records the correspondence between each application window and the seating area bound to that application window, on an application window basis. Unlike the method of determining seating area affiliation on a display screen basis, the window-level seating area binding table 21 allows different application windows on the same vehicle display screen 200 to be assigned to different seating areas.

[0032] Upon receiving the aforementioned request, the window registration module 20 reads the target seating area identifier carried in the request, establishes a correspondence between the application window of the application to be added and the seating area indicated by the identifier, and writes this correspondence into the window-level seating area binding table 21. When an application window that has been added for multi-window display exits, the window registration module 20 deletes the record corresponding to that application window from the window-level seating area binding table 21.

[0033] Taking the parallel display of three application windows on the vehicle display screen 200 as an example: application window 210 carries a navigation application, and the target seat area identifier carried in its join request indicates the driver's seat area; application window 220 carries a music application, and the target seat area identifier carried in its join request indicates the driver's seat area; application window 230 carries a video application, and the target seat area identifier carried in its join request indicates the front passenger seat area. After the three are added in sequence, the contents of the window-level seat area binding table 21 are shown in Table 1.

[0034]

[0035] Table 1 S103. Publish the seat area binding status, which includes the window-level seat area binding table, to the status bus.

[0036] Seat area binding status refers to a status defined by the window-level seat area binding table 21. The window registration module 20 is the publisher of this status. Whenever the content of the window-level seat area binding table 21 changes, the window registration module 20 publishes the seat area binding status containing the changed window-level seat area binding table 21 to the status bus 10.

[0037] S104. Obtain the seat area binding status from the state bus, and perform the first operation and the second operation according to the window-level seat area binding table therein.

[0038] Both the focus arbiter 40 and the audio arbiter 50 subscribe to the seat area binding state from the state bus 10. After obtaining the state from the state bus 10, they respectively execute the first operation (i.e., step S105) and the second operation (i.e., step S106) according to the window-level seat area binding table 21 in the state. The first operation and the second operation are based on the same window-level seat area binding table 21, and they do not directly call each other. The order of their execution does not affect their respective execution results. The focus arbiter 40 and the audio arbiter 50 can each initiate the acquisition from the state bus 10, or one of them can acquire the state and then share it with the other via the state bus 10.

[0039] S105. For each input source, determine whether the seating area bound to each application window matches the seating area corresponding to the input source, and determine the application window whose determination result matches as the application window that is allowed to operate on the input source; when receiving an input event from a target input source among multiple input sources, determine the application window that is allowed to operate on the target input source as the focus window, publish the focus ownership status to the status bus, and direct the input event to the focus window according to the focus ownership status.

[0040] An application window that is allowed to operate is an application window to which an input source can perform input operations. The focus arbiter 40, based on the window-level seating area binding table 21, retrieves each application window one by one for each input source and determines whether the seating area bound to the application window matches the seating area corresponding to the input source. If the result is a match, the application window is determined as an application window that is allowed to operate on by the input source; for application windows whose bound seating area does not match the seating area corresponding to the input source, the input source is not allowed to operate on the application window.

[0041] The focus window refers to the application window that receives input events at a given moment. The focus attribution state is a state indicating which application window is currently the focus window, and its content includes the identifier of that focus window. The focus arbitrator 40 is the issuer of this state. Attribution means that among multiple application windows displayed in parallel on the vehicle display screen 200 at the same time, the input event is directed to only one application window; the state records the application window to which the input event currently belongs.

[0042] The focus status is republished as the focus window changes, and the parties subscribing to the status perform corresponding processing: the audio arbitrator 50 determines the audio level corresponding to the focus window, and the window scheduler 30 displays the focus window on the vehicle display screen 200 with a highlighted border or other methods, so that the user can confirm which application window is currently being operated without having to stare at the vehicle display screen 200 for a long time.

[0043] Please see Figure 3 Continuing the previous example, input source 301 corresponds to the driver's seat area. Application windows 210 and 220 are both bound to the driver's seat area, which matches the seat area corresponding to input source 301. Therefore, input source 301 is allowed to operate application windows 210 and 220. Application window 230 is bound to the passenger seat area, which does not match the seat area corresponding to input source 301. Therefore, input source 301 is not allowed to operate application window 230. Correspondingly, input source 302 corresponds to the passenger seat area and is allowed to operate application window 230, but not application windows 210 and 220.

[0044] When the focus arbiter 40 receives an input event from the input source 301, the input source 301 becomes the target input source. The focus arbiter 40 determines only one of the application windows 210 and 220 as the focus window, publishes the identifier of the focus window as the focus ownership status to the status bus 10, and directs the input event to the focus window according to the focus ownership status. Therefore, the operation performed by the driver through the input source 301 will not fall into the application window 230 facing the front passenger.

[0045] In the implementation of focus orientation, the focus arbitrator 40 can set the focus window as the system's input focus window, and the input distribution component can distribute subsequent input events to the application window; alternatively, the focus arbitrator 40 can intercept input events at the input service layer and inject them directly into the focus window without changing the system's input focus.

[0046] S106. For each application window, determine the audio output channel corresponding to the seating area bound to the application window as the audio output channel of the application window, publish the audio authorization status to the status bus, and output the audio of the application window to the audio output channel of the application window according to the audio authorization status.

[0047] Audio license status refers to a status that defines the correspondence between each application window and its audio output channel, and the audio arbitrator 50 is the issuer of this status.

[0048] The audio output component of the vehicle terminal 100 subscribes to the audio authorization status on the status bus 10. The audio output component refers to the component in the vehicle terminal 100 used to send each audio channel into the corresponding audio output channel and complete the mixing. It does not belong to the scheduling system 1, but is connected to the status bus 10.

[0049] The audio arbitrator 50 determines the corresponding audio output channel for each application window based on the seat area bound to each application window as recorded in the window-level seat area binding table 21. Continuing the previous example, application windows 210 and 220 are bound to the driver's seat area, and their audio output channel is determined to be audio output channel 401; application window 230 is bound to the passenger seat area, and its audio output channel is determined to be audio output channel 402. The audio arbitrator 50 publishes this correspondence as an audio authorization status to the status bus 10, and according to this audio authorization status, directs the audio output of the navigation application and music application to audio output channel 401, and directs the audio output of the video application to audio output channel 402. Thus, the audio of the video application aimed at the passenger is only played at the headphone output on the passenger side, without entering the vehicle's common sound field and without interfering with the driver's listening to the navigation announcements.

[0050] In terms of implementing directional audio output, each seating area can be configured with an independent audio output bus. The audio hardware abstraction layer can send each audio stream to the corresponding output bus according to the correspondence recorded in the audio license status. Alternatively, the audio arbitrator 50 can bind different audio devices to the audio streams of each application window and the audio strategy component can complete the output.

[0051] Based on the above steps S101 to S106, the vehicle terminal 100 can also determine the display area and stacking order of each application window on the vehicle display screen 200 according to the window-level seat area binding table 21 in the seat area binding state obtained from the state bus 10, publish the window layout state to the state bus, and display each application window according to the window layout state. The window layout state includes the determined display area and stacking order of each application window.

[0052] Window layout state refers to a state in which the display area and stacking order of each application window on the vehicle display screen 200 are defined, and the window scheduler 30 is the issuer of this state. The display area refers to the position and size of the application window on the vehicle display screen 200, and the stacking order refers to the sequential relationship of each application window in the display depth direction.

[0053] The window scheduler 30 allocates display areas to each application window based on the number of application windows and their bound seating areas recorded in the window-level seating area binding table 21.

[0054] The window scheduler 30 has preset multiple display area allocation ratios, such as the ratio where the display area of ​​each application window is equal, the ratio where the display area of ​​one application window is a multiple of the other application windows, and the ratio where the display area of ​​one application window is zero and is shared by the other application windows. The window scheduler 30 selects one of the allocation ratios based on the number of application windows recorded in the window-level seat area binding table 21, and then determines the position of each display area on the vehicle display screen 200 based on the seat area bound to each application window, so that the display position of each application window matches the position of the passenger it faces.

[0055] The window scheduler 30 can also obtain the focus ownership status from the status bus 10, allocate a larger display area to the focus window indicated by the status, and allocate a smaller display area to the other application windows, so that the display area of ​​the application window currently operated by the user is larger than that of the other application windows.

[0056] Continuing the previous example, application windows 210 and 220 are bound to the driver's seat area and assigned to the display area on the vehicle display screen 200 closest to the driver; application window 230 is bound to the passenger seat area and assigned to the display area closest to the passenger, thus ensuring that the display position of each application window corresponds to the position of the passenger it faces. The window scheduler 30 publishes the determined display area and stacking order of each application window as a window layout state to the status bus 10, and displays each application window on the vehicle display screen 200 according to this window layout state.

[0057] In terms of display implementation, multiple parallel sub-display areas can be pre-set as window slots under the display area hierarchy corresponding to the vehicle display screen 200. The window scheduler 30 realizes the display layout by setting the boundaries and stacking order of each sub-display area. Alternatively, a free window approach can be adopted, where the window scheduler 30 directly sets the position and size of each application window. Or, a virtual display can be allocated to each application window, and the graphics compositing component can composite the images of each virtual display onto the vehicle display screen 200.

[0058] This embodiment reduces the seating area affiliation from the display screen granularity to the application window granularity. When an application adds multiple windows for display, a window-level seating area binding table 21 is established based on the target seating area identifier carried in its request. This ensures that each application window on the same vehicle display screen 200 has a clear seating area affiliation, overcoming the defect that application windows do not have seating area affiliation when multiple application windows are displayed in parallel on a single display screen.

[0059] Based on this, the window-level seat area binding table 21 is published via the state bus 10 in the form of seat area binding status. The determination of the application window that each input source is allowed to operate on and the determination of the audio output channel of each application window are both obtained from the state bus 10 and based on the same window-level seat area binding table 21. This achieves input isolation and audio isolation across seat areas within the same vehicle display screen 200. Input sources whose corresponding seat areas do not match the seat areas bound to the application window will not obtain the focus of the application window, and the audio of the application window will only be output to the audio output channel corresponding to its bound seat area. At the same time, the determination basis for both is the same data on the state bus 10, avoiding inconsistencies caused by the ownership information being stored separately in each subsystem, and the three do not need to call each other, shortening the linkage path. The display area and stacking order of each application window can also be determined according to the window-level seat area binding table 21, so that the display position of each application window matches the position of the occupant it faces.

[0060] In the previous embodiment, the application windows that each input source is allowed to operate on, and the audio output channels of each application window, are obtained in real time by comparing the seating area bound to the application window with the seating areas corresponding to the input source and the audio output channel. In this method, the window-level seating area binding table 21 only records the correspondence between the application window and the seating area, and the other information required for comparison is obtained separately each time it is executed.

[0061] In existing methods that determine ownership at the display screen level, the input source and audio output channel corresponding to a display screen are configured at system startup and do not require repeated confirmation during operation. However, when ownership is assigned at the application window level, the number of application windows on the in-vehicle display screen 200 and their associated seat areas change with the addition and exit of applications. If the seat area corresponding to the input source is re-compared with the seat areas bound to each application window every time an input event is received, and the seat areas bound to each application window are re-compared with the seat areas corresponding to each audio output channel every time the audio directional output changes, the determination path becomes lengthy when there are many application windows. Furthermore, in actual vehicle models, there may be situations where exceptions need to be configured for individual application windows, such as allowing the driver to operate the application window bound to the passenger seat area via input source 301 when the vehicle is stopped. Such exceptions cannot be derived from the comparison relationship between seat areas.

[0062] This embodiment is a further refinement of the window-level seating area binding table 21 obtained in step S102 of the previous embodiment, and the determination process in steps S105 and S106. The remaining steps are the same as those in the previous embodiment.

[0063] In this embodiment, the window-level seating area binding table 21 records not only the correspondence between each application window and the seating area bound to that application window, but also the correspondence between each application window and the identifier of the input source that is allowed to operate the application window, and the correspondence between each application window and the identifier of the audio output channel of that application window.

[0064] The input source allowed to operate a certain application window is the input source whose corresponding seating area matches the seating area bound to the application window; the audio output channel of a certain application window is the audio output channel corresponding to the seating area bound to the application window. In other words, the contents recorded in the two newly added columns in the window-level seating area binding table 21 are consistent with the results obtained through real-time comparison in the previous embodiment. The difference is that in this embodiment, the result is predetermined and recorded in the table during binding.

[0065] In step S102, after binding the application window of the application to be added to the seat area indicated by the target seat area identifier, the window registration module 20 further selects the input source that matches the corresponding seat area from each input source according to the seat area, and writes the identifier of the input source into the record where the application window is located in the window-level seat area binding table 21; and selects the audio output channel corresponding to the seat area from each audio output channel, and writes the identifier of the audio output channel into the same record. Subsequently, the window registration module 20 publishes the seat area binding status containing the above-mentioned window-level seat area binding table 21 to the status bus 10.

[0066] Continuing with the example from the previous embodiment, after application windows 210, 220 and 230 are added in sequence, the contents of the window-level seating area binding table 21 in this embodiment are shown in Table 2.

[0067] Specifically, for each application window, the input source allowed to operate the application window is the input source corresponding to the seating area bound to the application window, and the audio output channel of the application window is the audio output channel corresponding to the seating area bound to the application window.

[0068]

[0069] Table 2 Accordingly, in step S105 of the previous embodiment, it is determined whether the seat area bound to each application window matches the seat area corresponding to the input source, and the application window whose determination result matches is determined as the application window whose input source is allowed to operate. In this embodiment, it specifically includes: the focus arbitrator 40 reads the identifier of the input source that is allowed to operate each application window from the window-level seat area binding table 21, and determines the input source indicated by the read identifier as the input source that is allowed to operate the application window.

[0070] Continuing the previous example, the focus arbiter 40 reads from Table 2 that the input source identifiers corresponding to application windows 210 and 220 are both input source 301, and the input source identifier corresponding to application window 230 is input source 302. Based on this, it determines that input source 301 is allowed to operate on application windows 210 and 220, and input source 302 is allowed to operate on application window 230. When the focus arbiter 40 receives an input event from input source 301, it determines the focus window only among application windows 210 and 220.

[0071] In step S106 of the previous embodiment, the audio output channel corresponding to the seating area bound to the application window is determined as the audio output channel of the application window. In this embodiment, this specifically includes: the audio arbitrator 50 reads the identifier of the audio output channel of each application window from the window-level seating area binding table 21, and determines the audio output channel indicated by the read identifier as the audio output channel of the application window.

[0072] Continuing with the previous example, the audio arbitrator 50 reads from Table 2 that the audio output channel identifiers corresponding to application windows 210 and 220 are both audio output channel 401, and the audio output channel identifier corresponding to application window 230 is audio output channel 402. Based on this, the audio of the navigation application and the music application is directed to audio output channel 401, and the audio of the video application is directed to audio output channel 402.

[0073] Therefore, in this embodiment, the correspondence between the input source and the application window, as well as the correspondence between the application window and the audio output channel, is determined and recorded in the window-level seating area binding table 21 when the application window is bound to the seating area. The focus arbiter 40 and the audio arbiter 50 do not need to compare the seating areas again during execution; they can directly read the identifiers recorded in the table.

[0074] In a preferred embodiment, the window-level seat area binding table 21 may contain multiple identifiers for input sources that are allowed to operate a certain application window, so that the application window can be operated by multiple input sources. For example, the record of the application window 210 bound to the driver's seat area may simultaneously include the identifier of the knob and the identifier of the voice input device, so that the driver can operate the application window both by the knob and by voice.

[0075] In another embodiment, when writing the identifiers of allowed input sources, the window registration module 20 can further add or delete input source identifiers in the record based on preset configuration items, in addition to the input sources selected based on the seating area, thereby achieving exceptional configuration for individual application windows. For example, when the preset configuration item indicates that the driver is allowed to operate the application window on the passenger side, the window registration module 20 writes the identifier of input source 301 into the record of application window 230, so that input source 301 is also allowed to operate application window 230.

[0076] This embodiment further records the identifiers of the input sources allowed to operate each application window and the identifiers of the audio output channels of each application window in the window-level seating area binding table 21. The relationship between input operation permissions and audio directional output is changed from being obtained by comparing the seating area in real time each time it is executed to being determined and recorded in advance when the application window is bound to the seating area.

[0077] Therefore, when the focus arbiter 40 receives an input event, and when the audio arbiter 50 determines the audio directional output, there is no need to compare the seat areas bound to each application window with the seat areas corresponding to the input source and audio output channel. They can directly read the corresponding identifiers from the window-level seat area binding table 21, thus shortening the determination path. At the same time, since the relationship between input operation permissions and audio directional output is recorded independently in the table in the form of identifiers, it is possible to add or delete preset configuration items based on the seat area determination, thereby supporting the exception configuration of individual application windows that is different from their bound seat areas.

[0078] In any of the foregoing embodiments, upon receiving a request to add an application to a multi-window display, the window registration module 20 binds the application window of the application to be added to the corresponding seat area based on the target seat area identifier carried in the request. This process does not consider the vehicle's own operating conditions; any request carrying a valid target seat area identifier can be accepted.

[0079] Current in-vehicle terminals typically handle information display during driving by allowing each application to independently decide whether to collapse or simplify its interface, or by requiring the user to manually close applications unsuitable for viewing while the vehicle is stationary. When multiple application windows are displayed concurrently on a single in-vehicle screen, these methods have two shortcomings: First, since each application decides whether to collapse its window, the in-vehicle terminal cannot uniformly control this. If an application fails to do so, video application windows will still appear directly in front of the driver during driving, distracting the driver. Second, even if the application window is not located in the driver's seat area, the dynamic images and sounds from that application window may still enter from the edge of the driver's field of vision or the vehicle's sound field while the vehicle is moving, interfering with driving.

[0080] Furthermore, when the vehicle speed fluctuates around a threshold, if the vehicle is judged to be in a moving or stopped state solely based on whether the speed exceeds a certain fixed threshold, the judgment result will repeatedly flip with the slight fluctuations in speed, causing the application window to be repeatedly hidden and restored, and the screen to change frequently.

[0081] This embodiment is based on the aforementioned embodiment, adding an access verification between steps S101 and S102, and downgrading the application window when the vehicle enters driving mode. The remaining steps are the same as in the aforementioned embodiment. In this embodiment, the multiple seating areas divided in the vehicle include the driver's seat area.

[0082] Please see Figure 4 The method of this embodiment includes the following steps: S201, Receive a request to add the application to a multi-window display.

[0083] This step is the same as step S101 in the aforementioned embodiment, and the received request carries the target seating area identifier, which will not be described in detail here.

[0084] S202, Obtain vehicle driving status from the status bus.

[0085] Vehicle driving status refers to a state used to indicate whether a vehicle is in a moving or stationary state. The on-board terminal 100 acquires the vehicle speed, determines the vehicle driving status based on the speed, and publishes the vehicle driving status to the status bus 10.

[0086] The vehicle's driving status is published by the vehicle signal interface component of the vehicle terminal 100. The vehicle signal interface component refers to the component in the vehicle terminal 100 used to access the vehicle network and read vehicle signals such as speed and gear position. It does not belong to the dispatch system 1, but is connected to the status bus 10 and is the sole publisher of the vehicle's driving status. The window registration module 20, the window dispatcher 30, and the audio arbitrator 50 all subscribe to this vehicle driving status from the status bus 10.

[0087] Please see Figure 5 When determining the vehicle's driving status based on its speed, two different thresholds are used for determining the two directions respectively: when the vehicle speed rises above the first threshold and remains above it for a first duration, the vehicle's driving status is set to a driving state; when the vehicle speed drops below the second threshold and remains below it for a second duration, the vehicle's driving status is set to a stopped state; wherein the second threshold is lower than the first threshold.

[0088] The difference between the first threshold and the second threshold constitutes a range that does not cause a state reversal. When the vehicle speed fluctuates within this range, the vehicle's driving state remains unchanged. The first duration and the second duration require that the vehicle speed be maintained after exceeding the corresponding threshold, and a brief overshoot does not cause a state reversal.

[0089] In a specific example, the first threshold is set to 5 km / h, the second threshold is set to 3 km / h, and both the first and second durations are set to 1 second. These values ​​can be determined based on the vehicle's curb weight, starting characteristics, and signal refresh cycle; this embodiment does not limit their specific values.

[0090] In other embodiments, the vehicle's driving state can also be determined based on the vehicle's gear position. For example, when the gear is in parking gear, it is set to a stopped state. It can also be further subdivided based on the type of road the vehicle is on. For example, when the vehicle speed is higher than a first threshold, it can be set to different driving levels depending on whether the road is an urban road or a highway.

[0091] The vehicle driving status published by the vehicle signal interface component includes a degradation level. This degradation level only indicates the current level of constraint that should be achieved, without indicating which application window should be processed and what action should be taken. After obtaining the vehicle driving status from the state bus 10, the window scheduler 30 and the audio arbitrator 50 determine the action to be taken according to their respective preset strategies: the window scheduler 30 determines which application window's display area should be hidden or reduced to what size, and the audio arbitrator 50 determines which application window's audio gain should be muted or reduced to what level. Since the degradation level and the action to be taken are separate, adding a new degradation rule only requires adding the corresponding degradation level and supplementing the corresponding action in the respective strategies of the window scheduler 30 and the audio arbitrator 50; the existing implementations of both do not need to be modified.

[0092] S203. Determine whether the vehicle is in motion, the application type is video, and the target seating area is indicated by the driver's seat area.

[0093] Access verification refers to the verification performed on the request based on the vehicle's driving status, the type of application to be added, and the target seating area identifier before binding the application window to the seating area. This step is the judgment stage of access verification.

[0094] The window registration module 20 reads the acquired vehicle driving status, the type of the application to be added, and the target seating area identifier carried in the request. If all three conditions are met, step S204 is executed; otherwise, step S205 is executed. The type of the application to be added can be determined based on the category declared in the application's installation package or based on the media capabilities it requests.

[0095] S204. Request rejected.

[0096] When all three conditions are met, the window registration module 20 determines that the request has failed the access verification, rejects the request, and does not bind the application window of the application to be added to the seat area indicated by the target seat area identifier. At this time, the content of the window-level seat area binding table 21 remains unchanged, and the multi-window display on the vehicle display screen 200 remains unchanged.

[0097] S205. Bind the application window of the application to be added to the seat area indicated by the target seat area identifier.

[0098] If the above three conditions are not met simultaneously, the window registration module 20 determines that the request passes the admission verification, and binds the application window of the application to be added to the seat area indicated by the target seat area identifier according to step S102 of the aforementioned implementation method, thereby obtaining the window-level seat area binding table 21, and publishes the seat area binding status containing the table to the status bus 10 according to step S103.

[0099] Continuing with the examples in the aforementioned embodiments, when the vehicle is in motion, if a user requests to add a video application to a multi-window display and the target seat area identifier in the request indicates the driver's seat area, the request is rejected, and the video application window will not appear in the display area directly in front of the driver. If the target seat area identifier in the same video application request indicates the front passenger seat area, then all three conditions are not met simultaneously, the request passes the access verification, and application window 230 is bound to the front passenger seat area and added to the multi-window display. When the vehicle is stationary, regardless of which seat area the target seat area identifier indicates, the video application request passes the access verification.

[0100] S206. Set a degradable flag for the application window of the application to be added.

[0101] A degradable flag is a flag indicating that an application window should be degraded when the vehicle enters a driving state. When the application type to be added is video, the window scheduler 30 sets a degradable flag for the application window of that application. In an implementation with a window layout state, the degradable flag is added to the window layout state and published to the state bus 10.

[0102] Continuing from the previous example, application window 230 is marked as degradeable because the video application it hosts is a video application, while application windows 210 and 220 are not marked as such.

[0103] S207. Determine whether the vehicle's driving state has changed from a stationary state to a driving state.

[0104] The window scheduler 30 and the audio arbiter 50 continuously acquire the vehicle's driving status from the status bus 10. When the status changes from a stopped state to a driving state, step S208 is executed; otherwise, step S209 is executed.

[0105] S208. Hide the application window with the downgradeable flag, set the audio gain of the application window to the mute value, update the audio license status, and publish the updated audio license status to the status bus.

[0106] The window scheduler 30 locates the application window with a downgradeable flag and hides it; the audio arbiter 50 sets the audio gain of the application window to a mute value and updates it to the audio licensing state, then publishes the updated audio licensing state to the state bus 10. In an embodiment with a window layout state, the window scheduler 30 can also update the display area of ​​the application window to be hidden in the window layout state and publish the updated window layout state to the state bus 10.

[0107] Continuing with the previous example, when the vehicle changes from a stationary state to a moving state, the display area of ​​application window 230 is updated to be hidden, and the audio gain of the video application it carries is set to a mute value; since application windows 210 and 220 do not have a degradeable flag set, their display areas and audio gains do not change, and the navigation application and music application continue to be displayed and played normally.

[0108] Since degrading only requires finding the target application window based on the degradable flag, without having to re-execute the complete display layout decision, the degrading response path is shorter.

[0109] S209. Maintain the current layout.

[0110] When the vehicle's driving state has not changed from a stationary state to a moving state, the display area and audio gain of each application window remain unchanged.

[0111] In a preferred embodiment, in addition to video applications, the application windows of game applications can also be set to be degradeable; in another embodiment, the degrade method can be to update the display area to be hidden, or to shrink the display area of ​​the application window to a preset minimum size, or to hide only the dynamic screen in the application window while retaining its static controls.

[0112] This implementation adds an access check before binding the application window to the seating area. The system determines whether to accept the request based on the vehicle's driving status, the type of application to be added, and the target seating area identifier. This prevents video application windows facing the driver's seat area from being bound during vehicle operation. The constraints on information display during driving are uniformly applied by the vehicle terminal and no longer rely on individual applications to handle them.

[0113] For video application windows that pass the access verification, this implementation sets a downgrade flag for them. When the vehicle changes from a stationary state to a moving state, the display area of ​​the application is updated to be hidden and its audio gain is set to a mute value based on the flag. This prevents dynamic images and sounds facing other seating areas from entering the driver's field of vision and the in-vehicle sound field during vehicle movement. Meanwhile, application windows without the flag are unaffected. Since the downgrade is executed directly based on the flag, there is no need to re-make a complete display layout decision, resulting in a shorter response path.

[0114] In addition, this embodiment uses two different thresholds and sets a duration for each when determining the vehicle's driving status based on the vehicle speed, so that the vehicle's driving status remains unchanged when the vehicle speed fluctuates around the threshold, avoiding screen jumps caused by the application window being repeatedly hidden and restored.

[0115] In the previous embodiment, the binding relationship between application windows and seating areas is established and deleted as applications are added and exited, and this binding relationship is stored in the window-level seating area binding table 21. When the vehicle display screen 200 is turned off, the vehicle terminal 100 usually releases the resources occupied by each application window, and the records in the window-level seating area binding table 21 are cleared accordingly.

[0116] When the display screen is turned on again, existing in-vehicle terminals typically revert to the default interface or simply restart the single application that was previously in the foreground. When multiple application windows are displayed in parallel on a single in-vehicle display, this method requires users to add each application individually and specify the facing seat area for each application every time they get in the vehicle, which is cumbersome.

[0117] If, in order to solve the above problems, the ownership of each application window and its seat area before the screen is turned off is directly saved, and the binding relationship is directly rebuilt according to the saved content after the screen is turned on again, a new problem will arise: the vehicle's operating status may have changed between the screen being turned off and when it is turned on again. For example, the vehicle may be in a stationary state when the screen is turned off, but in a moving state when it is turned on again. In this case, directly rebuilding the binding relationship according to the saved content will cause video application windows that should not be displayed facing the driver's seat area during driving to reappear, bypassing the access verification added in the previous implementation method.

[0118] This embodiment is based on the previous embodiment, but adds the saving and restoration of the seat area binding state, and the remaining steps are the same as the previous embodiment.

[0119] Please see Figure 9 The method of this embodiment includes the following steps: S301. Before the vehicle display screen is turned off, obtain and save the seat area binding status from the status bus.

[0120] After receiving a screen-off notification from the power management component of the vehicle terminal 100 and before the vehicle display screen 200 is actually turned off, the window registration module 20 obtains the current seat area binding status from the status bus 10 and writes the application carried by each application window and the seat area bound to that application window, which are recorded in the window-level seat area binding table 21 in the status, into the non-volatile memory.

[0121] Continuing with the example in the aforementioned implementation, after the vehicle stops, the user turns off the screen and gets out of the car. At this time, the in-vehicle display screen 200 displays application windows 210, 220 and 230 in parallel. The saved content is: the navigation application corresponds to the driver's seat area, the music application corresponds to the driver's seat area, and the video application corresponds to the passenger seat area.

[0122] S302, the vehicle's display screen lights up again.

[0123] After the user gets back into the vehicle, the in-vehicle display screen 200 lights up. The window registration module 20 reads the content saved in step S301 and enters the recovery process.

[0124] S303. For each application window in the saved seat area binding state, generate a request to add the application carried by that application window to the multi-window display.

[0125] The window registration module 20 does not directly reconstruct the binding relationship based on the saved content. Instead, for each application window, it generates a request to add the application carried by that application window to the multi-window display. The target seating area identifier carried in the generated request indicates the seating area bound to that application window.

[0126] Continuing from the previous example, the window registration module 20 generates three requests in sequence: the first request is for a navigation application, and its target seat area identifier indicates the driver's seat area; the second request is for a music application, and its target seat area identifier indicates the driver's seat area; the third request is for a video application, and its target seat area identifier indicates the passenger seat area.

[0127] The generated request is identical in form and content to the request received in step S201, the only difference being that the former is generated by the window registration module 20 based on the saved content, while the latter is generated by user operation.

[0128] S304. Determine whether the generated request passes the admission verification.

[0129] The window registration module 20 executes steps S202 to S203 of the aforementioned implementation for each generated request, namely, obtaining the current vehicle driving status from the status bus 10, and making a judgment based on the vehicle driving status, the type of application corresponding to the request, and the target seating area identifier carried by the request. Requests that pass the access verification execute step S305, and requests that fail the access verification execute step S306.

[0130] Since step S202 obtains the current vehicle driving status, rather than the vehicle driving status when the screen is off, the recovery process is based on the actual operating status of the vehicle when it is turned on again.

[0131] S305. Bind the application window of the application corresponding to the access verification request to the seat area indicated by the target seat area identifier carried in the request.

[0132] The window registration module 20 completes the binding according to step S205 of the aforementioned embodiment, obtains the window-level seating area binding table 21, and publishes the seating area binding status containing the table to the status bus 10. After the window scheduler 30, focus arbitrator 40, and audio arbitrator 50 obtain the status from the status bus 10, they respectively execute steps S104 to S106 of the aforementioned embodiment, and the display layout, input operation permissions, and audio output channels of each application window are restored accordingly.

[0133] S306, Not bound.

[0134] Applications whose requests fail the access verification will not be added to the multi-window display, and their application windows will not be bound to any seating area.

[0135] Continuing with the previous example, if the vehicle is stationary when the screen is turned on again, all three requests pass the access verification, and application windows 210, 220, and 230 are restored. The vehicle display screen 200 returns to its display state before it was turned off. If the vehicle is in motion when the screen is turned on again, the video application corresponding to the third request is hidden immediately after restoration because it is marked as degradeable. If the target seating area identifier carried by the request indicates the driver's seat area, the request fails the access verification, the video application is not restored, and only the navigation and music application windows are restored.

[0136] In a preferred embodiment, the content saved in step S301 may also include the focus state and audio authorization state when the screen is off, so that the audio levels of the focus window and each application window are restored together after the screen is turned on again.

[0137] In another embodiment, the content saved in step S301 can be written to the non-volatile memory of the vehicle terminal 100 or to the persistent storage component on the vehicle side, so that the content saved after the vehicle terminal 100 is restored to factory settings is still retained.

[0138] This implementation saves the seat area binding state before the vehicle display screen 200 is turned off, and restores the multi-window display based on the saved content after it is turned on again, so that users do not need to add each application and specify the seat area they are facing when they get back into the vehicle.

[0139] More importantly, the restoration process in this embodiment does not directly rebuild the binding relationship based on the saved content. Instead, it converts each saved application window into a request to join the multi-window display, and then performs the same access verification as the normal joining path on the generated request. Thus, the restoration process is based on the actual operation of the vehicle when it is turned on again, rather than the operation when the screen is off. This avoids restoring video application windows facing the driver's seat area when the vehicle is already in motion, ensuring that the restoration path does not bypass the access verification added in the aforementioned embodiment.

[0140] In any of the foregoing embodiments, the state bus 10 dispatches each state on it in the same manner. After each subscriber obtains the state from the state bus 10, it performs its own corresponding processing. The processing of each subscriber is independent of each other and is completed sequentially.

[0141] During vehicle operation, sudden braking by the vehicle in front or lane departure will trigger the vehicle's safety system to issue a warning signal, and the in-vehicle terminal 100 needs to display the corresponding warning content on the in-vehicle display screen 200. When multiple application windows are displayed simultaneously on a single in-vehicle display screen, the existing processing methods have two shortcomings: First, whether the window containing the alarm content can be displayed above other application windows depends on the comparison between the stacking order requested by the alarm content and the stacking order requested by each application window. If an application window requests a higher stacking order, or if the stacking order of the window containing the alarm content has not yet taken effect while the application windows have already refreshed, the alarm content will be obscured by the application windows, and the driver will not be able to receive the alarm in a timely manner.

[0142] Secondly, even if the alarm window is positioned above other application windows, if those application windows can still receive input events and their audio is still playing, the driver's attention will still be distracted by the interaction and sound of those application windows. Furthermore, the placement of the alarm window on top, the clearing of the input areas of each application window, and the muting of the audio in each application window are all accomplished by different components, and the timing of these actions is inconsistent. This can result in an intermediate state where the alarm window is positioned above other application windows, but those application windows can still receive input events, a state that can be perceived by the user.

[0143] This embodiment is based on the aforementioned embodiment, adding preemptive release and two-stage dispatch of security alarm status; the remaining steps are the same as in the aforementioned embodiment. In this embodiment, the scheduling system 1 also includes a security protection module 60.

[0144] Please see Figure 1 and Figure 6 The processing in this embodiment is as follows: The safety protection module 60 receives alarm signals from the vehicle's safety system and obtains a safety alarm status based on the alarm signals.

[0145] Safety alarm status refers to a status defined by the alarm level and the alarm window identifier. The alarm window refers to the window on the vehicle display screen 200 used to display the alarm content indicated by the alarm signal. It does not support applications that are added to the multi-window display, and therefore is not recorded in the window-level seat area binding table 21, nor does it participate in the allocation of the display layout.

[0146] The vehicle's safety system typically operates on a controller outside the vehicle terminal 100. Its alarm signals are transmitted to the vehicle terminal 100 via the vehicle control interface, and the safety protection module 60 is the receiver of these alarm signals on the vehicle terminal 100 side. Upon receiving the alarm signal, the safety protection module 60 does not perform any business judgment but directly obtains the safety alarm status based on the alarm signal and issues it, thereby shortening the path from alarm generation to effectiveness.

[0147] The vehicle control interface can be a vehicle network interface or a shared storage space between the vehicle terminal 100 and the controller. When using shared storage space, the controller writes alarm signals to the shared storage space and notifies the safety protection module 60 to read them via an interrupt. During writing, two alternately used storage areas are used, and a sequence number is added for each write. The safety protection module 60 uses the sequence number to determine whether the read content is complete, so as to avoid content corruption caused by simultaneous reading and writing.

[0148] The security protection module 60 publishes security alarm states to the status bus 10 with a higher priority than other states on the status bus 10. The status bus 10 interrupts the dispatch of other states when dispatching a security alarm state, and continues to dispatch the interrupted other states after the security alarm state dispatch is completed.

[0149] The remaining states after an interruption are retained in the dispatch queue of state bus 10 and are not discarded. Therefore, the preemption of alarms will not result in the loss of window layout state, focus ownership state, or audio authorization state.

[0150] The alarm windows set by the window scheduler 30 are stacked at the levels reserved by the vehicle terminal 100, and the applications carried by each application window cannot request this level. Therefore, the result that the alarm windows are placed above the application windows is unrelated to the stacking order requested by each application window, and no application can obscure the alarm windows by requesting a higher stacking order.

[0151] Window scheduler 30, focus arbiter 40, and audio arbiter 50 all subscribe to the security alarm status from the status bus 10. After obtaining the status from the status bus 10, the window scheduler 30 places the alarm window on top of each application window in the stacking order, the focus arbiter 40 emptys the input areas of each application window so that input events from each input source are not directed to each application window, and the audio arbiter 50 sets the audio gain of each application window to a mute value.

[0152] The focus arbiter 40 clears the input areas of each application window, rather than blocking the application itself. During this time, each application can continue to draw its interface, but will no longer receive input events. Therefore, interaction can be restored without restarting the applications after the alarm is cleared.

[0153] In addition, the window registration module 20 also subscribes to the security alarm status from the status bus 10, and during the period when the status indicates that an alarm exists, it rejects the received request to add the application to the multi-window display, so as to avoid the addition of new application windows during the alarm display process and change the display layout.

[0154] To ensure that the stacking order of alarm windows and the input areas of each application window take effect simultaneously, the status bus 10 dispatches security alarm status in the following two stages.

[0155] In the first phase, the state bus 10 sends a preparatory message to each subscriber that has subscribed to the security alarm status. Upon receiving this preparatory message, each subscriber determines the stacking order of the alarm windows, the input area of ​​each application window, and the audio gain of each application window, and sends an acknowledgment back to the state bus 10. In this phase, each subscriber only completes the determination; the determined results have not yet been submitted and taken effect.

[0156] In the second phase, after receiving confirmations from each subscriber, the state bus 10 sends a commit message to each subscriber. Based on the commit message, each subscriber merges its determined stacking order and input region into the same display composition transaction and commits it, so that the stacking order and the input region take effect within the same rendering frame.

[0157] Since the stacking order and the input area are submitted in the same display composition transaction, they are presented together on the vehicle display screen 200 during the same screen refresh, and there will be no intermediate state where the alarm window is placed on top of each application window while each application window can still receive input events.

[0158] The status bus 10 sets a waiting time for acknowledgments. If no acknowledgment is received from any subscriber after the waiting time since the preparation message was sent, the status bus 10 will stop waiting, directly place the alarm window on top of each application window, and empty the input area of ​​each application window.

[0159] This process does not depend on any subscriber, so even if a subscriber experiences an anomaly or slow response, the result that the alarm window is placed above all application windows and the application windows do not receive input events still holds true. The set waiting time can be one screen refresh cycle of the vehicle display screen 200, or a fraction of that cycle; this embodiment does not limit its specific value.

[0160] The safety alarm status is dispatched by the first dispatch thread 15 of the status bus 10, while the remaining statuses on the status bus 10 are dispatched by the second dispatch thread 16 of the status bus 10. The scheduling priority of the first dispatch thread 15 is higher than that of the second dispatch thread 16. Therefore, the dispatch of the safety alarm status is not affected when the dispatch of other statuses is blocked.

[0161] When dispatching a security alarm state, the first dispatch thread 15 uses a message object that has been allocated before dispatch to carry the security alarm state. That is, the message object is established when the state bus 10 is initialized. No new storage space is requested during the dispatch process, thereby avoiding the allocation and reclamation of storage space being included in the alarm response time.

[0162] Upon receiving an alarm clearing signal, the security protection module 60 will publish a security alarm status indicating that no alarm exists to the status bus 10. Accordingly, the window scheduler 30 will remove the stacking order of alarm windows, the focus arbitrator 40 will restore the input areas of each application window, the audio arbitrator 50 will restore the audio gain of each application window, and the window registration module 20 will resume accepting join requests.

[0163] Continuing with the example in the aforementioned implementation, when the vehicle in front brakes suddenly during driving, triggering an alarm signal, the safety protection module 60 preemptively issues a safety alarm state. The state bus 10 interrupts the dispatch of the window layout state and dispatches the safety alarm state in two stages. The alarm window is built into the same rendering frame on top of application window 210, application window 220, and application window 230. The input areas of the three are set to empty, and the audio gain is set to a mute value. After the alarm is cleared, the display layout, input operation permissions, and audio output channels of the three are restored to the state before the alarm.

[0164] In this implementation, the security alarm status is published to the status bus 10 with a higher priority than other statuses. The status bus 10 interrupts the distribution of other statuses during the distribution process, ensuring that the alarm window is placed above each application window. This result is guaranteed by the distribution mechanism of the status bus 10 and no longer depends on the stacking order requested by each application window, thus overcoming the defect that the alarm content is obscured by the application window. The interrupted other statuses continue to be distributed after the alarm distribution is completed, and the preemption does not cause the loss of status.

[0165] This implementation further employs a two-stage process of preparation and submission to dispatch security alarm status. Each subscriber only completes confirmation during the preparation stage. After receiving all confirmations, the overlay order and input area are merged into the same display composition transaction during the submission stage, so that the two take effect within the same rendering frame. This eliminates the intermediate state where the alarm window is on top but each application window can still receive input events.

[0166] In addition, this implementation sets a waiting time for the confirmation and, after the timeout, the status bus 10 directly executes the top and empty operations, so that the above results still hold when a subscriber is abnormal or responds slowly; and the security alarm status is handed over to the first dispatch thread 15 with a higher scheduling priority and using a pre-allocated message object for dispatch, so that its dispatch is not affected by the blocking of other statuses and the allocation of storage space.

[0167] In any of the foregoing embodiments, the state bus 10 is responsible for transmitting the seat area binding status, window layout status, focus status, audio authorization status, vehicle driving status, and safety alarm status. Each module publishes and obtains the above statuses through the state bus 10, and there is no direct call between modules. It can be seen that the status transmitted by the state bus 10 is the sole basis for each module's judgment and execution.

[0168] This raises three issues that need to be addressed: Firstly, if the same status can be published by multiple modules, the content published by different modules may conflict with each other, and the content obtained by each subscriber will depend on the time of its acquisition, and the judgment criteria are no longer unique.

[0169] Secondly, during the dispatch process, the order of dispatches on the state bus 10 may change due to thread scheduling, causing subscribers to receive a newer publication first and an earlier publication later. Taking window layout state as an example, if a subscriber receives a new display layout and then receives a previous display layout and executes accordingly, the display areas of each application window will revert to the previous state, resulting in a display layout disorder.

[0170] Third, the process running on state bus 10 may exit abnormally. In this case, each subscriber will no longer receive any state updates. If each subscriber continues to use the state it last obtained, the state may no longer match the actual situation of the vehicle. After state bus 10 restarts, it will no longer retain any state, and each subscriber will not be able to obtain it again.

[0171] This embodiment is a further refinement of the state bus 10 in any of the foregoing embodiments, and the remaining steps are the same as those in the foregoing embodiments.

[0172] Please see Figure 7 The state bus 10 includes a topic registry 11, a publisher table 12, a subscriber table 13, a latest value cache 14, a first dispatch thread 15, a second dispatch thread 16, a version number verification unit 17, and an expiration timer unit 18.

[0173] When the vehicle terminal 100 starts up, the state bus 10 establishes a topic registry 11 and a publisher table 12 according to a preset configuration, thus completing the registration of each state and its attributes, as well as the establishment of the correspondence between each state and its publisher. Subsequently, each module of the scheduling system 1 subscribes to the states it needs from the state bus 10, and the state bus 10 records each module in the subscriber table 13. Only after each module has completed its subscription can it begin publishing and retrieving states. Therefore, the above initialization of the state bus 10 is a prerequisite for the various processes in the aforementioned embodiments to be performed.

[0174] The topic registry 11 is used to record the various states and their attributes transmitted on the status bus 10. The recorded attributes include the name of the state, the composition of its content, its priority, its consistency level, and its validity period.

[0175] Consistency level refers to the strictness of synchronization of the same state among its subscribers, including strong consistency, sequential consistency, and eventual consistency, which decrease in order. Strong consistency requires that the processing performed by each subscriber based on the state takes effect simultaneously within the same rendering frame. Security alarm states belong to this level, and their implementation has been described in the aforementioned embodiments. Sequential consistency requires that each subscriber executes processing in the order in which the state is published. Window layout states and focus ownership states belong to this level, and their implementation is the version number verification described below. Eventual consistency allows each subscriber to converge to consistency after a short delay. All other states belong to this level.

[0176] The publisher table 12 records the correspondence between each state and its publisher. Each state on the state bus 10 is published to the state bus 10 by only one publisher. Following the aforementioned implementation methods, the publisher for the seating area binding state is the window registration module 20, the publisher for the window layout state is the window scheduler 30, the publisher for the focus ownership state is the focus arbitrator 40, the publisher for the audio authorization state is the audio arbitrator 50, and the publisher for the security alarm state is the security protection module 60. The publisher table 12 is established during the initialization of the state bus 10. Any other modules will be rejected when publishing this state to the state bus 10, except for those listed in the table.

[0177] The subscriber table 13 is used to record the correspondence between each state and its subscribers. When a module subscribes to a state from the state bus 10, the state bus 10 records the module in the record corresponding to that state in the subscriber table 13; when the state is published, the state bus 10 determines which modules should be dispatched to based on the subscriber table 13.

[0178] Each state is published with a version number generated by the publisher of that state, and this version number increments with each subsequent publication of that state. The version number is an ordinal number used to indicate the order in which the same state is published.

[0179] Version number verification unit 17 is used to perform verification when the subscriber obtains the state: if the version number of the obtained state is not higher than the version number of the same state that the subscriber has already obtained, the obtained state is discarded.

[0180] Continuing with the example from the previous implementation, the window scheduler 30 publishes the window layout state twice, the first time with version number 3 and the second time with version number 4. The focus arbiter 40 first receives the window layout state with version number 4 and calculates the input area of ​​each application window accordingly. Subsequently, it receives the window layout state with version number 3. At this time, version number 3 is not higher than version number 4 already obtained by the focus arbiter 40, so this state is discarded. The focus arbiter 40 will not recalculate based on the previous display layout, and the input areas of each application window will not revert to the previous state.

[0181] The latest value cache 14 is used to store the most recently published value for each state on the state bus 10. When a module subscribes to a state on the state bus 10, it can directly obtain the current value of that state from the latest value cache 14 without waiting for the next publication of that state.

[0182] The validity period timing unit 18 is used to set the validity period for each state on the state bus 10 and record the time when each subscriber last acquired each state. The validity period refers to the length of time that a state can continue to be considered valid after it has been acquired. If a subscriber has not acquired a state again after the validity period of that state since it last acquired it, the default value of that state will replace the state that the subscriber has acquired.

[0183] While replacing the state with a preset default value, the subscriber also initiates a re-fetch request to the state bus 10. Upon receiving the re-fetch request, the state bus 10 responds with the value of the state stored in the latest value cache 14. If the state bus 10 is functioning normally and only the publisher of the state has failed to publish in a timely manner, the subscriber can restore the state to its most recently published value through this re-fetch without waiting for its next publication.

[0184] The default values ​​for each state are determined based on the principle of not causing danger. Continuing with the aforementioned implementation methods, the default value for the vehicle driving state is "driving state," the default value for the safety alarm state is "no alarm," the default value for the window layout state is "the display area of ​​each application window is hidden," and the default value for the audio authorization state is "the audio gain of each application window is muted."

[0185] When the process containing state bus 10 exits abnormally, each subscriber continues to run according to its last acquired state within the validity period. After the validity period expires, it switches to running according to the preset default value. Taking the vehicle driving state as an example, its preset default value is the driving state. Therefore, when state bus 10 is abnormal, the vehicle terminal 100 runs according to the constraints corresponding to the driving state, and the application window of video applications will not be added to the multi-window display.

[0186] After state bus 10 restarts, it republishes the values ​​stored in the latest value cache 14 to state bus 10. Each subscriber can obtain the republished values ​​after resubscribing to the corresponding state and update its acquired state accordingly, thus restoring the judgment criteria of each module.

[0187] The functions of the first dispatch thread 15 and the second dispatch thread 16 have been explained in the previous embodiments and will not be repeated here. The priority recorded in the subject registry 11 is the basis for the state bus 10 to determine whether a certain state should be dispatched by the first dispatch thread 15 or the second dispatch thread 16.

[0188] In a preferred embodiment, the validity period of each state is set according to its change frequency, and the validity period of the vehicle driving state and the safety alarm state is shorter than the validity period of the seat area binding state and the window layout state.

[0189] In another embodiment, the value stored in the latest value cache 14 can be written to non-volatile memory so that the stored value is not lost when the process of the status bus 10 exits abnormally and restarts.

[0190] This implementation limits each state on the state bus 10 to be published by only one publisher, ensuring that the source of the same state obtained by each subscriber is unique, thus avoiding content conflicts caused by multiple modules publishing the same state.

[0191] This implementation sets a version number for each state that increments with each release, and the subscriber discards states whose version number is not higher than the value it has already acquired. This ensures that if the distribution order is reversed, earlier states will not overwrite newer states, thus avoiding display layout errors.

[0192] This implementation also sets an expiration period for each state, so that if the subscriber does not obtain the state again after the expiration period, it will switch to the preset default value, thus avoiding the subscriber from continuing to use a state that does not match the actual situation of the vehicle during the abnormal period of the state bus 10; and the state bus 10 saves the most recently published value of each state, and republishes the saved value after it restarts, so that the judgment basis of each module can be restored.

[0193] In the aforementioned embodiments, the audio of each application window is directed to the corresponding audio output channel according to the seat area it is bound to, and the audio of multiple application windows bound to the same seat area is played through the same audio output channel.

[0194] Existing in-vehicle terminals use a mutually exclusive approach to audio on the same audio output channel: once an application initiates and is granted audio focus, it monopolizes that channel, and audio focus requests from other applications are denied. The denied applications automatically pause playback upon receiving a focus loss notification. This approach works fine when only one application is running on a single display screen. However, when multiple application windows are displayed concurrently on an in-vehicle display screen, and these windows are bound to the same seating area, only one application bound to that seating area can produce sound, while the others are forced to pause.

[0195] Following the example in the aforementioned embodiments, the navigation application carried by application window 210 and the music application carried by application window 220 are both bound to the driver's seat area, and the audio of both is broadcast through audio output channel 401. Under the aforementioned mutually exclusive method, when the navigation application initiates a broadcast, the music application is forced to pause, and after the broadcast ends, the music application needs to restart playback, resulting in repeated interruptions to music playback. If the music application obtains audio focus first, the navigation broadcast cannot be heard, and the driver cannot obtain timely information about intersection turns.

[0196] Furthermore, if each application controls its own volume to allow multiple applications to play audio simultaneously, then the applications have no way of knowing each other's playback status and cannot reduce their own volume accordingly when critical audio is being played.

[0197] This embodiment is a further refinement of step S106 in the aforementioned embodiment, and the remaining steps are the same as in the aforementioned embodiment.

[0198] Please see Figure 8 The processing in this embodiment is as follows: The audio arbitrator 50 receives audio focus requests initiated by the applications carried by each application window and returns permission for each audio focus request.

[0199] An audio focus request is a request initiated by an application to the in-vehicle terminal 100 before playing audio to obtain playback permission. Since the audio arbitrator 50 grants permission to all audio focus requests, no application receives a notification of focus loss and therefore does not automatically pause playback. The audio of multiple application windows bound to the same seating area can be played simultaneously through the same audio output channel.

[0200] The audio arbiter 50 assigns an audio level to each application window based on the type of application it carries, adds the correspondence between each application window and its audio level to the audio authorization status, and publishes the audio authorization status to the status bus 10.

[0201] Audio level refers to the hierarchy of an application window's audio when played simultaneously with the audio from other application windows. Audio levels are categorized into primary, secondary, and background levels, decreasing in importance.

[0202] The audio arbitrator 50 has a pre-set correspondence between application types and audio levels: applications that carry key driving information, such as navigation broadcasts, voice assistants, and vehicle prompts, are assigned to the primary level; media applications such as music and radio are assigned to the secondary level; and entertainment applications such as videos and games are assigned to the background level.

[0203] Continuing with the example in the aforementioned implementation, application window 210, which hosts a navigation application, is assigned the primary level; application window 220, which hosts a music application, is assigned the secondary level; and application window 230, which hosts a video application, is assigned the background level.

[0204] During the audio output of the primary application window, the audio arbitrator 50 reduces the audio gain of the secondary application window and the background application window according to the audio license status, and restores the reduced audio gain to its previous value after the output ends.

[0205] Please see Figure 8Continuing from the previous example, when the navigation application is silent, the music and video applications play at their original audio gain. When the navigation application starts broadcasting, the audio gain of application windows 220 and 230 is reduced, so the navigation broadcast can be clearly heard amidst the music. After the broadcast ends, the audio gain of application windows 220 and 230 returns to the value before it was reduced, and the music playback is not interrupted.

[0206] Since the navigation application and the music application are bound to the same seating area, their audio is played through the same audio output channel 401, and the aforementioned suppression and recovery are completed on this channel; the application window 230 is bound to the passenger seat area, and its audio is played through the audio output channel 402, and the suppression of its audio gain does not affect the audio on the audio output channel 401.

[0207] In a preferred embodiment, the reduced audio gain is a fraction of the original value, or it can be a preset fixed value. This embodiment does not limit the specific value.

[0208] The audio arbiter 50 obtains the focus assignment status from the status bus 10. If the audio level assigned to the focus window indicated by the focus assignment status is the background level, the audio level assigned to the focus window is adjusted to the secondary level, and the adjusted correspondence is updated to the audio authorization status and then published to the status bus 10.

[0209] Continuing the previous example, the front passenger operates application window 230 via input source 302. Focus arbitrator 40 identifies application window 230 as the focus window and publishes the focus ownership status. After acquiring this status, audio arbitrator 50 adjusts the audio level of application window 230 from background to secondary, ensuring that the audio of the video application being operated by the front passenger is no longer suppressed to the level corresponding to the background. When the focus shifts to another application window, the audio level of application window 230 reverts to the background level.

[0210] In another embodiment, the audio of the application window assigned a background level can be muted when the vehicle is in motion, as described in the foregoing embodiments.

[0211] This implementation grants permission to all audio focus requests initiated by the applications hosted in each application window, preventing each application from automatically pausing playback upon receiving a focus loss notification. Audio from multiple application windows bound to the same seating area can be played simultaneously through the same audio output channel, overcoming the defect of existing mutual exclusion methods where only one application can produce sound on the same channel; moreover, this processing requires no adaptation on the application side.

[0212] Based on this, this implementation method assigns a primary, secondary, and background level to each application window according to the type of application, and lowers the audio gain of the secondary and background levels during the primary level's sound output and restores it after the sound output ends, so that key driving audio such as navigation broadcasts can always be heard clearly, while the playback of media and entertainment applications is not interrupted.

[0213] This embodiment also adjusts the audio level of the focused window to the secondary level when the audio level assigned to it is the background level, so that the audio loudness of the application window being operated by the user matches their operation.

[0214] In step S105 of the aforementioned embodiment, when the focus arbitrator 40 receives an input event from an input source, it determines the focus window within the application windows where that input source is allowed to operate. When there are multiple application windows where the input source is allowed to operate, it is also necessary to determine how to select the focus window among them.

[0215] For touch input devices, users can select the target application window simply by touching the display area, thus avoiding the aforementioned problem. However, for input sources such as knobs and steering wheel buttons located in fixed positions, they do not point to a specific location on the vehicle display screen 200. Users can only move the focus between application windows through continuous switching actions. While the vehicle is in motion, the driver's gaze needs to remain on the road ahead. Touch input devices have small operating targets and require visual coordination, making knobs and steering wheel buttons the primary input methods for the driver while driving. The predictability of these switching actions directly affects operational efficiency.

[0216] The existing in-vehicle terminal has two shortcomings in handling the above-mentioned switching. First, the switching scope covers all application windows on the in-vehicle display screen 200, including application windows that the input source is not allowed to operate. After the focus is switched to such application windows, the user cannot get any response when continuing to operate, and needs to switch again to get past it, resulting in a situation where operation is performed but no application window responds.

[0217] Secondly, existing rotary input sources are typically only used to move the selected control between controls within an application window, without distinguishing between switching between application windows and switching within an application window. When multiple application windows are displayed in parallel on a vehicle display screen, if the two are not differentiated, when a user moves the selected control within an application window, the focus may shift out of that application window and into an adjacent application window; conversely, when a user wants to switch to an adjacent application window, the operation may be consumed one by one by the controls within that application window.

[0218] This eighth embodiment is a further refinement of step S105 in the aforementioned embodiment, which describes determining the focus window in the application window where the input source is allowed to operate. The remaining steps are the same as in the aforementioned embodiment.

[0219] The focus arbiter 40 sorts the application windows that the input source is allowed to operate on according to their respective arrangement order on the vehicle display screen 200 to obtain a window sequence.

[0220] A window sequence is an ordered set of application windows that are allowed to operate on a certain input source, arranged in the order of their display positions. The focus moves along this sequence under the operation of the input source. Application windows that are not allowed to operate on the input source do not participate in the above ordering and therefore do not enter the window sequence.

[0221] The arrangement order can be determined from left to right of the vehicle display screen 200 according to the display area of ​​each application window, or from top to bottom; when the display areas of each application window are distributed in all directions, the order can be determined first from top to bottom, and then from left to right.

[0222] When the window-level seating area binding table 21 or the window layout state changes, the focus arbiter 40 reconstructs the window sequence. Therefore, when an application joins or leaves the multi-window display, the newly added or deleted application window enters or leaves the window sequence accordingly; when the display area of ​​an application window is updated to hidden according to the aforementioned implementation, the application window will no longer enter the window sequence.

[0223] Please see Figure 3 Continuing with the example from the previous embodiment, input source 301 corresponds to the driver's seat area, where it is allowed to operate application windows 210 and 220, but not application window 230. The display area of ​​application window 210 is located to the left of the display area of ​​application window 220; therefore, the window sequence of input source 301 consists of application windows 210 and 220 in sequence, while application window 230 is not included in this window sequence. Correspondingly, the window sequence of input source 302 contains only application window 230.

[0224] After the window sequence is formed and before the input event from the input source is received, the focus arbitrator 40 determines the initial focus window from the window sequence in the following order: First, when the safety alarm status indicates that an alarm exists, the focus window points to the alarm window; second, when there is an application window in the window sequence that carries navigation, voice assistant, or vehicle control applications, the focus window points to that application window; third, when there is an application window in the window sequence that was recently added to the multi-window display, the focus window points to that application window; fourth, if none of the above conditions are met, the focus window points to the first application window in the window sequence.

[0225] The focus arbiter 40 also has preset weights for each input source. Weight refers to the degree to which a particular input source is preferentially bound to the focus window when multiple input sources are available simultaneously. The focus arbiter 40 obtains the vehicle's driving status from the state bus 10. When the state indicates the vehicle is in motion, it increases the weight of the knob, steering wheel buttons, and voice input device while decreasing the weight of the touch input device. When the state indicates the vehicle is stationary, it increases the weight of the touch input device. Since the driver's gaze needs to remain on the road ahead while the vehicle is in motion, and the operation target of the touch input device is small and requires eye contact, the above adjustments ensure that the focus during driving is controlled by input sources that do not require eye contact by default.

[0226] When an input event indicates a focus switch, the focus arbiter 40 switches the focus window from its current position in the window sequence to the next application window; and when the current position is the last position in the window sequence, it switches the focus window to the first application window in the window sequence.

[0227] Continuing the previous example, when the focus window is application window 210, the driver performs a switching action via input source 301, switching the focus window to application window 220. When the focus window is application window 220, it is at the end of the window sequence. If the driver performs another switching action, the focus window switches back to application window 210. Since application window 230 is not included in the window sequence, the focus will not switch to application window 230. Each switching action by the driver ensures that the focus falls on the application window that can be operated, preventing a situation where an operation is performed but no application window responds.

[0228] After each switch, the focus arbitrator 40 publishes the identifier of the new focus window as the focus ownership status to the status bus 10. After obtaining the status from the status bus 10, the window scheduler 30 displays the focus window on the vehicle display screen 200 by highlighting the border or other means, so that the driver can confirm which application window is currently being operated without having to stare at the vehicle display screen 200 for a long time.

[0229] Input events from the same input source include the first input event and the second input event.

[0230] When the input event is the first input event, the focus arbiter 40 switches the focus window from its current position in the window sequence to the next application window, that is, it performs a switch between application windows.

[0231] When the input event is the second input event, the focus arbiter 40 keeps the focus window unchanged and switches the selected control among the controls within the focus window, that is, performs a switch within the application window. The selected control refers to the control within the application window that is currently receiving the operation. It moves sequentially among the controls contained in the application window as the second input event occurs, and will not go beyond the application window.

[0232] The process of switching the selected control among the controls within the focus window can be performed by the focus arbitrator 40 or by the existing component in the vehicle terminal 100 used to move the selected control within an application window. When the latter is used, the focus arbitrator 40 is only responsible for switching between application windows and forwards the second input event to that component for processing. The input events processed by the two components do not overlap, thus preventing a situation where both simultaneously change the selected control.

[0233] Taking a knob as an example, the first input event can be the rotation of the knob while it is pressed, and the second input event can be the normal rotation of the knob. Alternatively, the triggering of a dedicated switching button on the steering wheel can be used as the first input event, and the rotation of the knob as the second input event. Continuing the previous example, when the driver presses the knob and rotates it one notch, the focus window switches from application window 210 to application window 220. Subsequently, when the knob is rotated directly, the focus window remains application window 220, and the selected control moves sequentially among the play, pause, previous track, and next track controls in the music application.

[0234] Therefore, switching between application windows and switching within an application window are triggered by different input events, and the two do not cross each other's boundaries: when moving the selected control within an application window, the focus window will not change, and when switching application windows, the controls within that application window will not be consumed one by one.

[0235] This embodiment arranges application windows that are allowed to operate on a certain input source into a window sequence according to the order of their display areas, and makes the focus window cycle between the last and the first position of the window sequence. Since application windows that are not allowed to operate on the input source do not enter the window sequence, the focus will not switch to application windows that the user cannot operate, overcoming the defect of non-responsive operation caused by the switching range including non-operable application windows in the existing method, and making the driver's switching actions during driving have predictable results.

[0236] This embodiment further uses a first input event to trigger the switching between application windows and a second input event to trigger the switching of the selected control within the application window. This allows the two-level switching to be handled by different input events, avoiding the situation where the focus goes out of the application window when moving the selected control within the application window, and the operation being consumed by the control within the application window when switching application windows.

[0237] In step S104 of the aforementioned embodiment, the window scheduler 30 allocates a display area for each application window, and each application window is displayed according to the allocated display area. Since multiple application windows are displayed in parallel on the vehicle display screen 200, the display area occupied by each application window is smaller than the full screen size of the vehicle display screen 200, and the size of the display area varies with the number of application windows displayed in parallel.

[0238] Most existing in-vehicle applications are designed with full-screen interfaces and declare in their installation packages that they are not scalable. When these applications are made to display in multiple windows, their interfaces are drawn at full-screen size, but the displayed area is smaller than the full screen, resulting in the interface being cropped or stretched and distorted. If the in-vehicle terminal 100 sends a configuration change event to notify that the display area has changed, these applications need to recreate their interfaces. Applications that have not been adapted for multi-window displays are prone to abnormal exits during the recreation process. As a result, the number of applications that can be made to display in multiple windows is limited, and the range of applications that users can choose from is narrow.

[0239] Furthermore, applications may become unresponsive or exit due to their own reasons during operation. When multiple application windows are displayed in parallel on a vehicle display screen, if the above situation occurs in one application, there will be no screen output in its corresponding display area, and the display area will become blank. If the vehicle terminal 100 reallocates the display layout for this reason, the display areas of the other application windows will change accordingly, and the operation of passengers using other applications will be interrupted.

[0240] This embodiment is based on the aforementioned embodiment, adding application interface compatibility handling and application exception handling; the remaining steps are the same as in the aforementioned embodiment. In this embodiment, the scheduling system 1 also includes an application compatibility module 70.

[0241] Before binding the application window of the application to be added to the seating area indicated by the target seating area identifier, the application compatibility module 70 detects whether the application to be added declares a scalable attribute.

[0242] The scalable attribute refers to a declaration made in its installation package that indicates its interface can be rearranged according to changes in the allocated display area. With this attribute declared, the interface can automatically rearrange itself according to the allocated display area and can be directly displayed according to step S104 of the aforementioned implementation method.

[0243] When the application to be added does not declare a scalable attribute, the application compatibility module 70 renders the interface of the application to be added to the off-screen buffer at the resolution declared by the application to be added. After obtaining the window layout state, the content in the off-screen buffer is scaled to the display area of ​​the application window of the application in the window layout state and then composited for display, without sending a configuration change event to the application to be added.

[0244] Off-screen buffer refers to an image storage space that is not directly output to the vehicle display screen 200; the content in it needs to be composited before it can be displayed.

[0245] Because the rendering uses the resolution declared by the application, the application's interface is still rendered at its designed size and is not cropped. Since no configuration change events are sent to the application, it will not recreate its interface due to changes in the display area, thus avoiding abnormal exits during the recreation process. Scaling occurs between the off-screen buffer content and the display area, handled by the graphics compositing component, which the application is unaware of, and therefore requires no adaptation for multi-window displays.

[0246] Continuing with the example from the previous implementation, the video application does not declare a scalable attribute; its declared resolution is the full-screen size of the in-vehicle display 200. The application compatibility module 70 renders the video application's interface to an off-screen buffer at this full-screen size, then scales the content in the off-screen buffer to the display area of ​​the application window 230 in the window layout state before compositing it for display. Throughout the process, the video application draws its interface at full-screen size without receiving any configuration change events.

[0247] In another embodiment, when the application compatibility module 70 detects that the application to be added has not declared a scalable attribute, it can also determine that the request has failed the admission verification and reject the request; or add the application window of the application to the multi-window display as a floating window superimposed on the other application windows, and the display area occupied by the floating window does not participate in the allocation of the display area of ​​the other application windows.

[0248] In a preferred embodiment, for applications that do not declare scalable attributes but whose interfaces are adaptable, the application compatibility module 70 can also configure the application to be considered as having declared scalable attributes, so that it can be displayed directly according to step S104 of the aforementioned embodiment, in order to avoid the loss of clarity caused by scaling.

[0249] In another embodiment, the application compatibility module 70 can also create multiple independent running instances for the same application, each running instance hosted in a different application window. In this case, the multiple application windows of the application can be bound to different seating areas. For example, the application window corresponding to one running instance of the same video application can be bound to the front passenger seat area, while the application window corresponding to another running instance can be bound to the rear seat area, and the two do not affect each other. The running instances are created as follows: when the application adds multi-window display, the application compatibility module 70 creates independent task stacks for each instance, ensuring that the running instances do not reuse the same task stack.

[0250] The application compatibility module 70 monitors the running status of the applications carried by each application window. When any application becomes unresponsive or exits, it obtains an indication of the abnormal running status of the application and publishes it to the status bus 10.

[0251] Application running status refers to a state in which the applications carried by each application window are running normally. The application compatibility module 70 is the publisher of this status. The application compatibility module 70 can determine whether an application has exited by monitoring whether the process of each application is alive, and it can also determine whether an application is unresponsive by monitoring whether the response of each application to input events times out.

[0252] The window scheduler 30 obtains the application's running status from the status bus 10. In the window layout state, a placeholder window is displayed in the display area of ​​the application window to replace the application window, while keeping the display area, stacking order, and audio output channel of the other application windows unchanged.

[0253] A placeholder window is a window provided by the vehicle terminal 100 to replace an application window that malfunctions. It displays prompts and provides an entry point to restart or switch applications.

[0254] Continuing with the previous example, when the music application becomes unresponsive, the application compatibility module 70 issues an indication of the application's abnormal running status, and the window scheduler 30 displays a placeholder window within the original display area of ​​the application window 220. At this time, the display area, stacking order, and audio output channels of application windows 210 and 230 remain unchanged, the navigation application continues to display and broadcast, and the front passenger's viewing of the video application is unaffected; the original display area of ​​application window 220 is occupied by the placeholder window, and no blank space appears.

[0255] In another embodiment, the window registration module 20 can also obtain the application running status from the status bus 10. When the status indicates that an application is abnormal, the module deletes the record corresponding to the application window of the application from the window-level seating area binding table 21, and republishes the seating area binding status containing the deleted window-level seating area binding table 21 to the status bus 10, so that the display area can be reused by other applications.

[0256] This implementation renders the interface of an application that has not declared a scalable attribute to an off-screen buffer at its declared resolution. Then, it scales the content in the off-screen buffer to the allocated display area and composites the display, without sending a configuration change event to the application. Thus, the application's interface is not cropped or recreated due to changes in the display area. Applications that have not been adapted for multi-window display can now be added to multi-window display, expanding the range of applications available to the user. Furthermore, this process requires no adaptation on the application side.

[0257] This embodiment also displays a placeholder window in the original display area of ​​the application window when any application becomes unresponsive or exits, while maintaining the display area, stacking order, and audio output channels of the remaining application windows unchanged. Thus, an anomaly in a single application does not cause blanking in that display area, nor does it alter the display layout of the remaining application windows, and passengers using other applications are unaffected.

[0258] Regarding the multi-task window scheduling method provided in the aforementioned embodiments, existing vehicle terminals delegate each step of the method to a window management subsystem, an input management subsystem, and an audio management subsystem, respectively. The subsystems transmit the required information to each other by calling each other. That is, when a subsystem needs information that another subsystem has, it directly calls the interface provided by that subsystem to obtain it.

[0259] In this approach, each subsystem forms a pairwise calling relationship, and each subsystem must be aware of the existence and interface of the other subsystems. When an in-vehicle display screen shows multiple application windows in parallel, and it is necessary to introduce driving status processing, safety alarm processing, and application compatibility processing in addition to the original window management, input management, and audio management, the newly introduced parts need to establish calling relationships with each of the existing subsystems one by one. The existing subsystems also need to be modified accordingly to accept the calls of the newly introduced parts. The number of calling relationships increases rapidly with the number of subsystems, and the scope of modification is quite large.

[0260] Based on this, this embodiment provides a multi-task window scheduling system for implementing the multi-task window scheduling method described in any of the foregoing embodiments.

[0261] Please see Figure 1 The dispatch system 1 is located in the vehicle terminal 100 and includes a state bus 10, a window registration module 20, a focus arbiter 40, and an audio arbiter 50. It may also include a window dispatcher 30, a safety protection module 60, and an application compatibility module 70. Each of the above modules is connected to the state bus 10. After any module generates a certain state, it publishes the state to the state bus 10. Modules that need to use the state obtain the state from the state bus 10. There is no direct calling relationship between modules.

[0262] The state bus 10 is used to transmit state via publish-subscribe. Its internal components and the handling of dispatch order, version number, validity period, and latest value have been described in the aforementioned embodiments.

[0263] When transmitting a security alarm status, the status bus 10 dispatches it with a higher priority than other statuses. When dispatching this status, the dispatch of other statuses is interrupted, and the dispatch of the interrupted other statuses continues after the dispatch of this status is completed. The dispatch process is divided into two stages: preparation and submission. A waiting time is set for the return confirmation. If no confirmation is received after the waiting time is exceeded, the process will stop waiting and directly execute the top and empty functions. The specific processing has been described in the aforementioned implementation.

[0264] The window registration module 20 receives requests to add applications to a multi-window display, reads the target seating area identifier carried in the request, binds the application window of the application to be added to the seating area indicated by the target seating area identifier, obtains a window-level seating area binding table 21, and publishes the seating area binding status containing the window-level seating area binding table 21 to the status bus 10. When an application window that has been added to the multi-window display exits, the window registration module 20 deletes the record corresponding to the application window from the window-level seating area binding table 21 and republishes the seating area binding status. Therefore, the window registration module 20 is the sole maintainer of the window-level seating area binding table 21 and the sole publisher of the seating area binding status.

[0265] The window registration module 20 is also used to obtain the vehicle driving status from the state bus 10, and perform an access verification on the request based on the vehicle driving status, the type of application to be added, and the target seat area identifier before binding; it is also used to obtain and save the seat area binding status from the state bus 10 before the vehicle display screen 200 is turned off, and generate a joining request based on the saved content and re-perform the access verification after the vehicle display screen 200 is turned on again.

[0266] The window registration module 20 is also used to obtain the security alarm status from the status bus 10, and during the period when the security alarm status indicates that an alarm exists, reject the received request to add the application to the multi-window display.

[0267] In an embodiment where the scheduling system 1 also includes a window scheduler 30, the window scheduler 30 is connected to the state bus 10 and is used to obtain the seat area binding status from the state bus 10, determine the display area and stacking order of each application window on the vehicle display screen 200 according to the window-level seat area binding table 21 therein, publish the window layout status to the state bus 10, and display each application window according to the window layout status; the window layout status includes the determined display area and stacking order of each application window.

[0268] The window scheduler 30 is also used to obtain the safety alarm status from the status bus 10 and place the alarm window stacking order above each application window; it is also used to obtain the vehicle driving status from the status bus 10, and update the display area of ​​the application window with the degradable flag to hidden when the status changes from the stopped state to the driving state; it is also used to obtain the application running status from the status bus 10, and when the status indicates that an application is abnormal, display a placeholder window in the display area of ​​the application window of that application to replace the application window.

[0269] The focus arbiter 40 is used to obtain the seat area binding status from the status bus 10, and according to the window-level seat area binding table 21 therein, for each input source, it determines whether the seat area bound to each application window matches the seat area corresponding to the input source, and determines the application window whose determination result matches as the application window that is allowed to operate on the input source, and forms a window sequence according to the arrangement order of the display area of ​​the application window; it is also used to determine the focus window in the window sequence when an input event is received from any input source, publish the identifier of the focus window as the focus ownership status to the status bus 10, and direct the input event to the focus window according to the focus ownership status.

[0270] The focus arbiter 40 is also used to obtain the security alarm status from the status bus 10 and to empty the input areas of each application window.

[0271] The audio arbiter 50 is used to obtain the seat area binding status from the status bus 10, determine the audio output channel of each application window as the audio output channel corresponding to the seat area bound to the application window according to the window-level seat area binding table 21, publish the correspondence between each application window and its audio output channel as the audio authorization status to the status bus 10, and direct the audio of each application window to the audio output channel of the application window according to the audio authorization status.

[0272] The audio arbitrator 50 is also used to receive audio focus requests initiated by the applications carried by each application window and return approval for each audio focus request; to assign audio levels to each application window according to the type of application carried by each application window; and to reduce the audio gain of the application windows assigned secondary and background levels during the audio output of the application window assigned primary level; it is also used to obtain the focus ownership status from the status bus 10 and adjust the focus window to secondary level if the audio level assigned to it is background level; it is also used to obtain the safety alarm status from the status bus 10 and set the audio gain of each application window to a mute value; and to obtain the vehicle driving status from the status bus 10 and set the audio gain of the application window with a downgradeable flag to a mute value.

[0273] The safety protection module 60 receives alarm signals from the vehicle's safety system, determines the safety alarm status based on the alarm signals, and publishes the safety alarm status to the status bus 10 with a higher priority than other statuses on the status bus 10. The safety protection module 60 is the sole publisher of the safety alarm status.

[0274] The application compatibility module 70 is used to detect whether the application to be added declares a scalable attribute before the window registration module 20 performs binding. If the application does not declare the attribute, the application's interface is rendered to the off-screen buffer at its declared resolution. After the window scheduler 30 obtains the display layout, the content in the off-screen buffer is scaled to the display area of ​​the application window of the application in the display layout and then composited for display. No configuration change event is sent to the application.

[0275] The application compatibility module 70 is also used to monitor the running status of the applications carried by each application window. When any application becomes unresponsive or exits, it receives an indication of the abnormal running status of that application and publishes it to the status bus 10. The application compatibility module 70 is the sole publisher of application running status.

[0276] In summary, the publishers and subscribers of each state in scheduling system 1 are shown in Table 3.

[0277]

[0278] Table 3 As shown in Table 3, each state has only one publisher, corresponding to the content recorded in Publisher Table 12 in the aforementioned embodiments; all information required by any module is obtained from the state bus 10, and all information generated by any module is published to the state bus 10. There is no direct calling relationship between modules. The vehicle signal interface component and the audio output component do not belong to the dispatch system 1, but they are connected to the state bus 10. Similarly, there is no direct calling relationship between them and the modules of the dispatch system 1.

[0279] In a preferred embodiment, each module of the scheduling system 1 is implemented as a software module running within the system service process of the vehicle terminal 100, and each module publishes and obtains status with the status bus 10 through in-process calls.

[0280] In another embodiment, each module can also be implemented as an independently running process, or even deployed on different processors within the vehicle terminal 100. In this case, each module communicates with the status bus 10 by publishing and obtaining status through an inter-process communication interface or an inter-processor communication interface. Since there is no direct call between the modules, the change in the above deployment method does not affect the implementation of each module itself.

[0281] In another embodiment, the scheduling system 1 may be supplemented with additional modules, such as a module for adjusting the availability of each seating area based on the seating situation of the occupants. The newly added module only needs to be connected to the state bus 10, subscribe to the state bus 10 for the state it needs, and publish the state it generates to the state bus 10, without modifying the existing modules.

[0282] In the scheduling system 1 provided in this embodiment, the window registration module 20, window scheduler 30, focus arbitrator 40, audio arbitrator 50, security protection module 60 and application compatibility module 70 are respectively connected to the state bus 10. The information required by each module is obtained from the state bus 10 and the information generated is published to the state bus 10. There is no direct calling relationship between the modules.

[0283] Therefore, each module does not need to know the existence of other modules or their interface forms. The connection between modules changes from being connected in pairs to each module being connected to the state bus 10. This overcomes the defect that the number of coupling relationships caused by the pairwise calls of the existing subsystems increases rapidly with the number of subsystems. When introducing new modules such as driving status processing, safety alarm processing and application compatibility processing, the new modules only need to connect to the state bus 10 and subscribe to the corresponding status. The existing modules do not need to be modified.

[0284] Meanwhile, each state has only one publisher in the scheduling system 1, and the information source relied upon by each module is unique. This works in conjunction with the publisher limitation described in the aforementioned implementation method to further ensure the consistency of the judgment criteria of each module.

[0285] Please see Figure 10 This embodiment provides an in-vehicle terminal 100, including an internal bus 110, a processor 120 and a memory 130.

[0286] The processor 120 and the memory 130 are communicatively connected via an internal bus 110. The memory 130 stores a computer program 131 that can be executed by the processor 120. When the processor 120 executes the computer program 131, it implements the multi-task window scheduling method described in any of the foregoing embodiments.

[0287] The vehicle terminal 100 also includes a display interface 140, an input interface 150, and an audio interface 160 connected via an internal bus 110. The display interface 140 is used to connect to the vehicle display screen 200, allowing the processor 120 to display various application windows on the vehicle display screen 200 according to the window layout. The input interface 150 is used to connect to various input sources 300, allowing the processor 120 to receive input events from each input source 300. The audio interface 160 is used to connect to various audio output channels 400, allowing the processor 120 to direct the audio of each application window to the corresponding audio output channel according to the audio authorization status.

[0288] The processor 120 can be a central processing unit, or other general-purpose processors, digital signal processors, application-specific integrated circuits, field-programmable gate arrays, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The number of processors 120 can be one or more; when there are multiple processors, the steps described in the foregoing embodiments can be executed by multiple processors 120 respectively.

[0289] The memory 130 can be a read-only memory, a random access memory, an erasable programmable read-only memory, a flash memory, or other forms of memory, or a combination of the above types of memory. In addition to storing the computer program 131, the memory 130 can also be used to store the seat area binding state saved before the vehicle display screen 200 is turned off in the aforementioned embodiments, as well as the most recently published value of each state saved in the latest value cache 14 in the aforementioned embodiments.

[0290] The state bus 10, window registration module 20, window scheduler 30, focus arbiter 40, audio arbiter 50, security protection module 60, and application compatibility module 70 described in the foregoing embodiments can each correspond to a program segment in computer program 131. When processor 120 executes the corresponding program segment, it implements the function of the corresponding module.

[0291] In a preferred embodiment, the vehicle terminal 100 can be the vehicle's central control unit, in which case the vehicle display screen 200 is the central control display screen connected to the central control unit; the vehicle terminal 100 can also be the vehicle's cockpit domain controller, in which case the vehicle display screen 200 is one of the multiple displays connected to the cockpit domain controller.

[0292] This embodiment also provides a non-volatile computer storage medium storing computer-executable instructions. When these computer-executable instructions are executed by one or more processors, the one or more processors can perform the multi-tasking window scheduling method described in any of the foregoing embodiments.

[0293] The non-volatile computer storage medium can be a read-only memory, flash memory, disk or optical disk, etc.; the computer executable instructions stored therein can also be transmitted to the vehicle terminal 100 via a communication network and written into its memory 130.

[0294] This embodiment carries the methods and systems described in the foregoing embodiments on specific hardware. The processor 120 executes the computer program 131 stored in the memory 130 to implement the method. The display interface 140, input interface 150 and audio interface 160 are respectively connected to the vehicle display screen 200, each input source 300 and each audio output channel 400, so that the processing described in the foregoing embodiments can be specifically implemented on the vehicle terminal 100.

[0295] This embodiment also carries the computer-executable instructions on a non-volatile computer storage medium, so that the methods described in the foregoing embodiments can be delivered separately in software form and can be written into the manufactured vehicle terminal 100 via a communication network, thereby enabling the function of displaying multiple application windows in parallel on a single vehicle display screen or supplementing the processing described in the foregoing embodiments without replacing the entire device.

[0296] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of this application as described above, which are not provided in detail for the sake of brevity; although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A multi-task window scheduling method applied to an in-vehicle terminal, the in-vehicle terminal including an in-vehicle display screen, multiple input sources, and multiple audio output channels, wherein the in-vehicle display screen displays multiple application windows in parallel to form a multi-window display, each application window being used to carry an application requesting to join the multi-window display; the vehicle in which the in-vehicle terminal is located is divided into multiple seating areas, each input source corresponding to one seating area, and each audio output channel corresponding to one seating area; characterized in that, The vehicle-mounted terminal is equipped with a state bus that transmits states via a publish-subscribe method, and the method includes: Receive a request to add the application to be added to the multi-window display; the request carries a target seating area identifier, which is used to indicate one of the multiple seating areas; The application window of the application to be added is bound to the seat area indicated by the target seat area identifier to obtain a window-level seat area binding table; the window-level seat area binding table records the correspondence between each application window and the seat area bound to that application window; The seat area binding status, which includes the window-level seat area binding table, is published to the status bus; The seat area binding status is obtained from the state bus, and the first operation and the second operation are performed according to the window-level seat area binding table therein; The first operation includes: for each input source, determining whether the seating area bound to each application window matches the seating area corresponding to the input source, and determining the application window whose determination result matches as the application window allowed to operate on the input source; upon receiving an input event from a target input source among the plurality of input sources, determining one of the application windows allowed to operate on the target input source as the focus window, publishing the focus attribution status to the status bus, and directing the input event to the focus window according to the focus attribution status; the focus attribution status includes the identifier of the focus window; The second operation includes: for each application window, determining the audio output channel corresponding to the seating area bound to the application window as the audio output channel of the application window, publishing the audio authorization status to the status bus, and directing the audio of the application window to the audio output channel of the application window according to the audio authorization status; the audio authorization status includes the correspondence between each application window and the audio output channel of the application window.

2. The method according to claim 1, characterized in that, The window-level seating area binding table also records the correspondence between each application window and the identifier of the input source that is allowed to operate the application window, as well as the correspondence between each application window and the identifier of the audio output channel of the application window. For each application window, the input source allowed to operate the application window is an input source that matches the seating area bound to the application window, and the audio output channel of the application window is an audio output channel corresponding to the seating area bound to the application window. In the first operation, determining the application window whose bound seating area matches the seating area corresponding to the input source as the application window that the input source is allowed to operate includes: reading the identifier of the input source that is allowed to operate each of the application windows from the window-level seating area binding table, and determining the input source indicated by the read identifier as the input source that is allowed to operate the application window; In the second operation, determining the audio output channel corresponding to the seating area bound to the application window as the audio output channel of the application window includes: reading the identifier of the audio output channel of each application window from the window-level seating area binding table, and determining the audio output channel indicated by the read identifier as the audio output channel of the application window.

3. The method according to claim 1, characterized in that, The method further includes: Based on the window-level seat area binding table in the seat area binding state, the display area and stacking order of each application window on the vehicle display screen are determined, the window layout state is published to the state bus, and each application window is displayed according to the window layout state; the window layout state includes the determined display area and stacking order of each application window.

4. The method according to claim 1, characterized in that, The plurality of seating areas includes a driver's seat area; the method further includes: The vehicle speed is obtained, and the vehicle driving status is determined based on the vehicle speed and published to the status bus; the vehicle driving status is used to indicate whether the vehicle is in a driving state or a stationary state. After receiving the request, the vehicle driving status is obtained from the status bus, and the request is verified for admission based on the vehicle driving status, the type of the application to be added, and the target seating area identifier. When the vehicle driving status indicates that the vehicle is in a driving state, the type of the application to be added is video, and the target seat area identifier indicates the driver's seat area, it is determined that the request has failed the access verification, the request is rejected, and the application window of the application to be added is not bound to the seat area indicated by the target seat area identifier; otherwise, it is determined that the request has passed the access verification, and the application window of the application to be added is bound to the seat area indicated by the target seat area identifier.

5. The method according to claim 4, characterized in that, The process of obtaining the vehicle's driving status based on the vehicle speed includes: When the vehicle speed increases to above a first threshold and remains above a first duration, the vehicle's driving state is set to a driving state; When the vehicle speed drops below a second threshold and remains below that threshold for a second duration, the vehicle's driving state is set to a stopped state; the second threshold is lower than the first threshold.

6. The method according to claim 4, characterized in that, The method further includes: when the request passes the admission verification and the type of the application to be added is video, setting a downgradeable flag for the application window of the application to be added; When the vehicle's driving state changes from a stopped state to a driving state, the application window with the downgraded flag is hidden, the audio gain of the application window is set to a mute value, the audio authorization state is updated, and the updated audio authorization state is published to the status bus.

7. The method according to claim 4, characterized in that, The method further includes: obtaining and saving the seat area binding status from the status bus before the vehicle display screen is turned off; After the vehicle display screen is turned on again, for each application window in the saved seat area binding state, a request is generated to add the application carried by the application window to the multi-window display; the target seat area identifier carried in the generated request is used to indicate the seat area bound to the application window. The generated request is subjected to the admission verification, and the application window of the application corresponding to the request that passes the admission verification is bound to the seat area indicated by the target seat area identifier carried in the request.

8. The method according to claim 1, characterized in that, The method further includes: receiving an alarm signal from the vehicle's safety system, and obtaining a safety alarm status based on the alarm signal; the safety alarm status includes an alarm level and an alarm window identifier, wherein the alarm window is a window on the vehicle display screen used to display the alarm content indicated by the alarm signal; The security alarm state is published to the state bus with a higher priority than other states on the state bus. The state bus interrupts the dispatch of other states when dispatching the security alarm state, and continues to dispatch the interrupted other states after the security alarm state has been dispatched. The system acquires the security alarm status from the status bus, places the alarm window on top of each application window in the stacking order, sets the input area of ​​each application window to empty so that input events from each input source are not directed to each application window, and sets the audio gain of each application window to a mute value; and during the period when the security alarm status indicates that an alarm exists, it rejects the received request to add the application to the multi-window display.

9. The method according to claim 8, characterized in that, The status bus dispatches the security alarm status, including: The state bus sends a preparation message to each subscriber that has subscribed to the security alarm state; After receiving the preparation message, each subscriber determines the stacking order of the alarm window, the input area of ​​each application window, and the audio gain of each application window, and sends an acknowledgment back to the status bus. After receiving confirmation from each subscriber, the status bus sends a commit message to each subscriber. Each subscriber, based on the submission message, merges its determined stacking order and input region into the same display composition transaction, so that the stacking order and the input region take effect within the same rendering frame.

10. The method according to claim 9, characterized in that, The method further includes: the status bus sets a waiting time for sending back the confirmation; If no confirmation is received from each of the subscribers after the specified waiting time since the preparatory message was sent, the status bus will stop waiting and directly place the alarm window on top of each of the application windows, and empty the input area of ​​each of the application windows.

11. The method according to claim 8, characterized in that, The security alarm status is dispatched by the first dispatch thread of the status bus, and the remaining statuses on the status bus are dispatched by the second dispatch thread of the status bus. The scheduling priority of the first dispatch thread is higher than that of the second dispatch thread. When the first dispatch thread dispatches the security alert status, it uses a message object that has been allocated before dispatch to carry the security alert status.

12. The method according to claim 1, characterized in that, The method further includes: receiving audio focus requests initiated by the applications carried by each of the application windows, and returning approval for each audio focus request; Based on the type of application carried by each application window, an audio level is assigned to each application window. The audio levels include a primary level, a secondary level, and a background level, which are successively reduced. The correspondence between each application window and its audio level is updated to the audio authorization status, and the updated audio authorization status is published to the status bus. During the audio output of the application window assigned to the primary level, the audio gain of the application window assigned to the secondary level and the application window assigned to the background level is reduced according to the audio authorization state, and the reduced audio gain is restored to the value before the output ends.

13. The method according to claim 12, characterized in that, The method further includes: obtaining the focus attribution status from the state bus; if the audio level assigned to the focus window indicated by the focus attribution status is the background level, adjusting the audio level assigned to the focus window to the secondary level; updating the adjusted correspondence to the audio authorization status; and publishing the updated audio authorization status to the state bus.

14. The method according to claim 1, characterized in that, The step of designating one of the application windows in which the target input source is allowed to operate as the focus window includes: The application windows that are allowed to operate from the target input source are sorted according to their respective arrangement order on the vehicle display screen to obtain a window sequence; When the input event indicates a focus switch, the focus window is switched from its current position in the window sequence to the next application window, and when the current position is the last position in the window sequence, the focus window is switched to the first application window in the window sequence.

15. The method according to claim 14, characterized in that, The input event includes a first input event and a second input event; When the input event is the first input event, the focus window is switched from the current position in the window sequence to the next position of the application window; When the input event is the second input event, the focus window remains unchanged, and the selected control is switched among the controls within the focus window.

16. The method according to claim 1, characterized in that, The method further includes: before binding the application window of the application to be added to the seating area indicated by the target seating area identifier, detecting whether the application to be added declares a scalable attribute; When the application to be added does not declare the scalable attribute, the interface of the application to be added is rendered to the off-screen buffer at the resolution declared by the application to be added. After obtaining the window layout state, the content in the off-screen buffer is scaled to the display area of ​​the application window of the application in the window layout state and then composited for display, and no configuration change event is sent to the application to be added.

17. The method according to claim 1, characterized in that, The method further includes: monitoring the running status of the applications carried by each application window, and when any application is unresponsive or exits, obtaining an application running status indicating that the application is abnormal and publishing it to the status bus; The application running state is obtained from the state bus. In the window layout state, a placeholder window is displayed in the display area of ​​the application window to replace the application window, while keeping the display area, stacking order and audio output channel of the other application windows unchanged.

18. The method according to claim 1, characterized in that, Each state on the state bus is published to the state bus by only one publisher, and each state carries a version number generated by the publisher of that state when it is published, and the version number increments each time the state is published. If a subscriber to the state bus acquires a state whose version number is not higher than the version number of the same state that the subscriber has already acquired, the subscriber shall discard the acquired state.

19. The method according to claim 1, characterized in that, The method further includes: setting an expiration period for each state on the state bus; if a subscriber of the state bus has not acquired a state again after the expiration period since the last time it acquired a state, the subscriber shall replace the state acquired by the subscriber with a preset default value of the state. The state bus stores the most recently published value for each state and republishes the stored value to the state bus after the state bus restarts.

20. A multi-task window scheduling system, the system being installed in a vehicle-mounted terminal, the vehicle-mounted terminal including a vehicle-mounted display screen, multiple input sources, and multiple audio output channels, the vehicle-mounted display screen displaying multiple application windows in parallel to form a multi-window display, each application window being used to carry an application requesting to join the multi-window display; the vehicle where the vehicle-mounted terminal is located is divided into multiple seating areas, each input source corresponding to one seating area, and each audio output channel corresponding to one seating area; characterized in that, The system includes: A state bus is used to transmit state in a publish-subscribe manner. A window registration module, connected to the status bus, is used to receive a request to add an application to the multi-window display. The request carries a target seating area identifier, which is used to indicate one of the multiple seating areas. The window registration module is also used to bind the application window of the application to be added to the seat area indicated by the target seat area identifier to obtain a window-level seat area binding table, and publish the seat area binding status containing the window-level seat area binding table to the status bus. The window-level seat area binding table records the correspondence between each application window and the seat area bound to that application window. A focus arbitrator, connected to the state bus, is used to obtain the seat area binding status from the state bus, and according to the window-level seat area binding table therein, for each input source, determine whether the seat area bound to each application window matches the seat area corresponding to the input source, and determine the application window whose determination result matches as the application window that is allowed to operate on the input source. The focus arbiter is further configured to, upon receiving an input event from a target input source among the plurality of input sources, determine a focus window within the application window where the target input source is allowed to operate, publish the focus attribution status to the status bus, and direct the input event to the focus window according to the focus attribution status; the focus attribution status includes an identifier of the focus window; An audio arbitrator, connected to the state bus, is used to obtain the seat area binding status from the state bus, and, based on the window-level seat area binding table therein, determine the audio output channel corresponding to the seat area bound to the application window as the audio output channel of the application window for each application window. The audio arbitrator is also used to publish the audio authorization status to the status bus, and to direct the audio of each application window to the audio output channel of the application window according to the audio authorization status; the audio authorization status includes the correspondence between each application window and the audio output channel of the application window.

21. The system according to claim 20, characterized in that, The system also includes a window scheduler; The window scheduler is connected to the state bus and is used to obtain the seat area binding status from the state bus, determine the display area and stacking order of each application window on the in-vehicle display screen according to the window-level seat area binding table therein, publish the window layout status to the state bus, and display each application window according to the window layout status; the window layout status includes the determined display area and stacking order of each application window.

22. The system according to claim 21, characterized in that, The system also includes a safety protection module connected to the state bus. The safety protection module receives alarm signals from the vehicle's safety system, obtains a safety alarm status based on the alarm signals, and publishes the safety alarm status to the state bus with a higher priority than other statuses on the state bus. The safety alarm status includes an alarm level and an alarm window identifier. The alarm window is a window on the vehicle display screen used to display the alarm content indicated by the alarm signal. The state bus is also used to interrupt the dispatch of other states when dispatching the security alarm state, and to continue dispatching the interrupted other states after the security alarm state has been dispatched. The window scheduler is also used to obtain the security alarm status from the status bus and place the alarm window on top of each of the application windows in the stacking order. The focus arbiter is also used to obtain the security alarm status from the status bus and empty the input area of ​​each application window; The audio arbitrator is also used to obtain the security alarm status from the status bus and set the audio gain of each application window to a mute value; The window registration module is also used to obtain the security alarm status from the status bus, and during the period when the security alarm status indicates that an alarm exists, reject the received request to add the application to the multi-window display.

23. The system according to claim 22, characterized in that, The state bus is also used to send a preparation message to the window scheduler, the focus arbiter, and the audio arbiter when dispatching the security alarm state, and to receive confirmations from the window scheduler, the focus arbiter, and the audio arbiter after they have determined the stacking order, the input area, and the audio gain, respectively; after receiving confirmations from the window scheduler, the focus arbiter, and the audio arbiter, the state bus sends a commit message so that the window scheduler and the focus arbiter merge their determined stacking order and input area into the same display composition transaction according to the commit message; The status bus is also used to set a waiting time for sending back the confirmation, and if all the confirmations are not received after the waiting time since the preparation message was sent, the alarm window is directly placed on top of each of the application windows, and the input area of ​​each of the application windows is set to empty.

24. The system according to claim 22, characterized in that, The state bus has a first dispatch thread and a second dispatch thread. The first dispatch thread is used to dispatch the security alarm state, and the second dispatch thread is used to dispatch the remaining states on the state bus. The scheduling priority of the first dispatch thread is higher than that of the second dispatch thread, and the first dispatch thread is used to carry the security alarm state with a message object that has been allocated before dispatch.

25. The system according to claim 21, characterized in that, The audio arbitrator is also used to receive audio focus requests initiated by the applications carried by each application window and return approval for each audio focus request, assign audio levels to each application window according to the type of the application carried by each application window, update the correspondence between each application window and its audio level to the audio authorization status, and publish the updated audio authorization status to the status bus; the audio levels include primary, secondary and background levels in descending order; The audio arbiter is also used to reduce the audio gain of the application window assigned to the secondary level and the application window assigned to the background level during the audio output of the application window assigned to the primary level, and to restore the reduced audio gain to its previous value after the output ends.

26. The system according to claim 21, characterized in that, The system also includes an application compatibility module, which is connected to the state bus. Before the window registration module binds the application window of the application to be added to the seat area indicated by the target seat area identifier, the application to be added detects whether the application to be added declares a scalable attribute. If the application to be added does not declare the scalable attribute, the application's interface is rendered to the off-screen buffer at the resolution declared by the application to be added. After the window scheduler obtains the window layout state, the content in the off-screen buffer is scaled to the display area of ​​the application window of the application in the window layout state and then composited for display. No configuration change event is sent to the application to be added. The application compatibility module is also used to monitor the running status of the applications carried by each application window. When any application is unresponsive or exits, it obtains the application running status indicating the abnormality of the application and publishes it to the status bus. The window scheduler is also used to obtain the application running state from the state bus, and in the window layout state, display a placeholder window in the display area of ​​the application window of the application to replace the application window, while keeping the display area and stacking order of the remaining application windows unchanged.

27. A vehicle-mounted terminal, characterized in that, include: At least one processor; A memory that is communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the multitasking window scheduling method as described in any one of claims 1-19.

28. A non-volatile computer storage medium, characterized in that, The non-volatile computer storage medium stores computer-executable instructions; The computer-executable instructions are executed by one or more processors, causing the one or more processors to perform the multitasking window scheduling method as described in any one of claims 1-19.

Images