Multi-screen application adaptation method and device, vehicle and computer readable storage medium

Abstract

The invention discloses a multi-screen application adaptation method and device, a vehicle and a computer readable storage medium, and the method comprises the steps: recognizing user behavior information in a vehicle cabin after a preset target application is triggered based on a preset touch operation, determining a target display mode corresponding to the preset target application according to the user behavior information; and loading and displaying the preset target application through the target display mode. According to the method, the user demand can be identified more accurately, then the application information is accurately displayed on the screen matched with the driving position of the user, application display services are provided for passengers in a targeted mode, and better multi-screen cooperative linkage interaction experience is brought to the user.

Description

Technical Field

[0001] The present invention relates to the field of smart cockpit technology, and in particular to a multi-screen application adaptation method, device, vehicle and computer-readable storage medium. Background Art

[0002] As the multi-task display function plays an increasingly important role in the user's car use, users have an increasingly strong demand for adaptive changes in the application of different areas of the screen and linkage between screens. In other words, the demand for coordinated linkage display between different areas of the screen is increasing: as vehicles are equipped with larger screen displays, passengers hope to be able to effectively coordinate and link displays between different areas of the screen.

[0003] In the related art, images can only be displayed independently on multiple single screens, but cannot be displayed collaboratively on multiple screens. For example, when a user displays application information on a car display, it is impossible to intelligently identify the user's needs and accurately load the application to the screen location required by the user. Currently, the user needs to manually operate to load and display the application in the screen area intended by the user, which brings a poor screen interaction experience to the user.

[0004] To sum up, the current problem is that the in-vehicle system cannot effectively support the coordinated and linked display between multiple screens. Summary of the invention

[0005] The main purpose of the present invention is to provide a multi-screen application adaptation method, device, vehicle and computer-readable storage medium, aiming to solve the technical problem that the current vehicle-mounted system cannot effectively support collaborative linkage display between multiple screens.

[0006] To achieve the above object, the present invention provides a multi-screen application adaptation method, which is applied to a vehicle display screen, wherein the vehicle display screen includes a plurality of independent split screens, wherein the independent split screens include a first screen corresponding to a main driver's seat and a second screen corresponding to a co-driver's seat, and a floating floating window is provided on the vehicle display screen, wherein the application loaded and displayed in the floating window is an application independent of the first screen and the application of the second screen, and the method includes:

[0007] After the preset target application is triggered based on the preset touch operation, identifying user behavior information in the vehicle cabin, and determining a target display mode corresponding to the preset target application to be displayed according to the user behavior information;

[0008] Loading and displaying the preset target application through the target display mode;

[0009] Among them, the target display mode is a first single-screen display mode displayed on the first screen, a second single-screen display mode displayed on the second screen, a full-screen display mode or a floating window display mode. The full-screen display mode is a mode in which the first screen and the second screen are combined into a large screen to jointly display an application. The floating window display mode is a mode in which the application is loaded and displayed through the floating window.

[0010] Optionally, the step of determining, according to the user behavior information, a target display mode corresponding to displaying the preset target application includes:

[0011] Determine a trigger device corresponding to the preset target application, and obtain user account information bound to the trigger device;

[0012] Determining user demand information for the preset target application based on the user behavior information and in combination with the user account information;

[0013] According to the user demand information, a target display mode corresponding to displaying the preset target application is determined.

[0014] Optionally, the step of identifying user behavior information in the vehicle cabin includes:

[0015] Dynamically collect image information inside the vehicle cabin through the vehicle-mounted camera;

[0016] Identify user action information based on the image information and in combination with an image recognition algorithm;

[0017] The user behavior information in the vehicle cabin is determined according to the user action information.

[0018] Optionally, the step of determining user behavior information in the vehicle cabin according to the user action information includes:

[0019] Dynamically collect audio information in the cabin through the on-board microphone;

[0020] The user behavior information in the vehicle cabin is identified based on the audio information and combined with the user action information.

[0021] Optionally, the step of identifying user behavior information in the vehicle cabin based on the audio information and the user action information includes:

[0022] Identify the user's seat position in the vehicle cabin through the seat sensor;

[0023] Detecting seat adjustment information corresponding to the vehicle seat seating position, wherein the seat adjustment information includes a seat back angle and a seat height;

[0024] User behavior information in the vehicle cabin is identified based on the seat adjustment information and the audio information in combination with the user action information.

[0025] Optionally, the step of identifying user behavior information in the vehicle cabin according to the seat adjustment information and the audio information in combination with the user action information includes:

[0026] The seat adjustment information, the audio information, and the user action information are input into a pre-trained behavior recognition neural network model to identify the user behavior information in the cabin.

[0027] In addition, the present application also provides a multi-screen application adaptation device, which is applied to a vehicle display screen, wherein the vehicle display screen includes multiple independent split screens, wherein the independent split screens include a first screen corresponding to a main driver's seat, and a second screen corresponding to a co-driver's seat, and a floating floating window is provided on the vehicle display screen, wherein the application loaded and displayed in the floating window is an application independent of the first screen and the application of the second screen, and the device includes:

[0028] A behavior recognition module, configured to recognize user behavior information in the vehicle cabin after a preset target application is triggered based on a preset touch operation, and determine a target display mode corresponding to the preset target application to be displayed according to the user behavior information;

[0029] An application display module, used for loading and displaying the preset target application through the target display mode;

[0030] Among them, the target display mode is a first single-screen display mode displayed on the first screen, a second single-screen display mode displayed on the second screen, a full-screen display mode or a floating window display mode. The full-screen display mode is a mode in which the first screen and the second screen are combined into a large screen to jointly display an application. The floating window display mode is a mode in which the application is loaded and displayed through the floating window.

[0031] In addition, the present application also provides a vehicle display screen, including a memory, a processor, and a multi-screen application adapter program stored in the memory and runnable on the processor, wherein the multi-screen application adapter program implements the steps of the multi-screen application adaptation method as described above when executed by the processor.

[0032] In addition, the present application also provides a vehicle, comprising the vehicle display screen as described above.

[0033] In addition, the present application also provides a computer-readable storage medium, on which a multi-screen application adaptation program is stored, and when the multi-screen application adaptation program is executed by a processor, the steps of the multi-screen application adaptation method described above are implemented.

[0034] When users display application information on the car display, the system cannot intelligently identify the user's needs and accurately display the application information at the corresponding screen position. Instead, the user needs to manually touch to select the corresponding target display mode, which brings a poor screen interaction experience to the user. For example, the screen corresponding to the main driver is screen A and the instrument screen (i.e. screen C), and the screen corresponding to the co-driver is screen B. When the passengers in the car display screen display information of different applications such as making calls, video projection or map navigation, how to accurately display different applications at the corresponding screen position (whether displayed on screen A, screen B or screen C, or cross-screen display of screen A and screen B, or full-screen display) without the user manually triggering adjustment to improve the user's screen interaction experience has become an urgent problem to be solved.

[0035] The present application arranges a vehicle display screen to include multiple independent split screens, the independent split screens include a first screen corresponding to the main driver's seat, and a second screen corresponding to the co-driver's seat, and a floating floating window is provided on the vehicle display screen, and the application loaded and displayed in the floating window is an application independent of the first screen and the application of the second screen. The method includes: after the preset target application is triggered based on the preset touch operation, identifying the user behavior information in the cabin, determining the target display mode corresponding to the preset target application according to the user behavior information, loading and displaying the preset target application through the target display mode, thereby intelligently identifying user needs, accurately displaying the application information on the screen matching the user's driving seat, and then providing application display services to the occupants in a targeted manner, thereby effectively guiding the passengers. Different applications such as navigation, multimedia, video, and telephone can be used to intelligently select the optimal target display mode for information display, and accurately and intelligently transfer user behavior information between the first screen and the second screen, giving users a better multi-screen collaborative linkage interactive experience. Compared with the traditional fixed split-screen mode, the above solution can dynamically adjust the display position of the application on the screen according to the needs of the passengers and scene requirements, to achieve a more flexible split-screen display effect. Through the linkage between applications, passengers can enjoy richer interactive experience and personalized services. This linkage and interaction method can meet the passengers' needs for personalization and intelligence, and improve the overall user experience, thereby enabling the present application to effectively solve the technical problem that the current in-vehicle system cannot effectively support collaborative linkage display between multiple screens. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0037] Figure 1 This is a flow chart of a first embodiment of a multi-screen application adaptation method of the present invention;

[0038] Figure 2 A schematic diagram of an interface of a preset target application displayed in a floating window in an embodiment of the present invention;

[0039] Figure 3 This is a schematic diagram of the interface of a vehicle display screen according to a first embodiment of the present invention;

[0040] Figure 4 A schematic diagram of an interface of a vehicle display screen according to a second embodiment of the present invention;

[0041] Figure 5 A schematic diagram of an interface of a vehicle display screen according to a third embodiment of the present invention;

[0042] Figure 6 This is a flow chart of a second embodiment of the multi-screen application adaptation method of the present invention;

[0043] Figure 7 A schematic diagram of the interface conversion of a vehicle display screen for generating a floating window in one embodiment of the present invention;

[0044] Figure 8 A schematic diagram of scene conversion for generating a floating window in an embodiment of the present invention;

[0045] Fig. 9 A schematic diagram of an interface of a vehicle display screen according to a third embodiment of the present invention;

[0046] Fig.10 A schematic diagram of a scene of a window state of a floating window in an embodiment of the present invention;

[0047] Fig.11 A schematic diagram of device modules of a multi-screen application adaptation device according to an embodiment of the present invention;

[0048] Fig.12 The figure is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present invention.

[0049] The realization of the purpose, functional features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. The above-mentioned drawings have shown clear embodiments of this application, which will be described in more detail later. These drawings and textual descriptions are not intended to limit the scope of the concept of this application in any way, but to illustrate the concept of this application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0052] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0053] The smart cockpit is an intelligent service system based on the car cockpit, which is used to proactively understand and meet user needs. From the perspective of end consumers, passengers not only do not need to worry about car driving issues, but can also get a comfortable experience from the smart cockpit entertainment system.

[0054] With the rapid development of smart cockpits, the display screens in cockpits have also changed from a single display screen to multiple displays to provide users with a better riding experience. During the user's riding experience, due to the increase in screen display area, passengers' demand for interaction between different areas of the screen has gradually increased. At the same time, with the gradual enrichment of vehicle infotainment systems and in-vehicle ecosystems, the use scenarios of applications in the in-vehicle interface have become more complex, and the multi-task display function has become increasingly important in the user's car use process. At the same time, the need for applications in different areas of the screen to make adaptive changes and to link between screens has become a function to enhance the user experience.

[0055] As the function of multi-task display becomes more and more important in the process of users using the car, users have an increasingly strong demand for adaptive changes in applications in different areas of the screen and linkage between screens. That is, the demand for coordinated linkage display between different areas of the screen is increasing: as vehicles are equipped with larger screen displays, passengers hope to be able to effectively coordinate and link displays between different areas of the screen. In related technologies, it is only possible to display images independently for multiple single screens, but it is not possible to coordinate and link multiple screens. For example, when users display application information on the car display screen, it is impossible to intelligently identify the user's needs and accurately load the application to the screen position required by the user. Currently, users need to manually operate to load and display the application in the screen area intended by the user, which brings a poor screen interaction experience to users. In summary, the current problem is that the in-vehicle system cannot effectively support the coordinated linkage display between multiple screens.

[0056] Based on this, the present invention provides a multi-screen application adaptation method, please refer to Figure 1 The multi-screen application adaptation method is applied to a vehicle display screen, the vehicle display screen includes multiple independent split screens, the independent split screens include a first screen corresponding to the main driver's seat, and a second screen corresponding to the co-driver's seat, a floating floating window is provided on the vehicle display screen, and the application loaded and displayed in the floating window is an application independent of the first screen and the application of the second screen, and the method includes:

[0057] Step S10, after the preset target application is triggered based on the preset touch operation, identifying user behavior information in the vehicle cabin, and determining a target display mode corresponding to the preset target application according to the user behavior information;

[0058] Step S20: loading and displaying the preset target application through the target display mode.

[0059] Among them, the target display mode is a first single-screen display mode displayed on the first screen, a second single-screen display mode displayed on the second screen, a full-screen display mode or a floating window display mode. The full-screen display mode is a mode in which the first screen and the second screen are combined into a large screen to jointly display an application. The floating window display mode is a mode in which the application is loaded and displayed through the floating window.

[0060] In this embodiment, if Figure 2As shown, the types of preset target applications displayed in the floating window include but are not limited to phone calls 11, video calls 12, ADAS (Advanced Driver Assistance System) applications 13, information notifications 14, video playback 15, music playback 16, and vehicle side and rear environment images 17, etc., which are not specifically limited in this embodiment. In this embodiment, various small window scenes can be presented on the vehicle display screen by generating a floating floating window, so as to facilitate the acceptance of various temporary tasks and facilitate the driver and the co-driver to quickly view and process.

[0061] In this embodiment, the user behavior information may include the user's seat position, whether the user is driving the vehicle, lying down in the vehicle, sleeping, playing with a mobile phone, and using the KTV mode or movie pure sound mode carried by the smart cabin, etc. This embodiment can identify the user behavior information in the cabin after the preset target application is triggered based on the preset touch operation, select the optimal target display mode according to the user behavior information, and display the preset target application to the user based on the optimal target display mode, thereby intelligently identifying user needs, accurately displaying the application information on the screen matching the user's driving seat according to the user's actual needs, and then providing targeted application display services for the passengers.

[0062] In this embodiment, after the generation of the floating window is triggered, the size of the generated floating window is the initial default size, which is smaller than the size of the first screen and smaller than the size of the second screen, so that the user can move the floating window within the screen area of ​​the first screen, the screen area of ​​the second screen, or across the screen between the first screen and the second screen based on the preset user touch drag operation. It can be understood that the user can expand the size of the window area of ​​the floating window on the vehicle display screen by performing a preset touch operation on the floating window, that is, after the floating window is generated, its initial default size can be edited and changed based on the user's touch operation instruction.

[0063] Those skilled in the art will know that the application loaded and displayed in the floating window is independent of the application on the first screen, and the application on the second screen means that the floating window is a floating window that can be moved between the first screen and the second screen based on the user's touch operation, and the application loaded and displayed in the floating window can be independent of application 1 (the application loaded and displayed on the desktop corresponding to the first screen) and application 2 (the application loaded and displayed on the desktop corresponding to the second screen) without interfering with each other. Figure 8As shown. Of course, when the vehicle display screen enters the full-screen display mode, that is, when the first screen and the second screen are combined into a large screen to display an application together, when the application on the first screen is interacted with by human-computer interaction and the interface of the application changes, the application on the second screen should also change accordingly. However, the application loaded and displayed in the floating window can still be independent of the application loaded and displayed on the large screen combined with the first and second screens without interfering with each other.

[0064] In one implementable manner, the display area of ​​the first screen includes a central control display area and an instrument area, wherein the default display image information of the instrument area is the instrument image information, the default display image information of the central control display area is user application information or multimedia information, and the initial default size of the floating window is smaller than the size of the central control display area.

[0065] To facilitate understanding, in one embodiment shown, Figure 3 As shown, the vehicle display screen includes a first screen 2 corresponding to the main driver's seat and a second screen 3 corresponding to the co-driver's seat, wherein the first screen 2 may include a central control display area 22 and an instrument area 21. In addition, the vehicle display screen also includes a floating floating window 1.

[0066] When users display application information on the car display, the system cannot intelligently identify the user's needs and accurately display the application information at the corresponding screen position. Instead, the user needs to manually touch to select the corresponding target display mode, which brings a poor screen interaction experience to the user. For example, the screen corresponding to the main driver is screen A and the instrument screen (i.e. screen C), and the screen corresponding to the co-driver is screen B. When the passengers in the car display screen display information of different applications such as making calls, video projection or map navigation, how to accurately display different applications at the corresponding screen position (whether displayed on screen A, screen B or screen C, or cross-screen display of screen A and screen B, or full-screen display) without the user manually triggering adjustment to improve the user's screen interaction experience has become an urgent problem to be solved.

[0067] The embodiment of the present application arranges a vehicle display screen to include multiple independent split screens, the independent split screens include a first screen corresponding to the main driver's seat, and a second screen corresponding to the co-driver's seat, and a floating floating window is provided on the vehicle display screen, and the application loaded and displayed in the floating window is an application independent of the first screen and the application of the second screen. The method includes: after the preset target application is triggered based on the preset touch operation, identifying the user behavior information in the cabin, determining the target display mode corresponding to the preset target application according to the user behavior information, loading and displaying the preset target application through the target display mode, thereby intelligently identifying the user's needs, accurately displaying the application information on the screen matching the user's driving seat, and then providing the occupants with targeted application display services, thereby effectively guiding the guide. Different applications such as navigation, multimedia, video, and telephone can be used to intelligently select the optimal target display mode for information display, and accurately and intelligently transfer user behavior information between the first screen and the second screen, so as to bring users a better multi-screen collaborative linkage interactive experience. Compared with the traditional fixed split-screen mode, the above solution can dynamically adjust the display position of the application on the screen according to the needs of the passengers and scene requirements, so as to achieve a more flexible split-screen display effect. Through the linkage between applications, passengers can enjoy richer interactive experience and personalized services. This linkage and interaction method can meet the passengers' needs for personalization and intelligence, and improve the overall user experience, thereby making the embodiment of the present application effectively solve the technical problem that the current in-vehicle system cannot effectively support the collaborative linkage display between multiple screens.

[0068] In one possible implementation, please refer to Figure 6 The step of determining the target display mode corresponding to the preset target application according to the user behavior information includes:

[0069] Step S11, determining a triggering device corresponding to the preset target application, and obtaining user account information bound to the triggering device;

[0070] Step S12, determining user demand information for the preset target application based on the user behavior information and the user account information;

[0071] Step S13: determining a target display mode corresponding to displaying the preset target application according to the user demand information.

[0072] Since the driver often sits in the driver's seat, the partner or wife usually sits in the front passenger seat. That is, the main driver's seat and the passenger seat are usually occupied by fixed people. For this riding scene, users in different front seats can pre-register and input user account information through their own electronic devices (such as mobile phones) in advance, so that users in corresponding seats can bind to their matching independent split screens. For example, the first screen is bound to the first user account (main driver's user account), and the second screen is bound to the second user account (co-pilot user account), so that it is convenient to identify the trigger device corresponding to the preset target application in the future, so as to accurately display the application information of the preset target application on the screen matching the user's driving seat, and then provide targeted application display services for the occupants. For example, the first screen corresponding to the main driver's seat is the main driving area (i.e., screen A) and the instrument area (i.e., screen C), and the second screen corresponding to the co-pilot is screen B. When Xiao Ming, a passenger in the car, displays information of different applications such as making calls, video projection, or map navigation on the vehicle display screen based on his mobile phone, the information can be displayed on the independent split screen corresponding to the seat where Xiao Ming sits, improving the user's linkage display experience of the vehicle display screen.

[0073] This embodiment determines the trigger device corresponding to the preset target application and obtains the user account information bound to the trigger device, determines the user demand information for the preset target application based on the user behavior information and the user account information, and determines the target display mode corresponding to the preset target application based on the user demand information. This allows the embodiment of the present application to integrate the trigger device corresponding to the preset target application, conduct a comprehensive analysis and evaluation of the user demand information, and more accurately identify user needs, thereby accurately displaying the application information on the screen that matches the user's driving seat, thereby providing targeted application display services to the occupants.

[0074] In a possible implementation, the step of identifying user behavior information in the vehicle cabin includes:

[0075] Step A10, dynamically collecting image information in the vehicle cabin through the vehicle-mounted camera;

[0076] Step A20, identifying user action information based on the image information in combination with an image recognition algorithm;

[0077] Step A30, determining the user behavior information in the vehicle cabin according to the user action information.

[0078] In this embodiment, by collecting image information in the cabin, based on the image information and combined with an image recognition algorithm, the user's position in the cabin, whether the user is operating the steering wheel, whether the seat angle is adjusted to a larger angle, whether the user has his eyes closed, whether the user is looking at the mobile phone, and whether the user is looking at the vehicle-mounted multimedia device and other user action information can be identified to identify the user's sitting position, whether the user is driving the vehicle, lying down in the car to rest, sleeping, playing with the mobile phone, or using the KTV mode or movie pure sound mode equipped in the smart cabin and other user behavior information.

[0079] This embodiment dynamically collects image information in the vehicle cabin through a vehicle-mounted camera, and then identifies user action information based on the image information in combination with an image recognition algorithm, thereby more accurately obtaining user behavior information in the vehicle cabin based on the user action information.

[0080] In a possible embodiment, the step of determining the user behavior information in the vehicle cabin according to the user action information includes:

[0081] Step B10, dynamically collecting audio information in the vehicle cabin through the vehicle microphone;

[0082] Step B20: identifying the user behavior information in the vehicle cabin based on the audio information and the user action information.

[0083] In this embodiment, the audio information in the cabin can be dynamically collected through the vehicle-mounted microphone. Based on the audio information and combined with the user action information, it is possible to identify whether the audio information in the cabin contains the roar of the engine, the navigation broadcast sound of the map, snoring, the sound of mobile games, the sound of video playback, or the sound of music playback, etc., to identify whether the user is driving the vehicle, lying down in the car to rest, sleeping, playing with a mobile phone, or using the KTV mode or movie pure sound mode equipped in the smart cabin, and other user behavior information.

[0084] This embodiment dynamically collects audio information in the cabin through the on-board microphone, and then identifies the user behavior information in the cabin based on the audio information and the user action information. As a result, the embodiment of the present application integrates the audio information in the cabin, conducts a comprehensive analysis and evaluation of the user behavior information, and can more accurately identify the user behavior information, so that the application information can be accurately displayed on the screen that matches the user's driving seat based on the accurate user behavior information, and targeted application display services can be provided to the occupants, effectively and intelligently selecting the optimal target display mode for different applications such as navigation, multimedia, and video for information display, so as to facilitate the accurate and intelligent flow between the first screen and the second screen based on the user behavior information, thereby bringing users a better multi-screen collaborative linkage interactive experience.

[0085] Furthermore, the step of identifying the user behavior information in the vehicle cabin based on the audio information and the user action information includes:

[0086] Step C10, identifying the user's seat position in the vehicle cabin through a seat sensor;

[0087] In one embodiment, the seat sensor may be a gravity sensor (or a pressure sensor), and the load-bearing information obtained by the seat sensor may be analyzed to obtain the user's seat position in the vehicle cabin. For example, if the weight information obtained by the seat sensor on the front passenger seat is 50 kg, it is determined that there is a user on the front passenger seat, and the front passenger seat is used as the user's seat position in the vehicle cabin.

[0088] In another embodiment, the seat sensor may be an infrared sensor, which detects the physical volume of the seat and detects whether there is a human body on the seat based on whether the physical volume of the seat is greater than a preset volume threshold, thereby identifying the user's seating position in the vehicle cabin.

[0089] In yet another embodiment, the seat sensor may be an inductive sensor for detecting whether a seat belt is buckled. By detecting whether the seat belt on the seat is fastened, it is determined whether there is a human body on the seat, thereby identifying the user's seat position in the vehicle cabin.

[0090] Step C20, detecting seat adjustment information corresponding to the vehicle seat seating position, wherein the seat adjustment information includes a seat back angle and a seat height;

[0091] Step C30, identifying the user behavior information in the vehicle cabin according to the seat adjustment information and the audio information in combination with the user action information.

[0092] Exemplarily, further, the step of identifying the user behavior information in the vehicle cabin according to the seat adjustment information and the audio information in combination with the user action information includes:

[0093] Step D10, inputting the seat adjustment information, the audio information, and the user action information into a pre-trained behavior recognition neural network model to identify the user behavior information in the cabin.

[0094] This embodiment uses a seat sensor to identify the user's seat position in the vehicle cabin, and then detects seat adjustment information corresponding to the seat position, wherein the seat adjustment information includes a seat back angle and a seat height, so as to facilitate identifying the user's seat position and the user's sitting posture on the current seat, such as whether sitting upright, lying flat or sitting at an angle (when the sitting posture is sitting at an angle, the angle information of the sitting angle is also determined by the seat adjustment information), etc., based on the sensing data of the seat sensor and the seat adjustment information, and combining the collected audio information in the vehicle cabin and the user's motion information, to accurately identify the user's seat position, whether the user is lying flat to rest in the vehicle, sleeping, driving a vehicle, playing with a mobile phone, or using the KTV mode or movie pure sound mode equipped in the smart vehicle cabin, and other user behavior information.

[0095] The embodiment of the present application integrates the seat adjustment information of the user's seat position in the vehicle cabin, conducts a comprehensive analysis and evaluation of the user behavior information, and can more accurately identify the user behavior information, so as to facilitate the subsequent accurate display of the application information on the screen that matches the user's driving seat based on the accurate user behavior information, thereby providing targeted application display services to the passengers, and then effectively selecting the optimal target display mode for different applications such as navigation, multimedia, video and telephone to display information, and accurately and intelligently transferring the information between the first screen and the second screen based on the user behavior information, bringing users a better collaborative and interactive experience between multiple screens.

[0096] In order to help understand the vehicle-mounted display screen of the present application, a specific embodiment 1 is listed:

[0097] (1) Multiple independent split screens that can interact with each other

[0098] The application is started on different independent split screens (including the first screen 2 and the second screen 3, wherein the first screen 2 includes the instrument area 21 and the central control display area 22), or a screen floating window (i.e., a floating window 1), such as Figure 7 As shown, the subsequent description directly uses the A screen (i.e., the central control display area 22), the B screen (i.e., the second screen 3), the C screen (i.e., the instrument area 21), the D screen (i.e., the floating window 1), etc. to distinguish the types and positions of different screens, which can be referred to Figure 7 .

[0099] When the application is launched on the screen, a launch icon may or may not exist on screen A, screen B, or screen C. The application supports multi-window display (i.e., supports display of multiple floating windows 1 ), and supports independent operation when multiple windows are displayed.

[0100] (2) Small window (i.e. floating window)

[0101] The small window exists as a virtual screen called D screen, which is independent of screen A, screen B, and screen C. It has a size different from other screens and can adapt to more scenarios. It is also more flexible, with multi-zone linkage and real-time switching between display areas for users in different positions (for example, if the driver receives a call and it is inconvenient to answer it, the call can be transferred from screen A to screen B at the co-pilot passenger seat through a three-finger sliding gesture).

[0102] Some applications support displaying in small windows. The application sends a command to the small window, calls up the small window D screen and displays it within the D screen range. The small window may appear in multiple locations. Fig. 9 The vehicle display screen includes a first screen 2 corresponding to the main driver's seat, and a second screen 3 corresponding to the co-driver's seat, wherein the first screen 2 may include a central control display area 22 and an instrument area 21. The small window (i.e., floating window 1) may specifically appear at the location where the application side initiates the command. For example, when a phone call comes in, the phone application side is limited to opening only the central control display area 22 on the main driver's side, and sends a command to the system side to call up the D screen (i.e., floating window 1) on the left side of the A screen (i.e., the central control display area 22) to display the phone application.

[0103] The small window has two display states: embedded small window and floating small window. Fig.10 .

[0104] 1. Embedded small window: When the small window appears, the system determines whether there is space for the small window to be embedded based on the applications currently opened in other areas. If the embedding conditions are met, the virtual screens in other areas and the corresponding applications on the screen will be adaptively changed to ensure that the small window and applications in other areas can be fully displayed. The embedded small window is displayed side by side with applications in other areas at the same level.

[0105] 2. Floating small window: When the small window appears, the system determines whether there is space for the small window to be embedded based on the applications currently opened in other areas. If the embedding conditions are not met, the virtual screen in other areas and the corresponding applications on the screen maintain the current size and remain unchanged. After the small window is opened, the level is above the application level.

[0106] Applications running in small windows in the same position are displayed in the order of opening time. Newly opened small windows cover existing small windows. The exit mechanism of each application in the small window frame is based on the disappearance mechanism of each application. When the close button in the small window is clicked, only the application currently covering the top layer is closed, and the covered application on the lower layer is displayed.

[0107] The two states of the small window will switch according to the specific situation, and there are several specific situations:

[0108] When a new small window is opened, if the opening position is different from the original small window position, it is necessary to determine the display position of the original small window and the new small window. If there is a small window displayed near the driver's side, the small window is embedded, and the other small windows are suspended. If there is no small window displayed near the driver's side, the small window opened later will be displayed in a suspended state according to the order in which the small windows are opened.

[0109] When the system recognizes the existence of multiple small window applications, if the newly opened small window and the already opened small window need to be opened in the same position, the newly opened small window will be displayed on top of the already opened small window.

[0110] It should be noted that the above-mentioned specific embodiment 1 is only used to understand the vehicle display screen of the present application, and does not constitute a limitation on the multi-screen application adaptation method of the present application. More forms of simple transformations based on this technical concept are all within the protection scope of the present application.

[0111] Further, to facilitate understanding, an exemplary multi-screen application adaptation scenario is listed, which can be referred to Figure 3 and Figure 4 , users can Figure 3 The floating window 1 of the vehicle display screen shown in the figure performs a first preset touch operation, such as sliding to the right with three fingers, and is converted into Figure 4 The interface of the vehicle display screen is shown, so that the floating window 1 moves from the first position in the first screen 2 to the second position in the second screen 3, and the floating window completes the cross-screen movement between the first screen 2 and the second screen 3. It should be noted that the multi-screen application adaptation scenario shown is only used to understand the technical concept or principle of the present application, and does not constitute a limitation on the interface of the vehicle display screen of the present application. More simple changes to the application interface of the vehicle display screen based on this technical concept are all within the scope of protection of the present application.

[0112] Furthermore, the floating window 1 can be converted into an embedded window, that is, the floating window 1 can be resized to align with other display areas in the second screen, such as Figure 5 As shown, an example of a schematic diagram of the interface of a vehicle display screen is given, in which the three floating windows 1 are aligned and adapted with other display areas (including the instrument area 21 and the central control display area 22) of the first screen 2. In addition, the display area of ​​the first screen 2 is aligned and adapted with the second screen 3.

[0113] In addition, the present application embodiment also proposes a multi-screen application adaptation device, referring to Fig.11 , Fig.11 This is a functional module diagram of an embodiment of a multi-screen application adaptation device of the present application.

[0114] In this embodiment, the multi-screen application adaptation device is applied to a vehicle display screen, the vehicle display screen includes multiple independent split screens, the independent split screens include a first screen corresponding to the main driver's seat, and a second screen corresponding to the co-driver's seat. A floating floating window is provided on the vehicle display screen, and the application loaded and displayed in the floating window is an application independent of the first screen and the application of the second screen. The device includes:

[0115] A behavior recognition module 10 is used to recognize user behavior information in the vehicle cabin after a preset target application is triggered based on a preset touch operation, and determine a target display mode corresponding to the preset target application to be displayed according to the user behavior information;

[0116] An application display module 20, configured to load and display the preset target application through the target display mode;

[0117] Among them, the target display mode is a first single-screen display mode displayed on the first screen, a second single-screen display mode displayed on the second screen, a full-screen display mode or a floating window display mode. The full-screen display mode is a mode in which the first screen and the second screen are combined into a large screen to jointly display an application. The floating window display mode is a mode in which the application is loaded and displayed through the floating window.

[0118] The multi-screen application adaptation device provided in this embodiment and the multi-screen application adaptation method provided in the above-mentioned embodiment belong to the same inventive concept. The technical details not fully described in this embodiment can be found in the embodiment of the above-mentioned multi-screen application adaptation method. This embodiment has the same beneficial effects as the embodiments of the multi-screen application adaptation method, which will not be repeated here.

[0119] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0120] like Fig.12 As shown, Fig.12 It is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present invention.

[0121] The terminal in the embodiment of the present invention is a vehicle.

[0122] like Fig.12As shown, the terminal may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0123] Optionally, the terminal may also include a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, and the like. Among them, the sensors include light sensors, motion sensors, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display screen according to the brightness of the ambient light, and the proximity sensor may turn off the display screen and / or backlight when the terminal device is moved to the ear. Of course, the terminal device may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., which will not be repeated here.

[0124] Those skilled in the art will understand that Fig.12 The terminal structure shown in the figure does not constitute a limitation on the terminal, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0125] like Fig.12 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a multi-screen application adapter.

[0126] exist Fig.12 In the terminal shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to the client (user end) and communicate data with the client; and the processor 1001 can be used to call the multi-screen application adapter stored in the memory 1005 and perform the following operations:

[0127] After the preset target application is triggered based on the preset touch operation, identifying user behavior information in the vehicle cabin, and determining a target display mode corresponding to the preset target application to be displayed according to the user behavior information;

[0128] Loading and displaying the preset target application through the target display mode;

[0129] Among them, the target display mode is a first single-screen display mode displayed on the first screen, a second single-screen display mode displayed on the second screen, a full-screen display mode or a floating window display mode. The full-screen display mode is a mode in which the first screen and the second screen are combined into a large screen to jointly display an application. The floating window display mode is a mode in which the application is loaded and displayed through the floating window.

[0130] In addition, the present invention also provides a vehicle, including a memory, a processor, and a multi-screen application adapter program stored in the memory and executable on the processor, wherein the multi-screen application adapter program implements the steps of the multi-screen application adaptation method as described above when executed by the processor.

[0131] The present invention also provides a computer-readable storage medium, on which a multi-screen application adaptation program is stored. When the multi-screen application adaptation program is executed by a processor, the steps of the multi-screen application adaptation method described above are implemented.

[0132] The specific implementation of the computer-readable storage medium of the present invention is basically the same as the above-mentioned embodiments of the multi-screen application adaptation method, and will not be repeated here.

[0133] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0134] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0135] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0136] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. All equivalent structural changes made using the contents of the present invention's specification and drawings, or directly / indirectly applied in other related technical fields, are included in the patent protection scope of the present invention.

Claims

1. A multi-screen application adaptation method, characterized in that: The multi-screen application adaptation method is applied to a vehicle display screen, the vehicle display screen includes a plurality of independent split screens, the independent split screens include a first screen corresponding to a main driver's seat, and a second screen corresponding to a co-driver's seat, a floating floating window is provided on the vehicle display screen, and the application loaded and displayed in the floating window is an application independent of the first screen and the application of the second screen, and the method includes: After the preset target application is triggered based on the preset touch operation, identifying user behavior information in the vehicle cabin, and determining a target display mode corresponding to the preset target application to be displayed according to the user behavior information; Loading and displaying the preset target application through the target display mode; Among them, the target display mode is a first single-screen display mode displayed on the first screen, a second single-screen display mode displayed on the second screen, a full-screen display mode or a floating window display mode. The full-screen display mode is a mode in which the first screen and the second screen are combined into a large screen to jointly display an application. The floating window display mode is a mode in which the application is loaded and displayed through the floating window.

2. The multi-screen application adaptation method according to claim 1, characterized in that: The step of determining, according to the user behavior information, a target display mode corresponding to the preset target application for displaying includes: Determine a trigger device corresponding to the preset target application, and obtain user account information bound to the trigger device; Determining user demand information for the preset target application based on the user behavior information and in combination with the user account information; According to the user demand information, a target display mode corresponding to displaying the preset target application is determined.

3. The multi-screen application adaptation method according to claim 1, characterized in that: The step of identifying user behavior information in the vehicle cabin includes: Dynamically collect image information inside the vehicle cabin through the vehicle-mounted camera; Identify user action information based on the image information and in combination with an image recognition algorithm; The user behavior information in the vehicle cabin is determined according to the user action information.

4. The multi-screen application adaptation method according to claim 3, characterized in that: The step of determining the user behavior information in the vehicle cabin according to the user action information comprises: Dynamically collect audio information in the cabin through the on-board microphone; The user behavior information in the vehicle cabin is identified based on the audio information and combined with the user action information.

5. The multi-screen application adaptation method according to claim 4, characterized in that: The step of identifying the user behavior information in the vehicle cabin according to the audio information and the user action information comprises: Identify the user's seat position in the vehicle cabin through the seat sensor; Detecting seat adjustment information corresponding to the vehicle seat seating position, wherein the seat adjustment information includes a seat back angle and a seat height; User behavior information in the vehicle cabin is identified based on the seat adjustment information and the audio information in combination with the user action information.

6. The multi-screen application adaptation method according to claim 5, characterized in that: The step of identifying the user behavior information in the vehicle cabin according to the seat adjustment information and the audio information in combination with the user action information comprises: The seat adjustment information, the audio information, and the user action information are input into a pre-trained behavior recognition neural network model to identify the user behavior information in the cabin.

7. A multi-screen application adaptation device, characterized in that: The multi-screen application adaptation device is applied to a vehicle display screen, the vehicle display screen includes a plurality of independent split screens, the independent split screens include a first screen corresponding to a main driver's seat, and a second screen corresponding to a co-driver's seat, a floating floating window is provided on the vehicle display screen, and the application loaded and displayed in the floating window is an application independent of the first screen and the application of the second screen, and the device includes: A behavior recognition module, configured to recognize user behavior information in the vehicle cabin after a preset target application is triggered based on a preset touch operation, and determine a target display mode corresponding to the preset target application to be displayed according to the user behavior information; An application display module, used for loading and displaying the preset target application through the target display mode; Among them, the target display mode is a first single-screen display mode displayed on the first screen, a second single-screen display mode displayed on the second screen, a full-screen display mode or a floating window display mode. The full-screen display mode is a mode in which the first screen and the second screen are combined into a large screen to jointly display an application. The floating window display mode is a mode in which the application is loaded and displayed through the floating window.

8. A vehicle display screen, characterized in that: The invention comprises a memory, a processor and a multi-screen application adapter program stored in the memory and executable on the processor, wherein the multi-screen application adapter program, when executed by the processor, implements the steps of the multi-screen application adaptation method according to any one of claims 1 to 6.

9. A vehicle, characterized in that: Comprising the vehicle display screen as claimed in claim 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a multi-screen application adapter program, and when the multi-screen application adapter program is executed by the processor, the steps of the multi-screen application adaptation method according to any one of claims 1 to 6 are implemented.

Images