Multi-screen linkage method, apparatus and device, computer-readable storage medium, and computer program product

Abstract

The present application discloses a multi-screen linkage method, apparatus and device, a computer-readable storage medium, and a computer application program. The method is applied to an application of an in-vehicle infotainment system, wherein the in-vehicle infotainment system comprises at least two screens. The method comprises: generating change parameters on the basis of an operation event for any screen; on the basis of the change parameters, obtaining a first display image corresponding to each screen from a view engine, wherein the view engine is used for, on the basis of the change parameters, updating camera parameters corresponding to each screen, and on the basis of the camera parameters corresponding to each screen and a same three-dimensional scenario, generating a first display image corresponding to each screen; and displaying the corresponding first display image in each screen.

Description

Multi-screen linkage method, device, equipment, computer-readable storage medium, and computer program product

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] The embodiments of this application are based on the Chinese patent application with application number 202311457450.9, application date November 3, 2023, and application name “Multi-screen linkage method, device, equipment and computer-readable storage medium”, and claim the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby introduced into this application as a reference. Technical Field

[0003] The present application relates to the field of vehicle technology, and in particular to a multi-screen linkage method, device, equipment, computer-readable storage medium, and computer program product. Background Art

[0004] With the development of in-vehicle multimedia terminals, vehicle configurations are becoming increasingly comprehensive. In addition to electronic devices for realizing driving functions on the main driver's seat, in-vehicle terminal devices with entertainment functions can also be configured on the co-driver's seat. Various views can be displayed on the screens of multiple devices.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide a multi-screen linkage method, apparatus, device, computer-readable storage medium, and computer program product, which improve the richness and diversity of interaction.

[0007] The technical solution of the embodiment of the present application is implemented as follows:

[0008] In a first aspect, an embodiment of the present application provides a multi-screen linkage method, which is applied to an application on a vehicle-mounted system, wherein the vehicle-mounted system includes at least two screens, and the method includes: generating change parameters based on an operation event for any screen; obtaining a first display image corresponding to each screen from a view engine based on the change parameters; the view engine is used to update the camera parameters corresponding to each screen based on the change parameters, and generate the first display image corresponding to each screen based on the camera parameters corresponding to each screen and the same three-dimensional scene; and displaying the corresponding first display image on each screen.

[0009] In a second aspect, an embodiment of the present application provides a multi-screen linkage method, which is applied to an application on a vehicle-mounted system. The vehicle-mounted system includes at least two screens, the screen includes a background area and a front area, the front area is used to display at least one function control, the front area includes a first area, and the at least one function control includes a vehicle control. The method includes: in response to the vehicle control being slid to the first area, generating a calling parameter of the vehicle model; based on the calling parameter, obtaining a third display image from a view engine; displaying the third display image, wherein, in the third display image, the vehicle model appears from an adjacent screen, drives into a target screen, and finally stops in the first area; the target screen is the screen where the at least one function control is located.

[0010] In a third aspect, an embodiment of the present application provides a multi-screen linkage method, which is applied to a view engine, and the method includes: receiving change parameters sent by an application on a vehicle system; the change parameters are generated in response to an operation event of any one of at least two screens corresponding to the vehicle system; updating the camera parameters corresponding to each screen based on the change parameters; generating a first display image corresponding to each screen based on the camera parameters corresponding to each screen and the same three-dimensional scene; sending data feedback carrying the first display image corresponding to each screen to the application on the vehicle system; the data feedback is used to instruct the application on the vehicle system to display the corresponding first display image in each screen.

[0011] In a fourth aspect, an embodiment of the present application provides a multi-screen linkage device, which is applied to an application on a vehicle-mounted system, and the vehicle-mounted system includes at least two screens. The device includes: a first generating part, configured to generate change parameters based on an operation event for any screen; an acquiring part, configured to acquire the first display image corresponding to each screen from a view engine based on the change parameters; the view engine is used to update the camera parameters corresponding to each screen based on the change parameters, and generate the first display image corresponding to each screen based on the camera parameters corresponding to each screen and the same three-dimensional scene; and a display part, configured to display the corresponding first display image in each screen.

[0012] In a fifth aspect, an embodiment of the present application provides a multi-screen linkage device, which is applied to an application on a vehicle-mounted system. The vehicle-mounted system includes at least two screens, the screen includes a background area and a front area, the front area is used to display at least one function control, the front area includes a first area, and at least one function control includes a vehicle control. The device includes: a first generating part, which is configured to generate a calling parameter of a vehicle model in response to the vehicle control being slid to the first area; an acquiring part, which acquires a third display image from a view engine based on the calling parameter; a display part, which displays the third display image, wherein, in the third display image, the vehicle model appears from an adjacent screen, drives into a target screen, and finally stops in the first area; the target screen is the screen where the at least one function control is located.

[0013] In the sixth aspect, an embodiment of the present application provides a multi-screen linkage device, which is applied to a view engine, and the device includes: a first receiving part, configured to receive change parameters sent by an application on the vehicle system; the change parameters are generated in response to an operation event of any one of the at least two screens corresponding to the vehicle system; an updating part, configured to update the camera parameters corresponding to each screen based on the change parameters; a second generating part, configured to generate a first display image corresponding to each screen based on the camera parameters corresponding to each screen and the same three-dimensional scene; a first sending part, configured to send data feedback carrying the first display image corresponding to each screen to the application on the vehicle system; the data feedback is used to instruct the application on the vehicle system to display the corresponding first display image in each screen.

[0014] In the seventh aspect, an embodiment of the present application provides a multi-screen linkage device, which is a vehicle-mounted system; the device includes: a memory configured to store an executable computer program; and a processor configured to execute the executable computer program stored in the memory to implement the multi-screen linkage method described in the first aspect above.

[0015] In the eighth aspect, an embodiment of the present application provides a multi-screen linkage device, which is a vehicle-mounted system; the device includes: a memory configured to store an executable computer program; and a processor configured to execute the executable computer program stored in the memory to implement the multi-screen linkage method described in the second aspect above.

[0016] In the ninth aspect, an embodiment of the present application provides a multi-screen linkage device, which is a view engine; the device includes: a memory configured to store an executable computer program; and a processor configured to execute the executable computer program stored in the memory to implement the multi-screen linkage method described in the third aspect above.

[0017] In a tenth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program for implementing the multi-screen linkage method described in the first aspect when executed by a processor.

[0018] In the eleventh aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program for implementing the multi-screen linkage method described in the second aspect when executed by a processor.

[0019] In the twelfth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program for implementing the multi-screen linkage method described in the third aspect when executed by a processor.

[0020] In the thirteenth aspect, an embodiment of the present application provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the multi-screen linkage method described in the first aspect above.

[0021] In the fourteenth aspect, an embodiment of the present application provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the multi-screen linkage method described in the second aspect above.

[0022] In the fifteenth aspect, an embodiment of the present application provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the multi-screen linkage method described in the third aspect above.

[0023] Embodiments of the present application provide a multi-screen linkage method, apparatus, device, computer-readable storage medium, and computer program product. According to the solution provided in the embodiments of the present application, the method is applied to an application on a vehicle-mounted system, the vehicle-mounted system comprising at least two screens. The method comprises: generating a change parameter based on an operation event on any of the screens; obtaining a first display image corresponding to each screen from a view engine based on the change parameter; the view engine being configured to update the camera parameters corresponding to each screen based on the change parameter, and generating a first display image corresponding to each screen based on the camera parameters corresponding to each screen and the same three-dimensional scene; and displaying the corresponding first display image on each screen. The change parameter can be understood as the absolute change in the virtual camera corresponding to the screen in the view engine, and the camera parameter can be understood as the relative change in the virtual camera in the view engine (relative to the original parameter or the change in the parameter at the previous moment). The view engine is configured to generate image data corresponding to different perspectives of different virtual cameras in the same three-dimensional scene based on the camera parameter. The application on the vehicle-mounted system obtains the image data from the view engine and controls the display and update of the image. Because the image presented on each screen is equivalent to a planar image from the perspective of a different virtual camera in the same three-dimensional scene, the images presented by at least two screens are associated and may be adjacent. When a user swipes on the in-vehicle system's screen, the control event is converted into a parameter change for the virtual camera. The view engine then reacquires image data and controls the display, achieving multi-screen interaction through software logic. In this solution, as a finger swipes on any screen, the virtual camera rotates or moves, driving the views from different perspectives to change simultaneously. This allows the images displayed on multiple screens to follow the changes simultaneously, achieving multi-screen interaction and increasing the richness and diversity of interactions. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] FIG1 is a flowchart of optional steps of a multi-screen linkage method provided in an embodiment of the present application;

[0025] FIG2 is an exemplary schematic diagram 1 of a screen view provided in an embodiment of the present application;

[0026] FIG3 is a second optional step flow chart of a multi-screen linkage method provided in an embodiment of the present application;

[0027] FIG4 is a second exemplary schematic diagram of a screen view provided in an embodiment of the present application;

[0028] FIG5 is a flowchart of optional steps of a multi-screen linkage method provided in an embodiment of the present application;

[0029] FIG6A is a fourth exemplary diagram of a screen view provided in an embodiment of the present application;

[0030] FIG6B is a fifth exemplary diagram of a screen view provided in an embodiment of the present application;

[0031] FIG6C is a sixth exemplary diagram of a screen view provided in an embodiment of the present application;

[0032] FIG7 is a third exemplary diagram of a screen view provided in an embodiment of the present application;

[0033] FIG8 is a fourth optional step flow chart of a multi-screen linkage method provided in an embodiment of the present application;

[0034] FIG9 is a fifth optional step flow chart of a multi-screen linkage method provided in an embodiment of the present application;

[0035] FIG10 is a timing diagram of interaction between various devices in a multi-screen linkage provided by an embodiment of the present application;

[0036] FIG11 is an overall flow chart of a multi-screen linkage method provided in an embodiment of the present application;

[0037] FIG12 is a first optional structural diagram of a multi-screen linkage device provided in an embodiment of the present application;

[0038] FIG13 is a second optional structural diagram of a multi-screen linkage device provided in an embodiment of the present application;

[0039] FIG14 is a schematic diagram of a structure of a multi-screen linkage device according to an embodiment of the present application;

[0040] FIG15 is a second schematic diagram of the structure of a multi-screen linkage device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. It should be understood that some of the embodiments described here are used to explain the technical solutions of the present application and are not used to limit the technical scope of the present application.

[0042] In order to better understand the multi-screen linkage method provided in the embodiment of the present application, before introducing the technical solution of the embodiment of the present application, the application background and related technologies are first explained.

[0043] Most of the in-vehicle multi-screen display solutions in related technologies use a multi-window mode, with different screens displaying different windows. The linkage of multiple windows (that is, multi-screen linkage) requires setting the view background in each window separately, which is a more complicated solution. You can also use a single-window mode and then use the underlying hardware logic to divide the scene data into data on multiple display areas, and then display them on multiple screens separately. This solution relies on the underlying hardware logic and is more complicated.

[0044] The present application provides a multi-screen linkage method, as shown in FIG1 , which is a flowchart of the steps of the multi-screen linkage method provided by the present application. The multi-screen linkage method is applied to an application on a vehicle system, which includes at least two screens. The multi-screen linkage method includes the following steps:

[0045] S101 : Generate a change parameter based on an operation event on any screen.

[0046] In the embodiment of the present application, the application on the vehicle system can also be referred to as software, program, application software, application program, etc. The operating system is, for example, Android, IOS, etc., and the multi-screen linkage method can be carried on the operating system of the vehicle system. In actual applications, the application on the vehicle system obtains image data of the perspectives corresponding to at least two virtual cameras in the same three-dimensional (3D) scene from the view engine, and displays them on each screen respectively, with one virtual camera corresponding to one screen (also called a window). Since the image presented by each screen is equivalent to a planar image from the perspective of different virtual cameras in the same three-dimensional scene, the images presented by at least two screens have an associated relationship and can be adjacent parts in the three-dimensional scene. In other words, the views under the corresponding perspectives displayed on each screen (i.e., the two-dimensional images after planarization in the three-dimensional scene) are associated with each other.

[0047] In an embodiment of the present application, the view engine may be a Unity engine, which is used to create a 3D scene. For example, an environment is created in a 3D space that includes a vehicle model, background, sound, special effects, and other elements. As shown in FIG2 , FIG2 is an exemplary schematic diagram of a screen view provided in an embodiment of the present application. The 3D scene rendered by the Unity engine is equivalent to a video. The dynamic elements created may include dynamic water flow, dynamic birds, etc., wherein some elements may change along the time axis, for example, the sun is displayed during the day and the moon and starry sky are displayed at night. FIG2 is shown using the night as an example. Since the screen is a two-dimensional screen, Unity needs to capture the view and "flatten" it for display. This is achieved in Unity by adding a camera (also known as a camera). In actual applications, multiple cameras can be arranged in a 3D scene based on parameters such as position, shooting angle, and shooting distance (depth). After the 3D scene is created, Unity packages the entire scene and outputs a library file for operating system integration. That is, the view of the area to be displayed is set according to the arranged cameras, and the integrated data of each camera output channel is generated. By integrating a Unity-generated library into the operating system, you can directly add the corresponding camera-generated views to different screens. This allows you to change the 3D scene, and the views generated by different cameras change simultaneously, so that the display on multiple screens follows the changes simultaneously.

[0048] In the embodiment of the present application, an operation event refers to an operation performed by a user on any screen, including but not limited to sliding, clicking, dragging, double-clicking, pinching, opening with two fingers, zooming in and out. The application on the vehicle system responds to the operation event and generates change parameters based on the operation event, that is, converting the operation event into a change parameter. The change parameter refers to the absolute value of the change of the virtual camera, including but not limited to camera rotation parameters and camera displacement parameters.

[0049] S102. Based on the change parameters, obtain the first display image corresponding to each screen from the view engine; the view engine is used to update the camera parameters corresponding to each screen based on the change parameters, and generate the first display image corresponding to each screen based on the camera parameters corresponding to each screen and the same three-dimensional scene.

[0050] In an embodiment of the present application, the application on the vehicle system transmits the change parameters to the view engine, which can be a Unity engine. The view engine updates the camera parameters of the virtual cameras corresponding to each screen according to the change parameters, and regenerates new image data (including image data of each screen) in the same three-dimensional scene based on each camera parameter, and transmits the new image data to the application on the vehicle system, which displays a new two-dimensional image (i.e., the first display image) on each screen according to the new image data. The view engine can also directly generate a new two-dimensional image, transmit the new two-dimensional image to the application on the vehicle system, and the application on the vehicle system directly displays the new two-dimensional image (i.e., the first display image) on each screen. The first display image includes the display images of each screen.

[0051] In the embodiments of this application, the change parameter can be understood as the absolute change in the virtual camera corresponding to the screen in the view engine, and the camera parameter can be understood as the relative change in the virtual camera in the view engine (relative to the original parameter or the change in the parameter at the previous moment). The view engine is used to generate image data corresponding to different perspectives of different virtual cameras in the same 3D scene based on the camera parameters. The application on the vehicle system obtains image data from the view engine and controls the display and update of the image.

[0052] S103: Display the corresponding first display image on each screen.

[0053] In this embodiment of the present application, the application on the vehicle system obtains the first display image corresponding to each screen in the view engine and then updates the displayed image on each screen. On the vehicle system side, the corresponding viewing angles of each screen simultaneously move in the direction corresponding to the sliding operation, so that the two-dimensional images displayed on each screen move in the direction corresponding to the operation event, realizing multi-screen linkage.

[0054] In an embodiment of the present application, when a user slides on the screen of the vehicle-mounted system, the application on the vehicle-mounted system converts the operation event into a change parameter of the virtual camera, transmits the change parameter to the view engine, and the view engine adjusts the camera parameters of its virtual camera according to the change parameter, and re-outputs the image data according to the camera parameter. The application on the vehicle-mounted system controls the display of a two-dimensional image based on the image data output by the view engine, thereby achieving the effect of multi-screen linkage through software logic. In this solution, as the finger slides on any screen, the virtual camera rotates or moves, driving the views from different perspectives to change together, so that the images displayed on multiple screens follow the changes at the same time, thereby realizing multi-screen linkage and improving the richness and diversity of interaction.

[0055] The in-vehicle multi-zone 3D scene linkage solution provided by the embodiments of this application is an improvement based on software control logic. It does not rely on the underlying hardware logic, does not require hardware changes, and reduces costs. Due to the linkage between cameras in the view engine, it can achieve the effect of multi-screen linkage. For example, if a multi-screen includes a main screen and a secondary screen, for example, sliding the 3D scene on the main screen will cause the 3D scene on the secondary screen to move accordingly.

[0056] According to the solution provided in an embodiment of the present application, the method is applied to an application on a vehicle-mounted system, the vehicle-mounted system including at least two screens. The method includes: generating a change parameter based on an operation event for any screen; obtaining a first display image corresponding to each screen from a view engine based on the change parameter; the view engine is configured to update the camera parameters corresponding to each screen based on the change parameter, and generate a first display image corresponding to each screen based on the camera parameters corresponding to each screen and the same three-dimensional scene; and displaying the corresponding first display image on each screen. The change parameter can be understood as the absolute change in the virtual camera corresponding to the screen in the view engine, and the camera parameter can be understood as the relative change in the virtual camera in the view engine (relative to the original parameter or the change in the parameter at the previous moment). The view engine is configured to generate image data corresponding to different perspectives of different virtual cameras in the same three-dimensional scene based on the camera parameter. The application on the vehicle-mounted system obtains the image data from the view engine and controls the display and update of the image. Since the image presented on each screen is equivalent to a planar image from the perspective of a different virtual camera in the same three-dimensional scene, the images presented by at least two screens are associated and may be adjacent in perspective or position. When a user swipes on the in-vehicle system's screen, the control event is converted into a parameter change for the virtual camera. The view engine then reacquires image data and controls the display, achieving multi-screen interaction through software logic. In this solution, as a finger swipes on any screen, the virtual camera rotates or moves, driving the views from different perspectives to change simultaneously. This allows the images displayed on multiple screens to follow the changes simultaneously, achieving multi-screen interaction and increasing the richness and diversity of interactions.

[0057] In some embodiments, based on Figure 1 above, the screen includes a background area and a front area, and the front area is used to display at least one functional control, as shown in Figure 3. Figure 3 is an optional step flow chart 2 of a multi-screen linkage method provided in an embodiment of the present application, and the multi-screen linkage method also includes S201 to S203.

[0058] S201: Generate a displacement parameter based on a sliding event on a front area of ​​a target screen; the target screen is a screen where at least one functional control is located.

[0059] In an embodiment of the present application, the screen includes a background area and a front area suspended on a two-dimensional image. The background area is the area of ​​the screen excluding the front area, and the front area is used to display at least one functional control; the functional control can be in the form of a card, which carries a reduced icon or text of a third-party application. The third-party applications include but are not limited to weather controls, music controls, map navigation controls, vehicle controls, etc. It can also be understood that the background area displays a two-dimensional image in a three-dimensional scene, and the front area is the area where multiple cards suspended on the two-dimensional image are located. The view engine also loads at least one functional control in the three-dimensional scene and suspends the functional control on the screen, for example, suspending the cards corresponding to multiple functional controls in the front area. In one embodiment, the cards corresponding to each functional control are also the entry interfaces of each third-party application, and the user can enter the application after clicking. A sliding event refers to a sliding operation performed by the user on any screen.

[0060] As shown in FIG4 , FIG4 is an exemplary schematic diagram of a screen view provided by an embodiment of the present application. FIG4 exemplarily shows cards corresponding to three functional controls (i.e., three applications), which are, from left to right, a map navigation control 41, a weather control 42, and a vehicle control 43, arranged sequentially in the front area in the form of square cards. Each card can be preset to display first information. The first information can be real-time information of the application or the information that the user is most interested in. The specific information can be set according to the actual scenario. Among them, a brief introduction to the map navigation control, the first information may include: arrival time 10:20, remaining time 56 minutes (min), remaining distance 2.3 kilometers (km), 400 meters into Zhongshan South Second Road; a brief introduction to the weather control, the first information may include: today's temperature 26°C, thunderstorm, Shanghai; a brief introduction to the vehicle control, the first information may include: mileage 598 km, driving time 23 hours (h), energy consumption 9.0 kilowatts per hour (kw / h).

[0061] In an embodiment of the present application, if a user calls a display button for a function control in an application on the vehicle system, the image data obtained by the application on the vehicle system in the view engine includes not only a three-dimensional image (as a background) but also a card corresponding to the function control (as a front view), and at least one function control is displayed on a certain screen (which can be a default screen, such as a home screen, or a screen operated when the user calls the display button, which is not limited in this embodiment of the present application). For ease of description, the screen where at least one function control is located is referred to as a target screen. The user slides on the front area of ​​the target screen to control the movement of the cards corresponding to multiple function controls. The application on the vehicle system responds to the sliding event and generates displacement parameters based on the sliding event, that is, converts the sliding event into displacement parameters.

[0062] S202. Based on the displacement parameters, obtain a second display image of the target screen from the view engine; the view engine is used to update the position of at least one functional control in the front area based on the displacement parameters to generate the second display image of the target screen.

[0063] In an embodiment of the present application, the application on the vehicle system transmits the displacement parameters to the view engine, the view engine updates the position of the function controls in the front area of ​​the target screen according to the displacement parameters, and regenerates a new two-dimensional image of the target screen (including the background and function controls), and transmits the new two-dimensional image to the application on the vehicle system, which displays the new two-dimensional image (i.e., the second display image) on the target screen.

[0064] It's important to note that the secondary display image is the entire screen (including the background and functional controls), meaning both the background and the cards. Although only the position of the cards in the front area is changed here, the Unity engine outputs all data. When the user swipes the floating card, the 3D background does not follow the swipe, but the card does.

[0065] S203: Display the second display image on the target screen.

[0066] In this embodiment of the present application, when a user swipes on the front area of ​​a target screen in a vehicle-mounted system, the vehicle-mounted system application converts the swipe event into a displacement parameter and transmits the displacement parameter to the view engine. The view engine then adjusts the position of at least one functional control in the front area based on the displacement parameter and re-outputs the image. The vehicle-mounted system application displays the image output by the view engine and controls the display of the two-dimensional image, thereby achieving a multi-screen linkage effect through software logic.

[0067] In an embodiment of the present application, an application on the vehicle system converts a sliding event acting on the front area of ​​the target screen into a displacement parameter; based on the displacement parameter, image data is obtained from the view engine; the image data represents image data in which the viewing angles corresponding to at least two virtual cameras do not change and the functional controls move; based on the image data, at least one functional control is updated and displayed on the front area of ​​the target screen, so that the at least one functional control displayed on the front area of ​​the target screen moves in the direction corresponding to the sliding event, and the two-dimensional image displayed on the background area of ​​the target screen does not follow the movement.

[0068] In an embodiment of the present application, the application on the vehicle system responds to a sliding event acting on the front area of ​​the target screen, and at least one function control is switched for display in the front area of ​​the target screen along the direction corresponding to the sliding event (which may be toward the left or right of the screen). The two-dimensional image in the background area of ​​the target screen does not move with the sliding event. By switching the function controls, an application scenario is achieved in which the background does not switch when the user slides a floating function control (in the form of a card display) on the same screen, thereby improving the richness and diversity of the interaction.

[0069] In some embodiments, the front area includes a first area and a second area; the multi-screen linkage method also includes the following steps: calling the first information and the second information of the function control; displaying the first information and the second information of the function control located in the first area in the first area; and displaying the first information of the function control located in the second area in the second area.

[0070] In an embodiment of the present application, the view engine is used to load functional controls related to third-party applications. Functional controls refer to controls used to implement specific functions. Functional controls may be presented in forms including, but not limited to, icons and cards. The first information for a third-party application may be real-time information from the third-party application, and the second information may include historical information and / or forecast information and / or user-defined information from the third-party application. For example, using a weather control, the first information may include the current day's real-time weather conditions; the second information may include yesterday's and tomorrow's weather conditions, or the weather conditions for user-defined days.

[0071] The first information and second information of the function control are called through interaction between the application on the vehicle system and the third-party application. The application on the vehicle system displays the first information of the corresponding function control and the second information of the function control in the first area; and displays the first information of the corresponding function control in the second area. The first area is the area located in the middle of the target screen. For example, three function controls are displayed on the front area. The function control in the middle displays the first information displayed on the card directly above and the second information below the card. The function controls on both sides display the first information through the card, and do not display the second information below the card.

[0072] In one embodiment, the functional controls corresponding to the first area and the second area can be obtained first, and then the first information and second information corresponding to the functional controls in the first area can be retrieved through the application on the vehicle system, and the first information corresponding to the functional controls in the second area can be retrieved. This can reduce the amount of data transmission and improve processing efficiency.

[0073] For example, as shown in Figure 4, the first area is located in the middle of the front area. Located in the first area is a weather control 42, which displays weather conditions to the user. The card displays the first information, namely the current real-time weather. The second information in the area below the card includes historical weather and future weather, exemplified by the following: Monday is 26°C, Tuesday is predicted to be 23°C, Wednesday is predicted to be 23°C, Thursday is predicted to be 21°C, and Friday is predicted to be 20°C. The second area is located on both sides of the front area, respectively containing the map navigation control and the vehicle control. These two functional controls display the first information through cards. The map navigation control card simply displays the arrival time, remaining time, remaining mileage, and distance to the next road of the current trip. The vehicle control displays vehicle data to the user, including the current mileage, driving speed, and fuel consumption.

[0074] For example, if the map navigation control is located in the first area, in addition to the first information on the card (arrival time, remaining time, remaining mileage, distance to the next road, etc. of the current trip), the second information below the card can be a virtual rendering of the navigation route. In this way, the user can see the navigation route without entering the application, which greatly improves the user experience.

[0075] In an embodiment of the present application, information of different levels of detail is displayed through different areas. When the user slides a function control in the first area, in addition to the first information displayed on the card, the second information is also displayed below the card, including historical information and / or forecast information and / or custom information, etc., so that the user can understand the function control that he is interested in or currently needs. The user can view more information about the corresponding third-party application by sliding to switch the function control, thereby improving the diversity and richness of the interactive forms.

[0076] In some embodiments, before S101 in Figure 1, the multi-screen linkage method also includes the following steps: obtaining the initial display image corresponding to each screen from the view engine; the view engine is used to generate the initial display image corresponding to each screen based on the initial camera parameters corresponding to each screen and the same three-dimensional scene; and displaying the corresponding initial display image on each screen.

[0077] In this embodiment of the present application, after the view engine creates the 3D scene, it generates initial display images for each screen based on the initial camera parameters corresponding to each screen and the same 3D scene. The application on the vehicle system then retrieves the initial display images for each screen from the view engine and displays them. In other words, the view engine outputs initial image data corresponding to the perspectives of at least two virtual cameras within the same 3D scene. The application on the vehicle system then displays the respective 2D images on the display areas of the respective screens based on the initial image data.

[0078] For example, in the case of two screens, the view engine sends data from two channels to the application on the vehicle system for display on both screens. The initial display image represents the data from the two channels output by the view engine at the initial stage.

[0079] In the embodiment of the present application, taking the case where multiple screens include a main screen and a secondary screen, and the view engine is the Unity engine as an example, if the Unity engine is used to generate data and display the data on each screen, it is easy for operations on the main screen to cover the content displayed on the secondary screen. The embodiment of the present application adopts the method of using the Unity engine to generate data and apply application control display, so that the operation content on the main screen does not affect the display on the secondary screen, and can realize independent operation of multiple screens. The operation of the function control on the main screen has no effect on the secondary screen. The effect of multi-screen linkage is achieved through software logic, which improves the richness and diversity of interaction.

[0080] In some embodiments, the screen includes a background area and a front area, the front area is used to display at least one functional control; the operation event is an operation event for the background area of ​​any screen, and at least one functional control does not move or scale with the operation event.

[0081] In an embodiment of the present application, the screen includes a background area and a front area suspended on a two-dimensional image, and the front area is used to display at least one functional control. The target screen is the screen where at least one functional control is located. For the target screen, since the target screen also displays the functional control, the operation event on the target screen is an operation event on the background area of ​​the target screen. S101 to S103 in Figure 1 above can be implemented in the following manner: based on the operation event on the background area of ​​the target screen, a change parameter is generated; based on the change parameter, the first display image corresponding to each screen is obtained from the view engine; and the corresponding first display image is displayed on each screen. That is, the application on the vehicle system responds to the operation event acting on the background area of ​​the target screen, and the corresponding perspectives of each window move simultaneously along the direction corresponding to the operation event, so that the two-dimensional image moving in the direction is displayed on each screen, and the functional control does not move with the operation event.

[0082] In an embodiment of the present application, by setting a background area and a front area, when a sliding event acts on the front area, the two-dimensional image in the background area does not move with the sliding event; when an operation event acts on the background area, the functional controls in the front area do not move with the operation event; this realizes an application scenario in which the user slides the background area excluding the functional controls (display form of cards) in any screen, and the backgrounds displayed in the two screens follow the sliding display, thereby improving the richness and diversity of the interaction.

[0083] Most in-vehicle multi-screen display solutions employ a multi-window mode, with different screens displaying different windows. Multi-window linkage (also known as multi-screen linkage) requires setting a view background for each window, which is a complex solution. Furthermore, directly displaying data on each screen can easily cause operations on the primary screen to overwrite content displayed on the secondary screen.

[0084] An embodiment of the present application provides a multi-screen linkage method, as shown in FIG5 , which is a flowchart of optional steps of a multi-screen linkage method provided in an embodiment of the present application. The multi-screen linkage method is applied to an application on a vehicle system. The vehicle system includes at least two screens, the screen includes a background area and a front area, the front area is used to display at least one functional control, the front area includes a first area, and the at least one functional control includes a vehicle control. The multi-screen linkage method includes the following steps:

[0085] S301: In response to the vehicle control being slid to the first area, generate calling parameters of the vehicle model.

[0086] In the embodiment of the present application, the first area is located in the middle of the front area. Functional controls refer to controls used to implement specific functions. Functional controls may be expressed in forms including, but not limited to, icons and cards. Functional controls may include, but are not limited to, vehicle controls. When a user slides a card corresponding to a vehicle control into the first area, the application on the vehicle system responds to the event and converts it into a call parameter. The call parameter is used to call the vehicle model. The vehicle model is a special effect rendered by the view engine. The vehicle model can be considered as part of the three-dimensional scene.

[0087] S302: Based on the calling parameters, obtain a third display image from the view engine.

[0088] In an embodiment of the present application, the view engine is used to update the position of the vehicle model based on the calling parameters, and generate a dynamic position change trajectory that appears from an adjacent screen and drives into a target screen, and finally stops in the first area; the target screen is the screen where the at least one functional control is located.

[0089] S303. Display the third display image, wherein, in the third display image, the vehicle model appears from the adjacent screen, drives into the target screen, and finally stops in the first area; the target screen is the screen where the at least one functional control is located.

[0090] In the embodiment of the present application, the third display image is one or more display images. The third display image can be used to display the trajectory of the vehicle model across at least two screens until it stops and appears below the card where the vehicle control is located. It should be noted that in this case, the second information corresponding to the vehicle control is the vehicle model that stops and appears below the card where the vehicle control is located.

[0091] In an embodiment of the present application, the application on the vehicle system transmits the calling parameters to the view engine, the view engine updates the position of the vehicle model according to the calling parameters, and generates a dynamic position change trajectory, generates a third display image common to the target screen and the adjacent screen, and transmits the third display image to the application on the vehicle system, which displays the third display image on the two screens. The overall visual effect presented is: when the user slides the card corresponding to the vehicle control to the first area, the vehicle model appears from the bottom of the co-pilot screen, drives toward the main driver screen, passes through the junction of the two screens, drives into the main driver screen, and finally stops directly below the card corresponding to the vehicle control.

[0092] Exemplarily, the following is a description of the process in which the vehicle model passes through the secondary screen and the main screen and is finally displayed on the main screen (corresponding to the target screen) in the order in which the vehicle model appears, through Figures 6A, 6B and 6C. In this embodiment, the main screen corresponds to the target screen and the secondary screen corresponds to the adjacent screen. As shown in Figure 6A, Figure 6A is an exemplary schematic diagram four of a screen view provided by an embodiment of the present application. In Figure 6A, the vehicle model 601 appears in the secondary screen 602. As shown in Figure 6B, Figure 6B is an exemplary schematic diagram five of a screen view provided by an embodiment of the present application. As shown in Figure 6C, Figure 6C is an exemplary schematic diagram six of a screen view provided by an embodiment of the present application. Figures 6B and 6C show that the vehicle model 601 passes through the secondary screen 602 and the main screen 603. After the vehicle model passes through the secondary screen 602 and the main screen 603, it is finally displayed on the main screen 603. It should be noted that the front area in Figure 6C should also have cards corresponding to the various functional controls as shown in Figure 4. The middle card should be the vehicle control. The vehicle model travels below each card and eventually stops below the vehicle control card. The cards corresponding to the various functional controls are not drawn in Figure 6C.

[0093] In this embodiment of the present application, the user slides the card corresponding to the vehicle control to the first area in the middle of the main driving screen to call up the vehicle model. The vehicle model appears in a manner that runs through the main and secondary screens, increasing the richness and diversity of the interaction. In this embodiment of the present application, the main screen can also be called the main driving screen, and the secondary screen can also be called the secondary driving screen.

[0094] In an embodiment of the present application, the corresponding perspectives of each screen move continuously along the direction corresponding to the continuous sliding event at the same time, so that each screen displays a two-dimensional image that continues to move in the direction until the vehicle model passes through two adjacent screens and the vehicle model 601 is displayed in the screen where the vehicle model is last displayed (for example, the main screen 603 in Figure 6C).

[0095] In some embodiments, the screen further includes an instrument screen. If the instrument screen is the target screen, the adjacent screen is the main driver's screen, or the main driver's screen and the passenger driver's screen. Where the target screen is the instrument screen and the adjacent screen is the main driver's screen, when the user slides the card corresponding to the vehicle control to the first area of ​​the front area of ​​the target screen, the vehicle model appears from the bottom of the main driver's screen, drives toward the instrument screen, passes through the junction between the main driver's screen and the instrument screen, enters the instrument screen, and finally stops directly below the card corresponding to the vehicle control. Where the target screen is the instrument screen and the adjacent screens are the main driver's screen and the passenger driver's screen, when the user slides the card corresponding to the vehicle control to the first area of ​​the front area of ​​the target screen, the vehicle model appears from the bottom of the passenger driver's screen, drives toward the instrument screen, passes through the junction between the passenger driver's screen and the main driver's screen, the main driver's screen, and the junction between the main driver's screen and the instrument screen, until it enters the instrument screen and finally stops directly below the card corresponding to the vehicle control.

[0096] Taking multiple screens including a main screen and a secondary screen as an example, if the Unity engine is used to generate data and the data is displayed directly on each screen, it is easy for operations on the main screen to overwrite the content displayed on the secondary screen. For example, opening the vehicle model under the car control on the main screen will force the music being played on the secondary screen to be displayed as a vehicle model. The embodiment of the present application uses the Unity engine to generate data and the Android operating system to control the display, so that the operation content on the main screen does not affect the display on the secondary screen, and independent operation of multiple screens is achieved. For example, the operation of the function control on the main screen has no effect on the secondary screen.

[0097] In some embodiments, the vehicle model corresponding to the vehicle system is relatively stationary with respect to the three-dimensional scene, as shown in FIG8 . FIG8 is an optional step flow chart four of a multi-screen linkage method provided in an embodiment of the present application. The multi-screen linkage method also includes S401 to S402 .

[0098] S401: In response to a first click event on the vehicle model or vehicle control in the first area, load multiple function buttons corresponding to multiple virtual components of the vehicle model and generate a fourth display image of the target screen. In one embodiment, step S401 is performed after step S303. In this embodiment of the present application, the first click event is an application on the vehicle computer responding to a click event on the vehicle model or vehicle control in the first area.

[0099] S402: Display a fourth display image on the target screen; each of the function buttons is suspended on the vehicle model.

[0100] For example, as shown in Figure 7, Figure 7 is an exemplary schematic diagram three of a screen view provided in an embodiment of the present application, showing the fourth display image of calling the vehicle model and displaying it in full screen after clicking the vehicle model or vehicle control, with the vehicle model 701 displayed in the middle of the target screen and the vehicle virtual component 702 displayed on the left.

[0101] In the embodiment of the present application, the multiple virtual components include: a left door, a right door, a trunk, a sunroof, a left window and a right window.

[0102] In an embodiment of the present application, at least one function control is displayed on the front area of ​​the target screen; a click event acting on a vehicle control or a vehicle model is converted into a call parameter of the vehicle model; image data is obtained from the view engine based on the call parameter; based on the image data, the vehicle model is displayed in full screen, and function buttons of the vehicle virtual components are displayed, and the function buttons are suspended on the various components of the vehicle model (not shown in FIG4 ). For example, a circular function button is suspended on the trunk of the vehicle model, and the user clicks the function button to open and close the trunk. The shape and style of the function button can be set by the technician to trigger the click event of the corresponding component of the vehicle model.

[0103] In an embodiment of the present application, the user calls the vehicle model by continuously sliding the two-dimensional image, and the camera perspective changes. The vehicle model appears in a manner that runs through the main and sub-screens, allowing the user to slide the background area excluding the function card on any screen, so that the vehicle model runs through the application scenarios of the main screen and sub-screen, thereby improving the richness and diversity of the interaction.

[0104] In an embodiment of the present application, in the scheme shown in Figure 5 above, the user slides the vehicle control to the middle of the main screen, calling up the vehicle model to travel from the secondary screen to the main screen; in the scheme shown in Figure 8 above, the user clicks on the vehicle model or vehicle control, the vehicle model follows the background, and the vehicle model is locked and enlarged by rotating the perspective until the vehicle model is displayed in full screen on the main screen, entering the vehicle control mode, and the user can flexibly control various vehicle components through function buttons.

[0105] In some embodiments, based on Figures 5 to 8 above, the interaction between the application on the vehicle system and the vehicle body controller is explained, and the multi-screen linkage method also includes the following steps: based on the second click event for the target function button, generating a control signal and simulation parameters; the target function button is any function button; sending the control signal to the vehicle body controller, so that the vehicle body controller controls the vehicle physical component corresponding to the target function button in the vehicle to perform a first switching action based on the control signal; the second click event is a click operation on the target function button; the first switching action represents the conversion of the vehicle physical component from a first state to a state opposite to the first state; based on the simulation parameters, obtaining the fifth display image of the target screen from the view engine; the view engine is used to update the first simulation action of the vehicle virtual component corresponding to the target function button in the vehicle model based on the simulation parameters, and generate the fifth display image of the target screen, the first simulation action representing the conversion of the vehicle virtual component from a first state to a state opposite to the first state; the fifth display image is displayed on the target screen.

[0106] In an embodiment of the present application, the view engine loads function buttons (which can be in the form of icons or buttons) related to certain functions in the vehicle controller. The specific implementation process and information of the function buttons require interaction between the vehicle-mounted system application and the vehicle controller. The vehicle controller is the actual vehicle controller, responsible for controlling the opening and closing of doors, windows, and trunk, among other functions. The vehicle-mounted system application converts the second click operation event for the target function button into a control signal and simulation parameters, and sends the control signal to the vehicle controller. Based on the control signal, the vehicle controller controls the physical vehicle component corresponding to the target function button to perform the first switching action. The vehicle-mounted system application sends the simulation parameters to the view engine. Based on the simulation parameters, the view engine updates the state of the target function button on the target screen and the simulated actions of the virtual vehicle components corresponding to the target function button in the vehicle model. It also regenerates a new two-dimensional image of the target screen (including the background, vehicle model, and function buttons) and transmits this new two-dimensional image to the vehicle-mounted system application, which then displays the new two-dimensional image (i.e., the fifth display image) on the target screen. The fifth display image is an image in which the vehicle virtual components of the vehicle model on the screen are changed, but the background does not change.

[0107] For example, assuming the target function button is the left front door button, and the corresponding vehicle physical component is the left front door, the control signal and simulation parameters represent clicking the left front door button (assuming the first state is closed), causing the vehicle controller to control the left front door in the vehicle to open. The view engine outputs image data of the left front door button and the left front door model in the vehicle model (i.e., the virtual vehicle component) switching from the closed state to the open state.

[0108] In an embodiment of the present application, in response to a second click event on a target function button, the vehicle physical component corresponding to the target function button in the vehicle is controlled to perform a first switching action, and the vehicle virtual component corresponding to the target function button in the vehicle model is controlled to perform a first simulation action, thereby realizing an application scenario in which the user clicks on the function button corresponding to the vehicle model (opening and closing doors, opening and closing windows, opening and closing trunks, etc.), and the vehicle model and the vehicle follow the action, thereby improving the richness and diversity of the interaction.

[0109] In some embodiments, based on Figures 5 to 8 above, the interaction between the application on the vehicle system and the vehicle body controller is explained, and the multi-screen linkage method also includes the following steps: when the target vehicle physical component undergoes a substantial switching action, receiving the analog component signal sent by the vehicle body controller; the target vehicle physical component is any vehicle physical component; based on the analog component signal, obtaining the sixth display image of the target screen from the view engine; the view engine is used to update the function button corresponding to the target vehicle physical component and the corresponding second analog action of the vehicle virtual component in the vehicle model based on the analog component signal, and generate the sixth display image of the target screen, the second analog action representing the conversion of the vehicle virtual component from the second state to the state opposite to the second state; and displaying the sixth display image on the target screen.

[0110] In an embodiment of the present application, when a physical component of the target vehicle undergoes an actual switching action, the vehicle body controller generates an analog component signal, sends the analog component signal to the application on the vehicle system, and the application on the vehicle system sends the analog component signal to the view engine. Based on the analog component signal, the view engine updates the status of the function buttons corresponding to the physical component of the target vehicle in the target screen, and the simulated action of the vehicle virtual component corresponding to the physical component of the target vehicle in the vehicle model, and regenerates a new two-dimensional image of the target screen (including background, vehicle model and function buttons), transmits the new two-dimensional image to the application on the vehicle system, and the application on the vehicle system displays the new two-dimensional image (i.e., the sixth display image) on the target screen. The sixth display image is an image in which the function buttons and the vehicle virtual components of the vehicle model on the screen are changed, but the background does not change.

[0111] For example, taking the target vehicle physical component as the trunk and the second state as closed, the corresponding vehicle virtual component is the trunk, the simulated component signal represents the user opening the trunk, and the view engine outputs the trunk button and the trunk model in the vehicle model (i.e., the vehicle virtual component) image data switching from the closed state to the open state.

[0112] In an embodiment of the present application, when the target vehicle physical component undergoes an actual switching action, the function button corresponding to the target vehicle physical component in the control vehicle and the vehicle virtual component corresponding to the target vehicle physical component in the vehicle model both perform a second simulation action, thereby realizing the action of the vehicle (actual opening and closing of doors, windows, trunk, etc.), and the vehicle model corresponds to the application scenario of the same simulation operation, thereby improving the richness and diversity of the interaction.

[0113] In some embodiments, the screen includes a background area and a front area, and the front area is used to display at least one function control. Based on Figures 1 to 4 above, the multi-screen linkage method also includes the following steps: based on a third click event for the target function control, generating a first shooting depth parameter; the target function control is any function control; based on the first shooting depth parameter, obtaining the seventh display image corresponding to each screen from the view engine; the view engine is used to update the first camera depth parameter corresponding to each screen based on the first shooting depth parameter, and generate the seventh display image corresponding to each screen based on the first camera depth parameter corresponding to each screen and the same three-dimensional scene; displaying the corresponding seventh display image in each screen until the target function control is enlarged to a size corresponding to the target screen, and the target screen is the screen where at least one function control is located.

[0114] In an embodiment of the present application, a user clicks on a target function control, and the application on the vehicle system responds to a third click event and converts the third click event into a first shooting depth parameter. The third click event is a click operation on the target function control itself, and the first shooting depth parameter is used to change the perspective of each virtual camera from far to near. The application on the vehicle system transmits the first shooting depth parameter to the view engine, which updates the first camera depth parameters corresponding to each screen based on the first shooting depth parameter, that is, the perspective of each virtual camera changes from far to near, and generates a new two-dimensional image of each screen based on the first camera depth parameters corresponding to each screen and the same three-dimensional scene (including the function control and background on the target screen that gradually increases or decreases from far to near, and the background on other screens that gradually increases). The new two-dimensional image is transmitted to the application on the vehicle system, and the application on the vehicle system displays the new two-dimensional image (i.e., the seventh display image) on each screen. The seventh display image is an image in which the background and function control on the target screen change from far to near, and the background on other screens changes from far to near, which is equivalent to a zoomed-in image.

[0115] It should be noted that since the virtual cameras corresponding to each screen are associated and can be enlarged or reduced at the same time, when the user clicks on the target function control and wants to enlarge the target function control, the background will grow along with the target function control. Therefore, although the user operates the screen where the target function control is located (ie, the target screen), the view engine needs to output two-dimensional images of each screen.

[0116] In an embodiment of the present application, in response to a third click event on a target function control on a target screen, the target function control gradually enlarges, and the two-dimensional image displayed on the target screen enlarges along with the target function control. Correspondingly, the two-dimensional images displayed on other screens also enlarge accordingly, until the target function control is enlarged to the size corresponding to the target screen (which can also be understood as filling the entire screen). The enlarged target function control is displayed on the target screen, and the enlarged two-dimensional images are displayed on the other screens. This implements an application scenario in which a user clicks any function control and the background enlarges accordingly until the function control is displayed in the entire screen area, thereby improving the richness and diversity of interaction.

[0117] In some embodiments, after the target function control is enlarged to the size corresponding to the target screen, the multi-screen linkage method also includes the following steps: generating a second shooting depth parameter based on a fourth click event for the target screen; obtaining an eighth display image corresponding to each screen from the view engine based on the second shooting depth parameter; the view engine is used to update the second camera depth parameter corresponding to each screen based on the second shooting depth parameter, and generate an eighth display image corresponding to each screen based on the second camera depth parameter corresponding to each screen and the same three-dimensional scene; displaying the corresponding eighth display image on each screen until the target function control is reduced to the size corresponding to the front area of ​​the target screen.

[0118] In an embodiment of the present application, the user clicks on the target function control, and the application on the vehicle system responds to the fourth click event and converts the fourth click event into a second shooting depth parameter. The second shooting depth parameter is used to change the perspective of each virtual camera from near to far. The application on the vehicle system transmits the second shooting depth parameter to the view engine, and the view engine updates the second camera depth parameter corresponding to each screen according to the second shooting depth parameter, that is, the perspective of each virtual camera changes from near to far, and generates a new two-dimensional image of each screen based on the second camera depth parameter corresponding to each screen and the same three-dimensional scene (including the function control and background of the target screen that gradually becomes smaller or from near to far, and the background of other screens that gradually becomes smaller), and transmits the new two-dimensional image to the application on the vehicle system, and the application on the vehicle system displays the new two-dimensional image (i.e., the eighth display image) on each screen. The eighth display image is an image in which the background and function control of the target screen change from near to far, and the background of other screens changes from near to far, which is equivalent to a distant perspective.

[0119] It should be noted that the fourth click event is a click operation on the target functional control itself, or a restore operation on the enlarged target functional control (for example, clicking a button representing a restore operation).

[0120] In an embodiment of the present application, in response to a fourth click event on a target function control on a target screen, the target function control gradually becomes smaller, and the two-dimensional image displayed on the target screen becomes smaller along with the target function control. Correspondingly, the two-dimensional images displayed on other screens also become smaller together, until the target function control is reduced to the size corresponding to the front area (which can also be understood as restoring to the initial display state in the front area); the restored target function control is displayed on the target screen, and the restored two-dimensional images are displayed on other screens. This implements an application scenario in which when a user clicks on an enlarged function control, the perspective of the virtual camera corresponding to the target screen (from near to far) becomes smaller along with the target function control until the function control is displayed in the front area of ​​the target screen, accompanied by the display of two-dimensional images from near to far, thereby improving the richness and diversity of interaction.

[0121] In some embodiments, the screen includes a background area and a front area, and the front area is used to display at least two function controls; the at least two function controls include a first function control and a last function control; based on Figures 1 to 4 above, the multi-screen linkage method also includes the following steps: based on a second continuous sliding event for the front area of ​​the target screen, generating a second continuous displacement parameter; the target screen is the screen where the at least two function controls are located; based on the second continuous displacement parameter, obtaining a ninth display image of the target screen from the view engine; the view engine is used to update the positions of the at least two function controls based on the second continuous displacement parameter, and generate a ninth display image of the target screen; based on the ninth display image, continuously displaying at least two function controls in the front area of ​​the target screen until reaching the first function control or the last function control.

[0122] In an embodiment of the present application, the user continuously slides the front area of ​​the target screen, and the application on the vehicle system responds to the second continuous sliding event and converts the second continuous sliding event into a second continuous displacement parameter. The application on the vehicle system transmits the second continuously changing parameter to the view engine, and the view engine updates the position of the function control in the front area of ​​the target screen according to the second continuously changing parameter, and regenerates a new two-dimensional image (including the background of the target screen and the function control) in the same three-dimensional scene, and transmits the new two-dimensional image to the application on the vehicle system, which continuously displays the new two-dimensional image (i.e., the ninth display image) on the front area of ​​the target screen. At least two function controls displayed on the front area of ​​the target screen continue to move in the direction corresponding to the second continuous sliding event until they reach the first function control or the last function control. In this scenario, the background area of ​​the target screen does not change, that is, the background does not move in the direction corresponding to the second continuous sliding event.

[0123] In the embodiment of the present application, the ninth display image is an image in which the background of each screen remains unchanged and the function controls continuously change in one direction. Multiple function controls have a head and a tail, and the head or the tail can be found by continuously sliding.

[0124] In an embodiment of the present application, in response to a continuous second continuous sliding event acting on the front area of ​​the target screen, at least two function controls are switched to be displayed in sequence in the front area until the first function control or the last function control is reached; continuing to respond to a continuous operation in the opposite direction of the second continuous sliding event acting on the front area of ​​the target screen, at least two function controls can be switched to be displayed in sequence in the front area until the last function control or the first function control is reached, thereby realizing an application scenario in which the user continuously slides the function controls without changing the camera perspective or the background, and the function controls are displayed in turn, thereby improving the richness and diversity of the ways in which vehicle models appear.

[0125] In some embodiments, the screen includes a background area and a front area, and the front area is used to display at least two function controls; the at least two function controls include function controls connected end to end; based on the above Figures 1 to 4, the multi-screen linkage method also includes the following steps: based on the third continuous sliding event of the front area of ​​the target screen, generating a third continuous displacement parameter; the target screen is the screen where the at least two function controls are located; based on the third continuous displacement parameter, obtaining the tenth display image of the target screen from the view engine; the view engine is used to update the positions of the at least two function controls based on the third continuous displacement parameter, and generate the tenth display image of the target screen; based on the tenth display image, cyclically displaying at least two function controls in the front area of ​​the target screen.

[0126] In an embodiment of the present application, the user continuously slides the front area of ​​the target screen, and the application on the vehicle system responds to the third continuous sliding event and converts the third continuous sliding event into a third continuous displacement parameter. The application on the vehicle system transmits the third continuously changing parameter to the view engine, and the view engine updates the position of the function control in the front area of ​​the target screen according to the third continuously changing parameter, and regenerates a new two-dimensional image (including the background of the target screen and the function control) in the same three-dimensional scene, and transmits the new two-dimensional image to the application on the vehicle system, which continuously displays the new two-dimensional image (i.e., the tenth display image) on the front area of ​​the target screen. At least two function controls displayed on the front area of ​​the target screen continue to move in a circular motion in the direction corresponding to the second continuous sliding event. In this scenario, the background area of ​​the target screen does not change, that is, the background does not move in the direction corresponding to the third continuous sliding event.

[0127] In an embodiment of the present application, the tenth display image is that the background of each screen remains unchanged, the function controls continuously change in one direction, multiple function cards are connected end to end, and continuous sliding can be used for cyclic display.

[0128] In an embodiment of the present application, in response to a third continuous sliding event acting on the front area of ​​the target screen, at least two function controls are displayed in a cyclic switching manner in the front area; in response to a continuous operation in the opposite direction of the third continuous sliding event acting on the front area of ​​the target screen, at least two function controls can be displayed in the front area again in a cyclic switching manner, thereby realizing an application scenario in which the user continuously slides the function controls without changing the camera perspective or the background, and the function controls are displayed in a cyclic manner, thereby improving the richness and diversity of the ways in which vehicle models appear.

[0129] The present application also provides a multi-screen linkage method, as shown in FIG9 , which is a fifth optional step flow chart of a multi-screen linkage method provided in the present application embodiment. The multi-screen linkage method is applied to a view engine and includes the following steps:

[0130] S501. Receive a change parameter sent by an application on the vehicle system; the change parameter is generated in response to an operation event on any one of at least two screens corresponding to the vehicle system.

[0131] S502: Update the camera parameters corresponding to each screen based on the changed parameters.

[0132] S503 : Generate a first display image corresponding to each screen based on camera parameters corresponding to each screen and the same three-dimensional scene.

[0133] S504 , sending data feedback carrying the first display image corresponding to each screen to the application on the vehicle system; the data feedback is used to instruct the application on the vehicle system to display the corresponding first display image on each screen.

[0134] In the embodiment of the present application, the change parameters are generated by the application on the vehicle system in response to the operation event. The view engine performs the following steps: according to the arranged virtual cameras (Camera), the view of the area to be displayed is set, and the integrated data of the output channels of each camera is generated. During the human-computer interaction process, the application on the vehicle system responds to the user's operation event and sends the change parameters to the view engine; the view engine updates the camera parameters of each camera based on the change parameters, and based on the updated camera parameters and the integrated data of the three-dimensional scene output channel, feedback is given to the application on the vehicle system, so that the application on the vehicle system displays the corresponding first display image on each screen.

[0135] In an embodiment of the present application, the vehicle system performs the following steps: (1) obtaining the integrated data of each camera output channel, displaying the integrated data on the screen based on each camera output channel, and controlling each screen to independently display its own content.

[0136] In an embodiment of the present application, the view engine adjusts the camera parameters based on the change parameters sent by the vehicle system, generates image data of the perspectives corresponding to different virtual cameras in the same three-dimensional scene based on the camera parameters, and sends it to the application on the vehicle system, so that the application on the vehicle system controls the display and update of the image. Since the image presented on each screen of the vehicle system is a planar image generated by the view engine under the perspectives of different virtual cameras in the same three-dimensional scene. When the user slides on the screen of the vehicle system, the operation event is converted into the change parameters of the virtual camera, the view engine re-outputs the image data, and the display is controlled by the application on the vehicle system, thereby achieving the effect of multi-screen linkage through software logic. In this solution, as the finger slides on any screen, the virtual camera rotates or moves, driving the views under different perspectives to change together, so that the images displayed on multiple screens follow the changes at the same time, thereby realizing multi-screen linkage and improving the richness and diversity of interaction.

[0137] In some embodiments, before S501 of Figure 9 above, the multi-screen linkage method also includes the following steps: rendering dynamic elements, vehicle models and vehicle virtual components to generate a three-dimensional scene; the vehicle virtual components include at least one of the following: left door, right door, trunk, sunroof, left window and right window; based on the shooting position, shooting angle and shooting depth, creating at least two virtual cameras in the three-dimensional scene; each virtual camera is used to generate a display image of the screen corresponding to the virtual camera based on the camera parameters.

[0138] For example, the Unity engine is used as the view engine. The Unity engine performs the following steps: (1) Rendering a 3D scene, which is equivalent to a video. Dynamic elements are created in the 3D scene, such as dynamic water flow, dynamic birds, etc. Rendering a vehicle model and virtual vehicle components. (2) Based on parameters such as shooting position, shooting angle, shooting depth (and shooting distance), multiple virtual cameras are arranged in the 3D scene. Each virtual camera corresponds to a different perspective, and each perspective is related to each other. (3) The view of the area to be displayed is set according to the arranged cameras, and the integrated data of each camera output channel is generated. It can also be understood that the field of view of each virtual camera is configured to generate configuration parameters; the configuration parameters are used to generate the view corresponding to each field of view during operation and are sent to each screen through each output channel.

[0139] In an embodiment of the present application, in the initial scenario, the view engine outputs data from two channels for display on two screens by the application on the vehicle system. Each virtual camera in the view engine corresponds to a different perspective, and each perspective is interconnected. When a user operates any screen, the view engine configures the field of view of each virtual camera based on camera parameters, determining the image corresponding to the perspective of each virtual camera in the same three-dimensional scene. Data feedback carrying the first display image corresponding to each screen is sent to the application on the vehicle system, and the application on the vehicle system updates the display of the first display image on each screen. This allows the two-dimensional images displayed on each screen to move in the direction corresponding to the operation event, achieving multi-screen linkage and increasing the richness and diversity of interaction.

[0140] In some embodiments, dynamic elements include elements that change over a timeline.

[0141] In an embodiment of the present application, certain elements rendered by the view engine can change along the timeline, such as displaying the sun during the day and the moon and starry sky at night, which increases the richness and interest of the scene.

[0142] In some embodiments, the multi-screen linkage method further includes the following steps: loading at least one functional control in the three-dimensional scene based on the shooting position, shooting angle and shooting depth.

[0143] For example, using Unity as the view engine, the Unity engine performs the following steps: For functional controls (third-party apps such as music, map navigation, vehicle controls, weather, etc.), based on parameters such as location, shooting angle, and shooting depth (shooting distance), its interface is loaded into the 3D scene and displayed in a floating manner above the 3D scene. Specific data is called by the application on the vehicle system. Correspondingly, the vehicle system performs the following steps: while the functional controls are also displayed on the screen, the data of the functional controls is obtained and displayed.

[0144] In some embodiments, the screen of the vehicle system includes a background area and a front area, and the front area is used to display at least one functional control; the multi-screen linkage method also includes the following steps: receiving a displacement parameter sent by an application on the vehicle system; the displacement parameter is generated by the application on the vehicle system in response to a sliding event on the front area of ​​a target screen, and the target screen is the screen where at least one functional control is located; based on the displacement parameter, the position of at least one functional control in the front area is updated to generate a second display image of the target screen; sending a first data feedback carrying the second display image of the target screen to the application on the vehicle system; the first data feedback is used to instruct the application on the vehicle system to display the second display image on the target screen.

[0145] In some embodiments, the front area includes a first area and a second area; the multi-screen linkage method also includes the following steps: generating an initial display image corresponding to each screen based on the initial camera parameters corresponding to each screen and the same three-dimensional scene; sending the initial display image corresponding to each screen to the application on the vehicle system, so that the application on the vehicle system displays the corresponding initial display image on each screen.

[0146] In an embodiment of the present application, each virtual camera in the view engine corresponds to a different perspective, and each perspective is interrelated; the vehicle system includes at least two screens, the screen includes a background area and a front area, the front area is used to display at least one functional control, the front area includes a first area, and the at least one functional control includes a vehicle control.

[0147] The view engine receives the calling parameters of the vehicle model sent by the application on the vehicle system; in an embodiment of the present application, the calling parameters are generated by the application on the vehicle system in response to the vehicle control being slid to the first area.

[0148] Based on the calling parameters, the view engine updates the position of the vehicle model and generates a dynamic position change trajectory, which appears from the adjacent screen and drives into the target screen, and finally stops under the card in the first area, to generate a third display image; in this embodiment of the present application, the view engine configures the field of view of each virtual camera based on the camera parameters, and determines the image of the perspective corresponding to each virtual camera in the same three-dimensional scene.

[0149] The view engine sends the third display image corresponding to each screen to the application on the vehicle system, so that the application on the vehicle system continues to display the corresponding third display image on each screen until the vehicle model passes through two adjacent screens, and then displays the vehicle model on the target screen.

[0150] In some embodiments, the vehicle model corresponding to the vehicle system is relatively stationary relative to the three-dimensional scene. The multi-screen linkage method further includes the following steps: receiving a first click event sent by an application on the vehicle system; the first click event is a response by the application on the vehicle system to a click on the vehicle model or vehicle control in the first area; updating a camera position corresponding to a target screen based on the first click event, loading multiple function buttons corresponding to multiple virtual components of the vehicle model, and generating a fourth display image of the target screen. Each function button is suspended above the vehicle model.

[0151] In some embodiments, the multi-screen linkage method also includes the following steps: receiving simulation parameters sent by an application on the vehicle system; the simulation parameters are generated by the application on the vehicle system in response to a second click event for a target function button, and the target function button is any function button; based on the simulation parameters, the first simulation action of the vehicle virtual component corresponding to the target function button in the vehicle model is updated to generate a fifth display image of the target screen, and the first simulation action represents the conversion of the vehicle virtual component from a first state to a state opposite to the first state; sending the fifth display image of the target screen to the application on the vehicle system, so that the application on the vehicle system displays the fifth display image on the target screen.

[0152] In some embodiments, the multi-screen linkage method also includes the following steps: receiving an analog component signal sent by an application on the vehicle system; the analog component signal is generated by the vehicle body controller when the target vehicle physical component undergoes a substantial switching action, and the vehicle body controller sends it to the application on the vehicle system, and the target vehicle physical component is any vehicle physical component; based on the analog component signal, updating the function button corresponding to the target vehicle physical component and the corresponding second analog action of the vehicle virtual component in the vehicle model, generating a sixth display image of the target screen, the second analog action representing the conversion of the vehicle virtual component from the second state to a state opposite to the second state; sending the sixth display image of the target screen to the application on the vehicle system, so that the application on the vehicle system displays the sixth display image on the target screen.

[0153] In some embodiments, the screen includes a background area and a front area, and the front area is used to display at least one function control. The multi-screen linkage method also includes the following steps: receiving a first shooting depth parameter sent by an application on the vehicle system; the first shooting depth parameter is generated by the application on the vehicle in response to a third click event for a target function control, and the target function control is any function control; updating the first camera depth parameter corresponding to each screen based on the first shooting depth parameter, and generating a seventh display image corresponding to each screen based on the first camera depth parameter corresponding to each screen and the same three-dimensional scene; sending the seventh display image corresponding to each screen to the application on the vehicle system, so that the application on the vehicle system displays the corresponding seventh display image on each screen until the target function control is enlarged to a size corresponding to the target screen, and the target screen is the screen where at least one function control is located.

[0154] In some embodiments, after the target function control is enlarged to the size corresponding to the target screen, the multi-screen linkage method also includes the following steps: receiving a second shooting depth parameter sent by an application on the vehicle system; the second shooting depth parameter is generated by the application on the vehicle in response to a fourth click event on the target screen; updating the second camera depth parameter corresponding to each screen based on the second shooting depth parameter, and generating an eighth display image corresponding to each screen based on the second camera depth parameter corresponding to each screen and the same three-dimensional scene; sending the eighth display image corresponding to each screen to the application on the vehicle system, so that the application on the vehicle system displays the corresponding eighth display image on each screen until the target function control is reduced to the size corresponding to the front area of ​​the target screen.

[0155] In some embodiments, the screen includes a background area and a front area, and the front area is used to display at least two function controls; the at least two function controls include a first function control and a last function control; the multi-screen linkage method also includes the following steps: receiving a second continuous displacement parameter sent by an application on the vehicle system; the second continuous displacement parameter is generated by the application on the vehicle in response to a second continuous sliding event for the front area of ​​the target screen, and the target screen is the screen where the at least two function controls are located; based on the second continuous displacement parameter, the positions of the at least two function controls are updated to generate a ninth display image of the target screen; the ninth display image of the target screen is sent to the application on the vehicle system, so that the application on the vehicle system continuously displays at least two function controls in the front area of ​​the target screen based on the ninth display image until it reaches the first function control or the last function control.

[0156] In some embodiments, the screen includes a background area and a front area, and the front area is used to display at least two function controls; the at least two function controls include function controls connected end to end; the multi-screen linkage method also includes the following steps: receiving a third continuous displacement parameter sent by an application on the vehicle system; the third continuous displacement parameter is generated by the application on the vehicle system in response to a third continuous sliding event in the front area of ​​the target screen, and the target screen is the screen where the at least two function controls are located; based on the third continuous displacement parameter, the positions of the at least two function controls are updated to generate a tenth display image of the target screen; the tenth display image of the target screen is sent to the application on the vehicle system, so that the application on the vehicle system cyclically displays at least two function controls in the front area of ​​the target screen based on the tenth display image.

[0157] It should be noted that the implementation method and technical effects achieved by the multi-screen linkage solution executed by the above-mentioned view engine side can be found in the above-mentioned description of the vehicle system side, and will not be described here again.

[0158] As used herein, the term "vehicle" or other similar terms encompasses a broad range of motor vehicles: for example, passenger cars including sport utility vehicles (SUVs), buses, trucks, various commercial vehicles; watercraft including various boats and ships; and aircraft; and includes hybrid vehicles, electric vehicles, hybrid electric vehicles, hydrogen-powered vehicles, and other alternative fuel vehicles (e.g., fuels derived from resources other than petroleum).

[0159] The following describes an exemplary application of the embodiments of the present application in a practical application scenario.

[0160] In an embodiment of the present application, as shown in Figure 10, Figure 10 is a timing diagram of the interaction between various devices in a multi-screen linkage provided by an embodiment of the present application; the view engine is illustrated as a Unity engine, and the interaction process between the Unity engine, the application on the vehicle system, the vehicle body controller, and the third-party application includes S1 to S37.

[0161] S1. Unity engine renders 3D scenes.

[0162] S2. The Unity engine arranges multiple cameras in a three-dimensional scene based on parameters such as position, shooting angle, and shooting depth.

[0163] For functional controls, S3 and Unity engines load their interfaces in the 3D scene based on parameters such as position, shooting angle, and shooting distance, and display them in a floating manner above the three-dimensional scene.

[0164] S4. The application on the vehicle system obtains the integrated data of each camera output channel from the Unity engine.

[0165] S5. The application on the vehicle system integrates data based on each camera output channel, displays it on the screen, and controls each screen to independently display its own content.

[0166] S6. The application on the vehicle system obtains function control data from the third-party application.

[0167] S7. The application on the vehicle system displays the first information of the function control according to the data of the function control.

[0168] S8. The user performs a sliding operation on the three-dimensional scene, and the application on the vehicle system converts the sliding event into camera rotation parameters (including camera displacement and rotation angle).

[0169] S9. The vehicle system should send the camera rotation parameters.

[0170] S10, Unity engine updates camera output channel integrated data for the first time.

[0171] S11. The Unity engine sends the first updated camera output channel integrated data to the application on the vehicle system.

[0172] S12. The application on the vehicle system is based on the integrated data control display of the camera output channel after the first update.

[0173] The above S8 to S12 realize the 3D scene following change of the main and sub screens. The above S8 to S12 correspond to S101 to S103 in Figure 1. The implementation process and technical effects achieved can be found in the description of Figure 1 above and will not be described here.

[0174] S13. The user performs a sliding operation on the function control, and the application on the vehicle system converts the sliding distance parameter in the sliding event into a displacement parameter.

[0175] It should be noted that the camera rotation parameters remain unchanged at this time, that is, the rotation angle and camera displacement remain unchanged.

[0176] S14. The application on the vehicle system sends the displacement parameter.

[0177] S15. The Unity engine updates the camera output channel integrated data for the second time.

[0178] S16. The Unity engine sends the camera output channel integrated data after the second update to the application on the vehicle system.

[0179] S17. The application on the vehicle system is based on the integrated data control display of the camera output channel after the second update.

[0180] The above S13 to S17 implement the sliding operation of the function control, and the three-dimensional (3D) scene does not change accordingly. The above S13 to S17 correspond to S201 to S203 in Figure 3. The implementation process and technical effects achieved can be found in the description of Figure 3 above and will not be described again here.

[0181] This example also includes the following steps: the user continuously slides the 3D scene, and the application on the vehicle system converts the continuous sliding events into continuous camera rotation parameters; the application on the vehicle system sends the continuous camera rotation parameters to the Unity engine; the Unity engine continuously updates the camera output channel integrated data; the Unity engine sends the continuously updated camera output channel integrated data to the application on the vehicle system; the application on the vehicle system controls the display based on the continuously updated camera output channel integrated data until the vehicle model appears. This example corresponds to S301 to S303 in Figure 5. Its implementation process and technical effects can be found in the description of Figure 5 above and will not be described again here.

[0182] S18. The user clicks on the function control to enlarge it, and the application on the vehicle system converts the click event into a camera shooting distance parameter.

[0183] S19. The application on the vehicle system sends the camera shooting distance parameters.

[0184] S20, Unity engine updates camera output channel integrated data for the third time.

[0185] S21. The Unity engine sends the camera output channel integrated data after the third update to the application on the vehicle system.

[0186] S22. The application on the vehicle system is based on the integrated data control display of the camera output channel after the third update.

[0187] The function control is enlarged, and more detailed content of the function control is displayed on the screen. This requires the application on the vehicle system to obtain data related to the function control, as shown in the following S23 to S24.

[0188] S23. The application on the vehicle system obtains detailed data related to the function control from the third-party application.

[0189] S24. The application on the vehicle system displays more detailed content of the function control on the screen based on the detailed data related to the function control.

[0190] The above S18 to S24 realize the enlargement of the function control, and the 3D scene changes accordingly to display detailed content.

[0191] This example also includes the following steps: when the user clicks on the enlarged function control and wants to shrink the function control to its original state, the application on the vehicle system converts the click event into the camera shooting distance parameters; the application on the vehicle system sends the camera shooting distance parameters to the Unity engine; the Unity engine updates the Camera output channel integrated data; the Unity engine sends the updated Camera output channel integrated data to the application on the vehicle system; the application on the vehicle system controls the display based on the updated Camera output channel integrated data; thereby, the function control is reduced in size and the 3D scene follows the changes.

[0192] Functional controls include vehicle controls. This example also includes the following steps: When a user clicks on a vehicle control to call a vehicle model, the application on the vehicle system converts the click event into call parameters for the vehicle model; the application on the vehicle system sends the call parameters to the Unity engine; the Unity engine updates the camera output channel integrated data; the Unity engine sends the updated camera output channel integrated data to the application on the vehicle system; the application on the vehicle system controls the display based on the updated camera output channel integrated data, thereby implementing the application scenario of calling a vehicle model. This example corresponds to S301 to S303 in Figure 5. Its implementation process and technical effects can be found in the description of Figure 5 above and will not be described again here.

[0193] S25. The user clicks on a function button (door, window, trunk, etc.), and the application on the vehicle system converts the click event into a control signal and simulation parameters.

[0194] S26. The application on the vehicle system sends a control signal to the vehicle body controller.

[0195] S27. The vehicle body controller controls the doors, windows, trunk and other equipment to close according to the control signal.

[0196] S28. The application on the vehicle system sends simulation parameters.

[0197] S29. The Unity engine updates the camera output channel integrated data for the fourth time.

[0198] S30. The Unity engine sends the fourth updated camera output channel integrated data to the application on the vehicle system.

[0199] S31. The application on the vehicle system is based on the integrated data control display of the camera output channel after the fourth update.

[0200] The above S26 to S31 realize the operation of the function buttons, control of the vehicle movement, and control of the operation of the vehicle virtual components in the vehicle model.

[0201] This example also includes the following steps: the user clicks on the virtual components of the vehicle (doors, windows, trunk, etc.) in the vehicle model, and the application on the vehicle system converts the click event into control signals and simulation parameters; the application on the vehicle system interacts with the vehicle body controller, and the application on the vehicle system sends the control signal to the vehicle body controller; the vehicle body controller controls the closing of devices such as doors or windows according to the control signal; the application on the vehicle system sends the simulation parameters to the Unity engine; the Unity engine updates the Camera output channel integrated data; the Unity engine sends the updated Camera output channel integrated data to the application on the vehicle system; the application on the vehicle system controls the display based on the updated Camera output channel integrated data; thereby realizing the operation of the virtual components of the vehicle in the vehicle model, controlling the actions of the vehicle (to realize actions such as closing doors and windows in reality), and controlling the operation of function buttons.

[0202] S32: The vehicle controller detects actions such as closing doors and windows in reality.

[0203] S33. The vehicle controller transmits the data to the application on the vehicle system.

[0204] S34. The application on the vehicle system sends data.

[0205] S35. Unity engine updates camera output channel integrated data for the fourth time.

[0206] S36. The Unity engine sends the fourth updated camera output channel integrated data to the application on the vehicle system.

[0207] S37. The application on the vehicle system is based on the integrated data control display of the camera output channel after the fourth update.

[0208] The above steps S32 to S37 realize the special effects of the function buttons and the virtual components of the vehicle in the vehicle model driven by the vehicle's actions.

[0209] It should be noted that the above S1 to S37 include the following technical solutions: S1 to S3, S4 to S5, S6 to S7, S8 to S12, S13 to S17, S18 to S24, S25 to S31, and S32 to S37. These solutions are not executed in sequence, and a portion of them can be selected for execution, or any combination of the solutions can be selected. This embodiment of the present application does not limit this.

[0210] The user scenarios of the embodiments of the present application are as follows: vehicle control, 3D music, 3D images, and other 3D scenes that require multi-screen interactive display. For example, under normal circumstances, the 3D image is on the main screen. When the co-driver wakes up the car computer with his voice, the 3D image moves from the main screen to the sub-screen and interacts with the co-driver's voice. When using Unity 3D to render the view, multiple window areas are required to display the 3D scene in a linked manner. When displaying on multiple screens, Unity 3D is required to support multi-screen view output at the same time. The multi-screen linkage method provided in the embodiments of the present application can achieve the following technical effects: creating an immersive 3D linkage experience for multiple screens in the car and increasing the sense of technology of the car computer.

[0211] It should be noted that the embodiment of the present application can also realize the output of video channels of different areas of the same view through the vehicle hardware to achieve the effect of multi-screen display linkage, and the embodiment of the present application does not limit this.

[0212] In an embodiment of the present application, the implementation principle is shown in Figure 11, which is an overall flow chart of a multi-screen linkage method provided in an embodiment of the present application; it is illustrated by taking two screens (screen 1 and screen 2), the view engine is the Unity engine, the car system is the Android system, and the camera is a camera as an example.

[0213] S601. Use Unity to build a scene, and the main camera renders the overall scene view.

[0214] In this example, Unity creates a 3D scene (an environment that creates a collection of elements such as models, backgrounds, sounds, and special effects in a 3D space), which will contain scene elements to be displayed on multiple screens.

[0215] S602: Create multiple cameras in Unity and set corresponding views of the area to be displayed.

[0216] In this example, multiple cameras can be created. For example, the Unity engine supports eight cameras. In Unity, the main camera (the rendering camera) renders the scene and performs other actions. Then, Camera1 (the view output camera responsible for outputting the regional view) is created to generate the view. This outputs the view to be displayed in Region 1 and loads it into the Android page view. For multi-region display, multiple Camera1s (view output cameras) are created, each configured for its own display region, and the view generation output is performed for the target view to be displayed in each region.

[0217] S603, Integrate the Unity generated library into the car Android application.

[0218] In this example, Unity packages the entire scene and outputs a library file for Android integration. The Unity-generated library is integrated into the Android application, and different screen windows directly add the corresponding Camera-generated views.

[0219] S6041, Android obtains the view generated by the first camera in Unity.

[0220] S6042: View 1 layout adds the view generated by camera 1.

[0221] S6043. The application on Android generates a complete view 1, and screen 1 displays the corresponding view.

[0222] S6051, Android obtains the view generated by the second camera in Unity.

[0223] S6052: View 2 layout adds the view generated by camera 2.

[0224] S6053. The application on Android generates a complete view 2, and screen 2 displays the corresponding view.

[0225] It should be noted that the above S6041 to S6043 and S6051 to S6053 are parallel technical solutions. When executing, there is no particular order and they can be executed simultaneously. This example does not limit this.

[0226] In this example, when the 3D scene changes, the views generated by different cameras change simultaneously, and the displays on multiple screens follow suit. By linking multiple screens at the software level, reliance on underlying hardware is reduced. This allows for the consistent display of the same scene across multiple screens, enhancing the car's sense of technology.

[0227] In order to implement the multi-screen linkage method on the vehicle system side in the embodiment of the present application, the embodiment of the present application also provides a multi-screen linkage device, as shown in Figure 12. Figure 12 is an optional structural schematic diagram of a multi-screen linkage device provided in the embodiment of the present application. The multi-screen linkage device 120 is applied to the vehicle system, and the vehicle system includes at least two screens. The multi-screen linkage device 120 includes: a first generating part 1201, which is configured to generate change parameters based on an operation event for any screen; an acquisition part 1202, which is configured to obtain the first display image corresponding to each screen from the view engine based on the change parameters; the view engine is used to update the camera parameters corresponding to each screen based on the change parameters, and generate the first display image corresponding to each screen based on the camera parameters corresponding to each screen and the same three-dimensional scene; the display part 1203, which is configured to display the corresponding first display image in each screen.

[0228] In some embodiments, the screen includes a background area and a front area, and the front area is used to display at least one functional control;

[0229] The first generating part 1201 is further configured to generate a displacement parameter based on a sliding event on a front area of ​​a target screen; the target screen is a screen where at least one functional control is located;

[0230] The acquisition portion 1202 is further configured to acquire a second display image of the target screen from the view engine based on the displacement parameter; the view engine is configured to update the position of at least one functional control in the front area based on the displacement parameter to generate the second display image of the target screen;

[0231] The display portion 1203 is further configured to display the second display image on the target screen.

[0232] In some embodiments, the front area includes a first area and a second area; the multi-screen linkage device 120 further includes a calling part 1204;

[0233] The calling portion 1204 is configured to call the first information of the function control and the second information;

[0234] The display portion 1203 is further configured to display the first information of the function control located in the first area and the second information in the first area; and to display the first information of the function control located in the second area in the second area.

[0235] In some embodiments, the acquiring portion 1202 is further configured to acquire an initial display image corresponding to each screen from a view engine; the view engine is configured to generate an initial display image corresponding to each screen based on initial camera parameters corresponding to each screen and the same three-dimensional scene;

[0236] The display portion 1203 is further configured to display a corresponding initial display image in each screen.

[0237] In some embodiments, the screen includes a background area and a front area, the front area is used to display at least one functional control; the operation event is an operation event for the background area of ​​any screen, and at least one functional control does not move or scale with the operation event.

[0238] In some embodiments, the screen includes a background area and a front area, the front area is used to display at least one functional control, the front area includes a first area, and the at least one functional control includes a vehicle control;

[0239] The first generating part 1201 is configured to generate a calling parameter of the vehicle model in response to the vehicle control being slid to the first area;

[0240] The acquisition portion 1202 is configured to acquire a third display image from a view engine based on the call parameters; the view engine is configured to update the position of the vehicle model based on the call parameters and generate a dynamic position change trajectory, which is a trajectory that appears from an adjacent screen, enters a target screen, and finally stops in the first area; the target screen is a screen where at least one functional control is located;

[0241] The display portion 1203 is configured to display a third display image.

[0242] In some embodiments, the vehicle model corresponding to the vehicle system is relatively stationary with respect to the three-dimensional scene;

[0243] The first generating part 1201 is further configured to load a plurality of function buttons corresponding to a plurality of virtual components of the vehicle model in response to a first click event on the vehicle model or the vehicle control in the first area;

[0244] The acquiring part 1202 is configured to generate a fourth display image of the target screen;

[0245] The display portion 1203 is further configured to display a fourth display image in the target screen; each function button is suspended on the vehicle model.

[0246] In some embodiments, the multi-screen linkage device 120 further includes a second sending part 1205;

[0247] The first generating part 1201 is further configured to generate a control signal and a simulation parameter based on a second click event on a target function button; the target function button is any function button;

[0248] The second sending part 1205 is configured to send a control signal to the vehicle body controller, so that the vehicle body controller controls the vehicle physical component corresponding to the target function button in the vehicle to perform a first switching action based on the control signal; the first switching action indicates that the vehicle physical component switches from a first state to a state opposite to the first state;

[0249] The acquisition portion 1202 is further configured to acquire, from the view engine, a fifth display image of the target screen based on the simulation parameters; the view engine is configured to update, based on the simulation parameters, a first simulation action of a vehicle virtual component corresponding to the target function button in the vehicle model, to generate the fifth display image of the target screen, wherein the first simulation action represents a transition of the vehicle virtual component from a first state to a state opposite to the first state;

[0250] The display portion 1203 is further configured to display the fifth display image on the target screen.

[0251] In some embodiments, the multi-screen linkage device 120 further includes a second receiving part 1206;

[0252] The second receiving part 1206 is configured to receive the analog component signal sent by the vehicle body controller when the target vehicle physical component undergoes a substantial switching action; the target vehicle physical component is any vehicle physical component;

[0253] The acquisition portion 1202 is further configured to acquire a sixth display image of the target screen from the view engine based on the simulated component signal; the view engine is configured to update, in the vehicle model, a function button corresponding to the target vehicle physical component and a second simulated action of the corresponding vehicle virtual component based on the simulated component signal, to generate the sixth display image of the target screen, wherein the second simulated action represents a transition of the vehicle virtual component from the second state to a state opposite to the second state;

[0254] The display portion 1203 is further configured to display the sixth display image on the target screen.

[0255] In some embodiments, the screen includes a background area and a front area, and the front area is used to display at least one functional control;

[0256] The first generating part 1201 is further configured to generate a first shooting depth parameter based on a third click event on a target function control; the target function control is any function control;

[0257] The acquisition portion 1202 is further configured to acquire, from the view engine, a seventh display image corresponding to each screen based on the first shooting depth parameter; the view engine is configured to update the first camera depth parameter corresponding to each screen based on the first shooting depth parameter, and generate the seventh display image corresponding to each screen based on the first camera depth parameter corresponding to each screen and the same three-dimensional scene;

[0258] The display portion 1203 is further configured to display the corresponding seventh display image in each screen until the target function control is enlarged to a size corresponding to the target screen, where the target screen is the screen where at least one function control is located.

[0259] In some embodiments, the first generating part 1201 is further configured to generate a second shooting depth parameter based on a fourth click event on the target screen;

[0260] The acquisition portion 1202 is further configured to acquire, from the view engine, an eighth display image corresponding to each screen based on the second shooting depth parameter; the view engine is configured to update the second camera depth parameter corresponding to each screen based on the second shooting depth parameter, and generate an eighth display image corresponding to each screen based on the second camera depth parameter corresponding to each screen and the same three-dimensional scene;

[0261] The display portion 1203 is further configured to display the corresponding eighth display image in each screen until the target function control is reduced to a size corresponding to the front area of ​​the target screen.

[0262] In some embodiments, the screen includes a background area and a front area, the front area is used to display at least two function controls; the at least two function controls include a first function control and a last function control;

[0263] The first generating part 1201 is further configured to generate a second continuous displacement parameter based on a second continuous sliding event on a front area of ​​a target screen; the target screen is a screen where at least two functional controls are located;

[0264] The acquiring portion 1202 is further configured to acquire, from the view engine, a ninth display image of the target screen based on the second continuous displacement parameter; the view engine is configured to update positions of at least two functional controls based on the second continuous displacement parameter to generate the ninth display image of the target screen;

[0265] The display portion 1203 is further configured to continuously display at least two function controls in the front area of ​​the target screen based on the ninth display image until the first function control or the last function control is reached.

[0266] In some embodiments, the screen includes a background area and a front area, the front area is used to display at least two function controls; the at least two function controls include function controls connected end to end;

[0267] The first generating part 1201 is further configured to generate a third continuous displacement parameter based on a third continuous sliding event of a front area of ​​a target screen; the target screen is a screen where at least two functional controls are located;

[0268] The acquiring portion 1202 is further configured to acquire, from the view engine, a tenth display image of the target screen based on the third continuous displacement parameter; the view engine is configured to update positions of at least two functional controls based on the third continuous displacement parameter to generate the tenth display image of the target screen;

[0269] The display portion 1203 is further configured to cyclically display at least two function controls in the front area of ​​the target screen based on the tenth display image.

[0270] In order to implement the multi-screen linkage method on the vehicle system side in the embodiment of the present application, the embodiment of the present application further provides a multi-screen linkage device. The multi-screen linkage device 120 is applied to the vehicle system. The vehicle system includes at least two screens, each screen including a background area and a front area. The front area is used to display at least one functional control. The front area includes a first area. The at least one functional control includes a vehicle control. The multi-screen linkage device includes:

[0271] a first generating portion configured to generate a calling parameter of the vehicle model in response to the vehicle control being slid to the first area;

[0272] The acquiring part is configured to acquire the third display image from the view engine based on the calling parameter;

[0273] The display part is configured to display the third display image, wherein, in the third display image, the vehicle model appears from the adjacent screen, drives into the target screen, and finally stops in the first area; the target screen is the screen where the at least one functional control is located.

[0274] In order to implement the multi-screen linkage method on the view engine side in the embodiment of the present application, the embodiment of the present application also provides a multi-screen linkage device, as shown in Figure 13. Figure 13 is an optional structural schematic diagram 2 of a multi-screen linkage device provided by the embodiment of the present application. The multi-screen linkage device 130 is applied to the view engine. The multi-screen linkage device 130 includes: a first receiving part 1301, configured to receive change parameters sent by an application on the vehicle system; the change parameters are generated in response to an operation event of any of the at least two screens corresponding to the vehicle system; an updating part 1302, configured to update the camera parameters corresponding to each screen based on the change parameters; a second generating part 1303, configured to generate a first display image corresponding to each screen based on the camera parameters corresponding to each screen and the same three-dimensional scene; a first sending part 1304, configured to send data feedback carrying the first display image corresponding to each screen to the application on the vehicle system; the data feedback is used to instruct the application on the vehicle system to display the corresponding first display image on each screen.

[0275] In some embodiments, the multi-screen linkage apparatus 130 further includes a creation portion 1305 ;

[0276] The second generating part 1303 is further configured to render the dynamic elements, the vehicle model, and the vehicle virtual components to generate a three-dimensional scene; the vehicle virtual components include at least one of the following: a left door, a right door, a trunk, a sunroof, a left window, and a right window;

[0277] The creation part 1305 is configured to create at least two virtual cameras in the three-dimensional scene based on the shooting position, shooting angle and shooting depth; each virtual camera is used to generate a display image on the screen corresponding to the virtual camera based on the camera parameters.

[0278] In some embodiments, dynamic elements include elements that change over a timeline.

[0279] In some embodiments, the multi-screen linkage device 130 further includes a loading portion 1306;

[0280] The loading part 1306 is configured to load at least one functional control in the three-dimensional scene based on the shooting position, shooting angle and shooting depth.

[0281] In some embodiments, the screen of the vehicle system includes a background area and a front area, and the front area is used to display at least one functional control;

[0282] The first receiving part 1301 is further configured to receive a displacement parameter sent by an application on the vehicle system; the displacement parameter is generated by the application on the vehicle system in response to a sliding event on a front area of ​​a target screen, where the target screen is a screen where at least one functional control is located;

[0283] The second generating part 1303 is further configured to update the position of at least one functional control in the front area based on the displacement parameter, and generate a second display image of the target screen;

[0284] The first sending part 1304 is further configured to send a first data feedback carrying the second display image of the target screen to the application on the vehicle system; the first data feedback is used to instruct the application on the vehicle system to display the second display image on the target screen.

[0285] In some embodiments, the front region includes a first region and a second region;

[0286] The second generating part 1303 is further configured to generate an initial display image corresponding to each screen based on the initial camera parameters corresponding to each screen and the same three-dimensional scene;

[0287] The first sending part 1304 is further configured to send the initial display image corresponding to each screen to the application on the vehicle system, so that the application on the vehicle system displays the corresponding initial display image on each screen.

[0288] In some embodiments, the first receiving part 1301 is further configured to receive a calling parameter of the vehicle model sent by an application on the vehicle system; the calling parameter is generated by the application on the vehicle system in response to the vehicle control being slid to the first area;

[0289] The second generating portion 1303 is further configured to update the position of the vehicle model based on the call parameter, and generate a dynamic position change trajectory, which is a third display image of the target screen, in which the vehicle model appears from an adjacent screen, enters a target screen, and finally stops in the first area; the target screen is a screen where at least one functional control is located;

[0290] The first sending part 1304 is also configured to send the third display image corresponding to each screen to the application on the vehicle system, so that the application on the vehicle system continues to display the corresponding third display image on each screen until the vehicle model passes through two adjacent screens, and then displays the vehicle model on the target screen.

[0291] In some embodiments, the vehicle model corresponding to the vehicle system is relatively stationary with respect to the three-dimensional scene;

[0292] The first receiving part 1301 is further configured to receive a first click event sent by an application on the vehicle system; the first click event is a click event of the vehicle model or vehicle control in the first area in response to the application on the vehicle system;

[0293] The updating part 1302 is further configured to update the camera position corresponding to the target screen based on the first click event, and load multiple function buttons corresponding to multiple virtual components of the vehicle model;

[0294] The second generating part 1303 is further configured to generate a fourth display image of the target screen;

[0295] The first sending part 1304 is further configured to send the fourth display image corresponding to the target screen to the application on the vehicle system, so that the application on the vehicle system displays the vehicle model and function buttons of the vehicle virtual components on the target screen.

[0296] In some embodiments, the first receiving part 1301 is further configured to receive simulation parameters sent by an application on the vehicle system; the simulation parameters are generated by the application on the vehicle system in response to a second click event on a target function button, where the target function button is any function button;

[0297] The second generating portion 1303 is further configured to update a first simulation action of the vehicle virtual component corresponding to the target function button in the vehicle model based on the simulation parameters, and generate a fifth display image of the target screen; the first simulation action represents that the vehicle virtual component is converted from a first state to a state opposite to the first state;

[0298] The first sending part 1304 is further configured to send the fifth display image of the target screen to the application on the vehicle system, so that the application on the vehicle system displays the fifth display image on the target screen.

[0299] In some embodiments, the first receiving part 1301 is further configured to receive an analog component signal sent by an application on the vehicle system; the analog component signal is generated by the vehicle body controller when a target vehicle physical component undergoes a substantial switching action, and the vehicle body controller sends the analog component to the application on the vehicle system. The target vehicle physical component is any vehicle physical component.

[0300] The second generating portion 1303 is further configured to update, in the vehicle model, the function button corresponding to the target vehicle physical component and the second simulated action of the corresponding vehicle virtual component based on the simulated component signal, and generate a sixth display image of the target screen, wherein the second simulated action represents that the vehicle virtual component transitions from the second state to a state opposite to the second state;

[0301] The first sending part 1304 is further configured to send the sixth display image of the target screen to the application on the vehicle system, so that the application on the vehicle system displays the sixth display image on the target screen.

[0302] In some embodiments, the screen includes a background area and a front area, and the front area is used to display at least one functional control;

[0303] The first receiving part 1301 is further configured to receive a first shooting depth parameter sent by an application on the vehicle system; the first shooting depth parameter is generated by the application on the vehicle system in response to a third click event on a target function control, where the target function control is any function control;

[0304] The updating part 1302 is further configured to update the first camera depth parameter corresponding to each screen based on the first shooting depth parameter;

[0305] The second generating part 1303 is further configured to generate a seventh display image corresponding to each screen based on the first camera depth parameter corresponding to each screen and the same three-dimensional scene;

[0306] The first sending part 1304 is also configured to send the seventh display image corresponding to each screen to the application on the vehicle system, so that the application on the vehicle system displays the corresponding seventh display image on each screen until the target function control is enlarged to a size corresponding to the target screen, and the target screen is the screen where at least one function control is located.

[0307] In some embodiments, the first receiving part 1301 is further configured to receive a second shooting depth parameter sent by an application on the vehicle system; the second shooting depth parameter is generated by the application on the vehicle system in response to a fourth click event on the target screen;

[0308] The updating part 1302 is further configured to update the second camera depth parameter corresponding to each screen based on the second shooting depth parameter;

[0309] The second generating part 1303 is further configured to generate an eighth display image corresponding to each screen based on the second camera depth parameter corresponding to each screen and the same three-dimensional scene;

[0310] The first sending part 1304 is also configured to send the eighth display image corresponding to each screen to the application on the vehicle system, so that the application on the vehicle system displays the corresponding eighth display image on each screen until the target function control is reduced to a size corresponding to the front area of ​​the target screen.

[0311] In some embodiments, the screen includes a background area and a front area, the front area is used to display at least two function controls; the at least two function controls include a first function control and a last function control;

[0312] The first receiving part 1301 is further configured to receive a second continuous displacement parameter sent by an application on the vehicle system; the second continuous displacement parameter is generated by the application on the vehicle system in response to a second continuous sliding event on a front area of ​​a target screen, where the target screen is a screen where at least two function controls are located;

[0313] The second generating part 1303 is further configured to update the positions of the at least two functional controls based on the second continuous displacement parameter to generate a ninth display image of the target screen;

[0314] The first sending part 1304 is also configured to send the ninth display image of the target screen to the application on the vehicle system, so that the application on the vehicle system continues to display at least two function controls in the front area of ​​the target screen based on the ninth display image until it reaches the first function control or the last function control.

[0315] In some embodiments, the screen includes a background area and a front area, the front area is used to display at least two function controls; the at least two function controls include function controls connected end to end;

[0316] The first receiving part 1301 is further configured to receive a third continuous displacement parameter sent by an application on the vehicle system; the third continuous displacement parameter is generated by the application on the vehicle system in response to a third continuous sliding event on the front area of ​​a target screen, where the target screen is a screen where at least two function controls are located;

[0317] The second generating part 1303 is further configured to update the positions of the at least two functional controls based on the third continuous displacement parameter to generate a tenth display image of the target screen;

[0318] The first sending part 1304 is also configured to send the tenth display image of the target screen to the application on the vehicle system, so that the application on the vehicle system cyclically displays at least two function controls in the front area of ​​the target screen based on the tenth display image.

[0319] It should be noted that the device provided in the above embodiment (including the multi-screen linkage device 120) is illustrated by the division of the above-mentioned program parts when performing multi-screen linkage. In actual applications, the above-mentioned processing can be assigned to different program parts as needed, that is, the internal structure of the device is divided into different program parts to complete all or part of the processing described above. In addition, the multi-screen linkage device provided in the above embodiment and the multi-screen linkage method embodiment executed by the corresponding device belong to the same concept. The specific implementation process and beneficial effects are detailed in the method embodiment and will not be described here. For technical details not disclosed in the embodiment of this device, please refer to the description of the method embodiment of this application for understanding.

[0320] In this embodiment and other embodiments, "part" can be part of a circuit, part of a processor, part of a program or software, etc., and of course it can also be a unit, a module, or a non-modular one.

[0321] In an embodiment of the present application, Figure 14 is a schematic diagram of the composition structure of the multi-screen linkage device proposed in the embodiment of the present application. As shown in Figure 14, the multi-screen linkage device 140 (also referred to as a vehicle-machine system) proposed in the embodiment of the present application includes a first processor 1401 and a first memory 1402 for storing an executable computer program. When the first processor 1401 is configured to execute the executable computer program stored in the first memory 1402, it implements the multi-screen linkage method provided on the vehicle-machine system side of the embodiment of the present application.

[0322] In some embodiments, the multi-screen linkage device 140 may further include a first communication interface 1403 and a first bus 1404 for connecting the first processor 1401 , the first memory 1402 and the first communication interface 1403 .

[0323] In the embodiment of the present application, the first bus 1404 is used to connect the first communication interface 1403, the first processor 1401 and the first memory 1402 to achieve mutual communication between these devices.

[0324] In an embodiment of the present application, Figure 15 is a second schematic diagram of the composition structure of the multi-screen linkage device proposed in the embodiment of the present application. As shown in Figure 15, the multi-screen linkage device 150 (also referred to as a view engine) proposed in the embodiment of the present application includes a second processor 1501 and a second memory 1502 for storing an executable computer program. When the second processor 1501 is configured to execute the executable computer program stored in the second memory 1502, it implements the multi-screen linkage method provided by the view engine side of the embodiment of the present application.

[0325] In some embodiments, the multi-screen linkage device 150 may further include a second communication interface 1503 and a second bus 1504 for connecting the second processor 1501 , the second memory 1502 and the second communication interface 1503 .

[0326] In the embodiment of the present application, the second bus 1504 is configured to connect the second communication interface 1503, the second processor 1501 and the second memory 1502 to achieve mutual communication between these devices.

[0327] In the embodiment of the present application, the processor (the first processor 1401 or the second processor 1501) may be at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, and a microprocessor. It is understood that for different devices, the electronic device used to implement the functions of the processor may also be other, and the embodiment of the present application does not specifically limit this.

[0328] The memory (first memory 1402 or second memory 1502) is configured to store executable computer programs and data, wherein the executable computer programs include computer operating instructions. The memory may include high-speed RAM memory and may also include non-volatile memory, such as at least two disk memories. In practical applications, the memory may be volatile memory (such as random-access memory (RAM); or non-volatile memory (such as read-only memory (ROM), flash memory, hard disk drive (HDD) or solid-state drive (SSD); or a combination of the above types of memory, and provides the processor with executable computer programs and data.

[0329] In addition, the various functional parts in this embodiment can be integrated into a single processing part, or each part can exist physically separately, or two or more parts can be integrated into a single part. The above-mentioned integrated parts can be implemented in the form of hardware or software functional parts.

[0330] If the integrated portion is implemented as a software functional portion and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method of this embodiment. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0331] An embodiment of the present application provides a computer-readable storage medium storing a computer program for implementing the multi-screen linkage method as described in any of the above vehicle system side embodiments when executed by a first processor; and for implementing the multi-screen linkage method as described in any of the above view engine side embodiments when executed by a second processor.

[0332] Exemplarily, the program instructions corresponding to a multi-screen linkage method in this embodiment can be stored on a storage medium such as a CD, a hard disk, or a USB flash drive. When the program instructions corresponding to a multi-screen linkage method in the storage medium are read or executed by an electronic device, the multi-screen linkage method described in any of the above embodiments can be implemented.

[0333] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0334] The present application is described with reference to the implementation flow diagram and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present application. It should be understood that each flow and / or box in the flow diagram and / or block diagram and the combination of the flow and / or box in the flow diagram and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processing machine or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one flow diagram or multiple flows and / or one block or multiple blocks of the block diagram.

[0335] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in implementing one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0336] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0337] The above description is a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Industrial Applicability

[0338] The present application discloses a multi-screen linkage method, apparatus, device, computer-readable storage medium, and computer application. The method is applied to an in-vehicle system comprising at least two screens. The method comprises: generating a change parameter based on an operation event on any one of the screens; obtaining a first display image corresponding to each screen from a view engine based on the change parameter; the view engine being configured to update the camera parameters corresponding to each screen based on the change parameter, and generating a first display image corresponding to each screen based on the camera parameters corresponding to each screen and the same three-dimensional scene; and displaying the corresponding first display image on each screen. When a user swipes on a screen of the in-vehicle system, the operation event is converted into a change parameter of the virtual camera, and image data is re-obtained from the view engine and the display is controlled, thereby achieving a multi-screen linkage effect through software logic. In this solution, as a finger swipes on any screen, the virtual camera rotates or moves, driving the views from different perspectives to change simultaneously, causing the images displayed on multiple screens to change simultaneously, thereby achieving multi-screen linkage and improving the richness and diversity of interaction.

Claims

1. A multi-screen linkage method, the method is applied to an application on a vehicle system, the vehicle system includes at least two screens, the method includes: Generate a change parameter based on an operation event on any screen; Based on the change parameter, obtaining a first display image corresponding to each screen from the view engine; The view engine is used to update the camera parameters corresponding to each screen based on the change parameters, and generate the first display image corresponding to each screen based on the camera parameters corresponding to each screen and the same three-dimensional scene; A corresponding first display image is displayed in each of the screens.

2. The method according to claim 1, wherein: The screen includes a background area and a front area, the front area is used to display at least one functional control, and the method further includes: generating a displacement parameter based on a sliding event on a front area of ​​a target screen; the target screen is a screen where the at least one functional control is located; Based on the displacement parameter, a second display image of the target screen is acquired from the view engine; the view engine is used to update the position of the at least one functional control in the front area based on the displacement parameter to generate the second display image of the target screen; A second display image is presented in the target screen.

3. The method according to claim 2, wherein: The front area includes a first area and a second area; the method further includes: Calling the first information and the second information of the function control; Displaying first information and second information of a functional control located in the first area in the first area; The first information of the functional control located in the second area is displayed in the second area.

4. The method according to claim 1, wherein: Before generating the change parameter based on the operation event on any screen, the method further includes: Acquire an initial display image corresponding to each screen from the view engine; the view engine is used to generate an initial display image corresponding to each screen based on initial camera parameters corresponding to each screen and the same three-dimensional scene; A corresponding initial display image is shown in each of the screens.

5. The method according to any one of claims 1 to 4, wherein: The screen includes a background area and a front area, the front area is used to display at least one functional control; the operation event is an operation event for the background area of ​​any screen, and the at least one functional control does not move or scale following the operation event.

6. The method according to claims 1 to 4, wherein: The screen includes a background area and a front area, the front area is used to display at least one functional control, the front area includes a first area, and the at least one functional control includes a vehicle control; the method further includes: In response to the vehicle control being slid to the first area, generating a calling parameter of a vehicle model; Based on the calling parameters, a third display image is obtained from the view engine; the view engine is used to update the position of the vehicle model based on the calling parameters, and generate a dynamic position change trajectory that appears from an adjacent screen and enters a target screen, and finally stops in the first area; the target screen is the screen where the at least one functional control is located; The third display image is displayed.

7. The method according to claim 6, wherein: The method further comprises: In response to a first click event on the vehicle model or the vehicle control in the first area, multiple function buttons corresponding to multiple virtual components of the vehicle model are loaded to generate a fourth display image of the target screen; A fourth display image is displayed in the target screen; each of the function buttons is suspended on the vehicle model.

8. The method according to claim 7, wherein: The method further comprises: Based on a second click event for a target function button, generating a control signal and a simulation parameter; the target function button is any function button; The control signal is sent to a vehicle body controller, so that the vehicle body controller controls a vehicle entity component corresponding to the target function button in the vehicle to perform a first switching action based on the control signal; the first switching action indicates that the vehicle entity component is converted from a first state to a state opposite to the first state; Based on the simulation parameters, a fifth display image of the target screen is acquired from the view engine; the view engine is used to update a first simulation action of a vehicle virtual component corresponding to the target function button in the vehicle model based on the simulation parameters, and generate a fifth display image of the target screen, wherein the first simulation action represents that the vehicle virtual component is converted from a first state to a state opposite to the first state; The fifth display image is displayed in the target screen.

9. The method according to claim 7, wherein: The method further comprises: When a target vehicle physical component undergoes a substantial switching action, receiving a simulated component signal sent by a vehicle body controller; the target vehicle physical component is any vehicle physical component; Based on the simulated component signal, a sixth display image of the target screen is acquired from the view engine; the view engine is used to update the function button corresponding to the target vehicle physical component and the second simulated action of the corresponding vehicle virtual component in the vehicle model based on the simulated component signal, and generate the sixth display image of the target screen, wherein the second simulated action represents that the vehicle virtual component is converted from a second state to a state opposite to the second state; The sixth display image is displayed in the target screen.

10. The method according to claims 1 to 4, wherein: The screen includes a background area and a front area, the front area is used to display at least one functional control, and the method further includes: Based on a third click event for a target functional control, generating a first shooting depth parameter; the target functional control is any functional control; Based on the first shooting depth parameter, obtaining a seventh display image corresponding to each screen from the view engine; the view engine is used to update the first camera depth parameter corresponding to each screen based on the first shooting depth parameter, and generate the seventh display image corresponding to each screen based on the first camera depth parameter corresponding to each screen and the same three-dimensional scene; The corresponding seventh display image is displayed in each of the screens until the target function control is enlarged to a size corresponding to a target screen, where the target screen is the screen where the at least one function control is located.

11. The method according to claim 10, wherein: After the target function control is enlarged to a size corresponding to the target screen, the method further includes: generating a second shooting depth parameter based on a fourth click event on the target screen; Based on the second shooting depth parameter, an eighth display image corresponding to each screen is obtained from the view engine; the view engine is used to update the second camera depth parameter corresponding to each screen based on the second shooting depth parameter, and generate the eighth display image corresponding to each screen based on the second camera depth parameter corresponding to each screen and the same three-dimensional scene; The corresponding eighth display image is displayed in each of the screens until the target function control is reduced to a size corresponding to the front area of ​​the target screen.

12. The method according to any one of claims 1 to 4, wherein: The screen includes a background area and a front area, and the front area is used to display at least two function controls; The at least two function controls include a first function control and a last function control; the method further includes: generating a second continuous displacement parameter based on a second continuous sliding event on a front area of ​​a target screen, wherein the target screen is a screen where the at least two functional controls are located; Based on the second continuous displacement parameter, a ninth display image of the target screen is acquired from the view engine; the view engine is used to update the positions of the at least two functional controls based on the second continuous displacement parameter to generate a ninth display image of the target screen; Based on the ninth display image, the at least two function controls are continuously displayed in the front area of ​​the target screen until the first function control or the last function control is reached.

13. The method according to any one of claims 1 to 4, wherein: The screen includes a background area and a front area, and the front area is used to display at least two function controls; The at least two functional controls include functional controls connected end to end; the method further includes: generating a third continuous displacement parameter based on a third continuous sliding event of the front area of ​​a target screen, wherein the target screen is a screen where the at least two functional controls are located; Based on the third continuous displacement parameter, acquiring a tenth display image of the target screen from the view engine; the view engine is used to update the positions of the at least two functional controls based on the third continuous displacement parameter to generate a tenth display image of the target screen; Based on the tenth display image, the at least two function controls are cyclically displayed in the front area of ​​the target screen.

14. A multi-screen linkage method, the method being applied to a view engine, the method comprising: Receive the change parameters sent by the application on the vehicle system; The change parameter is generated in response to an operation event on any one of the at least two screens corresponding to the vehicle system; Update the camera parameters corresponding to each screen based on the change parameters; Generate a first display image corresponding to each screen based on the camera parameters corresponding to each screen and the same three-dimensional scene; Sending data feedback carrying a first display image corresponding to each screen to an application on the vehicle system; The data feedback is used to instruct the application on the vehicle system to display the corresponding first display image on each screen.

15. The method according to claim 14, wherein: Before receiving the change parameter sent by the application on the vehicle system, the method further includes: Rendering the dynamic elements, the vehicle model and the vehicle virtual components to generate a three-dimensional scene; the vehicle virtual components include at least one of the following: a left door, a right door, a trunk, a sunroof, a left window and a right window; At least two virtual cameras are created in the three-dimensional scene based on the shooting position, shooting angle and shooting depth; each virtual camera is used to generate a display image of a screen corresponding to the virtual camera based on camera parameters.

16. The method according to claim 15, wherein: The dynamic elements include elements that change along the timeline.

17. The method according to claim 15 or 16, wherein: The method further comprises: At least one function control is loaded in the three-dimensional scene based on the shooting position, the shooting angle and the shooting depth.

18. A multi-screen linkage device, the device is applied to an application on a vehicle system, the vehicle system includes at least two screens, and the device includes: A first generating part is configured to generate a change parameter based on an operation event for any screen; An acquisition part, configured to acquire a first display image corresponding to each screen from a view engine based on the change parameter; The view engine is used to update the camera parameters corresponding to each screen based on the change parameters, and generate the first display image corresponding to each screen based on the camera parameters corresponding to each screen and the same three-dimensional scene; The display part is configured to display the corresponding first display image in each of the screens.

19. A multi-screen linkage device, the device being applied to a view engine, the device comprising: A first receiving part is configured to receive a change parameter sent by an application on the vehicle system; The change parameter is generated in response to an operation event on any one of the at least two screens corresponding to the vehicle system; An updating part, configured to update the camera parameters corresponding to each screen based on the change parameters; A second generating part is configured to generate a first display image corresponding to each screen based on the camera parameters corresponding to each screen and the same three-dimensional scene; The first sending part is configured to send data feedback carrying the first display image corresponding to each screen to the application on the vehicle system; the data feedback is used to instruct the application on the vehicle system to display the corresponding first display image on each screen.

20. A multi-screen linkage device, the device being a vehicle computer system; the device comprising: a memory configured to store an executable computer program; The processor is configured to implement the method according to any one of claims 1 to 13 when executing the executable computer program stored in the memory.

21. A computer-readable storage medium storing a computer program, configured to implement the method of any one of claims 1 to 13, or any one of claims 14 to 17 when executed by a processor.

22. A computer program product, comprising a computer program or instructions, which, when executed by a processor, implements the method of any one of claims 1 to 13, or any one of claims 14 to 17.

Images