Content suspension display method and device and in-vehicle infotainment system

Abstract

The embodiment of the invention relates to a content suspension display method and device and a vehicle-mounted infotainment system, and the method comprises the steps: setting a first display window and a second display window, enabling the display level of the first display window to be located below a display main interface of a receiving end, and rendering a video source sent by a source end into the first display window; setting a display level of a second display window to be located on a display main interface of the receiving end, and displaying Bitmap data through the second display window; therefore, the content display of the Miracast is realized. According to the method and the device, the display black edge condition or the content deformation condition caused by mismatching of the resolution ratios of the source end and the receiving end can be avoided, so that the second display window is accurately displayed on the display main interface in a suspension manner, and the user experience is improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of automotive electronic technology, and in particular to a content floating display method, device and vehicle system. Background Art

[0002] Miracast is a wireless display standard based on Wi-Fi Direct developed by the Wi-Fi Alliance in 2012. 3C devices that support this standard can share video images wirelessly. For example, a mobile phone can use Miracast to play videos or photos directly on a TV or other device without any connection cables or wireless hotspots.

[0003] The existing Miracast standard only supports one resolution, such as 1920*1080. If the receiver's screen physical resolution is not in the miracast display resolution list, when the receiver displays the video data provided by the source, a large black area will be displayed, or the displayed content will be distorted, thus affecting the user experience. Summary of the invention

[0004] In view of the above problems, the examples of the present invention provide a content floating display method, device and vehicle system, which are used to solve the problem that the Miracast standard in the prior art is only adapted to one resolution. If the physical resolution of the screen of the receiving end is not in the miracast display resolution list, then when the receiving end displays the video data provided by the source, a large black area will be displayed, or the displayed content will be deformed, thereby affecting the user experience.

[0005] According to one aspect of an example of the present invention, a method for floating display of content is provided, the method comprising: Establishing a connection between the source end and the receiving end, and setting a first resolution; Creating a first display window at the receiving end, setting the resolution of the first display window to a first resolution, setting the display level of the first display window to be located below the main display interface of the receiving end, and rendering the video source sent by the source end into the first display window; The Bitmap data of the first display window is obtained, and a second display window is created and set at the receiving end, and the display level of the second display window is set to be located above the main display interface of the receiving end; and the Bitmap data is displayed through the second display window.

[0006] In some optional embodiments, establishing a connection between the source end and the receiving end and setting the first resolution specifically includes: The source end and the receiving end are directly connected via WiFi; The source end sends a query request to the receiving end, wherein the query request includes inquiring about video supported formats; The receiving end receives the query request and replies with a confirmation instruction, wherein the confirmation instruction includes the video format of the receiving end.

[0007] In some optional embodiments, setting the display level of the first display window to be below the main display interface of the receiving end specifically includes: The display level of the first display window is set through the setZOrderOnTop function and the setZOrderMediaOverlay function.

[0008] In some optional embodiments, obtaining the Bitmap data of the first display window specifically includes: Create a Bitmap object with the same size as the first display window; Copy the content of the first display window to the Bitmap object through the PixelCopy.request function; In the OnPixelCopyFinishedListener interface, the copy result is processed through the onPixelCopyFinished function. If the copy result is successful, Bitmap data is generated, otherwise the copy is unsuccessful.

[0009] In some optional embodiments, setting the display level of the second display window to be located above the main interface displayed on the receiving end specifically includes: Creating a second display window, and setting layout parameters and display levels of the second display window; A window manager is obtained, and the second display window is added to the window manager.

[0010] In some optional embodiments, before displaying the valid content area of ​​the Bitmap data in the second display window, the Bitmap data is processed by a dichotomy method or an openCV algorithm.

[0011] In some optional embodiments, the Bitmap data is processed by a binary method, specifically including: Initialize the left and right boundaries of the Bitmap data; Find the left boundary of the Bitmap data by binary search, reset the left and right boundaries of the Bitmap data, and find the right boundary of the Bitmap data by binary search; After determining the left and right boundaries of the Bitmap data, the valid content area of ​​the Bitmap data is extracted.

[0012] In some optional embodiments, displaying the Bitmap data through the second display window specifically includes: The valid content area is saved as another Bitmap data, and displayed instead of the original Bitmap data; Alternatively, the resolution of the effective content area is extracted as the second resolution, and the second window is displayed at the second resolution.

[0013] According to another aspect of an embodiment of the present invention, a content floating display device is provided, the device comprising: A communication module, used to establish a connection between the source end and the receiving end, and set a first resolution; A background rendering module is used to create a first display window at the receiving end, set the resolution of the first display window to a first resolution, set the display level of the first display window to be located below the main display interface of the receiving end, and render the video source sent by the source end into the first display window; And a floating display module, used for obtaining the Bitmap data of the first display window, creating and setting a second display window at the receiving end, setting the display level of the second display window to be located above the main display interface of the receiving end; and displaying the Bitmap data through the second display window.

[0014] According to another aspect of an embodiment of the present invention, there is provided a vehicle system, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations such as the above-mentioned content floating display method.

[0015] The present invention provides a content floating display method, device and vehicle system, and its beneficial effects are: the content floating display method of the present invention, by setting a first display window and a second display window, sets the display level of the first display window to be located below the main display interface of the receiving end, and renders the video source sent by the source end into the first display window; sets the display level of the second display window to be located above the main display interface of the receiving end, and displays the Bitmap data through the second display window; thereby realizing Miracast content display. The present invention can avoid the display black edge or content deformation caused by the resolution mismatch between the source end and the receiving end, so that the second display window is accurately suspended and displayed on the main display interface, improving the user experience.

[0016] The above description is only an overview of the technical solution of the example of the present invention. In order to more clearly understand the technical means of the embodiment of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the example of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings: Figure 1 A schematic diagram showing a flow chart of a content suspension display method according to Embodiment 1 of the present invention is shown; Figure 2 The present invention provides a schematic flow chart of establishing a connection between a source end and a receiving end and setting a first resolution according to Embodiment 1; Figure 3 The present invention provides a schematic diagram of a process for obtaining Bitmap data of a first display window in Embodiment 1; Figure 4 The present invention provides a flowchart of setting the display level of the second display window to be located above the main display interface of the receiving end according to Embodiment 1; Figure 5 A schematic diagram showing the structure of a content suspension display device according to Embodiment 2 of the present invention is shown; Figure 6 The present invention provides a schematic structural diagram of a vehicle system according to Embodiment 3; Figure 7 A schematic diagram of display levels of a receiving end according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0018] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0019] Embodiment 1: Figure 1 , Figure 7 An embodiment of a content floating display method of the present invention is shown, and the method is used for multi-device screen projection. The method includes: 110, establish a connection between the source end and the receiving end, and set a first resolution; In step 110, establish a connection between the source end and the receiving end, and set a first resolution, specifically including: the source end and the receiving end are directly connected via WiFi; the source end sends a query request to the receiving end, wherein the query request includes inquiring about the supported video format; the receiving end receives the query request and replies with a confirmation instruction, wherein the confirmation instruction includes the video format of the receiving end. In addition, the source end can be a video or screen sharing source, such as a mobile terminal such as a mobile phone, tablet or smart watch; the receiving end can be a video or screen receiving display terminal, such as a vehicle-mounted display, a tablet computer or a PC computer, and the source end and the receiving end can be connected via a wireless communication method such as WiFi or a wired communication method.

[0020] 120, create a first display window at the receiving end, set the resolution of the first display window to the first resolution, set the display level of the first display window to be below the main display interface of the receiving end, and render the video source sent by the source end into the first display window; in step 120, set the display level of the first display window to be below the main display interface of the receiving end, specifically including: setting the display level of the first display window through the setZOrderOnTop function and the setZOrderMediaOverlay function. The video source or screen source of the source end is transmitted to the receiving end by wireless transmission. For example, in the Android system, the MediaCodec module can be used to decode and render the video data in the first display window.

[0021] 130, obtain the Bitmap data of the first display window, and create and set the second display window at the receiving end, set the display level of the second display window to be located above the main display interface of the receiving end; display the Bitmap data through the second display window. In step 130, obtain the Bitmap data of the first display window, specifically including: create a Bitmap object with the same size as the first display window; copy the content of the first display window to the Bitmap object through the PixelCopy.request function; process the copy result through the onPixelCopyFinished function in the OnPixelCopyFinishedListener interface, if the copy result is a successful copy, generate Bitmap data, otherwise the copy is unsuccessful. Setting the display level of the second display window to be located above the main display interface of the receiving end specifically includes: loading the Bitmap data; creating a second display window, and setting the layout parameters and display level of the second display window; obtaining the window manager, and adding the second display window to the window manager. Displaying the Bitmap data through the second display window specifically includes: saving the effective content area as another Bitmap data, and displaying it instead of the original Bitmap data; or extracting the resolution of the effective content area as the second resolution, and displaying it in the second window at the second resolution.

[0022] See also Figure 7 , 701 is the main display interface of the receiving end, 702 is the first display window, and 703 is the second display window. Figure 7 It can be seen that the first display window is located below the main display interface, and the second display window is located above the main display interface. Therefore, the second display window obtains the display content from the first display window and is displayed in a floating manner above the main display interface. The content floating display method of the present invention sets the display level of the first display window to be below the main display interface of the receiving end by setting the first display window and the second display window, and renders the video source sent by the source end into the first display window; sets the display level of the second display window to be above the main display interface of the receiving end, and displays the Bitmap data through the second display window; thereby realizing Miracast content display. The present invention can avoid the black edge display or content deformation caused by the resolution mismatch between the source end and the receiving end, so that the second display window is accurately displayed in a floating manner on the main display interface, thereby improving the user experience.

[0023] In a specific embodiment of step 110, establishing a connection between the source end and the receiving end and setting a first resolution specifically includes: 210, the source end and the receiving end are directly connected via WiFi; in step 210, the source end may be a video or screen sharing source, such as a mobile terminal such as a mobile phone, a tablet or a smart watch; the receiving end may be a video or screen receiving display end, such as a vehicle-mounted display, a tablet computer or a PC computer. The source end and the receiving end may be connected via wireless communication methods such as WiFi or wired communication methods.

[0024] 220, the source end sends a query request to the receiving end, wherein the query request includes inquiring about supported video formats; 230, the receiving end receives the query request and replies with a confirmation instruction, wherein the confirmation instruction includes the video format of the receiving end.

[0025] In steps 220-230, the source end can negotiate the WFD capability with the receiving end, and the first resolution can be negotiated through inquiry and reply confirmation. During the Miracast connection, the source end initiates a query request through GET_PARAMETER to query the properties and capabilities of the receiving end, one of which is the supported video format. The receiving end replies with a confirmation instruction to tell the source end the properties and capabilities it supports, such as the video format wfd_video_formats, which includes resolution.

[0026] In step 120, the display level of the first display window is set to be below the main display interface of the receiving end, specifically including: setting the display level of the first display window through the setZOrderOnTop function and the setZOrderMediaOverlay function. In an Android system example, the level of the first display window can be controlled by setting the Z Order of the first display window, such as using the setZOrderOnTop and setZOrderMediaOverlay methods. Among them, setZOrderOnTop(true): places the first display window at the top layer, which will block all other views. setZOrderMediaOverlay(false): places the first display window below the media overlay layer, allowing other views to cover it.

[0027] In step 130, the Bitmap data of the first display window is obtained, which specifically includes: 310, create a Bitmap object with the same size as the first display window; 320, copying the content of the first display window to the Bitmap object through the PixelCopy.request function; 330. In the OnPixelCopyFinishedListener interface, the copy result is processed through the onPixelCopyFinished function. If the copy result is successful, Bitmap data is generated. Otherwise, the copy is unsuccessful.

[0028] In steps 310-330, specifically in the Android system, the PixelCopy class can be used to save the content displayed in the first display window as Bitmap data. The Bitmap object creation function can be used to create the Bitmap bitmap, and the width, height, configuration and color mode of the Bitmap object are initialized when the Bitmap object is created. The content of the first display window is obtained through the PixelCopy.request function, and the copy result is processed in the onPixelCopyFinished method of the OnPixelCopyFinishedListener interface. If copyResult is PixelCopy.SUCCESS, it means that the copy is successful and the generated Bitmap data can be used.

[0029] In step 130, setting the display level of the second display window to be above the main interface displayed on the receiving end specifically includes: 410, create a second display window, and set the layout parameters and display level of the second display window; in step 410, create the second display window specifically through floatingImageView = new ImageView(this) and floatingImageView.setImageBitmap(bitmap); and set the layout parameters of the second display window through the WindowManager.LayoutParamsparams function.

[0030] 420, obtain the window manager, and add the second display window to the window manager. In step 420, obtain the window manager through the getSystemService function, and then add the second display window to the window manager through windowManager.addView.

[0031] In step 130, before the valid content area of the Bitmap data is displayed in the second display window, the Bitmap data is processed by the bisection method or the openCV algorithm. The layout of Bitmap data in memory is usually arranged by rows, and the size of each row is the image width multiplied by the number of bytes per pixel. By detecting each row, with the center of each row as the reference, the bisection method is used to search, and the edge of the black area is gradually searched and confirmed. In the following embodiments, the processing of Bitmap data by the bisection method is introduced.

[0032] In this embodiment, the processing of Bitmap data by the bisection method specifically includes: initializing the left and right boundaries of the Bitmap data; searching for the left boundary of the Bitmap data by the bisection method, resetting the left and right boundaries of the Bitmap data, and searching for the right boundary of the Bitmap data by the bisection method; after determining the left and right boundaries of the Bitmap data, extracting the valid content area of the Bitmap data. Among them, the first display window displays the video from the source end, and the second display window displays the Bitmap intercepted from the first display window. Before displaying the Bitmap data, the Bitmap data needs to be processed, that is, the black area is removed. Among them, the left boundary of the Bitmap data can be searched by the following function: while (left<right) {int mid = left + (right - left) / 2;if(isBlackColumn(bitmap, mid)) {left = mid + 1;} else {right = mid;}}intfinalLeft = left. To search for the right boundary of the Bitmap data by the bisection method, it can be implemented on the right boundary of the Bitmap data by referring to the search method of the left boundary of the Bitmap data. The valid content area is extracted by returnBitmap.createBitmap(bitmap, finalLeft, 0, finalRight - finalLeft + 1, height).

[0033] In the above embodiment, the Bitmap data can be displayed through the second display window by: saving the effective content area as another Bitmap data, and displaying it instead of the original Bitmap data. In addition, the Bitmap data can also be displayed through the second display window by extracting the resolution of the effective content area as the second resolution, and displaying it in the second window at the second resolution. In the above two schemes, when displaying the Bitmap data, the effective display area data can be saved as another Bitmap through the above calculation, or only the effective area resolution of the original Bitmap can be obtained. Both can realize the display of the second display window without black edges. The main difference is that when displaying the effective content area, the display origin is slightly different.

[0034] Embodiment 2: Figure 5 An embodiment of a content suspension display device is shown. The content suspension display device 500 includes a communication module 510, a background rendering module 520 and a suspension display module 530. The communication module 510 is used to execute step 110 in Example 1. The communication module 510 specifically includes: establishing a connection between the source end and the receiving end, and setting the first resolution includes: the source end and the receiving end are directly connected via WiFi; the source end sends a query request to the receiving end, wherein the query request includes inquiring about the supported video format; the receiving end receives the query request and replies with a confirmation instruction, wherein the confirmation instruction includes the video format of the receiving end. In addition, the source end can be a video or screen sharing source, such as a mobile terminal such as a mobile phone, tablet or smart watch; the receiving end can be a video or screen receiving display terminal, such as a terminal such as a car display, a tablet or a PC, and the source end and the receiving end can be connected via a wireless communication method such as WiFi or a wired communication method.

[0035] The background rendering module 520 is used to execute step 120 in Example 1. The background rendering module 520 specifically includes: setting the display level of the first display window to be located below the main display interface of the receiving end, specifically including: setting the display level of the first display window through the setZOrderOnTop function and the setZOrderMediaOverlay function. The video source or screen source of the source end is transmitted to the receiving end via wireless transmission. For example, in the Android system, the MediaCodec module can be used to decode and render the video data in the first display window.

[0036] The floating display module 530 is used to execute step 130 in Example 1. The floating display module 530 specifically includes: obtaining the Bitmap data of the first display window, specifically including: creating a Bitmap object of the same size as the first display window; copying the content of the first display window to the Bitmap object through the PixelCopy.request function; processing the copy result through the onPixelCopyFinished function in the OnPixelCopyFinishedListener interface, and generating Bitmap data if the copy result is a successful copy, otherwise the copy is unsuccessful. Setting the display level of the second display window to be located above the main display interface of the receiving end specifically includes: loading Bitmap data; creating a second display window, and setting the layout parameters and display level of the second display window; obtaining the window manager, and adding the second display window to the window manager. Displaying the Bitmap data through the second display window specifically includes: saving the effective content area as another Bitmap data, and displaying it instead of the original Bitmap data; or extracting the resolution of the effective content area as the second resolution, and displaying the second window at the second resolution.

[0037] Embodiment 3: Figure 6 The schematic diagram of the structure of the embodiment of the vehicle system of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the vehicle system.

[0038] like Figure 6 As shown, the vehicle system may include: a processor, a communication interface, a memory, and a communication bus.

[0039] The processor 610, the communication interface 640, and the memory 620 communicate with each other via the communication bus 630. The communication interface is used to communicate with other devices such as a client or other server network elements. The processor is used to execute the program 650, which can specifically execute the relevant steps in the above-mentioned embodiment of the vehicle-mounted key storage method.

[0040] Specifically, the program may include program code including computer executable instructions.

[0041] The processor may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the vehicle system may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0042] Memory is used to store programs. The memory may include high-speed RAM memory and may also include non-volatile memory, such as at least one disk storage.

[0043] The program can be called by the processor to enable the vehicle system to execute Figure 1 Step 110 to step 130.

[0044] The content floating display method of the present invention sets a first display window and a second display window, sets the display level of the first display window to be located below the main display interface of the receiving end, and renders the video source sent by the source end into the first display window; sets the display level of the second display window to be located above the main display interface of the receiving end, and displays the Bitmap data through the second display window; thereby realizing Miracast content display. The present invention can avoid the black edge display or content deformation caused by the resolution mismatch between the source end and the receiving end, so that the second display window is accurately suspended and displayed on the main display interface, thereby improving the user experience.

[0045] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system or other device. In addition, the embodiments of the present invention are not directed to any particular programming language.

[0046] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. Similarly, in order to simplify the present invention and help understand one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, the various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. Wherein, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself is a separate embodiment of the present invention.

[0047] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and further may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive.

[0048] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be understood as limitations on the order of execution.

Claims

1. A content suspension display method, characterized in that: The method comprises: Establishing a connection between the source end and the receiving end, and setting a first resolution; Creating a first display window at the receiving end, setting the resolution of the first display window to a first resolution, setting the display level of the first display window to be located below the main display interface of the receiving end, and rendering the video source sent by the source end into the first display window; The Bitmap data of the first display window is obtained, and a second display window is created and set at the receiving end, and the display level of the second display window is set to be located above the main display interface of the receiving end; and the Bitmap data is displayed through the second display window.

2. The content floating display method according to claim 1, characterized in that: The step of establishing a connection between the source end and the receiving end and setting the first resolution specifically includes: The source end and the receiving end are directly connected via WiFi; The source end sends a query request to the receiving end, wherein the query request includes inquiring about video supported formats; The receiving end receives the query request and replies with a confirmation instruction, wherein the confirmation instruction includes the video format of the receiving end.

3. The content floating display method according to claim 1, characterized in that: The step of setting the display level of the first display window to be below the main display interface of the receiving end specifically includes: The display level of the first display window is set through the setZOrderOnTop function and the setZOrderMediaOverlay function.

4. The content floating display method according to claim 1, characterized in that: Obtaining the Bitmap data of the first display window specifically includes: Create a Bitmap object with the same size as the first display window; Copy the content of the first display window to the Bitmap object through the PixelCopy.request function; In the OnPixelCopyFinishedListener interface, the copy result is processed through the onPixelCopyFinished function. If the copy result is successful, Bitmap data is generated, otherwise the copy is unsuccessful.

5. The content floating display method according to claim 4, characterized in that: The step of setting the display level of the second display window to be above the main display interface of the receiving end specifically includes: Creating a second display window, and setting layout parameters and display levels of the second display window; A window manager is obtained, and the second display window is added to the window manager.

6. The content floating display method according to claim 1, characterized in that: Before displaying the effective content area of ​​the Bitmap data in the second display window, the Bitmap data is processed by using a binary search method or an openCV algorithm.

7. The content floating display method according to claim 6, characterized in that: The Bitmap data is processed by a binary method, specifically including: Initialize the left and right boundaries of the Bitmap data; Find the left boundary of the Bitmap data by binary search, reset the left and right boundaries of the Bitmap data, and find the right boundary of the Bitmap data by binary search; After determining the left and right boundaries of the Bitmap data, the valid content area of ​​the Bitmap data is extracted.

8. The content floating display method according to claim 7, characterized in that: Displaying the Bitmap data through the second display window specifically includes: The valid content area is saved as another Bitmap data, and displayed instead of the original Bitmap data; Alternatively, the resolution of the effective content area is extracted as the second resolution, and the second window is displayed at the second resolution.

9. A content suspension display device, characterized in that: The device comprises: A communication module, used to establish a connection between the source end and the receiving end, and set a first resolution; A background rendering module is used to create a first display window at the receiving end, set the resolution of the first display window to a first resolution, set the display level of the first display window to be located below the main display interface of the receiving end, and render the video source sent by the source end into the first display window; And a floating display module, used for obtaining the Bitmap data of the first display window, creating and setting a second display window at the receiving end, setting the display level of the second display window to be located above the main display interface of the receiving end; and displaying the Bitmap data through the second display window.

10. A vehicle computer system, characterized in that: include: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation of the content floating display method as described in any one of claims 1-8.

Images