Method and circuit system for displaying delivery content through split window

Abstract

A method and a circuit system for displaying delivery content with segmented windows are provided, where the circuit system is applied to a display device which operates in a multi-window display mode, by executing the method, the circuit system receives an image transmitted by an image delivery technology from a user device, obtains an active area of the image after an edge searching procedure, and displays the delivery content in the multi-window display mode. In other words, the non-active area is cut off to obtain the image of the active area, and the image is displayed in one of the segmented windows of the display device after the display scale is adjusted according to the length-width ratio of the image of the active area.

Description

Technical Field

[0001] The specification discloses a method for projecting display content, and particularly relates to a method and circuit system for cutting out areas without content when projecting a vertical image to a display device, so as to display the projected content in a split window. Background Art

[0002] The display screen of a user device such as a mobile phone or tablet usually displays a general user interface and browsing related content in portrait display mode, and can be switched to landscape display mode when browsing specific video content.

[0003] Furthermore, the user can also use a video projection technology to project the content displayed on a mobile phone or tablet onto a computer screen or a TV top. The video projection technology used can be a wired connection (such as USB Type-C or HDMI). TM ) to display the image on an external device, using screen mirroring technology (such as ), or as Google The content delivery technology of the present invention wirelessly displays the image of the user device on the screen of the external device.

[0004] For example, if the screen resolution of a user device projected onto an external device using screen mirroring technology is 1920x1080, the image will be displayed at 1920x1080 resolution regardless of whether it is projected in portrait or landscape mode. However, when projecting images in portrait mode, large black areas will appear on both sides of the image displayed on the external device screen, which is generally designed for landscape display. These areas are areas where no content is displayed.

[0005] like Figure 1 The user device 10 uses screen mirroring technology to project its displayed vertical image onto the display screen of the external device 100. The vertical image 101 displayed on the external device 100 is flanked by black areas 102 and 103, with only the center area displaying the content projected by the user device 10. If the user device 10 is rotated to landscape orientation, the projected image is a landscape image, and the black area displayed on the external device 100 is smaller.

[0006] In another case, when the vertical image is projected onto the screen of the external device for display, if the external device 100 executes the multi-window mode, such as Figure 2As shown, the external device 100 divides the screen into two, wherein the split window 1 201 displays a television program, and the split window 202 displays the image projected from the user device 10 to the external device 100 via the screen mirroring technology.

[0007] according to Figure 2 In the display example, once the external device 100 enters the multi-window mode, the user device 10 projects a vertical image into one of the split windows displayed on the external device 100, such as split window 202, wherein the displayed image is as follows: Figure 1 The vertical image 101 and the black areas 102 and 103 on both sides of the example, thus, the image projected by the user device 10 in the split window 202 will be relatively small, which is not conducive to viewing. Currently, there is no screen mirroring technology that can improve this type of viewing experience. Summary of the Invention

[0008] In order to properly display the image projected by the user device onto the display device, especially when the display device operates in a multi-window display mode, this specification proposes a method and circuit system for displaying projected content in split windows.

[0009] According to an embodiment of a circuit system for use in a display device, the circuit system includes an image processing circuit that executes a method for displaying content in split windows. In the method, when the display device operates in a multi-window display mode, an image can be received from a user device and an active area of ​​the image containing display content can be obtained. By obtaining the image of the active area and adjusting the display ratio of the image, the image can be displayed in one of the split windows of the display device.

[0010] The display device provides at least two split windows in a multi-window display mode, respectively displaying a display image obtained from an image source and an image of an active area obtained from a user device.

[0011] According to an embodiment, the circuit system can implement an image processing device connected to a display device, and can receive an image transmitted by an image projection technology from a user device.

[0012] Furthermore, the user device may be a handheld device having a portrait display mode and a landscape display mode. When the user device operates in the portrait display mode, an image having a portrait ratio may be transmitted to the display device via image projection technology.

[0013] When the active area is determined to be an image with a horizontal ratio, one or more inactive areas can be directly cropped based on the image in the active area. When the active area is determined to be an image with a vertical ratio, one or more inactive areas determined by an edge finding process are cropped, and then the aspect ratio of the active area in the split window is determined. The display ratio of the image in the active area can be adjusted based on this aspect ratio before being displayed in the split window.

[0014] Furthermore, according to an embodiment of the step of obtaining an image of the active area through a circuit system, screen information of the display device used to display the image transmitted by the user device is first obtained, and window information of the split window used to display the image is also obtained. Then, a coordinate system corresponding to the split window is established. The boundary of the image can be obtained based on the image characteristics of the image, and the active area can be obtained. Then, based on the coordinate parameters defining the active area, an image cropping process is performed to obtain the image of the active area.

[0015] When the active area is obtained for the first time, the coordinate parameters of the active area of ​​the image obtained by the edge finding program are stored in a memory. When processing the image obtained after the active area is obtained for the first time, the coordinate parameters of the active area obtained for the first time can be obtained from the memory to execute the subsequent display program.

[0016] To further understand the features and technical contents of the present invention, please refer to the following detailed description and drawings of the present invention. However, the drawings provided are only for reference and illustration and are not intended to limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram showing the prior art of projecting a vertical image onto a screen of an external device for display;

[0018] Figure 2 A schematic diagram showing a conventional external device displaying images in a multi-window mode;

[0019] Figure 3 A diagram showing a system scenario embodiment of a method for displaying delivered content in split windows;

[0020] Figure 4 A schematic diagram showing an embodiment of a circuit system for executing a method for displaying delivered content in split windows;

[0021] Figure 5 A flow chart showing an embodiment of a method for displaying delivered content in a split window;

[0022] Figure 6 A diagram showing an embodiment of a method for displaying content in a split window by cutting out an active area; and

[0023] Figure 7A flow chart of an embodiment of determining whether to use stored parameters to display a picture in a method for displaying delivered content in split windows is shown.

[0024] Explanation of symbols

[0025] 10: User device

[0026] 100: External device

[0027] 101: Vertical Screen

[0028] 102,103: Black area

[0029] 201: Split Window 1

[0030] 202: Split Window 2

[0031] 30: User device

[0032] 310: Image processing device

[0033] 300: Display device

[0034] 301: Split Window 1

[0035] 302: Split Window 2

[0036] 40: Circuit system

[0037] 41: Wireless streaming unit

[0038] 43: Storage unit

[0039] 45: Output interface unit

[0040] 400: Image Signal Processor

[0041] 401: Edge search process

[0042] 402: Screen judgment program

[0043] 403: Screen cropping process

[0044] 42: Display device

[0045] 600: Active area

[0046] 60: Screen

[0047] Steps S501 to S511 display the delivered content in split windows

[0048] S501 Obtaining the split window screen information

[0049] S503 Obtain the image projected by the user device

[0050] S505 executes edge search program

[0051] S507 obtains the active area

[0052] S509 determines whether the image is in a vertical or horizontal orientation

[0053] S511 cuts out inactive areas

[0054] Steps S701 to S717 use the stored active area parameters to run the process DETAILED DESCRIPTION

[0055] The following is an explanation of the embodiments of the present invention through specific embodiments. Those skilled in the art can understand the advantages and effects of the present invention from the contents disclosed in this specification. The present invention can be implemented or applied through other different specific embodiments, and the details in this specification can also be modified and changed based on different viewpoints and applications without departing from the concept of the present invention. In addition, the drawings of the present invention are only simple schematic illustrations and are not depicted in actual size. It is stated in advance. The following embodiments will further explain the relevant technical content of the present invention in detail, but the disclosed content is not intended to limit the scope of protection of the present invention.

[0056] It should be understood that although terms such as "first," "second," and "third" may be used herein to describe various components or signals, these components or signals should not be limited by these terms. These terms are primarily used to distinguish one component from another, or one signal from another. In addition, the term "or" as used herein may include any one or more combinations of the associated listed items, depending on the actual situation.

[0057] In order to solve the problem of improper screen ratio when projecting a vertical image to an external device in a multi-window mode in the existing image projection technology, this specification proposes a method and circuit system for displaying the projected content in a split window. The system scenario of the method for displaying the projected content in a split window can be referred to. Figure 3 The embodiment diagram is shown.

[0058] The figure shows that the user device 30 uses image casting technology to process the displayed image through an image processing device 310 (such as a set-top box) and then projects it onto the display screen of the display device 300, wherein the image processing device 310 can be integrated into the display device 300 to implement a device such as a television.

[0059] The user device 30 generally refers to a user's handheld computer device, such as a smartphone, tablet computer, and laptop computer. The image processing device 310 can be a stand-alone set-top box or a circuit module integrated into the display device 300. The display device 300 can be a television device that provides a large screen for viewing content. The image projection technology can be a wired connection (such as USB Type-C or HDMI) TM ) to display the image on an external device, or using screen mirroring technology (such as ) or Google The content delivery technology of the present invention projects the image displayed by the user device 30 on the screen of the display device 300 in a wireless streaming manner.

[0060] When content displayed on user device 30 is projected onto the display screen of display device 300 using image projection technology, it is necessary to address the issue of areas on both sides of the vertical image being displayed without content. These areas are typically represented by black areas, which can affect the viewing experience, particularly in multi-window display mode. As shown in the figure, display device 300 enters multi-window mode, with the screen displaying split window 1 301 and split window 2 302. Split window 1 301 displays the program image generated by display device 300 itself, while split window 2 302 displays the image projected by user device 30 using image projection technology. Specifically, when the vertical image generated by user device 30 is projected onto split window 2 302 of display device 300, the circuit system therein executes the method for displaying content using split windows as proposed in this specification, cropping the detected areas without content (typically black areas) to display a more optimal proportion of the vertical image in split window 2 302.

[0061] The circuit system proposed in this manual can refer to Figure 4 The embodiment diagram shown in the figure shows a circuit system 40 which can be hardware or firmware running in an image processing device, and can implement the image processing device connected to the display device proposed in the above embodiment, which includes at least one image processing circuit, such as the image signal processor 400 shown in the figure, and the method of displaying content in a split window is executed by the collaboration of hardware and software means.

[0062] According to an embodiment, circuit system 40 utilizes an image signal processor (ISP) 400 as its primary processing circuitry. The circuit system 40 includes, through software and hardware collaboration, an edge detection program 401 for detecting areas without displayed content, a picture determination program 402 for determining whether a projected image is in portrait or landscape orientation, and a picture cropping program 403. Circuit system 40 also includes a wireless streaming unit 41 that implements a specific wireless communication protocol to receive image data transmitted from a user device using image projection technology. After processing by image signal processor 400, the image data can be output to display device 42 via output interface unit 45 for display. Specifically, the image projected by the user device can be displayed in one of the split windows of a multi-window display mode operated by display device 42.

[0063] The image signal processor 400 receives the user device (eg Figure 3 The user device 30 used for displaying the image data is transmitted using image projection technology, and the user device is a handheld device with a portrait display mode and a landscape display mode. For example, the user device uses a built-in sensor to sense the direction of the device, such as an inertial measurement unit (IMU) implemented by a combination of an accelerometer and a gyroscope, and determines the direction of the user device by the angular velocity and each axial acceleration sensed by the inertial measurement unit. In this way, when the user device is running in portrait display mode, an image with a portrait ratio is transmitted to the display device via image projection technology; conversely, when the user device is running in landscape display mode, an image with a landscape ratio is transmitted to the display device via image projection technology.

[0064] The circuit system in the display device, upon receiving an image transmitted from a user device, first determines whether the received image is in portrait or landscape proportions. One method involves executing an edge detection program 401 in an image signal processor 400 to detect active regions containing display content. Regions other than the active region are inactive regions. The circuit system 40 stores the detection results, including the coordinate parameters of the active region, in a memory unit 43. A frame determination program 402 in the image signal processor 400 then identifies whether the image displayed by the user device is in portrait or landscape proportions based on the characteristics of the active region. For example, the difference or ratio between the width and height of the active region (i.e., the aspect ratio) can be determined to determine whether the image displayed by the user device is in portrait or landscape proportions.

[0065] Afterwards, the image cropping program 403 executed by the image signal processor 400 can remove the area without display content by image cropping for the portrait image. According to an embodiment, when the active area is determined to be an image with a horizontal ratio, one or more inactive areas can be directly cropped based on the image information (such as coordinate parameters) of the active area. When the active area is determined to be an image with a vertical ratio, because when the display device 42 displays the active area, especially when it is displayed in one of the split windows, in addition to the active area, other inactive areas will be filled with black (or other colors), it is necessary to use the edge finding program 401 to find the black edge, thereby determining the active area and one or more inactive areas, and then cropping the one or more inactive areas by the image cropping program 403. Then, the image with the adjusted display ratio is output to the display device 42 via the input interface unit 45.

[0066] Related implementation procedures can be referred to Figure 5 Flowchart of the method for displaying content in a split window as shown, and refer to Figure 6 The reference diagram for edge finding is shown.

[0067] In a circuit system executing the method, image information of a display screen of a display device and multiple split windows in a multi-window display mode is obtained (step S501), wherein the display device provides at least two split windows in the multi-window display mode, which can respectively display a display screen obtained from an image source (e.g., a television program received from a set-top box) and an image of an active area obtained from the image received from the user device.

[0068] One of the split windows provided by the display device is as follows Figure 6 A screen 60 having a height (H) and a width (W) is displayed, and the screen 60 is described by a coordinate system (x, y coordinates), with the origin (0, 0) of the screen 60 defined by the pixel origin. A continuous image transmitted from a user device using image projection technology is then obtained from the content of the screen 60 in the split window (step S503), wherein the active area of ​​the image received from the user device is an image with a portrait ratio or an image with a landscape ratio.

[0069] The edge finding process 401 is executed based on the image features of the active area with displayed content and the inactive area without displayed content in the frame 60 (step S505). According to an embodiment, before executing the edge finding process 401, a coordinate system corresponding to the split window is first established. The edge finding process 401 determines the image boundary based on the image features, thereby obtaining the active area 600 with displayed content in the frame image and obtaining the coordinate parameters of the active area 600 (step S507). For example, based on the origin (0,0) of the coordinate system, the coordinate parameters of the active area 600 are described using the top coordinate, the bottom coordinate, the left coordinate, and the right coordinate, etc., and then stored in the memory unit 43 of the circuit system 40. Thus, it can be seen that the active area 600 determined by the edge finding process 401 is the image projected by the user device using the image projection technology, while the inactive area is the area without displayed content.

[0070] The size of the active area 600 occupied by the screen 60 can be obtained based on coordinate parameters such as the top coordinate (top), the bottom coordinate (bottom), the left coordinate (left) and the right coordinate (right). Figure 6 In the example shown, the width of active area 600 is obtained by subtracting the left coordinate (left) from the right coordinate (right), and the height of active area 600 is obtained by subtracting the top coordinate (top) from the bottom coordinate (bottom). Next, the image determination procedure 402 is executed. According to an embodiment, the difference or ratio between the width and height of active area 600 (e.g., width greater than height indicates a landscape orientation; width less than height indicates a portrait orientation) can be used to determine whether the image projected by the user device using the image projection technology is a portrait or landscape orientation, corresponding to whether the user device is displaying content in portrait or landscape mode (step S509).

[0071] For example, common smartphones or tablets have display screen ratios of (but not limited to) 21:9, 16:9, and 4:3, so the width-to-height ratio of the active area 600 can be used to determine whether the split window displays a portrait or landscape image.

[0072] According to an embodiment, the method for displaying content in a split window may execute the image cropping procedure 403 only for the portrait orientation to crop out one or more inactive areas outside the active area (step S511). In this step, if the image characteristics received from the user device cannot determine whether the projected image is portrait or landscape, the subsequent cropping procedure may not be performed until it is determined to be a portrait orientation, at which time the subsequent cropping procedure is performed. If it is a landscape orientation, no cropping is required, or a small cropping is performed. The circuit system then outputs the cropped image of the active area 600 to the display. The display ratio of the image of the active area may be adjusted based on the aspect ratio of the active area 600 occupied by the split window before displaying it in the split window, so that the active area 600 is displayed in the split window with a better proportion.

[0073] Furthermore, when the circuit system obtains the active area of ​​the image projected by the user device for the first time, the coordinate parameters of the active area can be stored in the memory of the circuit system, such as Figure 4 The storage unit 43 is shown, and there is no need to repeatedly perform edge finding and cutting procedures in the method flow, which causes system instability.

[0074] Figure 7 A flow chart of an embodiment of determining whether to use stored active area parameters to display an image in a method for displaying content in a split window is shown.

[0075] In this process, continuous frames are received from a user device in a streaming manner, and the active area with display content is detected based on the characteristics of each frame (step S701), and the aspect ratio of the active area is calculated (step S703).

[0076] Next, the active area's aspect ratio is determined to be a valid aspect ratio (step S705), thereby eliminating unprocessable or erroneous image data. For example, common image aspect ratios include 21:9, 16:9, 4:3, 1:1, 4:5, and 9:16 (this list is provided here for illustrative purposes only and is not intended to limit the scope of implementation). Therefore, aspect ratios between 21:9 and 9:16 can be considered valid. If the active area is determined not to have a valid aspect ratio (no), the image frame is ignored (step S707). Conversely, if the active area is determined to have a valid aspect ratio (yes), the process continues by determining whether this is the first time the active area has been processed (step S709). If this is the first time the active area has been determined in an image processing procedure (yes), the process continues by determining the display mode of the display device, including determining whether the display is in multi-window display mode, and determining the number of split windows and the image information of each split window (step S711).

[0077] Based on the obtained split window information, such as the split window number, ratio, and size, as well as the image information of the active area, such as whether the image is portrait-oriented or landscape-oriented, a determination is made as to whether a cropping process should be performed (step S713). If the active area of ​​the received image is portrait-oriented, the inactive area is cropped; if the active area of ​​the received image is landscape-oriented, a small cropping step or no cropping is performed. The information obtained in the above steps and the active area parameters are then stored in a memory (step S715) for subsequent frame processing.

[0078] On the other hand, if the active area is not obtained for the first time in an image processing program, the coordinate parameters of the active area and the related display information stored previously can be obtained from the memory in the program for displaying the image of the active area after the active area is obtained for the first time, that is, the display ratio of the active area image can be adjusted and then displayed in the split window (step S717), and the repeated execution steps can be omitted.

[0079] It should be noted that according to the above embodiment, when the user device continuously projects an image onto the display device in the same display mode, it indicates that the same image processing process is in progress. Therefore, the steps of repeatedly obtaining the active area coordinate parameters and adjusting the display in the split windows can be omitted, and the image can be successfully projected into one of the split windows of the display device. When the user device switches the display mode, for example, during the method, the circuit system determines based on the received image that the user device has switched from portrait to landscape mode, or vice versa, it is necessary to re-execute the image processing process, such as determining the active area of ​​the image and re-adjusting the display ratio to output the cropped image of the active area, so that the image is displayed in the cropped window with the appropriate ratio.

[0080] According to the aforementioned embodiments of the method and circuit system for displaying content in a split window, the method can determine whether an image transmitted by a user device via image projection technology is in portrait or landscape orientation based on its characteristics, and provide different processing methods for portrait and landscape orientations. Specifically, for a portrait image projected by the user device, the image is cropped based on the boundaries determined by the edge-finding process to determine an active area containing display content, which can be displayed in the split window after being appropriately scaled. Conversely, if the image projected by the user device is in landscape orientation, indicating that the display device has a smaller or no inactive area to fill when displaying, the image projected by the user device can be displayed or only slightly cropped. This allows the image projected by the user device to be displayed on the display device at an appropriate scale, and also avoids display instability caused by repeated detection and image cropping during the circuit system's image processing process.

[0081] The contents disclosed above are only preferred feasible embodiments of the present invention and do not limit the claims of the present invention. Therefore, all equivalent technical changes made using the contents of the present invention description and drawings are included in the claims of the present invention.

Claims

1. A circuit system, applied to a display device, comprising: An image processing circuit executes a method for displaying content in split windows, comprising: When the display device operates in a multi-window display mode, receiving an image from a user device; obtaining an active region of the image; and An image of the active area is acquired and displayed in one of the split windows of the display device after adjusting the display ratio. 2 . The circuit system of claim 1 , wherein the display device provides at least two split windows in the multi-window display mode, respectively displaying a display image obtained from an image source and an image of the active area. 3 . The circuit system of claim 2 , wherein the circuit system implements an image processing device connected to the display device to receive the image transmitted from the user device using an image projection technology.

4. The circuit system of claim 3, wherein the user device is a handheld device having a portrait display mode and a landscape display mode, and when the user device operates in the portrait display mode, an image having a portrait ratio is transmitted to the display device via the image projection technology.

5. The circuit system of claim 4, wherein: When it is determined that the active area is an image with a horizontal ratio, one or more inactive areas are directly cropped according to the image information of the active area.

6. The circuit system of claim 4, wherein: When it is determined that the active area is the image with a vertical proportion, one or more inactive areas obtained by a side-finding process are cropped, and then an aspect ratio of the active area occupied by the split window is obtained. That is, the display ratio of the image of the active area is adjusted according to the aspect ratio and then displayed in the split window; wherein, when the active area is obtained for the first time, the coordinate parameters of the active area of ​​the image obtained by the side-finding process are stored in a memory.

7. The circuit system according to any one of claims 1 to 6, wherein the step of acquiring the image of the active area by the circuit system comprises: Obtaining screen information of the display device for displaying the image transmitted by the user device; Obtaining window information of the split window for displaying the image; Establishing a coordinate system corresponding to the segmented window, obtaining the boundary of the image according to the image features of the image, and obtaining the active area; as well as An image of the active area is acquired after image cropping according to coordinate parameters defining the active area.

8. A method for displaying content in split windows, executed in a circuit system of a display device, comprising: When the display device operates in a multi-window display mode, receiving an image from a user device; obtaining an active region of the image; as well as An image of the active area is acquired and displayed in one of the split windows of the display device after adjusting the display ratio.

9. The method for displaying content in a split window as claimed in claim 8, wherein the active area of ​​the image received from the user device is an image with a portrait ratio or a landscape ratio, and the user device is a handheld device with a portrait display mode and a landscape display mode, the user device transmits the image with a portrait ratio or the image with a landscape ratio to the display device via an image projection technology, and determines whether the image projected by the user device is an image with a portrait ratio or an image with a landscape ratio based on the difference or ratio between the width and height of the active area.

10. The method for displaying content in split windows according to claim 8 or 9, wherein the step of acquiring the image of the active area by the circuit system comprises: Obtaining screen information of the display device for displaying the image transmitted by the user device; Obtaining window information of the split window for displaying the image; Establishing a coordinate system corresponding to the segmented window, obtaining the boundary of the image according to the image features of the image, and obtaining the active area; as well as An image of the active area is acquired after image cropping according to coordinate parameters defining the active area.

Images