Image processing method and display device

Abstract

The present disclosure relates to the technical field of multimedia post-processing, and in particular to an image processing method and a display device, which are used for alleviating the problems of the current operation process of displaying a video picture in a split manner by means of a plurality of displays being complex and the interaction experience of a user being poor. The method comprises: when setting information sent by a split-screen casting application has been received, switching a video channel to a target channel, wherein the setting information comprises a set of coordinates, which correspond to a split image actually displayed on a display device, in a frame image, the split image is a part of the frame image, and the target channel comprises any one of a multimedia channel, an analog signal channel and a high-definition multimedia interface channel; when start information sent by the split-screen casting application and used for instructing the start of special effect processing has been received, capturing the frame image from the target channel; performing special effect processing on the frame image, so as to obtain the split image, which corresponds to the set of coordinates, in the frame image; and rendering the split image, and displaying rendered split image.

Description

Image processing method and display device

[0001] Cross-reference to Related Applications

[0002] The present disclosure claims priority to Chinese application No. 202410703186.0, filed on May 31, 2024, the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0003] The present disclosure relates to the technical field of multimedia post-processing, and in particular, to an image processing method and a display device. BACKGROUND

[0004] When a user splits a video picture to display through multiple displays, the user can connect multiple displays through an HDMI (High Definition Multimedia Interface) splitter, so that each display can display a part of the video picture. However, at present, when the user splits the video picture to display through multiple displays, the original video picture needs to be processed, so that the display pictures displayed by the multiple displays can be combined into a complete video picture. This results in a relatively complex operation process for splitting the video picture to display through multiple displays at present, and the user's interactive experience is poor. SUMMARY

[0005] In a first aspect, the present disclosure provides a display device, which can include: a display; an external device interface for connecting the display device and an external device, so that the display device body receives multimedia playing content of the external device and presents on the display; at least one processor connected with the display and the external device interface, configured to execute computer instructions to enable the display device to perform: when receiving setting information sent by a split screen casting application, switching a video channel to a target channel; wherein the setting information includes a coordinate set corresponding to a split image in a frame image in actual display of the display device, the split image is part of the frame image, and the target channel includes any one of a multimedia channel, an analog signal channel, and an HDMI (High Definition Multimedia Interface) channel; when receiving start information sent by the split screen casting application for indicating starting special effect processing, grabbing a frame image from the target channel; performing special effect processing on the frame image to obtain a split image of the coordinate set on the frame image; rendering the split image to obtain a rendered split image, and controlling the display to display the rendered split image.

[0006] In a second aspect, the present disclosure provides an image processing method, which can include: switching a video channel to a target channel when receiving setting information sent by a split screen projection application; wherein the setting information includes a coordinate set corresponding to a split image actually displayed by the display device in a frame image, the split image being a part of the frame image, and the target channel including any one of a multimedia channel, an analog signal channel, and an HDMI channel; capturing a frame image from the target channel when receiving start information sent by the split screen projection application and indicating start of special effect processing; performing special effect processing on the frame image to obtain a split image of the coordinate set on the frame image; rendering the split image to obtain a rendered split image, and displaying the rendered split image.

[0007] In a third aspect, the present disclosure provides a display device, which can include: a memory and a processor, the memory being used to store a computer program, and the processor being used to, when executing the computer program, enable the display device to implement any one of the image processing methods provided in the first aspect.

[0008] In a fourth aspect, the present disclosure provides a computer readable storage medium, which can include: a computer program stored on the computer readable storage medium, and when the computer program is executed by a computing device, the computer program can enable the computing device to implement any one of the image processing methods provided in the first aspect.

[0009] In a fifth aspect, the present disclosure provides a computer program product, which, when running on a computer, can enable the computer to execute the image processing method provided in any one of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0010] FIG. 1 is a flow diagram of an existing image processing according to some embodiments;

[0011] FIG. 2 is a schematic diagram of an image processing method applied to a split display system according to some embodiments;

[0012] FIG. 3 is a schematic diagram of a structure of a display device in an image processing method according to some embodiments;

[0013] FIG. 4 is a schematic diagram of a structure of a display device in an image processing method according to some embodiments;

[0014] FIG. 5 is a flow diagram of an image processing method according to some embodiments;

[0015] FIG. 6 is a flow diagram of an image processing method according to some embodiments;

[0016] FIG. 7 is a flow diagram of an image processing method according to some embodiments;

[0017] FIG. 8 is a structural schematic diagram of a display device according to some embodiments;

[0018] FIG. 9 is a schematic diagram of a chip system according to some embodiments;

[0019] FIG. 10 is a structural schematic diagram of a computer program product of an image processing method according to some embodiments. DETAILED DESCRIPTION

[0020] In order to enable a more clear understanding of the above-mentioned objects, features and advantages of the present disclosure, the schemes of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0021] In the following description, a large number of specific details are set forth in order to facilitate a thorough understanding of the present disclosure, but the present disclosure can also be implemented in other manners different from those described herein; obviously, the embodiments described in the specification are only a part of the embodiments of the present disclosure, and not all the embodiments.

[0022] The display device provided by the embodiments of the present disclosure can have various implementation forms, for example, can be a television, a smart television, a laser projection device, a monitor, an electronic bulletin board, an electronic table, etc. FIG. 3 is a specific embodiment of the display device of the present disclosure.

[0023] It should be noted that, in this document, relational terms such as "first" and "second", and the like, are used solely to distinguish one entity or action from another entity or action, without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element defined by the phrase "comprising a..." does not exclude the existence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0024] In some embodiments, the Surface in the embodiments of the present disclosure is a basic component for graphics rendering in the Android system, and an application program can draw image content or display view content through the Surface.

[0025] In some embodiments, the OSD layer image in the embodiments of the present disclosure generally corresponds to an image drawn by a user interface (UI), without a quality effect.

[0026] The EGL in the embodiments of the present disclosure is an intermediate interface layer between an OpenGL ES rendering API and a native window system.

[0027] The framebuffer in the embodiments of the present disclosure is used to drive a video display device from a memory buffer containing complete frame data with a video output device.

[0028] In some embodiments, the existing image processing method can be as shown in FIG. 1, including an OSD layer, a VIDEO layer, an image processing module, a display processing module, and screen display. Among them, the OSD layer generally corresponds to an image drawn by a UI, without a quality effect, while the VIDEO layer corresponds to a video image played under a multimedia (MM), DTV, HDMI, etc. channel, which can be subjected to PQ quality processing such as SNR (signal-to-noise ratio), TNR (temporal noise reduction), ACM (adaptive contrast), etc., to show the user a clearer and higher quality image; after being scaled to the display resolution size by Scalar, the image is superimposed with the image of the OSD layer after the display processing module, and the superimposed signal is transmitted to the screen for screen display. The display device displayed by the above scheme can only filter the image displayed by the OSD layer, and cannot realize the user's customized feature processing, resulting in poor user experience. Therefore, the embodiments of the present disclosure provide an image processing method to solve the above problems, and the specific implementation process is as follows:

[0029] FIG. 2 is a schematic diagram of a scene of a display device according to some embodiments, as shown in FIG. 2, a user can control at least two display devices to display the same video signal through an electronic device 1, and the display device and the electronic device 1 can establish a communication connection through a wired or wireless manner, when connected through a wired manner, the connection can be made through a high-definition multimedia interface (HDMI) 3.

[0030] In some embodiments, taking two display devices as an example, namely display device 2-1 and display device 2-2, electronic device 1 is connected to the external device interface (e.g., HDMI interface) of display device 2-1 via an HDMI cable, and simultaneously connected to the external device interface (e.g., HDMI interface) of display device 2-2 via an HDMI cable. In this way, electronic device 1 can transmit the same two multimedia signals to display devices 2-1 and 2-2. Then, display devices 2-1 and 2-2 project their respective display images onto the display background 4. As shown in Figure 2, since both display devices 2-1 and 2-2 display the same multimedia signal, to make the displayed image larger, the projected images of display devices 2-1 and 2-2 can be stitched together, resulting in the effect shown in Figure 2. To ensure the completeness of the display image, it is usually necessary to stitch the display images of display devices 2-1 and 2-2 together, thereby obtaining display images with different aspect ratios (e.g., 16:9).

[0031] In some embodiments, when a user needs to display a frame image across multiple display devices, the user can set the number of display devices to be displayed in the split-screen projection application. Then, by setting the display area of ​​each display device in the split display (e.g., when there are 2 display devices, the display area can be set to include either the left or right side; or when there are 4 display devices, the display area can be set to include either the upper left, lower left, upper right, or lower right side) and the video resolution within the display area, the setting information (at least including the display area and video resolution) for each display device is obtained. After obtaining the setting information for each display device, the split-screen projection application sends the setting information to the corresponding display device. When the display device receives the setting information sent by the split-screen projection application, it switches its video channel to the target channel. The setting information includes the set of coordinates corresponding to the split image actually displayed by the display device in the frame image; the split image is a portion of the frame image; and the target channel includes any one of a multimedia channel, an analog signal channel, and a high-definition multimedia interface channel. Subsequently, when the user performs a start operation on the split-screen display application to indicate the start of special effects processing, the display device receives the start information sent by the split-screen projection application to indicate the start of special effects processing, and captures a frame image from the target channel; the display device performs special effects processing on the frame image to obtain a segmented image corresponding to the coordinate set on the frame image; wherein, the special effects processing includes at least image cropping; the display device renders the segmented image to obtain the rendered segmented image, and displays the rendered segmented image.

[0032] In some embodiments, the display device may be a laser TV or a projector, or other device that can be used for projection display.

[0033] As can be seen, the image processing method provided in this embodiment involves the following steps during image segmentation and display: Each display device switches its video channel to the target channel upon receiving setting information from the segmentation and projection application; upon receiving start information from the segmentation and projection application indicating the start of special effects processing, the display device captures a frame image from the target channel; the display device performs special effects processing on the frame image to obtain a segmented image corresponding to the coordinate set on the frame image; the display device renders the segmented image to obtain a rendered segmented image and displays the rendered segmented image. In this way, each display device can display the corresponding segmented image on the frame image according to the setting information sent by the segmentation and projection application. Furthermore, the rendered segmented images displayed by all display devices can form a complete frame image, eliminating the need for manual pre-setting of the frame images to be displayed for each display device, thereby improving the user experience.

[0034] Figure 3 is a hardware configuration block diagram of a display device according to some embodiments. As shown in Figure 3, the display device includes at least one of a tuner / demodulator 210, a communicator 220, a detector 230, an external device interface 240, a processor 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface 280. The processor includes a central processing unit, a video processor, an audio processor, a graphics processor, RAM, ROM, and a first to nth interface for input / output. The display 260 may be a touch-enabled display, such as a touch screen display. The tuner / demodulator 210 receives broadcast television signals via wired or wireless reception and demodulates audio and video signals, such as EPG data signals, from multiple wireless or wired broadcast television signals. The detector 230 is used to collect signals from the external environment or signals interacting with the external environment. The processor 250 and the tuner / demodulator 210 may be located in different separate devices; that is, the tuner / demodulator 210 may also be located in an external device of the main device where the processor 250 is located, such as an external set-top box.

[0035] In some embodiments, the external device interface 240 may include, but is not limited to, one or more interfaces such as: High Definition Multimedia Interface (HDMI), analog or data high-definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, etc. It may also be a composite input / output interface formed by multiple interfaces mentioned above.

[0036] In some embodiments, the display 260 may be a liquid crystal display, an OLED display, or a projection display, and may also be a projection device and a projection screen. For example, when the display device is a projector, the display 260 may be a projection screen, or when the display device is a laser TV, the display 260 may be a projection display.

[0037] In some embodiments, the processor 250 controls the operation of the display device and responds to user operations through various software control programs stored in memory. The processor 250 controls the overall operation of the display device.

[0038] In some embodiments, taking the display device of one or more embodiments as a projector 1 and the operating system of the projector 1 as an Android system as an example, as shown in Figure 4, the projector 1 can be logically divided into an application layer (hereinafter referred to as "application layer") 21, an application framework layer (hereinafter referred to as "framework layer") 22, an Android runtime and system library layer (hereinafter referred to as "system runtime library layer") 23, and a kernel layer 24.

[0039] The application layer 21 includes one or more applications. These applications can be system applications or third-party applications. For example, the application layer 21 includes a first application that can execute the image processing method provided in this embodiment. The framework layer 22 provides an application programming interface (API) and programming framework for the applications in the application layer 21. The system runtime library layer 23 provides support for the upper layer, i.e., the framework layer 22. When the framework layer 22 is used, the Android operating system runs the C / C++ libraries contained in the system runtime library layer 23 to implement the functions to be performed by the framework layer 22. The kernel layer 24 acts as a software middleware between the hardware layer and the application layer 21, used to manage and control hardware and software resources.

[0040] In some embodiments, when a user needs to display frame images in a split manner using multiple projectors 1, the user can set the number of projectors 1 to be displayed in the split-screen application. Then, by setting the position of each projector 1 in the split display, the setting information for each projector 1 is obtained. After obtaining the setting information for each projector 1, the split-screen application sends the setting information to the corresponding projector 1. When the receiving unit 202 receives the setting information sent by the split-screen application, the processing unit 201 of the first application of projector 1 controls the VIDEO layer of the frame layer 22 to switch the video channel to the target channel through the OSD layer of the frame layer 22. The image quality processing module of the frame layer 22 preprocesses the image frames of the target channel, such as PQ processing and Scalar processing, and inputs the preprocessed frame images to the display processing module of the frame layer 22. The display processing module of the frame layer 22 inputs the preprocessed frame images to the background display module of the frame layer 22 for background display. During this period, when the user performs a start operation on the split-screen application to indicate the start of special effects processing, when the receiving unit 202 receives the start information sent by the split-screen projection application indicating the start of special effects processing, the processing unit 201 of the projector 1 controls the VIDEO layer image capture module of the image processing module of the frame layer 22 to capture frame images from the target channel through the VIDEO layer graphics recording control (e.g., copying (or recording) the pre-processed frame image output by the image quality processing module of the frame layer 22, thereby completing the capture of frame images from the target channel). Afterwards, the processing unit 201 of the first application controls the VIDEO layer graphics recording of the image processing module of the frame layer 22 to buffer the frame images captured by the VIDEO layer image capture module of the frame layer 22 into the image frame buffer queue of the image processing module of the frame layer 22. Then, the yuvfilter of the image processing module of the frame layer 22 reads the buffered frame images from the image frame buffer queue and transmits the read frame images to the customfilter of the image processing module of the frame layer 22. The processing unit 201 of the first application of the projector 1 controls the custom filter of the image processing module of the frame layer 22 to perform special effects processing on the frame image, obtaining a segmented image corresponding to the coordinate set on the frame image; the processing unit 201 of the first application of the projector 1 controls the framebuffer of the image processing module of the frame layer 22 to render the segmented image output by the custom filter, obtaining the rendered frame image. Then, the processing unit 201 of the first application of the projector 1 transmits the rendered frame image to the front-end display module of the frame layer 22 through the EGL of the image processing module of the frame layer 22, so that the front-end display module can control the display unit 203 to display the rendered segmented image in the display area.

[0041] In some embodiments, the OSD layer provides a surface for fusion processing, which can perform different special effects processing based on different user needs. The segmented projection application provides the final processed surface for graphic display and switches to the VIDEO channel to be processed in the background, such as the HDMI channel. Customized image special effects processing is performed through the "special effects processing module," such as in fusion display scenarios involving multiple display devices.

[0042] In some embodiments, the target channel includes any one of Multimedia (MM), Digital Television (DTV), and HDMI.

[0043] In some embodiments, the VIDEO layer capture module is used to perform image recording services according to user-defined settings parameters (such as video resolution, frame rate, etc.).

[0044] In some embodiments, VIDEO layer image recording is mainly used to start the image recording service of the VIDEO layer and capture frame images of the VIDEO layer based on the set parameters (such as video resolution, frame rate, etc.), that is, capture frame images from the target channel.

[0045] In some embodiments, the image frame buffer queue mainly serves as a buffer, used to buffer the graphics captured by the VIDEO layer graphics recording and hand them over to the segmented projection application (i.e., the graphics consumer) for special effects processing.

[0046] In some embodiments, OpenGLES image processing involves receiving frame images captured from the target channel stored in the frame buffer queue of externally captured images via a YUV filter. If the frame image is a YUV420 image, a custom filter performs image effects processing on the YUV420 image, such as adding target effects (e.g., dizziness), adding stickers, text, or image blending. Finally, the processed YUV420 image is stored in the framebuffer.

[0047] In some embodiments, EGL is used to bind the Surface set by the split-screen projection application, and finally display the frame image of the OpenGLES image processing onto the Surface. Ultimately, the image with special effects processing can be displayed through the foreground window, i.e., the OSD layer. However, the image displayed by the OSD layer includes both the image quality processing effects of the VIDEO layer and the customized image effects processing effects of the application.

[0048] In some embodiments, special effects processing includes one or more of image cropping and image fusion. It can be seen that the difference from traditional solutions lies in the fact that the image processing method provided in this embodiment adds a VIDEO layer capture module to the VIDEO layer path. This VIDEO layer capture module can simultaneously preserve the image quality processing effects. The main function of the VIDEO layer capture module is to capture images based on user-set parameters such as resolution and frame rate, providing image recording services. This function is relatively simple and easy to implement. The more flexible image special effects processing module is controlled by the segmented projection application, with the capture module only serving as input to the special effects processing module, i.e., a graphics producer.

[0049] Specifically, the storage unit 204 of the projector 1 is used to store data such as computer programs for the image processing method provided in the embodiments of this disclosure.

[0050] In the following embodiments, the execution subject of the image processing method provided in the embodiments of this disclosure is the above-mentioned display device, and the display device is the projector 1, as an example to illustrate the method of the embodiments of this disclosure.

[0051] This disclosure provides an image processing method, as shown in FIG5, which may include S11-S14.

[0052] S11. Upon receiving the setting information sent by the split-screen projection application, switch the video channel to the target channel; wherein, the setting information may include the set of coordinates corresponding to the split image actually displayed by the display device in the frame image, the split image may be a part of the frame image, and the target channel may include any one of the multimedia channel, analog signal channel and high-definition multimedia interface channel.

[0053] In some embodiments, the projector 1 can receive setting information sent by the split-screen application, as well as start information sent by the split-screen application to indicate the start of special effects processing, through an external device interface 240, such as an HDMI high-definition multimedia interface.

[0054] In some embodiments, the target channel may be a channel for transmitting frame images that the user needs to process with special effects, such as any one of a multimedia channel, an analog signal channel, and a high-definition multimedia interface channel.

[0055] In some embodiments, the OSD layer image generally corresponds to the image drawn by the UI and has no image quality effect, while the VIDEO layer image corresponds to the video image played under MM, DTV, HDMI and other channels. It can be subjected to PQ image quality processing, such as Signal to Interference plus Noise Ratio (SNR), Temporal Noise Reduction (TNR), Auto Color Management (ACM) etc., to present a clearer and higher quality image to the user. After being scaled to the display resolution by Scalar, it passes through the display processing module and is superimposed with the OSD layer image. The superimposed signal is then transmitted to the screen for display.

[0056] In some embodiments, the user can set the number of screens to be split and projected on the display interface of electronic device 1, such as 2, 4, 6, 8, etc. Then, the user can set the overlap ratio of the images displayed on each display device within the display area, that is, the proportion of the overlapping image to the actual split image displayed on the display device. For example, when the number of screens to be split and projected is 2, it is assumed that the split image displayed on display device 1 on the left is the left side of the frame image. The segmented image displayed on the right side of display device 2 is the frame image on the right. At this time, the overlap ratio of display device 1 and display device 2 is 50%.

[0057] Assuming the frame image has a length of 640 and a width of 480, and a Cartesian coordinate system is established with the bottom left corner of the frame image as the origin, when the number of screen projections is 2, then assuming the display device 1 on the left displays the segmented image on the left side of the frame image. The segmented image actually displayed by display device 1 has four vertices in the frame image that are (0, 0), (0, 640), and (0, 640), respectively. If (320, 640) is the coordinate set of the segmented image displayed by display device 1, then the set of coordinates corresponding to the region enclosed by the four vertices (0, 0), (0, 640), (320, 0), and (320, 640) is the set of coordinate points. The segmented image actually displayed by display device 2 has the following four vertices in the frame image: Given the vertices (480, 160), (160, 640), and (480, 640), the set of coordinates corresponding to the segmented image displayed by display device 2 is the set of coordinate points enclosed by the area bounded by the four vertices (160, 0), (480, 160), (160, 640), and (480, 640). Thus, it can be seen that the four vertices of the overlapping image displayed by display devices 1 and 2 are (160, 0), (320, 0), (160, 640), and (320, 640). The ratio of the area of ​​the image with vertices (160, 0), (320, 0), (160, 640), and (320, 640) to the area of ​​the segmented image actually displayed by display device 1 is 50%, meaning the overlap ratio of display device 1 is 50%. Meanwhile, the area of ​​the image with four vertices (160, 0), (320, 0), (160, 640), and (320, 640) is 50% of the area of ​​the segmented image actually displayed by the display device 2, that is, the overlap ratio of the display device 2 is 50%.

[0058] In some embodiments, the length of a frame image refers to the number of pixels in the image from top to bottom, while the width of a frame image refers to the number of pixels in the image from left to right.

[0059] In some embodiments, the overlap ratio is greater than or equal to 0.5 and less than or equal to 1.

[0060] S12. Upon receiving the start information sent by the split-screen projection application to indicate the start of special effects processing, capture frame images from the target channel.

[0061] In some embodiments, when a start message indicating the start of special effects processing is received from a split-screen projection application and a frame image is captured from the target channel, the data stream of the target channel can be copied (e.g., through image recording) to obtain the data stream of the target channel. Then, the data stream of the target channel is parsed to obtain the corresponding frame image.

[0062] S13. Perform special effects processing on the frame image to obtain the segmented image corresponding to the coordinate set on the frame image;

[0063] This special effect processing includes at least image cropping.

[0064] In some embodiments, a Cartesian coordinate system can be established with the lower left corner of the frame image as the origin. Then, when cropping the frame image, the region image corresponding to the coordinate set of the display device on the frame image can be determined in the Cartesian coordinate system according to the coordinate set of the display device. The region image in the frame image is then cropped, and the region image is used as the segmented image corresponding to the coordinate set on the frame image.

[0065] In some embodiments, since the actual frame image size is different from the video resolution set by the user, the frame image needs to be cropped so that the image size of the segmented image is the same as the video resolution set by the user.

[0066] In some embodiments, to enrich the aspect ratio of the rendered segmented image, the special effects processing also includes image fusion. This allows for the stitching of images output from the projector, thereby changing the aspect ratio of the stitched image. However, when stitching images output from the projector, to prevent incomplete display, the images need to be overlapped to ensure display integrity. For the overlapping portions of the projector output images, image fusion is required to ensure that the overlapping and non-overlapping portions display identically. Therefore, the parameters also include the overlap ratio.

[0067] In some embodiments, a Cartesian coordinate system is established with the top-left corner of the frame image as the origin, and the coordinates of the four vertices of the frame image are (0, 0), (0, 1), (1, 0), and (1, 1), respectively. When the number of screen segments is 2 and the overlap ratio is 0.6, the regions of the image output from the left projector that are not subject to image fusion (x_left, y_left, w_left, h_left) can be determined, where x_left equals 0, y_right equals 0, and w_left equals 0. x indicates the overlap ratio, and h_left equals 1. The regions of the image output from the projector on the right that are not subject to image fusion (x_right, y_right, w_right, h_right), where x_right equals 1. y is equal to 0 on the right, w is equal to h is set to 1. Then, for the areas requiring image fusion, image parameters (such as brightness and contrast) are adjusted to ensure that the overlapping and non-overlapping parts of the image display identically.

[0068] S14. Render the segmented image to obtain the rendered segmented image, and display the rendered segmented image.

[0069] In some embodiments, the display device needs to have the capability to record images at the VIDEO layer so that the split-screen projection application can control the recording of frame images at the OSD layer and then apply special effects to the recorded VIDEO layer frame images. The split-screen projection application provides user parameter settings, such as the current terminal video display area, the resolution and frame rate of the VIDEO layer video recording, stickers for image effects processing, etc., and provides the Surface that needs to be displayed after the graphics effects processing.

[0070] The split-screen casting application switches to the channel containing the video for which special effects are to be applied, such as the HDMI channel, and mutes the sound. This way, although the HDMI content is playing in the background, the user is unaware of it because the application interface is displayed in the foreground. When the user clicks "start," the special effects are applied, and the user is redirected to a new page for the graphical display of the processed effects.

[0071] First, the image recording graphics reading thread is started, and the "VIDEO layer image recording service" is opened. Based on the user-set frame rate, the recorded video data is cyclically acquired and placed into the image frame buffer queue, and a frame availability signal is sent. For example, if the user-set frame rate is 25fps, then graphics data is read every 40ms.

[0072] Secondly, the graphics rendering thread for image recording is started. It checks if there is YUV data in the image frame buffer queue. If not, it waits for a frame availability signal from the graphics reading thread for image recording. Otherwise, or after receiving the signal, it reads the YUV data from the image frame buffer queue and performs customized effects processing based on the OpenGL ES programmable pipeline. At the same time, it checks if it is the first frame to be rendered. If so, it cancels the sound mute operation.

[0073] Finally, the processed frame image is output to the Surface provided by the segmented projection application via EGL.

[0074] By following the steps above, you can continuously read the graphics data from the graphics frame buffer queue, perform customized effects processing, and finally display it on the screen.

[0075] As described above, the image processing method provided in this embodiment involves the following steps during image segmentation and display: Each display device switches its video channel to the target channel upon receiving setting information from the segmentation and projection application; upon receiving start information from the segmentation and projection application indicating the start of special effects processing, the display device captures a frame image from the target channel; the display device performs special effects processing on the frame image to obtain a segmented image with coordinates corresponding to the frame image; the display device renders the segmented image and displays the rendered segmented image in the display area. In this way, each display device can display the corresponding segmented image on the frame image in its corresponding display area according to the setting information sent by the segmentation and projection application. Furthermore, the rendered segmented images displayed by all display devices can form a complete frame image, eliminating the need for manual pre-setting of the frame image to be displayed for each display device, thereby improving the user experience.

[0076] In some embodiments, as shown in FIG5 and FIG6, the image processing method of this disclosure further includes S15, and the above-mentioned S14 can be specifically implemented by the following S140.

[0077] S15. Upon receiving the settings information sent by the split-screen projection application, switch the video channel to the target channel and mute the output audio of the target channel.

[0078] S140. Render the segmented image. When the segmented image is determined to be the first image frame, unmute the output audio of the target channel and display the rendered segmented image.

[0079] In some embodiments, the effects processing further includes image blending and effects addition, whereby effects addition includes adding target animation effects and adding target effects, where the target effects include one or more of stickers and text.

[0080] In some embodiments, referring to FIG5 and as shown in FIG7, the above-mentioned S14 can be specifically implemented by the following S141 and S142.

[0081] S141. Bind the segmented image to the target component corresponding to the display area, and render the segmented image through the target component to obtain the rendered segmented image;

[0082] S142. Display the rendered segmented image in the target component.

[0083] The target component is the Surface component mentioned earlier.

[0084] Through the above process, when performing image segmentation display, each display device switches its video channel to the target channel upon receiving the setting information sent by the segmentation projection application; upon receiving the start information from the segmentation projection application indicating the start of special effects processing, the display device captures a frame image from the target channel; the display device performs special effects processing on the frame image to obtain the segmented image corresponding to the coordinate set on the frame image; the display device renders the segmented image to obtain the rendered segmented image, and displays the rendered segmented image. In this way, each display device can display the corresponding segmented image on the frame image according to the setting information sent by the segmentation projection application, and the rendered segmented images displayed by all display devices can form a complete frame image. This eliminates the need for manual pre-setting of the frame images to be displayed for each display device, thereby improving the user experience and alleviating the current problems of complex operation processes and poor user interaction when displaying video images on multiple monitors.

[0085] The foregoing primarily describes the solutions provided by the embodiments of this disclosure from a methodological perspective. To achieve the aforementioned functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0086] This disclosure embodiment can divide the image processing device into functional modules according to the above method example. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing unit. The integrated modules can be implemented in hardware or as software functional modules. It should be noted that the module division in this disclosure embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0087] Figure 8 is a schematic diagram of a display device according to some embodiments. As shown in Figure 8, the display device may include an external device interface 101, at least one processor 102, and a display 103.

[0088] Monitor 103;

[0089] External device interface 101 is used to connect the display device and an external device so that the display device can receive multimedia playback content from the external device and display it on the screen.

[0090] At least one processor 102, connected to a display 103 and an external device interface 101, is configured to execute computer instructions to cause the display device to perform the following: upon receiving setting information sent by a split-screen projection application, switching the video channel to the target channel; wherein the setting information includes the coordinate set corresponding to the split image actually displayed by the display device in the frame image, the split image being a portion of the frame image, and the target channel including any one of a multimedia channel, an analog signal channel, and a high-definition multimedia interface channel; upon receiving start information sent by the split-screen projection application indicating the start of special effects processing, capturing a frame image from the target channel; performing special effects processing on the frame image to obtain a split image corresponding to the coordinate set in the frame image; rendering the split image to obtain a rendered split image, and controlling the display 103 to display the rendered split image.

[0091] All relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and their functions will not be repeated here.

[0092] Of course, the display device provided in this embodiment includes, but is not limited to, the modules described above. For example, the display device may also include a memory 104. The memory 104 may be used to store the program code of the display device, and may also be used to store data generated by the display device during operation, such as data in write requests.

[0093] Referring to Figure 4, the receiving unit 202 in the projector 1 performs the same function as the external device interface 101, the processing unit 201 performs the same function as the processor 102, the display unit 203 performs the same function as the display 103, and the storage unit 204 performs the same function as the memory 104.

[0094] As shown in Figure 9, this disclosure also provides a chip system that can be applied to the display device in the foregoing embodiments. The chip system includes at least one processor 1501 and at least one interface circuit 1502. The processor 1501 may be the processor in the aforementioned display device. The processor 1501 and the interface circuit 1502 are interconnected via a circuit. The processor 1501 can receive and execute computer instructions from the memory of the aforementioned display device through the interface circuit 1502. When the computer instructions are executed by the processor 1501, the display device can perform the various steps performed by the display device in the foregoing embodiments. Of course, the chip system may also include other discrete devices, and this disclosure does not specifically limit this.

[0095] This disclosure also provides a computer-readable storage medium for storing computer instructions for operating the aforementioned display device.

[0096] As shown in Figure 10, this disclosure also provides a computer program product, including computer instructions for running the aforementioned display device.

[0097] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A display device, comprising: a display; an external device interface configured to connect the display device with an external device, so that the display device receives multimedia playing content of the external device and presents the content on the display; at least one processor connected with the display and the external device interface, configured to execute computer instructions to cause the display device to perform: switching a video channel to a target channel upon receiving setting information sent by a split screen projection application, wherein the setting information comprises a coordinate set corresponding to a split image in a frame image, the split image being a part of the frame image, and the target channel comprising any one of a multimedia channel, an analog signal channel and an HDMI channel; grabbing a frame image from the target channel upon receiving start information sent by the split screen projection application, the start information indicating start of special effect processing; performing special effect processing on the frame image to obtain a split image of the coordinate set on the frame image; rendering the split image to obtain a rendered split image, and controlling the display to display the rendered split image. 2.The display device of claim 1, wherein the at least one processor is further configured to execute computer instructions to cause the display device to perform: switching a video channel to a target channel upon receiving setting information sent by a split screen projection application, and muting output audio of the target channel; rendering the split image, and when the split image is determined to be a first image frame, unmuting the output audio of the target channel, and controlling the display to display the rendered split image. 3.The display device of claim 1, wherein the at least one processor is further configured to execute computer instructions to cause the display device to perform: binding the segmented image with a target component corresponding to the display area, and rendering the segmented image through the target component to obtain a rendered segmented image; wherein the target component being a basic component for graphic rendering in an Android system; controlling the display to display the rendered split image in the target component. 4.The display device of claim 1, wherein the at least one processor is specifically configured to execute computer instructions to cause the display device to grab a frame image from the target channel upon receiving start information sent by the split screen projection application, the start information indicating start of special effect processing, by: performing a specified copy operation on the target channel to obtain a data stream of the target channel; and parsing the frame image from the data stream; wherein the specified copy operation comprising an image recording operation. 5.The display device of claim 1, wherein the special effect processing comprises image cropping, and the at least one processor is specifically configured to execute computer instructions to cause the display device to perform the special effect processing on the frame image to obtain a split image of the coordinate set on the frame image, by: establishing a rectangular coordinate system with a lower left corner of the frame image as an origin; determining a region image corresponding to the coordinate set on the frame image in the rectangular coordinate system; cropping the region image in the frame image, and taking the region image as the split image of the coordinate set on the frame image.

6. The display device of claim 5, wherein the image size of the split image is the same as a current video resolution of the display device.

7. The display device of any one of claims 1-6, wherein the special effect processing further comprises image fusion and special effect addition, and the special effect addition comprises adding a target animation effect and adding a target special effect, and the target special effect comprises one or more of a sticker and text.

8. An image processing method, comprising: switching a video channel to a target channel upon receiving setting information sent by a split screen projection application, wherein the setting information comprises a coordinate set corresponding to a split image in a frame image, the split image being a part of the frame image, and the target channel comprises any one of a multimedia channel, an analog signal channel, and an HDMI channel; grabbing a frame image from the target channel upon receiving start information sent by the split screen projection application, the start information indicating start of special effect processing; performing special effect processing on the frame image to obtain a split image of the coordinate set on the frame image; rendering the split image to obtain a rendered split image, and displaying the rendered split image.

9. A computer-readable storage medium comprising: A computer program is stored on the computer readable storage medium, and when the computer program is executed by a computing device, the computing device implements the image processing method of claim 8.

10. A computer program product, comprising: A computer program is stored on the computer program product, and when the computer program is executed by a computing device, the computing device implements the image processing method of claim 8.

Images