A screen positioning method, a terminal device, and a display device

Abstract

The present invention discloses a screen positioning method, a terminal device and a display device. The terminal device responds to a customized wallpaper operation, sends indication information to the display device and displays a shooting preview interface; the shooting preview interface includes a positioning wireframe and shooting controls, and the shooting controls are in a prohibited response state when the screen border is not completely within the positioning wireframe; the indication information is used to instruct the display device to display a screen map card, and the screen map card at least includes a first color area and a second color area; according to the color of the background area in the shooting preview interface, a target color area is matched from the first color area and the second color area, and the screen border is positioned and tracked according to the target color area; when it is detected that the screen border is completely within the positioning wireframe, the shooting controls are switched to a responsive state, and a prompt message for prompting the user to trigger the shooting controls to shoot a target image is displayed. This application can quickly and accurately capture the position of the screen border, improving the efficiency of target image acquisition and the effect of the fusion screen.

Description

Technical Field

[0001] The present invention relates to the field of display devices, and in particular, to a screen positioning method, a terminal device, and a display device. Background Art

[0002] In some application scenarios, wallpaper may be pasted on a TV background wall. The wallpaper may be a solid color or have a pattern texture feature. To enhance the visual effect of the screensaver display, a wallpaper image that is the same as the color / pattern of the wallpaper can be customized and displayed. In this way, in the user's view, the TV background wall and the screen wallpaper have a fused and matching effect, that is, a fused screen is achieved.

[0003] The user can use a mobile terminal to capture a target image including the TV screen and the wallpaper within a certain background area around it. After the target image is uploaded to the server, the server identifies features such as the color, texture, and pattern of the wallpaper from the target image, customizes and generates a screen wallpaper, and sends the screen wallpaper to the display device for display. When the mobile terminal captures the target image, a test card is generally displayed on the TV screen. In the shooting preview state, only when the test card is completely within a preset screen positioning wireframe can the mobile terminal respond to the user's operation to capture the target image. Through the test card, the screen border of the TV can be more accurately positioned and identified, and the transitional fusion effect between the customized screen wallpaper and the wallpaper around the screen can be improved.

[0004] During the research and practice process of the applicant, it was found that when the wallpaper is a solid color and is the same as or similar to the color of the test card, no matter how the user adjusts the shooting position and angle of the mobile terminal, the mobile terminal cannot quickly and accurately capture the edge contour of the test card, and thus cannot accurately identify whether the test card is completely within the screen positioning wireframe. This not only results in a long time-consuming for capturing the target image, but also causes problems such as a poor fusion effect between the screen wallpaper and the wallpaper. Summary of the Invention

[0005] To solve the problems existing in the above background art, the present invention provides a screen positioning method, a terminal device, and a display device.

[0006] The terminal device provided by the first aspect embodiment includes:

[0007] A display for displaying an interface related to a customized wallpaper image;

[0008] An image collector for collecting a target image, where the target image includes a screen test card displayed by the display device and a background area outside the screen test card;

[0009] A communicator for communicatingly connecting with the display device;

[0010] A controller configured to execute:

[0011] In response to a custom wallpaper operation input by the user, send indication information to the display device and control the display to show a shooting preview interface; wherein, the shooting preview interface includes a positioning wireframe and shooting controls, and the shooting controls are in a non-responsive state when the screen border is not completely within the positioning wireframe; the indication information is used to instruct the display device to display a screen map card, and the screen map card includes at least a first color area and a second color area;

[0012] Match a target color area from the first color area and the second color area according to the color of the background area in the shooting preview interface, and position and track the screen border of the display device according to the target color area;

[0013] When it is detected that the screen border is completely within the positioning wireframe, switch the shooting controls to a responsive state, and display a prompt message for prompting the user to trigger the shooting controls to shoot the target image.

[0014] Optionally, the controller is configured to position the screen border of the display device in the following manner:

[0015] Pre-generate a preview image according to the shooting preview interface, and extract the target color area from the preview image;

[0016] Perform edge detection and line detection on the target color area in the preview image in sequence to obtain the maximum circumscribed rectangle corresponding to the target color area;

[0017] Obtain the screen border according to the maximum circumscribed rectangle.

[0018] Optionally, the controller is configured to position the screen border of the display device in the following manner:

[0019] Pre-generate an RGB image according to the shooting preview interface and convert the RGB image into an HSV image;

[0020] Extract the target color area from the HSV image;

[0021] Perform binary processing and color inversion processing on the HSV image in sequence to obtain a first image, and the color of the target color area in the first image is converted into black;

[0022] Perform edge detection and line detection on the target color area in the first image in sequence, and obtain the maximum circumscribed rectangle according to each edge segment detected from the first image;

[0023] Obtain the screen border according to the maximum circumscribed rectangle.

[0024] Optionally, the controller is configured to obtain the screen border as follows:

[0025] Expand the maximum circumscribed rectangle into the screen border according to the characteristic information of the screen graphics card;

[0026] Wherein, the characteristic information of the screen graphics card includes the position distribution and ratio information between the first color area and the second color area, and the ratio information includes the area ratio of the two color areas and / or the length ratio of the co-directional edge line segments.

[0027] Optionally, before performing edge detection on the target color area, the controller is further configured to execute:

[0028] Perform a dilation-first and erosion-second process on the first image based on a preset convolution kernel to eliminate the target pixel points in the first image;

[0029] Wherein, the target pixel points include the pixel points in the background area of the first image that are similar to the target color, and the scattered points that appear due to the edge expansion phenomenon caused by the light emission mapping of the display device.

[0030] Optionally, after positioning the screen border, the controller is further configured to execute:

[0031] If it is detected that the screen border is not completely within the positioning wireframe, set the color of the positioning wireframe to the first preset color, and display the first action adjustment information; the first action adjustment information is used to prompt the user to continue adjusting the shooting angle, shooting orientation, and / or the distance from the display device of the image collector;

[0032] If it is detected that the screen border is completely within the positioning wireframe and the screen border matches the positioning wireframe in size, set the color of the positioning wireframe to the second preset color, switch the shooting control to a responsive state, and display the prompt information;

[0033] If it is detected that the screen border is completely within the positioning wireframe but the screen border does not match the positioning wireframe in size, set the color of the positioning wireframe to the first preset color, and display the second action adjustment information, where the second action adjustment information is used to prompt the user to shorten the distance between the image collector and the display device.

[0034] The display device provided by the second aspect embodiment includes:

[0035] A display for displaying a screen graphics card;

[0036] A communicator for communicating with the terminal device;

[0037] A controller, configured to execute:

[0038] In response to receiving the indication information sent by the terminal device, control the display to display a screen card; the indication information is sent by the terminal device after receiving the user input of the customized wallpaper operation;

[0039] Wherein, the screen card at least includes a first color area and a second color area, and the screen card is used to enable the mobile terminal to, after displaying a shooting preview interface in response to the customized wallpaper operation, match a target color area from the first color area and the second color area according to the color of the background area in the shooting preview interface. The shooting preview interface includes a positioning wireframe and a shooting control, and the shooting control is in a prohibited response state when the screen border is not completely within the positioning wireframe;

[0040] The screen card is further used to enable the mobile terminal to locate and track the screen border of the display device according to the target color area, and when it is detected that the screen border is completely within the positioning wireframe, switch the shooting control to a responsive state, and display a prompt message for prompting the user to trigger the shooting control to capture a target image; the target image includes the screen card and the background area outside the screen card.

[0041] Optionally, after displaying the screen card, the controller is further configured to execute:

[0042] Receive a customized wallpaper image generated and sent by the server or the terminal device according to the target image, and control the display to display the customized wallpaper image;

[0043] Or, receive the target image sent by the terminal device, generate a customized wallpaper image according to the target image, and control the display to display the customized wallpaper image.

[0044] Optionally, after receiving the wallpaper image sent by the server, the controller is further configured to execute:

[0045] Save the customized wallpaper image into a wallpaper sequence; the wallpaper sequence is a collection of wallpaper images saved by the display device.

[0046] The screen positioning method in the terminal device provided by the third aspect embodiment includes:

[0047] In response to the user input of the customized wallpaper operation, send indication information to the display device and display a shooting preview interface; wherein, the shooting preview interface includes a positioning wireframe and a shooting control, and the shooting control is in a prohibited response state when the screen border is not completely within the positioning wireframe; the indication information is used to instruct the display device to display a screen card, and the screen card at least includes a first color area and a second color area;

[0048] Match a target color area from a first color area and a second color area according to the color of the background area in the shooting preview interface, and locate and track the screen border of the display device according to the target color area;

[0049] When it is detected that the screen border is completely within the positioning wireframe, switch the shooting control to a responsive state, and display a prompt message for prompting the user to trigger the shooting control to capture a target image; the target image includes the screen card and the background area outside the screen card.

[0050] The screensaver display method in the display device provided by the fourth aspect embodiment includes:

[0051] In response to receiving the indication information sent by the terminal device, display a screen card; the indication information is sent by the terminal device after receiving the user input of the customized wallpaper operation;

[0052] Wherein, the screen card at least includes a first color area and a second color area, and the screen card is used to enable the mobile terminal to match a target color area from the first color area and the second color area according to the color of the background area in the shooting preview interface after the shooting preview interface is displayed in response to the customized wallpaper operation. The shooting preview interface includes a positioning wireframe and a shooting control, and the shooting control is in a non-responsive state when the screen border is not completely within the positioning wireframe;

[0053] The screen card is further used to enable the mobile terminal to locate and track the screen border of the display device according to the target color area, and when it is detected that the screen border is completely within the positioning wireframe, switch the shooting control to a responsive state, and display a prompt message for prompting the user to trigger the shooting control to capture a target image; the target image includes the screen card and the background area outside the screen card.

[0054] The technical solution of the present application is applicable to display devices and terminal devices that support the customized integrated screen wallpaper function. Locating the TV screen border is an important link in the process of customizing and displaying the integrated screen wallpaper, which determines the acquisition efficiency of the target image and the display effect of the integrated screen wallpaper. This link is mainly processed and executed by the terminal device, and the display device focuses on generating and displaying a card that facilitates the terminal to locate the screen border.

[0055] After the user inputs the operation of customizing the wallpaper through the terminal device, the terminal sends an indication message to the display device. After receiving the indication message, the display device displays the screen card. At this time, the terminal device enters the shooting preview interface. The user adjusts the terminal to a suitable position through the shooting preview. The terminal needs to identify whether the screen card is completely within the preset positioning wireframe according to the dynamic position change of the screen card. If so, the shooting condition is met, and the user is prompted to trigger the shooting button, so that the image acquisition device of the terminal acquires the target image, which is convenient for generating the fused screen wallpaper according to the target image subsequently.

[0056] Among them, the screen card displayed on the display device side is no longer a conventional monochromatic card. This application adopts a multi-color card mode, that is, the screen card includes at least a first color area and a second color area. The colors of these two areas are different and have a relative position distribution, such as up and down distribution or left and right distribution, etc. The proportion of the two color areas relative to the card can also be preset. When the wallpaper of the TV background wall is solid color, it may be the same as or similar to the color of the screen card, resulting in the inability to locate the screen border. In this application, through the two-color contrast in the card, the target color with a greater contrast and difference from the wallpaper color is matched from the two colors, and the screen border of the display device is located and tracked according to the target color area. For example, the first color area uses green, and the second color area uses blue. If the background color of the wallpaper is green, blue can be used as the target color; if the wallpaper is purple, green with a more significant color difference from purple can be selected as the target color. After the target color area is determined, since the first color area and the second color area are related in position distribution, the framing area of the screen border can be expanded according to the target color area, that is, the position of the screen border can be located, so as to detect whether the screen border is completely within the positioning wireframe. If so, the shooting control can be triggered to respond, and the user clicks the shooting control to shoot the target image and upload it to the server to customize the wallpaper.

[0057] As can be seen from the above technical solutions, through the color difference contrast of the two-color card in this application, when the background color of the wallpaper is solid color, the position of the screen border can be quickly and accurately captured, so as to accurately detect whether the shooting condition is met, thereby reducing the adjustment action of the user on the camera, improving the acquisition efficiency of the target image, making the acquisition of the target image more accurate, enhancing the fusion effect of the screen wallpaper and the wallpaper, and enhancing the functional experience of the user in customizing the fused screen wallpaper. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required to be accessed in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0059] Figure 1 illustrates a usage scenario of a display device according to some embodiments;

[0060] Figure 2 illustrates a hardware configuration block diagram of a control device 100 according to some embodiments;

[0061] Figure 3 illustrates a hardware configuration block diagram of a display device 200 according to some embodiments;

[0062] Figure 4 illustrates a software configuration diagram in a display device 200 according to some embodiments;

[0063] Figure 5 illustrates an icon control interface display diagram of an application program in a display device 200 according to some embodiments;

[0064] Figure 6 exemplarily shows a schematic diagram of a setting page of a wallpaper application on the display device side;

[0065] Figure 7 exemplarily shows a schematic diagram of a tutorial page when the display device intelligently customizes a fused screen wallpaper;

[0066] Figure 8 exemplarily shows a schematic diagram of an operation instruction page for customizing a fused screen wallpaper on a terminal device;

[0067] Figure 9 exemplarily shows a schematic diagram of a display card interface on the display device side;

[0068] FIG. 10(a) exemplarily shows a schematic diagram of a terminal device displaying a first shooting preview interface;

[0069] FIG. 10(b) exemplarily shows a schematic diagram of a terminal device displaying a second shooting preview interface;

[0070] Figure 11 exemplarily shows a schematic diagram of a terminal device shooting a target image;

[0071] Figure 12 exemplarily shows a schematic diagram of a target image upload interface of a terminal device;

[0072] Figure 13 exemplarily shows a schematic diagram of an interface when a terminal device successfully uploads a target image;

[0073] Figure 14 exemplarily shows a schematic diagram of an interface on the display device side when a terminal device successfully uploads a target image;

[0074] Figure 15 Exemplarily shown is a schematic diagram of a setting page after the display device receives the fused screen wallpaper image;

[0075] Figure 16 Exemplarily shown is a schematic diagram of the display device displaying the fused screen wallpaper image;

[0076] Figure 17 Exemplarily shown is a flowchart of a screen positioning method executed by the terminal device;

[0077] Figure 18 Exemplarily shown is a schematic diagram of the first image obtained after the terminal device preliminarily processes the HSV image;

[0078] Figure 19 Exemplarily shown is a schematic diagram of the image obtained after the terminal device further purifies the first image;

[0079] Figure 20 Exemplarily shown is the effect diagram after the terminal device performs edge detection on the first image;

[0080] Exemplarily shown in FIG. 21(a) is a schematic diagram of the relative position relationship one between the screen border and the positioning wireframe;

[0081] Exemplarily shown in FIG. 21(b) is a schematic diagram of the relative position relationship two between the screen border and the positioning wireframe;

[0082] Exemplarily shown in FIG. 21(c) is a schematic diagram of the relative position relationship three between the screen border and the positioning wireframe;

[0083] Figure 22 Exemplarily shown is a schematic diagram of the extension wireframe;

[0084] Exemplarily shown in FIG. 23(a) is a schematic diagram of the relative position relationship between the screen border and the positioning wireframe detected using the difference area in case A;

[0085] Exemplarily shown in FIG. 23(b) is a schematic diagram of the relative position relationship between the screen border and the positioning wireframe detected using the difference area in case B;

[0086] Exemplarily shown in FIG. 24(a) is a schematic diagram of the relative position relationship between the screen border and the positioning wireframe detected using the difference area in case C;

[0087] Exemplarily shown in FIG. 24(b) is a schematic diagram of the relative position relationship between the screen border and the positioning wireframe detected using the difference area in case D;

[0088] Exemplarily shown in FIG. 24(c) is a schematic diagram of the relative position relationship between the screen border and the positioning wireframe detected using the difference area in case E;

[0089] FIG. 24(d) exemplarily shows a schematic diagram of the relative position relationship between the screen frame and the positioning wire frame by using the difference region detection in case F;

[0090] Figure 25 Exemplarily shows a schematic diagram of dividing small strip regions within each segment of the difference region. Detailed implementation manners

[0091] To make the purpose and implementation manners of this application clearer, the following will clearly and completely describe the exemplary implementation manners of this application in combination with the drawings in the exemplary embodiments of this application. Obviously, the described exemplary embodiments are only a part of the embodiments of this application, rather than all of the embodiments.

[0092] It should be noted that the brief description of the terms in this application is only for the convenience of understanding the subsequent described implementation manners, rather than intending to limit the implementation manners of this application. Unless otherwise specified, these terms should be understood in their ordinary and general meanings.

[0093] The terms "first", "second", "third", etc. in the description, claims and above-mentioned drawings of this application are used to distinguish similar or homogeneous objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that such terms can be interchanged under appropriate circumstances.

[0094] The terms "comprising" and "having" and any variations thereof are intended to cover but not exclusively include. For example, a product or device comprising a series of components does not necessarily have to be limited to all the clearly listed components, but may include other components not clearly listed or inherent to these products or devices.

[0095] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or a combination of hardware or / and software code that can perform functions related to that element.

[0096] Figure 1 Is a schematic diagram of the usage scenario of the display device according to the embodiment. As Figure 1 shown, the display device 200 also communicates with the server 400, and the user can operate the display device 200 through the smart device 300 or the control device 100.

[0097] In some exemplary implementation manners, the control device 100 can be a remote control. The communication between the remote control and the display device includes at least one of infrared protocol communication or Bluetooth protocol communication, and other short-distance communication manners, and controls the display device 200 in a wireless or wired manner. The user can input user instructions to control the display device 200 through at least one of the buttons on the remote control, voice input, control panel input, etc.

[0098] In some exemplary embodiments, the smart device 300 may include any one of a mobile terminal, a tablet computer, a computer, a laptop, an AR / VR device, etc.

[0099] In some exemplary embodiments, the smart device 300 may also be used to control the display device 200. For example, an application running on the smart device is used to control the display device 200.

[0100] In some exemplary embodiments, the smart device 300 and the display device may also communicate data.

[0101] In some exemplary embodiments, the display device 200 may also be controlled in ways other than the control device 100 and the smart device 300. For example, it may directly receive a user's voice command for control through a module for obtaining voice commands configured inside the display device 200, or may receive a user's voice command for control through a voice control device provided outside the display device 200.

[0102] In some exemplary embodiments, the display device 200 also communicates data with the server 400. The display device 200 is allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 may provide various contents and interactions to the display device 200. The server 400 may be a cluster or multiple clusters, and may include one type or multiple types of servers.

[0103] In some exemplary embodiments, the software steps executed by one step execution entity may be migrated to another step execution entity that communicates data therewith for execution as required. Exemplarily, the software steps executed by the server may be migrated to the display device that communicates data therewith for execution, and vice versa.

[0104] Figure 2 An exemplary configuration block diagram of the control device 100 according to an exemplary embodiment is shown. As Figure 2 shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 can receive a user's input operation instruction and convert the operation instruction into an instruction recognizable and responsive by the display device 200, acting as an interaction intermediary between the user and the display device 200.

[0105] In some exemplary embodiments, the communication interface 130 is used for external communication and includes at least one of a WIFI chip, a Bluetooth module, an NFC, or a replaceable module.

[0106] In some exemplary embodiments, the user input / output interface 140 includes at least one of a microphone, a touchpad, a sensor, a button, or an alternative module.

[0107] Figure 3 The hardware configuration block diagram of the display device 200 according to an exemplary embodiment is shown.

[0108] In some exemplary embodiments, the display device 200 includes at least one of a tuner-demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0109] In some exemplary embodiments, the controller includes a central processing unit, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and first to nth interfaces for input / output.

[0110] In some exemplary embodiments, the display 260 includes a display screen component for presenting a picture, and a driving component for driving image display, a component for receiving an image signal output from the controller, and for displaying video content, image content, and a menu control interface, as well as a user control UI interface, etc.

[0111] In some exemplary embodiments, the display 260 can be at least one of a liquid crystal display, an OLED display, and a projection display, and can also be a projection device and a projection screen.

[0112] In some exemplary embodiments, the tuner-demodulator 210 receives a broadcast television signal by wired or wireless reception, and demodulates an audio-video signal from a plurality of wireless or wired broadcast television signals, such as an EPG data signal.

[0113] In some exemplary embodiments, the communicator 220 is a component for communicating with an external device or a server according to various communication protocol types. For example: the communicator can include at least one of a Wifi module, a Bluetooth module, a wired Ethernet module, and other network communication protocol chips or near-field communication protocol chips, and an infrared receiver. The display device 200 can establish the sending and receiving of control signals and data signals with the control device 100 or the server 400 through the communicator 220.

[0114] In some exemplary embodiments, the detector 230 is used to collect signals of the external environment or for external interactions. For example, the detector 230 includes a light receiver, a sensor for collecting the ambient light intensity; alternatively, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures. Or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0115] In some exemplary embodiments, the external device interface 240 may include, but is not limited to, the following: any one or more of a High-Definition Multimedia Interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input / output interface formed by the above multiple interfaces.

[0116] In some exemplary embodiments, the controller 250 and the tuner demodulator 210 may be located in different separate devices, that is, the tuner demodulator 210 may also be in an external device of the main device where the controller 250 is located, such as an external set-top box, etc.

[0117] In some exemplary embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200. For example: in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 can perform operations related to the object selected by the user command.

[0118] In some exemplary embodiments, the object may be any one of the selectable objects, such as a hyperlink, an icon or other operable controls. Operations related to the selected object include: operations such as displaying a page, document, image, etc. connected to the hyperlink, or performing an operation corresponding to the program of the icon.

[0119] In some exemplary embodiments, the controller includes at least one of a Central Processing Unit (CPU), a video processor, an audio processor, a Graphics Processing Unit (GPU), a Random Access Memory (RAM), a Read-Only Memory (ROM), a first interface to an nth interface for input / output, a communication bus (Bus), etc.

[0120] CPU Processor. It is used to execute the operating system and application program instructions stored in the memory, and execute various application programs, data, and content according to various interaction instructions received from external inputs, so as to finally display and play various audio and video contents. The CPU processor may include multiple processors. For example, it includes a main processor and one or more sub-processors.

[0121] In some exemplary embodiments, a graphics processor is used to generate various graphic objects, such as at least one of icons, operation menus, and graphic displays of user input instructions, etc. The graphics processor includes an arithmetic unit that performs arithmetic operations by receiving various interaction instructions input by the user and displays various objects according to display attributes; it also includes a renderer that renders various objects obtained based on the arithmetic unit, and the rendered objects are used to be displayed on the display.

[0122] In some exemplary embodiments, a video processor is used to receive an external video signal and perform at least one of video processing operations such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis according to the standard codec protocol of the input signal, and a signal that can be directly displayed or played on the display device 200 can be obtained.

[0123] In some exemplary embodiments, the video processor includes at least one of a demultiplexing module, a video decoding module, an image synthesis module, a frame rate conversion module, a display formatting module, etc. Among them, the demultiplexing module is used to perform demultiplexing processing on the input audio and video data stream. The video decoding module is used to process the demultiplexed video signal, including decoding and scaling processing, etc. The image synthesis module, such as an image synthesizer, is used to superimpose and mix the GUI signal generated by the graphics generator according to user input or itself with the video image after scaling processing to generate an image signal for display. The frame rate conversion module is used to convert the input video frame rate. The display formatting module is used to change the received video output signal after frame rate conversion to a signal that conforms to the display format, such as outputting an RGB data signal.

[0124] In some exemplary embodiments, an audio processor is used to receive an external audio signal and perform at least one of decompression, decoding, noise reduction, digital-to-analog conversion, and amplification processing, etc. according to the standard codec protocol of the input signal to obtain a sound signal that can be played on the speaker.

[0125] In some exemplary embodiments, if a user enters a user command through a graphical user interface (GUI) displayed on a display 260, the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, if a user enters a user command by inputting a specific sound or gesture, the user input interface receives the user input command by identifying the sound or gesture through a sensor.

[0126] In some exemplary embodiments, a "user interface" is a media interface for interaction and information exchange between an application or an operating system and a user, and it realizes the conversion between the internal form of information and the form acceptable to the user. The common manifestation form of a user interface is a graphical user interface (GUI), which refers to a user interface related to computer operations displayed in a graphical manner. It can be an interface element such as an icon, a window, or a control displayed on the display screen of an electronic device, where the control can include at least one of visible interface elements such as an icon, a button, a menu, a tab, a text box, a dialog box, a status bar, a navigation bar, or a Widget.

[0127] In some exemplary embodiments, the user interface 280 is an interface that can be used to receive control inputs (such as physical buttons on the display device body, or others).

[0128] In some exemplary embodiments, the system of a display device may include a Kernel, a shell (command parser), a file system, and applications. The Kernel, shell, and file system together form the basic operating system structure, which allows users to manage files, run programs, and use the system. After power-on, the Kernel starts, activates the Kernel space, abstracts the hardware, initializes the hardware parameters, etc., and runs and maintains the virtual memory, scheduler, signals, and inter-process communication (IPC). After the Kernel starts, the Shell and user applications are loaded. The application is compiled into machine code after startup to form a process.

[0129] See Figure 4 , in some exemplary embodiments, the system is divided into four layers, from top to bottom: the application layer (abbreviated as the "application layer"), the application framework layer (abbreviated as the "framework layer"), the Android runtime and the system library layer (abbreviated as the "system runtime library layer"), and the Kernel layer.

[0130] In some exemplary embodiments, at least one application runs in the application layer. These applications can be window programs, system setting programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementation, the application packages in the application layer are not limited to the above examples.

[0131] The framework layer provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that decides the actions of the applications in the application layer. Through the API interface, an application can access the resources in the system and obtain the services of the system during execution.

[0132] As Figure 4 shown, in the embodiment of the present application, the application framework layer includes Managers, Content Provider, etc. Among them, the Managers include at least one of the following modules: The ActivityManager is used to interact with all the activities running in the system; the Location Manager is used to provide access to the system location service for system services or applications; the Package Manager is used to retrieve various information related to the application packages currently installed on the device; the NotificationManager is used to control the display and clearing of notification messages; the Window Manager is used to manage the icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0133] In some exemplary embodiments, the ActivityManager is used to manage the life cycles of various applications and the usual navigation back functions, such as controlling the exit, opening, and backward movement of applications. The Window Manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, taking screenshots, and controlling the changes of the display window (such as shrinking the display window, jittering the display, distorting the display, etc.).

[0134] In some exemplary embodiments, the system runtime layer provides support for the upper layer, i.e., the framework layer. When the framework layer is used, the Android operating system runs the C / C++ libraries included in the system runtime layer to implement the functions that the framework layer needs to achieve.

[0135] In some exemplary embodiments, the kernel layer is the layer between hardware and software. As Figure 4As shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0136] In some exemplary embodiments, after the display device is started, it can directly enter the interface of the preset video on demand program. The interface of the video on demand program can be as follows: Figure 5 As shown in FIG, the application layer includes at least a navigation bar 510 and a content display area below the navigation bar 510. The content displayed in the content display area changes according to the selected control in the navigation bar. Applications in the application layer can be integrated into the video-on-demand application and displayed through a control in the navigation bar, or further displayed after the application control in the navigation bar is selected.

[0137] In some exemplary embodiments, after the display device is started, it can directly enter the display interface of the last selected signal source, or the signal source selection interface, where the signal source can be a preset video on demand program, or at least one of an HDMI interface, a live TV interface, etc. After the user selects a different signal source, the display can display content obtained from different signal sources.

[0138] The above description describes the hardware / software architecture and functional implementation of the display device. In some application scenarios, the display device can be installed with a wallpaper application. Users can set the wallpaper theme and display effect in the wallpaper application. They can set preset wallpapers and also support intelligent customization of fusion screen wallpapers based on the TV wall. In this application, wallpapers can be used for screen savers, startup images, and home screen backgrounds.

[0139] In some exemplary embodiments, after the user opens the wallpaper application, he can enter the following Figure 6 An example of a settings page supports users to set a timed screen saver. For example, when the timed screen saver is set to 2 minutes, the display device will start the screen saver when it detects no user input within 2 minutes. Users can also select a screen saver mode, such as album mode or TV wall mode. Album mode is a conventional screen saver mode, while TV wall mode can achieve a fusion match between screen wallpaper and wall wallpaper. For example, when TV wall mode is selected, a wallpaper sequence will be displayed on the page. The wallpaper sequence generally includes the preset wallpaper image of the display device (i.e., the wallpaper that comes with the device when it leaves the factory), the wallpaper image regularly updated and issued by the server corresponding to the wallpaper application (i.e., the wallpaper pushed by the application background), and the user's intelligently customized fusion screen wallpaper image. Figure 6Shown is a case where the wallpaper sequence only includes preset wallpaper images. The wallpaper image with a focus in the wallpaper sequence is in the selected state. When the condition for displaying the screen saver is met, the currently selected wallpaper image is loaded and displayed.

[0140] In some exemplary embodiments, when the user selects the TV wall mode, a matching integrated screen wallpaper image can be intelligently customized according to the current wallpaper and background effect of the TV wall. The settings page of the wallpaper application has a first wallpaper customization control, that is Figure 6 the "AI customization" control in. When the user clicks this control, a wallpaper customization operation (which can be named the first customized wallpaper operation for easy distinction) is input on the display device side. At this time, the display device can jump to a tutorial page as shown in Figure 7 The tutorial page will display information such as the operation process and precautions for guiding the user to intelligently customize the integrated screen wallpaper. In the intelligent customization of the wallpaper in the TV wall mode in this application, the interaction and cooperation of the terminal device are required. The user uses the terminal device to take a photo of the TV wall including the TV screen and reports it to the server for recognition and generation of a matching wallpaper image. This requires that the display device and the terminal device are connected to the same WiFi network. Since there may be multiple WiFi networks set up in the home scene, the WiFi currently connected to the display device can be displayed on the tutorial page, such as TP-LINK_09F8, to facilitate the user to connect to this WiFi on the mobile phone side. The user can cooperate with the intelligent customization of the TV wall wallpaper image through the corresponding target application program of the terminal device. For example, open the JUHAOKAN application and enter the relevant operation interface for customizing the integrated screen wallpaper through the operation path of "My" - "Common Tools" - "TV Wall", so as to complete the wallpaper customization according to the application operation prompts in the terminal device. Figure 7 The tutorial page can display the QR code of the target application program in the mobile terminal. The user scans this QR code with the terminal device to install and use the target application program. In this application, the terminal device includes mobile terminals such as smart phones and tablet computers.

[0141] In some exemplary embodiments, after the user views the tutorial page displayed on the display device, the user can scan the QR code to install the target application program. If the target application program has been installed before, directly access the operation entry for customizing the integrated screen wallpaper, and then the terminal device jumps to the operation instruction page as shown in Figure 8 The operation instruction page can display prompt information, such as "Take a photo of the TV wall with the mobile phone and upload it, and the intelligent recognition will generate a wallpaper that matches the home background". A second wallpaper customization control is set on the operation instruction page, that is Figure 8In "Start Customization", when the user clicks "Start Customization", that is, the user inputs the operation of customizing the wallpaper at the terminal device end (for the convenience of distinction, it can be named the second wallpaper customization operation), the terminal device automatically starts the shooting preview of the camera and jumps to the shooting preview interface. At the same time, the terminal device also needs to send an indication message to the display device. After receiving the indication message, the display device changes the UI from Figure 7 the tutorial page of Figure 9 to the card interface shown in

[0142] In the card interface, the screen card is displayed full screen. Conventional screen cards are generally single solid colors. When the color of the screen card is the same as or similar to the color of the TV wall wallpaper, the terminal cannot quickly and accurately separate the screen card and the background area. To this end, in some exemplary embodiments, Figure 9 shows a card interface on the display device side of the present application. The screen card can adopt a multi-color mode, that is, the screen card at least includes a first color area and a second color area, and the colors of these two areas are different. Through color difference comparison, the defect of a single solid color card can be well solved. The colors used in each color area of the screen card are not limited and can be selected according to actual applications. In addition, the number of color areas in the screen card, the area ratio between each color area, and the relative position distribution are not limited. For the convenience of description, each implementation mode of the present application takes a two-color card as an example.

[0143] In some exemplary embodiments, according to the color of the background area, a target color area can be matched from the first color area and the second color area. Taking the first color area as green and the second color area as blue as an example, if the background area of the TV wall is green, there is a color difference from the second color area of the screen card, then the second color area (blue) is matched as the target color area; if the background area of the TV wall is not green, such as other colors such as pink and blue, then the first color area (green) is matched as the target color area. After matching the target color area, the position of the screen border can be located according to the target color area. Since the relative position of the screen border in the shooting preview interface changes as the user adjusts the orientation, angle, and distance of the terminal device, etc., the screen border of the TV can be located and tracked, so as to determine whether the position of the screen border meets the shooting conditions. This part of the content will be described in detail later.

[0144] In some exemplary embodiments, in addition to two colors, the screen card also has some other characteristic information. The characteristic information at least further includes the position distribution and ratio information between two color areas. There is a certain position distribution between the first color area and the second color area, such as Figure 9In the [figure], the two color regions are presented in an up-and-down distribution, that is, in the form of "two rows and one column". The positional distribution of the two color regions is not limited. For example, it can also be presented in a left-and-right distribution, that is, in the form of "one row and two columns", etc. For the convenience of subsequently expanding the screen border according to the target color region, the positional distribution between the two color regions is preferably not too complex, such as using a simplified up-and-down or left-and-right distribution.

[0145] In some exemplary embodiments, there is a certain proportional relationship between the first color region and the second color region, such as the area ratio of the two color regions, and / or, the length ratio of the co-directional edge line segments, etc. Referring to Figure 9 the positional distribution shown, for example, the area ratio of the two color regions is 1:1. Since the two color regions are green above and blue below and are both rectangles, and the lengths of the two color regions in the horizontal direction are the same, the widths of the two color regions in the vertical direction are also the same. If the target color region is the first color region (green), after extracting the edge contour of the green region, expanding the lower edge line of the green region downward by the same width can locate the lower edge line of the screen border, thereby obtaining the overall contour of the screen border; or for example, the two color regions are green on the left and blue on the right and are both rectangles, and the preset length ratio of the green region to the blue region in the horizontal direction is P. If the target color region is the second color region (blue), since the widths of the two color regions in the vertical direction are the same, after extracting the edge contour of the blue region, expanding the left edge line of the blue region to the left by a length of P*L2 can locate the left edge line of the screen border, thereby obtaining the overall contour of the screen border, where L2 is the length of the blue region and is also the length of the upper or lower edge line of the blue region. Thus, it can be seen that through the positional distribution and proportional relationship between the two-color regions in the screen card, the screen border can be located according to the target color region.

[0146] In some exemplary embodiments, after the user clicks the second wallpaper customization control, the terminal device first displays a first shooting preview interface as shown in FIG. 10(a). The first shooting preview interface includes a rectangular positioning wireframe and shooting controls. The positioning wireframe should be consistent with the aspect ratio of the display device screen. The positioning wireframe is used to position the screen card. Initially, the shooting controls are in a disabled response state. In this state, even if the user clicks the shooting controls, the target image cannot be captured. The first shooting preview interface also displays prompt information for guiding the user's operation, such as "Please adjust the mobile phone to ensure that the screen card displayed on the TV is exactly within the positioning wireframe". After the user views this prompt information, the image collector of the terminal device (usually the rear camera) is aligned with the TV background wall and the TV screen. By adjusting their own body movements, the user can dynamically adjust the orientation, angle, and distance between the terminal camera and the TV screen, so that the screen card in the shooting preview is just completely within the positioning wireframe. The background area outside the boundary line of the positioning wireframe is a partial background area of the TV wall. This part of the background area will carry attribute features such as the color, texture, and pattern of the TV wall, which is convenient for the subsequent server to customize and generate a matching wallpaper image.

[0147] In some exemplary embodiments, after a preset time interval, the preset time interval can be set according to the actual application, such as 5 seconds. The terminal device displays a second shooting preview interface as shown in FIG. 10(b). The prompt information for guiding the user's operation in the second shooting preview interface has changed, such as switching to "Ensure that the screen card is clearly displayed", so that the user knows that when the screen card is completely within the positioning wireframe and the screen card is displayed clearly enough, the condition for capturing the target image is met.

[0148] If the screen card is not within the positioning wireframe or not completely within the positioning wireframe, the shooting controls are unavailable, thus preventing the user from accidentally triggering the shooting controls when the shooting position is not properly positioned, resulting in errors in target image acquisition. Among them, the positioning wireframe can be set in the center of the shooting preview interface or in other positions of the shooting preview interface. The position of the positioning wireframe in the shooting preview interface does not affect the display effect of the final fusion screen.

[0149] In some exemplary embodiments, the terminal device can display other pages in the portrait mode. When the shooting preview interface is displayed, it automatically adjusts to the landscape mode to improve the shooting effect of the target image.

[0150] In some exemplary embodiments, the positioning wireframe in these two shooting preview interfaces is set to a first preset color, such as white. When the user observes that the positioning wireframe is white, it can be known that the screen card is not completely within the positioning wireframe currently, and the terminal camera still needs to be adjusted continuously until the color of the positioning wireframe changes to display a second preset color, such as green, indicating that the screen card has been completely adjusted to be within the positioning wireframe. At this time, the shooting control becomes responsive, and the user can click the shooting control to capture the target image. In this embodiment, through the color of the positioning wireframe, the user can intuitively judge whether the shooting conditions are met, avoiding the user from continuously adjusting the terminal camera when the shooting conditions are met, improving the acquisition efficiency of the target image, and simplifying the user's cumbersome operations.

[0151] In some exemplary embodiments, as Figure 11 shown, the user points the camera of the terminal device at the display device. At this time, the display device shows a two-color screen card, and the periphery of the TV screen / screen card is the TV background wall, which can be a solid color or have texture effects such as patterns and patterns; the terminal device enters the shooting preview at this time. The user adjusts the shooting position, angle, and distance of the terminal by operating their own body, so that the screen card in the shooting preview is just completely located within the positioning wireframe. When the terminal device recognizes that the screen card is completely within the positioning wireframe, it switches the positioning wireframe to the second preset color and changes the shooting control to a responsive state, which can prompt the user to click the shooting control to capture the target image, such as displaying a shooting prompt message of "Please click the shooting button on the right".

[0152] In some exemplary embodiments, after the user clicks the shooting control, the terminal device controls the UI to display a target image upload interface as Figure 12 shown. This interface displays the target image, an upload control, and a reshoot control. The target image includes both the screen card and the background area outside the screen card. After the terminal device captures the target image, if it receives the operation of the user clicking the upload control, the terminal device uploads the target image to the server so that the server can customize the fused screen wallpaper according to the target image; if it receives the operation of the user clicking the reshoot control, it does not upload the target image captured this time, but returns to the shooting preview interface to execute the foregoing shooting process again.

[0153] In some exemplary embodiments, after the terminal device captures the target image, it may not display the Figure 12 shown target image upload interface, but directly uploads the target image to the server by default. In this embodiment, after the user clicks the shooting control, the terminal device automatically uploads the target image to the server and does not support the user to reshoot, so the requirements for the accuracy of screen card positioning and target image acquisition are higher.

[0154] In some exemplary embodiments, after the terminal device captures and uploads a target image to the server, if the upload is successful, the terminal device switches to the interface as shown in Figure 13 . In this interface, information indicating that the customization operation on the terminal device side is completed can be prompted, such as "Completed. Go to the TV side to view the customized wallpaper", indicating that the target image has been uploaded to the server. When the user clicks "Complete", the terminal device ends the process of customizing the fused screen wallpaper. Since the server needs to perform computational processing on the target image and generate a customized fused screen wallpaper image after receiving the target image, the display device side currently shows the waiting and loading effect as shown in Figure 14 . After the display device side receives the fused screen wallpaper image, it can successfully load and display the fused screen wallpaper image.

[0155] In some exemplary embodiments, when the display device receives the fused screen wallpaper image generated and sent by the server, it stores the fused screen wallpaper image. As shown in Figure 15 , the fused screen wallpaper image can be saved to the wallpaper sequence, for example, saved to the first position or other positions in the wallpaper sequence, and at the same time, the fused screen wallpaper image is marked as the selected state. Then, when the condition for displaying the screensaver is met, the display device will display the currently selected fused screen wallpaper image, so that the customized fused screen wallpaper image has a merged effect with the TV background wall. If the intelligent information switch is turned on, the user's preset intelligent information also needs to be superimposed and displayed on the fused screen wallpaper image.

[0156] In some exemplary embodiments, the wallpaper sequence can be sorted according to a certain priority. For example, the fused screen wallpaper image customized by the user intelligently is placed at the first position and the front positions in the wallpaper sequence, and then the preset wallpaper images and the wallpaper images regularly updated and distributed by the server are arranged respectively. That is, the wallpaper customized by the user intelligently has the highest priority, followed by the preset wallpaper and the wallpaper distributed by the server. The display order of the wallpaper sequence is not limited.

[0157] In some exemplary embodiments, when the user clicks the "AI Customization" control, a fused screen wallpaper image that matches the TV background wall can be customized; if the user clicks any wallpaper in the wallpaper sequence, the focus of the selected state is switched, and a preview image of the clicked wallpaper is displayed.

[0158] In some exemplary embodiments, if the display device receives the fused screen wallpaper image, then as shown in Figure 16 , the fused screen wallpaper image is displayed on the screen, and the fused screen wallpaper image may also have a setting control, such as Figure 16 "Go to Settings" in. When the user clicks the setting control on the display interface of the fused screen wallpaper image, they can directly enter Figure 15The shown setting page facilitates users to quickly switch the wallpaper image and adjust the relevant options in the wallpaper settings.

[0159] In some exemplary embodiments, the display device may not receive the composite screen wallpaper image sent by the server due to certain abnormal factors. For example, the server itself fails to successfully generate the composite screen wallpaper image, or the composite screen wallpaper image fails to be sent due to network anomalies. In this case, the display device cannot display the currently customized composite screen wallpaper image. The display device can display a prompt message indicating that the wallpaper image download has failed, such as "The current network is abnormal, and the wallpaper image download has failed. Please check the network and try again." in the display device interface, and a network setting control is also displayed in this interface. When the user clicks "Set Network", the network can be checked and reconnected. After the network returns to normal, if the composite screen wallpaper image can be received, it is displayed on the screen interface. If the composite screen wallpaper image still cannot be received, it may indicate that the failure to obtain the customized wallpaper is caused by non-network factors, and the previous process can be retried to re-customize.

[0160] In some exemplary embodiments, due to factors such as network anomalies, the terminal device may fail to upload the target image to the server. In this case, the terminal device can display a message in the interface to prompt the user that the network is abnormal and ask the user to retry, such as "The network is having problems. Please take the photo again." And a re-shooting control is also set in this interface. When the user clicks the re-shooting control, the terminal device automatically returns to the shooting preview interface. The user can re-shoot the target image and upload it according to the previous process until the target image is successfully uploaded to the server, and the process of customizing the wallpaper on the terminal device ends.

[0161] In some exemplary embodiments, the relevant algorithm process for customizing the composite screen wallpaper image is not limited to being executed on the server side. For example, after the terminal device finishes shooting the target image, it can generate a customized wallpaper image (i.e., the aforementioned composite screen wallpaper image) on the local side according to the target image and the algorithm, and then send the customized wallpaper image to the display device for display; or, after the terminal device finishes shooting the target image, it directly sends the target image to the display device, and the display device generates and displays the customized wallpaper image according to the target image and the algorithm. The customization algorithm for the composite screen wallpaper image is not limited.

[0162] The foregoing embodiments mainly describe in detail the process of intelligent customization of the composite screen wallpaper through communication and interaction among the display device, the terminal device, and the server from the UI perspective, and introduce the attribute characteristics and functions of the two-color screen card. From the above content, it can be seen that the efficiency and accuracy of target image acquisition directly affect the subsequent wallpaper customization efficiency of the server and the fusion effect between the wallpaper and the TV background wall.

[0163] The following content will detail the key aspects of the fusion screen wallpaper customization process from the perspective of the device's control logic, namely how to quickly and accurately locate the screen border of the display device based on a specially designed two-color screen card, and identify whether the shooting conditions of the target image are met according to the screen border and the positioning wireframe, thereby improving the acquisition efficiency of the target image and the customization effect of the fusion screen wallpaper.

[0164] Referring to the foregoing description of UI interaction, Figure 17 A screen positioning method is provided. The method is mainly executed on the terminal device side and includes the following program steps:

[0165] Step S10, in response to a custom wallpaper operation input by the user, send an indication message to the display device and display a shooting preview interface.

[0166] Referring to Figure 8 , when the user clicks the second wallpaper customization control on the terminal interface, the terminal receives the custom wallpaper operation and needs to send an indication message to the display device. After receiving the indication message, the display device controls the display to show Figure 9 the two-color screen card shown. The screen card includes a first color area and a second color area. The two color areas have different colors, and have associated position distributions and proportional relationships. These attribute characteristics of the screen card are preset. After receiving the indication message, the display device can directly retrieve the preset template of the screen card for display.

[0167] At the same time, the terminal device switches to display the shooting preview interface. The shooting preview interface includes a positioning wireframe for positioning the screen border and a shooting control for shooting the target image. When the screen card is not completely within the positioning wireframe, the shooting control is always in a disabled response state, that is, no matter how the user clicks the shooting control, the terminal device cannot receive and respond to the operation of shooting the target image.

[0168] Step S20, according to the color of the background area in the shooting preview interface, match the target color area from the first color area and the second color area, and, according to the target color area, locate and track the screen border of the display device.

[0169] In some exemplary embodiments, the key of the detection mechanism of the present application lies in: generating a current preview image in the shooting preview interface in advance, extracting the target color area from the preview image, and then obtaining the maximum circumscribed rectangle corresponding to the target color area. The actual screen border can be extended through the maximum circumscribed rectangle.

[0170] In some exemplary embodiments, the user adjusts the camera of the terminal device to face the display device and the background wall. The terminal device first pre-generates a preview image according to the shooting preview interface. The preview image is an RGB image including a screen card and a background area. The RGB image is preview-generated for positioning the screen border and is not the target image captured by the user triggering the shooting control. Optionally, for facilitating image processing calculations, the RGB image is converted into an HSV image.

[0171] In some exemplary embodiments, then the target color area is matched according to the color of the background area. Taking the first color area as green and the second color area as blue as an example, if the background area is green, the target color is set to blue, that is, the second color area is matched as the target color area; if the background area is not green, the target color is set to green, that is, the first color area is matched as the target color area. In practical applications, the target color area can be adaptively matched according to the two preset colors in the screen card and the color of the background wall, so that there is a relatively significant color difference contrast between the target color area in the screen card and the background area, thus facilitating the subsequent accurate segmentation of the screen border from the background area.

[0172] In some exemplary embodiments, after determining the target color, it is necessary to identify the target color in the HSV image to extract the target color area. In specific implementation, color extraction can be performed through the inrange algorithm. The three channel values of H, S, and V should simultaneously fall between the minimum and maximum values of the target color in HSV. The inrange algorithm can refer to the prior art and will not be elaborated in this application. Additionally, the color extraction algorithm is not limited to the examples of this application.

[0173] In some exemplary embodiments, after extracting the target color area, the HSV image is binarized, and in the resulting image, the target color area appears as white and other color areas appear as black.

[0174] In some exemplary embodiments, when performing edge detection on the white target color area, the effect may not be good. To improve the edge detection effect and accuracy, optionally, the binarized image is inverted, that is, the colors are inverted, to obtain Figure 18 the first image as shown, where the target color area in the first image is converted to black and other areas are converted to white.

[0175] However, on the one hand, when extracting the target color region in the HSV image, regions similar to the target color may be extracted together. For example, if the target color is green and the overall tone of the background region is light pink, but there are green or colors very similar to green in some local patterns of the background region, then these regions that actually do not belong to the green region in the screen card will also be extracted together. On the other hand, based on the imaging principle of the display device, when its screen emits light, some light will diverge towards the periphery of the screen. Referring to Figure 18 , there are some scattered black dots outside the black region in the first image. These black dots are scattered points caused by the edge expansion phenomenon generated by the light emission mapping of the display device. These points are equivalent to noise. These two factors will cause the target color region in the first image that is not in the screen card to appear black, affecting the accuracy of screen border positioning and recognition.

[0176] In response, in some exemplary embodiments, the first image is processed using a filtering algorithm of first dilation and then erosion based on a preset convolution kernel. First, dilation is performed using white, which can eliminate the target pixel points in the first image. Here, the target pixel points include the pixel points in the background region that are similar to the target color, and the scattered pixel points in the edge expansion region generated by the light emission mapping of the display device, so that these target pixel points that affect the positioning accuracy of the screen card are eliminated. Then, erosion is performed again to obtain the purified first image as shown in Figure 19 . Figure 19 Finally, only the target color region (taking the first color region as an example) matched in the screen card is converted to black, and other regions all appear white, thus eliminating the interference points.

[0177] In some exemplary embodiments, after obtaining the first image, or further performing dilation-erosion purification processing on the first image, canny edge detection is performed on the black region in the first image (i.e., the target color region corresponding to the screen card) to obtain the edge contour of the target color region. Referring to Figure 9 , since the prompt information "Upload your own background according to the mobile phone operation guide" is displayed in the original target color region (taking the first color region as an example), edge segments will be detected at the position where the prompt information is located, and a rectangular edge will also be detected in the target color region. Therefore, the detection result shown in Figure 20 is presented, Figure 20 which includes the prompt information and multiple edge segments corresponding to the target color region.

[0178] In some exemplary embodiments, after obtaining the edge detection result, line detection needs to be further performed. The line detection method is not limited. For example, the Hough transform, LSD (Least Significant Difference), etc. can be used. After the line detection is completed, the coordinate information of each edge segment can be obtained. The coordinate information includes the starting coordinate and the ending coordinate of each edge segment. According to the coordinate information of each edge segment, a maximum circumscribed rectangle corresponding to the target color region is constructed. In a specific embodiment, for example, the minimum abscissa, the minimum ordinate, the maximum abscissa, and the maximum ordinate are selected from the coordinate information of each edge segment. The abscissa is the coordinate along the x-axis direction, and the ordinate is the coordinate along the y-axis direction. In this way, two diagonal points of the maximum circumscribed rectangle can be obtained, that is, the diagonal points in the upper left and lower right positions. Then, based on the rectangle attributes, the unique maximum circumscribed rectangle can be extended, and the rectangle edge contour generated by the prompt information is filtered out. Therefore, the maximum circumscribed rectangle is the actual edge contour of the target color region. Figure 20 In Figure 20 , the abscissa is the coordinate along the x-axis direction, and the ordinate is the coordinate along the y-axis direction. In this way, two diagonal points of the maximum circumscribed rectangle can be obtained, that is, the diagonal points in the upper left and lower right positions. Then, based on the rectangle attributes, the unique maximum circumscribed rectangle can be extended, and the rectangle edge contour generated by the prompt information is filtered out. Therefore, the maximum circumscribed rectangle is the actual edge contour of the target color region.

[0179] In some exemplary embodiments, the region of the maximum circumscribed rectangle is one of the color blocks of the entire screen card. Therefore, according to the preset feature information of the screen card, the maximum circumscribed rectangle can be correspondingly expanded into a complete screen border. Among them, the feature information of the screen card includes the position distribution and ratio information between the first color region and the second color region. The position distribution of the two color regions can be in the form of, for example, arranged vertically or horizontally. The ratio information includes the area ratio of the two rectangular color regions in the screen card, and / or the length ratio of the co-directional (horizontal or vertical) edge segments.

[0180] Taking Figure 20 the maximum circumscribed rectangle in Figure 20 as an example, this maximum circumscribed rectangle is obtained according to the first color region in the screen card. Since the first color region and the second color region are distributed vertically, it is necessary to expand the maximum circumscribed rectangle downward, query the area matching or the width ratio in the vertical direction of the two color regions, and expand the maximum circumscribed rectangle downward according to the ratio to locate the position of the lower edge line of the entire screen border. For example, the area ratio or width ratio of the two color regions is 1:1. According to the coordinate information of the maximum circumscribed rectangle, the width of the maximum circumscribed rectangle can be obtained. Starting from the lower edge line of the maximum circumscribed rectangle as the starting reference, extending a distance equal to the width downward can locate the lower edge line of the screen border, that is, satisfying y2 = y j +w j , where y2 is the ordinate of the lower edge line of the screen border, y j is the ordinate of the lower edge line of the maximum circumscribed rectangle, and w j is the width of the maximum circumscribed rectangle in the vertical direction.

[0181] In some exemplary embodiments, after obtaining the complete screen border according to the maximum circumscribed rectangle, the screen border can be positioned and marked in the shooting preview interface. For example, the edge contour line of the screen border can be highlighted or enhanced in color, so that the user can more clearly and intuitively compare the position deviation between the screen border and the positioning line frame, and thus can more quickly adjust the screen border to completely enter the positioning line frame, improving the efficiency of the user in adjusting the terminal camera.

[0182] When the user adjusts the terminal camera, the relative position of the screen border in the shooting preview interface will change dynamically, while the relative position of the positioning line frame in the shooting preview interface remains fixed. Therefore, by positioning and tracking the position of the screen border of the display device and comparing it with the position of the positioning line frame, it can be determined whether the shooting conditions for the target image are met.

[0183] Step S30, when it is detected that the screen border is completely within the positioning line frame, switch the shooting control to a responsive state, and display a prompt message for prompting the user to trigger the shooting control to capture the target image.

[0184] In some exemplary embodiments, the relative position of the positioning line frame is constant, and the screen border has been positioned through the maximum circumscribed rectangle. Both the screen border and the positioning line frame are rectangles. Therefore, by comparing the coordinate information of the screen border and the positioning line frame, it can be determined whether the screen border completely enters the positioning line frame.

[0185] In some exemplary embodiments, Figure 21(a) to 21(c) The relative position relationship between the screen border and the positioning line frame is shown: Fig. 21(a) shows the case where the screen border is completely within the positioning line frame, that is, the screen border area is a subset of the positioning line frame area; Fig. 21(b) shows the case where only a part of the screen border is within the positioning line frame and the other part is outside the positioning line frame, that is, the screen border area is not a subset of the positioning line frame area, but there is an intersection between the two; Fig. 21(c) shows the case where the screen border is completely outside the positioning line frame, that is, there is no intersection between the screen border area and the positioning line frame area. In Fig. 21(c), the difference in size and position between the screen border and the positioning line frame is relatively large, which is caused by the relatively large distance and relative position deviation between the terminal camera and the display device. Only when the limiting situation of Fig. 21(a) is met can the shooting conditions for the target image be satisfied, the shooting control can be triggered and responded to, and the user is prompted to click the shooting control to complete the shooting task of the target image.

[0186] In some exemplary embodiments, the vertex coordinates of the screen border and the positioning line frame can be compared. Let the four vertex coordinates of the screen border be the upper left (x s1 , y s1 ), the upper right (x s2 , y s1)、Lower left (x s1 , y s2 ) and lower right (x s2 , y s2 ). The four vertex coordinates of the positioning wireframe are upper left (x d1 , y d1 ), upper right (x d2 , y d1 ), lower left (x d1 , y d2 ) and lower right (x d2 , y d2 ).

[0187] Let condition A be x s1 ≥x d1 , condition B be x s2 ≤x d2 , condition C be y s1 ≥y d1 , condition D be y s2 ≤y d2 [[ID=]43], only when A∩B∩C∩D holds can the relative position relationship 1 in Fig. 21(a) be satisfied, so as to detect that the screen border is completely located within the positioning wireframe, switch the color of the positioning wireframe to the second preset color, change the shooting control to a responsive state, and display a shooting prompt message to prompt the user to click the shooting control to shoot the target image.

[0188] If that is, at least one of the above conditions A to D does not hold, it belongs to the relative position relationship 2 in Fig. 21(b) or the relative position relationship 3 in Fig. 21(c). The screen border is not completely located within the positioning wireframe, but partially located within the positioning wireframe or completely located outside the positioning wireframe. Then, the positioning wireframe still maintains the first preset color set initially, the shooting control still maintains the disabled response state, and the first action adjustment information can be displayed in the shooting preview interface. When the user views the first action adjustment information, they know that the current shooting conditions for the target image are not met, and then continue to adjust the shooting angle, shooting orientation, and / or the distance between the terminal and the display device until A∩B∩C∩D holds.

[0189] In some exemplary embodiments, another scheme for detecting the relative position relationship between the screen border and the positioning wireframe is provided. As Figure 22 shown, an extended wireframe is set outside the positioning wireframe, that is, the positioning wireframe is extended by a preset width in the up, down, left, and right directions respectively, and there is a difference area between the obtained extended wireframe and the positioning wireframe. Among them, the preset value can be set according to the actual situation. For example, the preset value is 3 pixels. Since the extended wireframe is extended in the peripheral neighborhood of the positioning wireframe, the difference area can be used to detect the relative position relationship between the screen border and the positioning wireframe.

[0190] In some exemplary embodiments, FIGS. 23(a) and 23(b) illustrate several situations where the target color can be detected within the difference region. Among them, in FIG. 23(a), part of the screen border is within the positioning frame, and the other part is outside the positioning frame. Moreover, there is an intersection between the target color region (the maximum circumscribed rectangle) of the card within the screen border region and the difference region. Therefore, the target color will definitely exist within the difference region, so this belongs to situation A where the shooting conditions are not met. In FIG. 23(b), there is also an intersection between the target color region (the maximum circumscribed rectangle) of the card within the screen border region and the difference region, but the screen border is completely outside the positioning frame. Therefore, this belongs to situation B where the shooting conditions are not met.

[0191] Thus, it can be seen that regardless of which situation of FIGS. 23(a) and 23(b) it belongs to, as long as the target color is detected within the difference region, the screen border will definitely not be completely within the positioning frame. Then, the positioning frame still maintains the initially set first preset color, the shooting control still remains in the disabled response state, and the first action adjustment information can be displayed in the shooting preview interface. When the user views the first action adjustment information, they will know that the current shooting conditions for the target image are not met, and then continue to adjust the shooting angle, shooting orientation, and / or the distance between the terminal and the display device of the terminal camera until the screen border is completely within the positioning frame.

[0192] In some exemplary embodiments, Figure 24(a) to Figure 24(d) illustrate several situations where the target color is not detected within the difference region. Among them, in FIG. 24(a), the card within the screen border includes a first color region and a second color region. One of the two color regions is matched as the target color region, and the other is the non-target color region. If there is no intersection between the target color region and the difference region, but there is an intersection between the non-target color region and the difference region, then although the target color cannot be detected within the difference region, it still belongs to situation C where the shooting conditions are not met. In FIG. 24(b), the screen border is outside the extended border, and there is no intersection between the screen border region and the difference region. Then, although the target color cannot be detected within the difference region, the screen border is outside the positioning frame, belonging to situation D where the shooting conditions are not met.

[0193] As can be seen, for the cases of FIGS. 24(a) and 24(b), although the target color is not detected in the difference region, the screen border does not completely enter the positioning wireframe. Therefore, the positioning wireframe still maintains the first preset color set initially, the shooting control still maintains the disabled response state, and the first action adjustment information can be displayed in the shooting preview interface. The first action adjustment information is, for example, "Please continue to adjust the mobile phone to ensure that the picture card is within this wireframe." When the user views the first action adjustment information, they can know that the relative position deviation of the current picture card is relatively large and does not meet the shooting conditions for the target image. Then, the shooting angle, shooting orientation, and / or the distance between the terminal and the display device of the terminal camera are continuously adjusted until the screen border completely enters the positioning wireframe.

[0194] In some exemplary embodiments, in FIGS. 24(c) and 24(d), the screen border is completely within the positioning wireframe, that is, the screen border region is a subset of the positioning wireframe region, and there is no intersection between the screen border region and the difference region. Therefore, the target color cannot be detected in the difference region. Among them, let the width of the screen border be W1, and the width of the positioning wireframe be W2. W1 is calculated based on the characteristic information of the maximum circumscribed rectangle and the screen picture card, and W2 is a fixed value. Then, the ratio S = W1 / W2 is calculated. When the distance between the terminal camera and the display device changes, the size of the screen border presented in the shooting preview interface will also change accordingly. The smaller the distance between the two, the relatively larger the screen border, and the larger the distance between the two, the relatively smaller the screen border. And the size of the screen border is proportional to the change in the ratio S, that is, the larger the screen border, the larger the ratio S, and the smaller the screen border, the smaller the ratio S. Therefore, the ratio S can be used to measure the relative distance between the terminal camera and the display device.

[0195] In some exemplary embodiments, a threshold T can be preset. The threshold T can be specifically set according to the size deviation between the positioning wireframe and the screen border. For example, the threshold T is 0.9. In FIG. 24(c), the threshold T ≤ S ≤ 1, indicating that while the screen border completely enters the positioning wireframe, the size matching degree between the screen border and the positioning wireframe is higher, and the screen border is closer to the degree of coinciding with the positioning wireframe. When S = 1, the screen border completely coincides with the positioning photo frame, which is the most ideal state. Therefore, when the threshold T ≤ S ≤ 1 is satisfied, it belongs to the case E that meets the shooting conditions. Then, the color of the positioning wireframe is switched to the second preset color, the shooting control is changed to the responsive state, and the shooting prompt information for prompting the user to click the shooting control to shoot the target image is displayed.

[0196] In some exemplary embodiments, in FIG. 24(d), S is less than the threshold T, indicating that the terminal camera is far from the display device. Although the screen border is completely within the positioning wireframe, the screen border is too small and the sizes of the screen border and the positioning wireframe do not match, resulting in a poor wallpaper customization effect. Therefore, when S is less than the threshold T, it belongs to the situation F where the shooting condition is not satisfied. Then, the positioning wireframe still maintains the first preset color set initially, the shooting control still remains in the disabled response state, and the second action adjustment information can be displayed in the shooting preview interface. The second action adjustment information is, for example, "You are far from the TV. Please get closer." When the user views the second action adjustment information, they know that the current terminal device is far from the display device. Then, without changing the shooting angle and orientation of the terminal camera, only shorten the distance between the terminal camera and the display device until the threshold T≤S≤1 is satisfied. It should be noted that for situations E and F, when the screen border is within the positioning wireframe, it is impossible for S to be greater than 1 because once S is greater than 1, there must be some areas of the screen border that exceed the defined range of the positioning border.

[0197] The above embodiments analyze the relative position relationship between the screen border and the positioning wireframe based on the difference area and provide several possible situations. In practical applications, the relative position relationship between the screen border and the positioning wireframe is not limited to those described in the embodiments of the present application, and will not be listed one by one here. For the situations A to F listed above, the specific calculation and control methods of the terminal device are given below.

[0198] In some exemplary embodiments, after obtaining the maximum circumscribed rectangle, based on the maximum circumscribed rectangle and the characteristic information of the screen card, the target edge line of the screen border is positioned, and the width W1 of the screen border is calculated. The target edge line is an extended edge line obtained based on the maximum circumscribed rectangle and the characteristic information of the screen card, and the extended edge line belongs to one of the edge outlines of the screen border.

[0199] For example, when the screen card has green on top and blue below, and the width ratio of the two color areas along the vertical direction is 1:1, and the target color is green, the maximum circumscribed rectangle is the edge outline of the green area in the card. After extending the lower edge line of the maximum circumscribed rectangle downward by the same width, the positioned target edge line is the lower edge line of the screen border, and the width W1 of the screen border is twice the width of the maximum circumscribed rectangle. In this case, the difference areas adjacent to the maximum circumscribed rectangle are the upper, left, and right three segments of the overall difference area.

[0200] For another example, when the screen graphic card uses green on the left and blue on the right, the horizontal length ratio of the two color areas is 1:2, and when the target color is blue, the maximum circumscribed rectangle is the edge contour of the blue area in the graphic card. After expanding the left edge line of the maximum circumscribed rectangle by half of its length to the left, the located target edge line is the left edge line of the screen border, and the width W1 of the screen border is equal to the width of the maximum circumscribed rectangle. In this case, the difference areas adjacent to the maximum circumscribed rectangle are the upper, lower, and right three segments of the overall difference area.

[0201] In some exemplary embodiments, let condition E be "no target color is detected in the difference area adjacent to the maximum circumscribed rectangle", condition F be "the target edge line is inside the positioning wireframe", and condition G be "threshold T ≤ S ≤ 1", where S = width W1 of the screen border / width W2 of the positioning wireframe. Only when E ∩ F ∩ G holds can the relative position relationship in FIG. 24(c) be satisfied, so as to detect that the screen border is completely inside the positioning wireframe and the size matching degree between the screen border and the positioning wireframe is relatively high. Then, the color of the positioning wireframe is switched to the second preset color, the shooting control is changed to a responsive state, and a shooting prompt message for prompting the user to click the shooting control to shoot the target image is displayed.

[0202] In some exemplary embodiments, if that is, at least one of the above conditions E, condition F, and condition G does not hold, it may belong to the relative position relationships in FIGS. 23(a), 23(b), 24(a), 24(b), or 24(d).

[0203] Among them, FIG. 24(d) shows the case where the screen border is inside the positioning wireframe, but the size of the screen border is smaller than that of the positioning wireframe, that is, then the positioning wireframe still maintains the first preset color set initially, the shooting control still maintains the disabled response state, and the second action adjustment information can be displayed in the shooting preview interface. When the user views the second action adjustment information, it is known that the distance between the current terminal device and the display device is relatively far. Then, the distance between the terminal camera and the display device is shortened until E ∩ F ∩ G holds.

[0204] For In other cases, it means that the screen border is not completely within the positioning wireframe. It may be partially within the positioning wireframe or completely outside the positioning wireframe. In this case, the positioning wireframe still maintains the first preset color set initially, the shooting control still remains in the disabled response state, and the first action adjustment information can be displayed in the shooting preview interface. When the user views the first action adjustment information, they can know that the relative position deviation of the current picture card is relatively large and does not meet the shooting conditions of the target image. Then, continue to adjust the shooting angle, shooting orientation, and / or the distance between the terminal and the display device until E ∩ F ∩ G is established. Compared with A ∩ B ∩ C ∩ D, the discrimination condition of E ∩ F ∩ G is more accurate.

[0205] In some exemplary embodiments, the foregoing discrimination rule of A ∩ B ∩ C ∩ D only considers that the screen border is completely within the positioning wireframe and does not consider the size matching between the screen border and the positioning wireframe. Therefore, to improve the accuracy of screen positioning, the discrimination rule of A ∩ B ∩ C ∩ D ∩ G can be set. When A ∩ B ∩ C ∩ D ∩ G is established, that is, it is detected that the screen border is completely within the positioning wireframe and the screen border matches the positioning wireframe in size, the color of the positioning wireframe is set to the second preset color, the shooting control is switched to the responsive state, and a prompt message is displayed to prompt the user to click the shooting control to shoot the target image. When is established, it means that the screen border is completely within the positioning wireframe, but the screen border does not match the positioning wireframe in size and the screen border is smaller. In this case, the color of the positioning wireframe is set to the first preset color, and the second action adjustment information is displayed to prompt the user to shorten the distance between the terminal camera and the display device until A ∩ B ∩ C ∩ D ∩ G is established. For the case where, it means that the screen border is not completely within the positioning wireframe. In this case, the color of the positioning wireframe is set to the first preset color, and the first action adjustment information is displayed to prompt the user to continue to adjust the shooting angle, shooting orientation, and / or the distance from the display device until A ∩ B ∩ C ∩ D ∩ G is established.

[0206] In the foregoing embodiments, when determining whether condition E is established, it is necessary to traverse whether there is a target color in the difference region within the three-way neighborhood of the maximum circumscribed rectangle. For example, when the screen picture card has green on top and blue at the bottom and the target color is green, it is necessary to traverse the upper, left, and right three segments of the difference region.

[0207] In some exemplary embodiments, to simplify the operation, such as Figure 25As shown, according to the detected maximum circumscribed rectangle, small strip regions can be respectively segmented in three of the segmented regions of the difference region. For example, when the screen graphics card adopts the style of green on top and blue at the bottom, if the maximum circumscribed rectangle corresponds to the green region in the graphics card (i.e., the target color is green), small strip regions are set in the upper segmented region, the left segmented region, and the right segmented region of the difference region; if the maximum circumscribed rectangle corresponds to the blue region in the graphics card (i.e., the target color is blue), small strip regions are set in the lower segmented region, the left segmented region, and the right segmented region of the difference region. That is, based on the position distribution of the two-color region in the screen graphics card and the target color matched based on the background color of the video wall, the three segmented regions in the difference region where small strip regions need to be set can be determined.

[0208] In some exemplary embodiments, after detecting the maximum circumscribed rectangle corresponding to the target color region, the size of each small strip region in the three segmented regions can be adaptively set according to the maximum circumscribed rectangle. Figure 25 Taking the screen graphics card as the style of green on top and blue at the bottom and the target color being green as an example, in this scenario, the detected maximum circumscribed rectangle is the upper half of the screen border. Therefore, small strip region 1 is set in the upper segmented region of the difference region, small strip region 2 is set in the left segmented region, and small strip region 3 is set in the right segmented region. Let the short side width of small strip regions 1 to 3 be all W t , the long side length of small strip region 1 be L t1 , and the long side lengths of small strip regions 2 and 3 be L t2 , then W t is equal to the preset value used when constructing the extended wireframe, L t1 = L m * Q1, L t2 = W m * Q2, where L m is the length of the maximum circumscribed rectangle, W m is the width of the maximum circumscribed rectangle, Q1 is the preset proportion of the small strip region in the horizontal direction, Q2 is the preset proportion of the small strip region in the vertical direction, the value ranges of Q1 and Q2 are, for example, from one quarter to three quarters, the values of Q1 and Q2 can be the same or different, and the values of Q1 and Q2 are not limited.

[0209] In some exemplary embodiments, the positions of the small strip regions 1 to 3 within their respective segments can be set. For example, the small strip regions 1 to 3 can be fixedly set at the center of the segment, or the relative positions of the small strip regions within their respective segments can be adaptively adjusted according to the relative position of the maximum circumscribed rectangle, etc. The setting of the relative positions of the small strip regions within the segment is not limited. The detection of whether the screen border is within the positioning wireframe is based on the aforementioned discrimination rule of E∩F∩G and is not affected by the relative positions of the small strip regions within the segment. By setting the small strip regions, when determining whether condition E is satisfied, it is not necessary to traverse the difference regions of the three segments including the neighborhood of the maximum circumscribed rectangle, but only to traverse whether the target color exists within the small strip regions segmented from these three segments, thereby reducing the computational amount of the terminal device and improving the efficiency of screen border positioning and detection.

[0210] It should be noted that after the screen border is located based on a multi-color (i.e., the number of color regions is greater than or equal to 2) color chart, the detection method of the relative position relationship between the screen border and the positioning wireframe is not limited to that described in the embodiments of the present application. Additionally, the customization process of the fusion screen wallpaper is not limited to the description in the UI and related embodiments of the present application. Based on the screen positioning and detection mechanism of the present application for a TV screen, other links in the wallpaper customization process can be adaptively extended or changed. The display device in the present application is not limited to a TV, but also includes other devices with wallpaper customization and display functions that can be used in the fusion screen scenario. The terminal device includes, but is not limited to, mobile terminals such as smart phones and tablet computers that support wallpaper customization and camera functions. Additionally, the image processing algorithms involved in the present application are not limited. When the screen color chart has more than two colors, the screen positioning and detection mechanism is similar to that of the aforementioned two-color color chart, and specific adaptive adjustments can be made based on the factors such as the number of color regions, colors, region ratios, and relative position distributions in the screen color chart on the basis of the screen positioning method for a two-color color chart.

[0211] This application mainly focuses on how to locate the screen border according to the multi-color color chart displayed by the display device and accurately identify the relative position relationship between the screen border and the positioning wireframe during the process of the terminal device capturing a target image after the display device shows a multi-color screen color chart, so as to determine whether the shooting conditions of the target image are met. Through the color difference comparison of the multi-color color chart, when the background color of the wall wallpaper is a solid color, the position of the screen border can be quickly and accurately captured, so as to accurately detect whether the screen border is completely within the positioning wireframe, thereby reducing the adjustment actions of the user on the camera, improving the acquisition efficiency of the target image, making the acquisition of the target image more accurate, enhancing the fusion effect of the screen wallpaper and the wall wallpaper, and improving the functional experience of the user in customizing the fusion screen wallpaper.

[0212] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. In a specific implementation, the present invention further provides a computer storage medium, which can store a program. When the computer storage medium is located in a display device or a terminal device, the program steps involved in the screen positioning method configured to be executed by the controller of any one of the two devices can be included when the program is executed. Among them, the computer storage medium can be a magnetic disk, an optical disk, a read-only memory (abbreviation: ROM), a random access memory (abbreviation: RAM), etc.

[0213] The screen positioning methods executed by each single-end device and its configuration in this application have been listed in the UI interaction and control logic. For the same and similar parts among the embodiments, reference can be made to each other, and the relevant content will not be elaborated. In addition, the UI interfaces of the terminal device and the display device in this application are only exemplary displays, and the specific ones are subject to the actual design and application.

[0214] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

[0215] For the sake of convenience of explanation, the above description has been made in combination with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. According to the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are for better explaining the principles and practical applications, so that those skilled in the art can better use the embodiments and various different modified embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that, Comprising: A display for displaying a user interface; A communicator for communicatively connecting with a server and a terminal device; A controller configured to execute: In response to receiving indication information sent by a terminal device, controlling the display to display a screen card; the indication information is sent by the terminal device after receiving a customized wallpaper operation input by a user; Wherein, the screen card at least includes a first color area and a second color area, and the screen card is used to enable a mobile terminal to, after displaying a shooting preview interface in response to a customized wallpaper operation, match a target color area from the first color area and the second color area according to the color of a background area in the shooting preview interface, the shooting preview interface includes a positioning wireframe and a shooting control, and the shooting control is in a non-responsive state when the screen border is not completely within the positioning wireframe; The screen card is further used to enable the mobile terminal to locate and track the screen border of a display device according to the target color area, and when detecting that the screen border is completely within the positioning wireframe, switch the shooting control to a responsive state, and display a prompt message for prompting the user to trigger the shooting control to capture a target image; the target image includes the screen card and a background area outside the screen card.

2. The display device according to claim 1, wherein After displaying the screen card, the controller is further used to execute: Receiving a customized wallpaper image generated and sent by a server or a terminal device according to the target image, and controlling the display to display the customized wallpaper image; Or, receiving the target image sent by the terminal device, generating a customized wallpaper image according to the target image, and controlling the display to display the customized wallpaper image.

3. The display device according to claim 2, characterized in that, After receiving the wallpaper image sent by the server, the controller is further used to execute: Saving the customized wallpaper image into a wallpaper sequence; the wallpaper sequence is a set of wallpaper images saved by the display device.

4. A terminal device, characterized in that, Comprising: A display for displaying an interface related to a customized wallpaper image; An image collector for collecting a target image, the target image including a screen card displayed by a display device and a background area outside the screen card; A communicator for communicatively connecting with the display device; A controller configured to execute: In response to a customized wallpaper operation input by a user, sending indication information to the display device and controlling the display to display a shooting preview interface; wherein, the shooting preview interface includes a positioning wireframe and a shooting control, and the shooting control is in a non-responsive state when the screen border is not completely within the positioning wireframe; the indication information is used to instruct the display device to display a screen card, and the screen card at least includes a first color area and a second color area; Matching a target color area from the first color area and the second color area according to the color of a background area in the shooting preview interface, and positioning and tracking the screen border of the display device according to the target color area; When detecting that the screen border is completely within the positioning wireframe, switching the shooting control to a responsive state, and displaying a prompt message for prompting the user to trigger the shooting control to capture the target image.

5. The terminal device according to claim 4, wherein The controller is configured to position the screen border of the display device in the following manner: Pre-generate a preview image according to the shooting preview interface, and extract the target color area from the preview image; Perform edge detection and line detection on the target color area in the preview image in sequence to obtain the maximum circumscribed rectangle corresponding to the target color area; Obtain the screen border according to the maximum circumscribed rectangle.

6. The terminal device according to claim 5, wherein The controller is configured to obtain the screen border in the following manner: Expand the maximum circumscribed rectangle into the screen border according to the characteristic information of the screen graphics card; Wherein, the characteristic information of the screen graphics card includes the position distribution and ratio information between the first color area and the second color area, and the ratio information includes the area ratio of the two color areas and / or the length ratio of the co-directional edge line segments.

7. The terminal device according to claim 5, wherein Before performing edge detection on the target color area, the controller is further configured to execute: Perform pre-dilation and post-erosion processing on the first image based on a preset convolution kernel to eliminate the target pixel points in the first image; Wherein, the target pixel points include the pixel points in the background area of the first image that are similar to the target color, and the scattered points that appear due to the edge expansion phenomenon caused by the light emission mapping of the display device.

8. The terminal device according to any one of claims 4 to 7, characterized in that After positioning the screen border, the controller is further configured to execute: If it is detected that the screen border is not completely within the positioning wireframe, set the color of the positioning wireframe to the first preset color, and display the first action adjustment information; the first action adjustment information is used to prompt the user to continue to adjust the shooting angle, shooting orientation and / or the distance from the display device of the image collector; If it is detected that the screen border is completely located within the positioning wireframe and the screen border matches the positioning wireframe in size, set the color of the positioning wireframe to the second preset color, switch the shooting control to a responsive state, and display the prompt information; If it is detected that the screen border is completely located within the positioning wireframe but the screen border does not match the positioning wireframe in size, set the color of the positioning wireframe to the first preset color, and display the second action adjustment information, and the second action adjustment information is used to prompt the user to shorten the distance between the image collector and the display device.

9. A screen positioning method in a terminal device, characterized in that, Including: In response to the customized wallpaper operation input by the user, send an indication message to the display device and display the shooting preview interface; wherein, the shooting preview interface includes a positioning wireframe and a shooting control, and the shooting control is in a prohibited response state when the screen border is not completely within the positioning wireframe; the indication message is used to instruct the display device to display the screen graphics card, and the screen graphics card includes at least a first color area and a second color area; Match the target color area from the first color area and the second color area according to the color of the background area in the shooting preview interface, and position and track the screen border of the display device according to the target color area; When it is detected that the screen border is completely within the positioning wireframe, switch the shooting control to a responsive state, and display a prompt message for prompting the user to trigger the shooting control to capture a target image; the target image includes the screen card and the background area outside the screen card.

10. A screen positioning method in a display device, characterized in that, including: In response to receiving the indication information sent by the terminal device, display a screen card; the indication information is sent by the terminal device after receiving the user input of the custom wallpaper operation; wherein, the screen card at least includes a first color area and a second color area, and the screen card is used to enable the mobile terminal to match a target color area from the first color area and the second color area according to the color of the background area in the shooting preview interface after displaying the shooting preview interface in response to the custom wallpaper operation. The shooting preview interface includes a positioning wireframe and a shooting control, and the shooting control is in a non-responsive state when the screen border is not completely within the positioning wireframe; The screen card is further used to enable the mobile terminal to locate and track the screen border of the display device according to the target color area, and when it is detected that the screen border is completely within the positioning wireframe, switch the shooting control to a responsive state, and display a prompt message for prompting the user to trigger the shooting control to capture a target image; the target image includes the screen card and the background area outside the screen card.

Images