Image Processing Method, Apparatus, Device, and Computer-Readable Storage Medium

Abstract

The present application provides an image processing method, apparatus, device, and computer-readable storage medium; the method includes: in response to a screen capture trigger operation implemented based on a remote control device, performing screen capture processing on the screen displayed by the television device, displaying the screen capture image, and sending the screen capture image to a mobile terminal; wherein, the screen capture image is used for synthesis with the captured image of the mobile terminal. Through the present application, it is possible to implement screen capture-based image processing and interaction of the television device, enhancing the user experience.

Description

Technical Field

[0001] This application relates to image processing technology in the field of the Internet, and particularly to an image processing method, apparatus, device, and computer-readable storage medium. Background Art

[0002] With the wide application of image processing technology, screenshot technology has been widely developed. In a mobile terminal, related technologies can capture the currently displayed screen in the intelligent mobile terminal by triggering corresponding shortcut keys or shortcut key combinations, gesture actions, etc. However, in a television device, related technologies cannot efficiently implement the image processing function based on screenshots, resulting in poor user interaction of the television device and seriously affecting the user experience. Summary of the Invention

[0003] Embodiments of this application provide an image processing method, apparatus, device, computer-readable storage medium, and computer program product, which can implement image processing and interaction based on screenshots of a television device and enhance the user experience.

[0004] The technical solution of the embodiments of this application is implemented as follows:

[0005] Embodiments of this application provide an image processing method, including:

[0006] Responding to a screenshot trigger operation implemented based on a remote control device, performing a screenshot process on the screen displayed by the television device, displaying a screenshot image, and

[0007] Sending the screenshot image to the mobile terminal; wherein, the screenshot image is used for synthesis with a captured image of the mobile terminal.

[0008] Embodiments of this application provide an image processing apparatus applied to a television device, including:

[0009] A screenshot module, configured to respond to a screenshot trigger operation implemented based on a remote control device, perform a screenshot process on the screen displayed by the television device, and display a screenshot image;

[0010] A sending module, configured to send the screenshot image to the mobile terminal; wherein, the screenshot image is used for synthesis with a captured image of the mobile terminal.

[0011] Embodiments of this application provide a television device, and the television device includes:

[0012] A memory, configured to store executable instructions;

[0013] A processor, configured to implement the image processing method provided by the embodiments of this application when executing the executable instructions stored in the memory.

[0014] An embodiment of the present application provides a mobile terminal, which includes:

[0015] A memory for storing executable instructions;

[0016] A processor for implementing the image processing method provided by the embodiment of the present application when executing the executable instructions stored in the memory.

[0017] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for causing a processor to implement the image processing method provided by the embodiment of the present application when executed.

[0018] An embodiment of the present application provides a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the image processing method described above in the embodiment of the present application.

[0019] The embodiment of the present application has the following beneficial effects:

[0020] By using a remote control device to take a screenshot of the picture displayed on a television device, and while displaying the screenshot image on the television device, sending the screenshot image to a mobile terminal to be synthesized with the captured image of the mobile terminal. In this way, the screenshot function is realized on the television device, and the screenshot image of the television device is synthesized with the picture in the mobile terminal, so that image processing and interaction based on screenshots can be realized for the television device, enhancing the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figures 1A - 1B is a schematic structural diagram of an image processing system architecture provided by an embodiment of the present application;

[0022] Figures 2A - 2B is a schematic structural diagram of an image processing device provided by an embodiment of the present application;

[0023] Figures 3A - 3D is a schematic flowchart of an image processing method provided by an embodiment of the present application;

[0024] Figures 4A - 4L is a schematic interface diagram of an image processing method provided by an embodiment of the present application;

[0025] Figures 5A - 5F is a schematic principle diagram of an image processing method provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] To make the objectives, technical solutions and advantages of this application clearer, the following will further describe this application in detail with reference to the accompanying drawings. The described embodiments should not be construed as limitations on this application. All other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of this application.

[0027] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0028] In the following description, the terms "first / second / third" are only used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first / second / third" can be interchanged with a specific order or sequence when permitted, so that the embodiments of this application described here can be implemented in an order other than that illustrated or described here.

[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0030] Before further elaborating on the embodiments of this application, the nouns and terms involved in the embodiments of this application are explained. The nouns and terms involved in the embodiments of this application are subject to the following explanations.

[0031] 1) Telecontrol equipment: Also known as radio remote control equipment, it usually consists of two major parts: a transmitting circuit and a receiving circuit. The transmitting circuit in the telecontrol equipment controls the remotely controlled equipment by transmitting command signals to the remotely controlled equipment. Communication between the telecontrol equipment and the remotely controlled equipment is usually carried out in a wireless connection manner. For example, common telecontrol equipment includes TV remote controls and air conditioner remote controls, and common remotely controlled equipment includes televisions, air conditioners, etc.

[0032] 2) Screen shot: It is a way to capture pictures or text and is also a computer application technology. Through the screen shot technology, one can capture and download pictures of articles of interest from the Internet for one's own use and viewing, which can help people better understand and use knowledge. It is a technology that everyone can use and learn. Moreover, the screen shot function can be achieved through some software.

[0033] 3) TV equipment: For example, Smart TV is a new TV product with a fully open platform and an operating system. Users can install and uninstall various application software while enjoying ordinary TV content, and continuously expand and upgrade functions. TV equipment can continuously bring users a rich and personalized experience that is different from using cable digital TV receivers (set-top boxes).

[0034] 4) Graphic code: Common graphic codes include two-dimensional bar code, bar code, etc. Two-dimensional code uses a certain geometric figure to record data symbol information in black and white distributed in a plane (two-dimensional direction) according to a certain rule; in the code compilation, the concept of "0" and "1" bit streams that constitute the internal logic basis of the computer is cleverly used, and several geometric shapes corresponding to binary are used to represent text numerical information, which is automatically read by image input equipment or photoelectric scanning equipment to realize automatic information processing; it has some common features of bar code technology: each code system has its own specific character set; each character occupies a certain width; it has a certain verification function, etc. At the same time, it also has the function of automatically identifying information in different rows and processing graphic rotation change points. Bar code is a graphic identifier that arranges multiple black bars and spaces of different widths according to certain coding rules to express a group of information.

[0035] 5) Local Area Network: It is a computer communication network that connects various computers, external devices and databases within a local geographical area (such as a school, factory and office), referred to as LAN. It can be connected to a remote local area network, database or processing center through a data communication network or a dedicated data circuit to form a large-scale information processing system.

[0036] With the widespread application of image processing technology, screenshot technology has been widely developed. In mobile terminals, related technologies can capture the current screen displayed in the smart mobile terminal by triggering corresponding shortcut keys or shortcut key combinations, gesture actions, etc. However, in TV devices, related technologies usually cannot realize the screenshot function, and cannot capture the current screen displayed on the TV device and send it to the mobile terminal.

[0037] Moreover, in mobile terminals, the screenshot function that can be achieved by related technologies is relatively simple. If you want to integrate the captured image and the screenshot image together, you often need to use third-party image processing software to integrate them manually. The operation process is relatively complicated, which seriously affects the user experience.

[0038] The functions that can be achieved by the screenshot method in the related art are relatively single, and it is impossible to synthesize the screenshot image of the TV device with the screen in the mobile terminal, resulting in poor user interaction of the TV device and seriously affecting the user experience.

[0039] The embodiments of the present application provide an image processing method, device, equipment, computer-readable storage medium and computer program product, which can realize the image processing and interaction based on screenshots of the TV device and enhance the user experience. The following describes the exemplary applications of the TV device and the mobile terminal provided by the embodiments of the present application. The TV device provided by the embodiments of the present application can be implemented as a smart TV, a smart display, a tablet computer, an in-vehicle TV device, etc. The mobile terminal provided by the embodiments of the present application can be implemented as a mobile smartphone, a mobile wearable device, a tablet computer, a laptop computer, etc. The following will describe the exemplary applications when the TV device and the mobile terminal are implemented as terminals.

[0040] In some embodiments, refer to Figure 1A , Figure 1A FIG. 10 is a schematic diagram of the architecture of the image processing system 100 provided by the embodiments of the present application. The image processing system 100 includes: a mobile terminal 500, a TV device 600, a remote control device 700, a server 800, and a network 400. The mobile terminal 500 and the TV device 600 can be connected to the server 800 through the network 400. The network 400 can be a wide area network or a local area network, or a combination of the two.

[0041] As an example, refer to Figure 1A FIG. 11. A client with a projection function (referred to as a projection client, such as a client for projecting video or images) runs on the mobile terminal 500 and the TV device 600. When it is necessary to project the multimedia content on the mobile terminal 500 onto the TV device 600 for playback, the TV device 600 will obtain the identity information of the mobile terminal 500 and the identifier of the multimedia content to be projected on the mobile terminal 500 from the mobile terminal 500. The remote control device 700 can remotely control the content displayed on the TV device 600 (for example, the remote control device 700 remotely controls the TV device 600 to switch the program being played). The mobile terminal 500 and the remote control device 700 are different terminal devices in the image processing system 100.

[0042] In other embodiments, refer to Figure 1B , Figure 1B FIG. 12 is a schematic diagram of the architecture of the image processing system 200 provided by the embodiments of the present application. The mobile terminal 500 and the TV device 600 are connected to the server 800 through the network 400. The network 400 can be a wide area network or a local area network, or a combination of the two.

[0043] As an example, refer to Figure 1B, an APP in the TV version runs on the television device 600 (for example, a video APP, an electronic photo album APP, an instant messaging client, etc.). When the television device displays a picture independently, the displayed picture of the television device can be a video picture or the displayed picture of any APP. The mobile terminal 500 can remotely control the content displayed on the television device 600 (for example, switch the program being played, switch the running APP). In the image processing system 200, the mobile terminal 500 can implement the function of remotely controlling the television device 600.

[0044] In some embodiments, the server 800 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The mobile terminal 500 and the remote control device 700 can be smart phones, tablet computers, laptop computers, smart speakers, smart watches, vehicle terminals, etc., but are not limited thereto. The mobile terminal 500 and the server 800 can be directly or indirectly connected through wired or wireless communication methods, and there is no limitation in the embodiments of the present application.

[0045] In some embodiments, refer to Figure 2A , Figure 2A is a schematic structural diagram of the mobile terminal 500 provided by the embodiments of the present application. Figure 2A The mobile terminal 500 shown includes: at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. Each component in the mobile terminal 500 is coupled together through a bus system 440. It can be understood that the bus system 440 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clear illustration, in Figure 2A all kinds of buses are labeled as the bus system 440.

[0046] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0047] The user interface 430 includes one or more output devices 431 enabling the presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components facilitating user input, such as keyboards, mice, microphones, touchscreen displays, cameras, and other input buttons and controls.

[0048] The memory 450 can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memories, hard disk drives, optical disc drives, etc. The memory 450 optionally includes one or more storage devices physically located away from the processor 410.

[0049] The memory 450 includes volatile memory or non-volatile memory, and may also include both volatile and non-volatile memory. The non-volatile memory can be a read-only memory (ROM), and the volatile memory can be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0050] In some embodiments, the memory 450 is capable of storing data to support various operations. Examples of such data include programs, modules, and data structures, or subsets or supersets thereof, which are illustratively described below.

[0051] The operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and handling hardware-based tasks;

[0052] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420. Exemplary network interfaces 420 include: Bluetooth, Wireless Fidelity (WiFi), and Universal Serial Bus (USB), etc.;

[0053] The presentation module 453 is used to enable the presentation of information (such as a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 associated with the user interface 430 (such as display screens, speakers, etc.);

[0054] The input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more of the one or more input devices 432.

[0055] In some embodiments, the image processing apparatus provided by the embodiments of the present application may be implemented in software. Figure 2A Shown is an image processing apparatus 455 stored in a memory 450, which may be software in the form of a program and plug-ins, etc., including the following software modules: a screenshot module 4551 and a sending module 4552. These modules are logical, so they can be arbitrarily combined or further split according to the implemented functions. The functions of each module will be described below.

[0056] In some embodiments, referring to Figure 2B , Figure 2B is a schematic structural diagram of a television device 600 provided by the embodiments of the present application. Figure 2B The shown television device 600 includes: at least one processor 660, a memory 650, at least one network interface 620, and a user interface 630. Each component in the television device 600 is coupled together through a bus system 640. It can be understood that the bus system 640 is used to realize the connection and communication between these components. In addition to including a data bus, the bus system 640 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clear illustration, in Figure 2B all kinds of buses are labeled as the bus system 640.

[0057] The processor 660 may be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or any conventional processor, etc.

[0058] The user interface 630 includes one or more output devices 631 capable of presenting media content, including one or more speakers and / or one or more visual display screens. The user interface 630 also includes one or more input devices 632, including user interface components that facilitate user input, such as a keyboard, a mouse, a microphone, a touch screen display, a camera, other input buttons, and controls.

[0059] The memory 650 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memories, hard disk drives, optical disk drives, etc. The memory 650 optionally includes one or more storage devices that are physically located away from the processor 660.

[0060] The memory 650 includes volatile memory, non-volatile memory, or both volatile and non-volatile memory. The non-volatile memory can be a read-only memory (ROM), and the volatile memory can be a random access memory (RAM). The memory 650 described in the embodiments of this application is intended to include any suitable type of memory.

[0061] In some embodiments, the memory 650 is capable of storing data to support various operations. Examples of such data include programs, modules, and data structures, or subsets or supersets thereof, which are illustrated below.

[0062] The operating system 651 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, the core library layer, the driver layer, etc., for implementing various basic services and handling hardware-based tasks;

[0063] The network communication module 652 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 620. Exemplary network interfaces 620 include: Bluetooth, Wi-Fi (Wireless Fidelity), and USB (Universal Serial Bus), etc.;

[0064] The presentation module 653 is used to enable the presentation of information (such as a user interface for operating peripheral devices and displaying content and information) via one or more output devices 631 associated with the user interface 630 (such as a display screen, a speaker, etc.).

[0065] The input processing module 654 is used to detect and translate one or more user inputs or interactions from one of one or more input devices 632.

[0066] In some embodiments, the image processing device provided in the embodiments of this application can be implemented in software. Figure 2B An image processing device 655 stored in the memory 650 is shown, which can be software in the form of programs and plugins, etc., and includes the following software modules: a receiving module 6551, a display module 6552, and a generating module 6553. These modules are logical, so they can be combined arbitrarily or further split according to the functions to be implemented. The functions of each module will be described below.

[0067] In some other embodiments, the image processing apparatus provided in the embodiments of the present application may be implemented in a hardware manner. As an example, the image processing apparatus provided in the embodiments of the present application may be a processor in the form of a hardware decoding processor, which is programmed to execute the image processing method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor may adopt one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs) or other electronic components.

[0068] In some embodiments, a mobile terminal or a television device may implement the image processing method provided in the embodiments of the present application by running a computer program. For example, the computer program may be a native program or a software module in an operating system; it may be a local (Native) application (APP), that is, a program that needs to be installed in the operating system to run, such as a live broadcast APP or an instant messaging APP; it may also be a small program, that is, a program that only needs to be downloaded to a browser environment to run; it may also be a small program that can be embedded in any APP. In short, the above computer program may be any form of application program, module or plug-in.

[0069] The image processing method provided in the embodiments of the present application will be described in conjunction with the exemplary applications and implementations of the television device provided in the embodiments of the present application.

[0070] See Figure 3A , Figure 3A is a schematic flowchart of the image processing method provided in the embodiments of the present application, which will be described in conjunction with Figure 3A the steps 101 to 103 shown.

[0071] In step 101, the television device performs a screenshot process on the screen displayed by the television device in response to a screenshot trigger operation implemented based on a remote control device.

[0072] In some embodiments, the remote control device is also known as a radio remote control device. Generally, the remote control device consists of two major parts: a transmitting circuit and a receiving circuit. The transmitting circuit in the remote control device controls the device to be remotely controlled by transmitting command signals to it, and wireless connection is usually adopted for communication between the remote control device and the device to be remotely controlled. In the embodiments of the present application, the device to be remotely controlled is a television device, that is, the remote control device is used to remotely control the television device.

[0073] In some embodiments, the remote control device is a dedicated remote control; or, the mobile terminal and the remote control device are the same device, and the mobile terminal is used to remotely control the television device by transmitting wireless signals; or, the remote control device is another mobile terminal different from the mobile terminal, and the other mobile terminal controls the television device based on wireless control signals (for example, infrared signals, Bluetooth signals, ultrasonic signals).

[0074] In some embodiments, the screen capture trigger operation can be a pressing operation on at least one button in the remote control device, or a somatosensory operation on the remote control device (for example, detecting a user gesture through a somatosensory device in the remote control device, such as the somatosensory device detecting that the user gesture is a upward wave), or a touch operation, voice control operation, etc. on the corresponding screen capture button in the remote control device.

[0075] As an example, see Figure 4A , Figure 4A which is a schematic diagram of the interface of the image processing method provided by the embodiments of the present application. By controlling the content 610 displayed on the television device through the remote control device, the television device performs a screen capture process on the displayed screen in response to the screen capture trigger operation implemented based on the remote control device, and obtains a screen capture image 601.

[0076] In this way, the screen capture process of the screen displayed on the television device is completed through the remote control device, enabling the user to capture video frames at any time according to their own needs when watching videos through the television device, meeting the user's personalized needs for screen capture of the television device.

[0077] In step 102, the television device displays the screen capture image.

[0078] As an example, see Figure 4E , Figure 4E which is a schematic diagram of the interface of the image processing method provided by the embodiments of the present application, and the screen capture image 601 can be displayed on the display interface of the television device.

[0079] In step 103, the television device sends the screen capture image to the mobile terminal; wherein, the screen capture image is used for synthesis with the captured image of the mobile terminal.

[0080] As an example, the TV device sends a screenshot image to the mobile terminal. After receiving the screenshot image, the mobile terminal further synthesizes the received screenshot image with the captured image of the mobile terminal to obtain the final synthesized image.

[0081] In some embodiments, referring to Figure 3B , Figure 3B is a schematic flowchart of the image processing method provided by an embodiment of the present application; as Figure 3B shown, before the above step 101, that is, before the TV device responds to the screenshot trigger operation implemented based on the remote control device, the TV device can display a graphic code through step 104.

[0082] In step 104, when the automatic connection condition is met, the TV device displays a graphic code, which is used for the mobile terminal to scan and establish a communication connection with the TV device.

[0083] In some embodiments, the automatic connection condition includes at least one of the following: the TV device is powered on, the TV device is awakened from the sleep mode, the TV device plays an interesting program, the TV device plays a program that has been screenshot.

[0084] As an example, the TV device can display a graphic code when the automatic connection condition is met before receiving the screenshot trigger operation implemented based on the remote control device, where the graphic code is used for the mobile terminal to scan and establish a communication connection with the TV device.

[0085] As an example, the TV device can display a graphic code after receiving the screenshot trigger operation implemented based on the remote control device and before responding to the screenshot trigger operation implemented based on the remote control device, where the graphic code is used for the mobile terminal to scan and establish a communication connection with the TV device.

[0086] As an example, when the TV device is in the powered-on state or the TV device is awakened from the sleep mode, it indicates that the TV device can operate normally at this time and execute corresponding functions (for example, the playback function, the signal reception function). At this time, a corresponding graphic code is displayed on the display interface of the TV device for the mobile terminal to scan and establish a connection with the mobile terminal.

[0087] As an example, when the TV device is playing a program that the user is interested in, it indicates that the possibility of the user taking a screenshot is relatively high (it can be understood that the user takes a screenshot of a certain frame in the interesting program to obtain the video picture that the user is interested in). At this time, a corresponding graphic code is displayed on the display interface of the TV device to meet the possible screenshot requirements of the user. When the user needs to take a screenshot, the user can scan the graphic code on the display interface of the TV device through the mobile terminal to establish a connection with the mobile terminal.

[0088] In some embodiments, the program of interest may be determined by the television device according to the user's historical usage habits or historical behavior portrait. The program of interest may also be a program subscribed by the user according to his own interests (that is, the subscribed program can push the corresponding prompt message to the user each time the program (program episode) is updated, or the subscribed program is placed at the top of the program list for easy viewing by the user). The program of interest may also be a program whose broadcast period exceeds the period threshold (for example, the broadcast period of program A is 10 episodes, the broadcast period of program B is 15 episodes, and the period threshold is 12 episodes, then program B is a program of interest). The program of interest may also be a program whose broadcast duration exceeds the duration threshold (for example, the broadcast duration of program A is 102 minutes, the broadcast duration of program B is 150 minutes, and the duration threshold is 120 minutes, then program B is a program of interest). The program of interest may also be a shared program (for example, program A has been shared by the user to his relatives and friends through a mobile terminal or television device, and program B has not been shared by the user to his relatives and friends through a mobile terminal or television device, then program A is a program of interest).

[0089] As an example, when a TV device is playing a program that has been screenshotted before, it also indicates that the user is likely to need to take a screenshot (it is understandable that the user takes a screenshot of a frame in a program that has been screenshotted to obtain the video screen of interest). At this time, the corresponding graphic code is displayed on the display interface of the TV device to meet the user's possible screenshot needs. When the user needs to take a screenshot, the user can scan the graphic code displayed on the display interface of the TV device through the mobile terminal to establish a connection with the mobile terminal.

[0090] In some embodiments, the television device can record each time a program is screenshotted and generate a historical screenshot program list. When the television device is playing a program, the currently playing program is matched with the programs in the historical screenshot program list. When a program identical to the currently playing program exists in the historical screenshot program list, the currently playing program is determined to be a program that has been screenshotted.

[0091] As an example, see Figure 4A , Figure 4A 602 is a schematic diagram of an interface of an image processing method provided by an embodiment of the present application. When the automatic connection condition is met, a graphic code 602 is displayed in the display interface of the television device, and the graphic code 602 is used for the mobile terminal to scan and establish a communication connection with the television device.

[0092] In this way, when the connection conditions are met, a graphic code is displayed for the mobile terminal to scan and establish a communication connection with the television device, thereby effectively avoiding the situation where the graphic code is always displayed in the human-computer interaction interface of the television device to affect the user's viewing experience.

[0093] In some embodiments, the graphic code is used for a mobile terminal to scan and identify the address and port of the TV device in the local area network, and establish a communication connection with the TV device in the local area network according to the address and port.

[0094] As an example, refer to Figure 4A , the mobile terminal scans the graphic code 602 displayed on the display interface of the TV device, so that the mobile terminal can identify the address and port of the TV device in the wireless local area network, and then establish a communication connection with the TV device in the local area network according to the address and port of the TV device in the local area network. For example, the communication connection method between the TV device and the mobile terminal can be a socket connection. A socket is an abstraction of the endpoints for two-way communication between application processes on different hosts in the network. A socket is one end of the communication between processes on the network, providing a mechanism for application layer processes to exchange data using network protocols. In terms of its position, a socket is connected to the application process above and the network protocol stack below. It is the interface for the application program to communicate through the network protocol and the interface for the application program to interact with the network protocol root.

[0095] In some embodiments, before the TV device receives the screenshot trigger operation implemented based on the remote control device in the above step 101, or while the TV device displays the graphic code in the above step 104. The TV device broadcasts or (in the group participated by the TV device) multicasts connection information based on the device discovery service, and the connection information includes the address and port of the TV device in the local area network; wherein, the connection information is used for the mobile terminal to resolve the address and port, and establish a communication connection with the TV device in the local area network based on the address and port.

[0096] As an example, refer to Figure 5A , Figure 5A is a schematic diagram of the principle of the image processing method provided in the embodiments of the present application. The TV device 600 sends connection information to the server 800 in a broadcast or multicast manner based on the device discovery service (mDNS service). The mobile terminal 500 obtains the connection information from the server 800 based on the device discovery protocol (searching for mDNS), and parses the obtained connection information based on the device discovery protocol, so as to resolve the address and port of the TV device included in the connection information in the local area network. Thus, the mobile terminal 500 establishes a communication connection with the TV device 600 in the local area network based on the address and port. As Figure 5A shown, the mobile terminal 500 and the TV device 600 complete private protocol interaction by establishing a connection.

[0097] Among them, the device discovery protocol is also known as the Multicast DNS protocol (mDNS). The mDNS protocol mainly enables hosts within a local area network to discover and communicate with each other without a traditional DNS server. It uses port 5353, complies with the DNS protocol, and uses the existing DNS information structure, name syntax, and resource record types. And no new operation codes or response codes are specified. The device discovery service provides services for the device discovery protocol.

[0098] In some embodiments, the TV device broadcasts connection information based on the device discovery service, or multicasts connection information in a group to which the TV device belongs. Among them, broadcasting refers to data transmission in a multi-point delivery manner. Multicast, also known as multi-casting, the original intention of multicast technology is to send information to a certain target group in the form of "best effort" in an IP network. This target group is called a multicast group. In this way, when a source host sends an information request to multiple target hosts, the source host only sends one copy of the data, and the destination address of the data is the multicast group address. In this way, all members belonging to this group can receive a copy of the data sent by the source host. In the multicast mode, only the members who really need the information will receive the information, and other hosts will not receive the information.

[0099] In some embodiments, before the TV device responds to the screen capture trigger operation implemented based on the remote control device in step 101 above, the screen capture trigger operation can be determined in the following way: obtain the sensitivity configuration information of the screen capture function from the server; among them, the sensitivity configuration information includes the number of presses and the pressing interval time of the screen capture trigger operation for at least one key; receive the wireless control signal sent by the remote control device; among them, the wireless control signal is sent by the remote control device in response to the pressing operation for at least one key, and includes the number of presses and the pressing interval time of the pressing operation; when the number of presses of the pressing operation is the same as the number of presses included in the configuration information, and the pressing interval time of the pressing operation is the same as the pressing interval time included in the configuration information, determine that the pressing operation is the screen capture trigger operation.

[0100] As an example, see Figure 5A , the TV device 600 obtains the sensitivity configuration information of the screen capture function from the server 800. Among them, the screen capture configuration information includes the number of presses and the pressing interval time of the screen capture trigger operation for at least one key. For example, it can be one press or multiple consecutive presses of at least one key in the remote control device, or pressing 2 or more keys in sequence or simultaneously.

[0101] As an example, a television device receives a wireless control signal sent by a remote control device. For example, the wireless control signal can be an infrared signal, an ultrasonic signal, etc. The remote control device sends the wireless control signal to the television device in response to a pressing operation on at least one button. After receiving the wireless control signal, the television device determines whether the number of presses and the pressing interval time of the pressing operation included in the wireless control signal are the same as the number of presses and the pressing interval time in the configuration information respectively. If both the number of presses and the pressing interval time are the same, it is determined that the pressing operation on at least one button of the remote control device is a screenshot trigger operation.

[0102] In this way, by obtaining the sensitivity configuration information and then determining whether the pressing operation on at least one button of the remote control device is a screenshot trigger operation according to the sensitivity configuration information, it effectively avoids misidentifying the pressing operation sent by the remote control device as a screenshot trigger operation, and effectively improves the accuracy of the screenshot trigger operation.

[0103] In some embodiments, in step 101 above, the screenshot processing of the screen displayed by the television device can be implemented in the following manner: call the first screenshot interface of the operating system of the television device to perform screenshot processing on the screen displayed by the television device; wherein, the first screenshot interface is an interface opened by the operating system to external applications; when the call to the first screenshot interface fails, perform reflection processing on the second screenshot interface of the operating system of the television device, and call the reflected second screenshot interface to perform screenshot processing on the screen displayed by the television device; wherein, the second screenshot interface is an interface that the operating system of the television device does not open for external application calls.

[0104] As an example, see Figure 5B , Figure 5B is a schematic diagram of the principle of the image processing method provided in the embodiments of the present application. Call the first screenshot interface of the operating system of the television device to perform screenshot processing on the screen displayed by the television device. When the call to the first screenshot interface fails and when the number of calls exceeds the retry count threshold, perform reflection processing on the second screenshot interface of the operating system of the television device, call the reflected second screenshot interface to perform screenshot processing on the screen displayed by the television device, and save the screenshot image obtained by the screenshot processing. When the call to the first screenshot interface fails and when the number of calls does not exceed the retry count threshold, continue to call the first screenshot interface of the operating system of the television device to perform screenshot processing on the screen displayed by the television device (i.e., retry). When the call to the first screenshot interface is successful, save the screenshot image obtained by the screenshot processing. When the call to the reflected second screenshot interface fails, prompt a screenshot error message on the screen displayed by the television device to notify the user that there is an error in taking the screenshot.

[0105] In some embodiments, reflection processing is in the running state, for any class, all the properties and methods of this class can be known; for any object, any method and property of it can be called. This processing method of dynamically obtaining information and dynamically calling objects is called reflection processing.

[0106] In this way, the screen displayed by the TV device is captured by calling the interface opened to external applications and the interface not opened to external applications by the operating system of the TV device, thereby providing double protection for the normal progress of the screenshot processing and ensuring that the screenshot processing can be carried out normally.

[0107] In some embodiments, the screenshot processing of the picture displayed by the TV device in the above step 101 can be achieved in the following way: the TV device takes a screenshot of the picture displayed by the TV device at the operation time point according to the operation time point of the screenshot trigger operation; wherein the screenshot area of the picture is the entire area of the picture, or a partial area set by the screenshot operation.

[0108] As an example, see Figures 4F - 4G , Figures 4F - 4G is an interface diagram of the image processing method provided by the embodiment of the present application. According to the operation time point of the screenshot trigger operation, the screen displayed by the television device at the operation time point is screenshot processed, for example, see Figure 4F and Figure 4G , the operation time point of the screenshot trigger operation is 21:52:15, and the screen displayed by the television device at the operation time point 21:52:15 is screenshotted. Moreover, the screenshot area of the screen can be the entire screen area 601, and the screenshot area of the screen can be a partial screen area 610. Figure 4F , when the captured area of the screen is the entire area 601 of the screen, "Issue 10: Champion..." is displayed in the captured area; see Figure 4G When the screenshot area of the picture is the partial area 610 of the picture, “Issue 10: Champion…” is not displayed in the screenshot area.

[0109] In this way, by taking a screenshot of the picture displayed by the TV device at the operation time point according to the operation time point of the screenshot trigger operation, it is ensured that the operation time point is consistent with the display time of the screenshot picture, and it is ensured that the screenshot picture is the picture that the user needs to capture, thereby effectively avoiding the inaccuracy of the screenshot image caused by operation delay.

[0110] In some embodiments, the screenshot processing of the screen displayed by the TV device in step 101 above can be implemented in the following manner: Determine a screenshot time window according to the operation time point of the screenshot trigger operation, where the screenshot time window starts, has its midpoint, or ends at the operation time point; Perform screenshot processing on multiple screens displayed by the TV device within the screenshot time window to correspondingly obtain multiple screenshot images.

[0111] In some embodiments, if the screenshot time window starts at the operation time point, the screenshot image within the screenshot time window can be the screen to be displayed in the TV device, that is, the screen that the TV device is about to display. If the screenshot time window has its midpoint at the operation time point, the screenshot image within the screenshot time window can be the screen to be displayed and the previously (historically) displayed screens in the TV device. If the screenshot time window ends at the operation time point, the screenshot image within the screenshot time window can be the previously (historically) displayed screens in the TV device.

[0112] As an example, refer to Figure 4H , Figure 4H is a schematic interface diagram of the image processing method provided by the embodiments of the present application. If the screenshot time window ends at the operation time point, the screenshot image within the screenshot time window can be the previously (historically) displayed screens in the TV device. Determine a screenshot time window with the operation time point as the end point according to the operation time point of the screenshot trigger operation, and perform screenshot processing on multiple screens displayed by the TV device within the screenshot time window to correspondingly obtain multiple screenshot images 611.

[0113] As an example, when the screenshot time window starts at the operation time point, the screenshot image within the screenshot time window can be a series of screens to be displayed in the TV device, so that multiple screenshot processes can be performed on a series of screens to be displayed in the TV device to obtain multiple screenshot images.

[0114] In some embodiments, function tabs with the operation time point as the start point, midpoint, and end point can be displayed in the display screen of the TV device, and in response to a selection operation for at least one function option in the function tabs for the start point, midpoint, and end point, set the corresponding screenshot time window.

[0115] In this way, through the setting of the screenshot time window, continuous multiple screenshot processes can be performed on the screen displayed by the TV device to obtain multiple screenshot images, thus effectively avoiding the situation of missing screenshot images and effectively improving the effect of screenshot processing.

[0116] In other embodiments, after the TV device obtains multiple screenshot images, all the obtained multiple screenshot images can be sent to the mobile terminal.

[0117] In other embodiments, after the above-mentioned television device obtains multiple screenshot images, the screenshot image to be sent to the mobile terminal can also be determined in the following manner: in response to a selection operation on multiple screenshot images, the selected screenshot image from the multiple screenshot images is marked as a screenshot image to be sent to the mobile terminal.

[0118] In this way, by setting the screenshot time window, the screen displayed on the TV device can be captured multiple times continuously to obtain multiple screenshot images, and then selection can be made among the multiple screenshot images to determine the screenshot image required by the user, and the selected screenshot image can be sent to the mobile terminal, thereby effectively avoiding the omission of the screenshot image.

[0119] In some embodiments, the above-mentioned multiple screenshot images are pictures that meet the filtering conditions among all the pictures displayed within the screenshot time window, and the filtering conditions include at least one of the following: the display ratio of the foreground to the background in the screenshot image is greater than the display ratio threshold; the number of objects in the foreground of the screenshot image is greater than the number threshold; the posture angle of the object in the foreground of the screenshot image is within the set angle range.

[0120] As an example, when N-3 screenshots out of N (where N is a positive integer greater than 3) screenshots satisfy that the display ratio of the foreground to the background in the screenshot image is greater than the display ratio threshold, then the multiple screenshots are the N-3 screenshots that meet the screening condition. For example, if the display ratio of the foreground to the background in the screenshot image is 1 / 3, and the display ratio threshold is 1, then the display ratio of the foreground to the background in the screenshot image is less than the display ratio threshold. It can be understood that when the display ratio is 1, it means that the display ratio of the foreground and the background in the screenshot image is the same.

[0121] As an example, when the number of objects in the foreground of a screenshot image is 10 and the number threshold is 8, the number of objects in the foreground of the screenshot image is greater than the number threshold. This indicates that the amount of information in the screenshot is relatively large, that is, the picture is relatively rich. At this time, the screenshot images in which the number of objects in the foreground of the screenshot image is greater than the number threshold are determined as multiple screenshot images, thereby realizing the screening of screenshot images through the dimension of image information amount.

[0122] As an example, the posture angle may be at least one of a pitch angle, a yaw angle, and a roll angle in a right-handed Cartesian coordinate system in a three-dimensional space. By determining the posture angle of the object in the foreground of the screenshot image, it is determined whether the posture angle of the object in the foreground of the screenshot image is within a set angle range, thereby implementing screening of the screenshot image.

[0123] In some embodiments, the screenshot processing of the picture displayed on the TV device in step 101 above can be implemented in the following manner: Determine a screenshot time window according to the operation time point of the screenshot trigger operation, where the screenshot time window starts, ends, or has the operation time point as its midpoint; Perform screenshot processing on multiple pictures displayed by the TV device within the screenshot time window, and splice the corresponding multiple screenshot images into a complete screenshot image; Wherein, the complete screenshot image is used to be sent to the mobile terminal.

[0124] As an example, when the screenshot time window starts from the operation time point, the screenshot images within the screenshot time window can be a series of pictures to be displayed in the TV device. Thus, multiple screenshot processes can be performed on a series of pictures to be displayed in the TV device to obtain multiple screenshot images, and the multiple screenshot images are spliced into a complete screenshot image, and the spliced complete image is sent to the mobile terminal for subsequent processing.

[0125] In this way, by splicing multiple screenshot images into a complete screenshot image, it is convenient to comprehensively and completely display multiple screenshot images, effectively improving the user experience.

[0126] In some embodiments, before the TV device displays the graphic code through step 104 above, the relevant content can also be displayed in the following manner: Display the following candidate function items: single screenshot function, synthesis function, cancel function; In response to the selection operation for the synthesis function, display the graphic code.

[0127] As an example, see Figures 4B - 4D , Figures 4B - 4D is a schematic diagram of the interface of the image processing method provided by the embodiments of the present application. In the human-computer interaction interface of the TV device, the following candidate function items are displayed: single screenshot function (only screenshot), synthesis function (synthesis), and cancel function. The user can click the candidate function options displayed in the human-computer interaction interface of the TV device through the corresponding controls in the remote control device. See Figure 4B , when clicking the function option synthesis function (synthesis), in response to the selection operation for the synthesis function, the graphic code 602 is displayed in the human-computer interaction interface of the TV device. See Figure 4C , when clicking the function item single screenshot function (only screenshot), in response to the selection operation for the single screenshot function (only screenshot), the corresponding screenshot image 601 is displayed in the human-computer interaction interface of the TV device.

[0128] In this way, by displaying the candidate function items in the human-computer interaction interface of the TV device, the needs of different users can be better met, and the user can select the corresponding function option according to their own needs, thus significantly improving the user experience.

[0129] In some embodiments, while responding to the selection operation for the single screenshot function, the following processing may also be performed: In response to the selection operation for the single screenshot function, at least one of the following operations is performed: saving the screenshot image in the TV device; sending the screenshot image to the server for saving; sending the screenshot image to the mobile terminal for saving; in response to the selection operation for the cancel function, stopping the display of the screenshot image.

[0130] As an example, refer to Figure 4C , when the function item single screenshot function (only screenshot) is clicked, in response to the selection operation for the single screenshot function (only screenshot), the corresponding screenshot image 601 is displayed in the human-machine interaction interface of the TV device, and the screenshot image 601 is saved in the TV device, and the screenshot image is sent to the server and the mobile terminal respectively for saving. Refer to Figure 4D , when the function item cancel is clicked, in response to the selection operation for the cancel function, the display of the screenshot image 601 and the candidate function items is stopped in the human-machine interaction interface of the TV device, and the screenshot image is deleted.

[0131] In some embodiments, when the TV device sends the screenshot image to the mobile terminal, the communication connection between the TV device and the mobile terminal may be established in advance, or a graphic code may be displayed after the save function is selected for the mobile terminal to scan to establish the communication connection. When sending the screenshot image to the server for saving, the server may save the screenshot image to the network disk of the login account of the TV device.

[0132] In this way, by sending the screenshot image to the server and the mobile terminal respectively for saving, it is convenient for the server and the mobile terminal to perform the next processing on the screenshot image.

[0133] In some embodiments, while the TV device displays the screenshot image in the above step 102, a graphic code is displayed; wherein, the graphic code is used for the mobile terminal to scan to establish a communication connection with the TV device.

[0134] As an example, refer to Figure 4E , when the screenshot image 601 is displayed, the graphic code 602 may be floatingly displayed above the screenshot image (for example, the graphic code may be displayed in the form of a floating window above the screenshot image). When the display of the screenshot image 601 is cancelled, the display of the graphic code 602 may be cancelled synchronously or the graphic code 602 may continue to be displayed; or, when the communication connection is established between the mobile terminal and the TV device, the display of the graphic code 602 is cancelled.

[0135] In some embodiments, after a communication connection is established between the mobile terminal and the TV device, the communication connection between the mobile terminal and the TV device can be maintained all the time (i.e., the connection is not disconnected after being established), or the communication connection can be maintained within a preset waiting duration after the communication connection between the mobile terminal and the TV device is established. When the connection duration reaches the preset waiting duration, the communication connection between the mobile terminal and the TV device is released (disconnected).

[0136] In some embodiments, in step 103 above, the TV device can send the screenshot image to the mobile terminal in the following manner: split the screenshot image into multiple data shards, sequentially send the multiple data shards to the mobile terminal through the communication connection, and display the transmission speeds of the multiple data shards; wherein, the multiple data shards are used for the mobile terminal to assemble to obtain the screenshot image.

[0137] As an example, the screenshot image includes multiple data (for example, pixel point data), so that the screenshot image can be split into multiple data shards, and each data shard includes partial pixel point data of the screenshot image. Sequentially send the multiple data shards to the mobile terminal through the communication connection, thereby completing the sending of the pixel point data of the entire screenshot image, and at the same time display the transmission speeds of the multiple data shards on the display interface of the TV device (for example, the transmission speeds of the multiple data shards can be displayed in the form of a progress bar). Then, after receiving the data shard, the mobile terminal assembles the data shard to reproduce the screenshot image.

[0138] In this way, the screenshot image is sent to the mobile terminal in the form of data shards, so that the transmission speed can be displayed in real time on the display interface of the TV device, which is convenient for the user to grasp the transmission situation in real time and effectively enhances the user experience.

[0139] In some embodiments, before the TV device sends the screenshot image to the mobile terminal in step 103 above, a synthesis instruction can be sent to the mobile terminal; wherein, the synthesis instruction is used to trigger the mobile terminal to display a synthesis interface.

[0140] As an example, by sending a synthesis instruction from the TV device to the mobile terminal, after receiving the synthesis instruction, the mobile terminal displays a synthesis interface on the human-computer interaction interface of the mobile terminal.

[0141] In this way, while taking a screenshot of the picture displayed on the TV device through the remote control device and displaying the screenshot image in the TV device, the screenshot image is sent to the mobile terminal to be synthesized with the captured image of the mobile terminal. In this way, the screenshot function is realized in the TV device, and the screenshot image of the TV device is synthesized with the picture in the mobile terminal, thereby significantly improving the user interaction of the TV device and enhancing the user experience.

[0142] The exemplary application and implementation of the mobile terminal provided in the embodiments of the present application will be combined to illustrate the image processing method provided in the embodiments of the present application.

[0143] Refer to Figure 3C , Figure 3C which is a schematic flowchart of the image processing method provided in the embodiments of the present application, and will be described in combination with Figure 3C the steps 201 to 203 shown.

[0144] In step 201, the mobile terminal receives a screenshot image sent by the TV device; wherein, the screenshot image is obtained by taking a screenshot of the screen displayed by the TV device.

[0145] As an example, the TV device takes a screenshot of the screen it displays, obtains the screenshot image, and sends the screenshot image to the mobile terminal, and the mobile terminal receives the screenshot image sent by the TV device.

[0146] In step 202, the mobile terminal displays the screenshot image and the shooting preview screen of the mobile terminal in the image synthesis interface.

[0147] As an example, refer to Figure 4I , Figure 4I which is a schematic interface diagram of the image processing method provided in the embodiments of the present application. The mobile terminal displays the received screenshot image 501 and the shooting preview screen 502 of the mobile terminal in the image synthesis interface 503.

[0148] In some embodiments, the above step of displaying the screenshot image and the shooting preview screen of the mobile terminal in the image synthesis interface in step 202 can be implemented in the following ways: performing one of the following operations: the mobile terminal respectively displays the screenshot image and the shooting preview screen of the mobile terminal in different areas of the image synthesis interface; performing a transparency processing on the background in the shooting preview screen, and superimposing the screenshot image on the shooting preview screen after the transparency processing; superimposing the screenshot image on the shooting preview screen with a set transparency applied.

[0149] As an example, refer to Figure 4J , Figure 4J which is a schematic interface diagram of the image processing method provided in the embodiments of the present application. The screenshot image 501 and the shooting preview screen 502 of the mobile terminal are respectively displayed in different areas of the image synthesis interface 503 (refer to Figure 4J the left figure). Performing a transparency processing on the background of the shooting preview screen 502, and superimposing the screenshot image 501 on the shooting preview screen 502 after the transparency processing (refer to Figure 4J the right figure). Superimposing the screenshot image 501 on the shooting preview screen 502 with a set transparency applied (refer to Figure 4J the right figure).

[0150] In this way, after making the background in the shooting preview screen transparent, the screenshot image is superimposed on the shooting preview screen after the transparency processing, thereby effectively enhancing the degree of fusion between the screenshot image and the shooting preview screen, and enabling the screenshot image and the shooting preview screen to be more closely integrated together.

[0151] In step 203, the mobile terminal responds to the shooting trigger operation, synthesizes the shooting image generated based on the current shooting preview screen with the screenshot image, and displays the synthesized image in the synthesis interface.

[0152] As an example, refer to Figure 4K , Figure 4K is a schematic diagram of the interface of the image processing method provided by the embodiment of the present application. The mobile terminal responds to the shooting trigger operation, synthesizes the shooting image generated based on the current shooting preview screen with the screenshot image, and displays the synthesized image 504 in the synthesis interface 503.

[0153] In this way, by synthesizing and displaying the shooting image and the screenshot image of the mobile terminal in the synthesis interface of the mobile terminal, the user can complete the synthesis display of the screenshot image of the television device in the synthesis interface of the mobile terminal, thereby significantly improving the user interaction of the television device and enhancing the user experience.

[0154] In some embodiments, before receiving the shooting trigger operation, the shooting preview screen of the mobile terminal can also be periodically refreshed.

[0155] As an example, before receiving the shooting trigger operation, the shooting preview screen of the mobile terminal is refreshed at a cycle of 1 second, so that the content of the shooting preview screen can be collected in real time before receiving the shooting trigger operation.

[0156] In some embodiments, a synthesis instruction sent by the television device is received; according to the synthesis instruction, the step of displaying the screenshot image and the shooting preview screen of the mobile terminal in the image synthesis interface is executed.

[0157] As an example, the mobile terminal receives the synthesis instruction sent by the television device, and the mobile terminal, according to the synthesis instruction, transfers to execute the above step 202 to display the screenshot image and the shooting preview screen of the mobile terminal in the image synthesis interface.

[0158] In some embodiments, the synthesis of the shooting image generated based on the current shooting preview screen and the screenshot image in the above step 203 can be achieved in the following ways: splicing the screenshot image and the shooting image in the width direction or the height direction; or superimposing the foreground in the shooting image on the screenshot image.

[0159] As an example, the screenshot image and the captured image are spliced in the width direction of the screenshot image and the width direction of the captured image. After splicing, the synthesized image obtained may be that the screenshot image is displayed at the top of the synthesis interface and the captured image is displayed at the bottom of the screenshot image; or, after splicing, the synthesized image obtained may be that the screenshot image is displayed at the bottom of the synthesis interface and the captured image is displayed at the top of the screenshot image. The screenshot image and the captured image are spliced in the height direction of the screenshot image and the height direction of the captured image. After splicing, the synthesized image obtained may be that the screenshot image is displayed on the left part of the synthesis interface and the captured image is displayed on the right part of the screenshot image; or, after splicing, the synthesized image obtained may be that the screenshot image is displayed on the right part of the synthesis interface and the captured image is displayed on the left part of the screenshot image.

[0160] As an example, refer to Figure 4L , Figure 4L the schematic diagram of the interface of the image processing method provided by the embodiment of the present application. The foreground 505 in the captured image (i.e., Figure 4L the face image showing only the head of the person in Figure 4L ) is superimposed on the screenshot image 501 (i.e.,

[0161] the face image showing the upper body of the person in

[0162] In some embodiments, the above-mentioned superimposing the foreground in the captured image on the screenshot image can be implemented in the following manner: identifying the foreground in the captured image, taking the part other than the foreground in the captured image as the background in the captured image, and modifying the color value of each pixel included in the background in the captured image to a transparent color value; wherein, the foreground includes at least one object; superimposing the screenshot image on the modified captured image.

[0163] The exemplary applications and implementations of the mobile terminal, the television device, and the remote control device provided by the embodiments of the present application will be combined to illustrate the image processing method provided by the embodiments of the present application.

[0164] Refer to Figure 3D , Figure 3D which is the schematic flowchart of the image processing method provided by the embodiment of the present application, and will be combined with Figure 3DSteps 301 to 310 shown will be described.

[0165] In step 301, when the automatic connection condition is met, a graphic code is displayed, which is used for a mobile terminal to establish a communication connection with a TV device after scanning.

[0166] As an example, the TV device determines whether the connection condition is met. When the automatic connection condition is met, a graphic code is displayed in the TV device, which is used for a mobile terminal to establish a communication connection with the TV device after scanning.

[0167] In step 302, the mobile terminal scans the graphic code to establish a communication connection with the TV device.

[0168] As an example, the mobile terminal scans the graphic code displayed on the TV device and the mobile terminal establishes a communication connection with the TV device.

[0169] In step 303, the remote control device generates a wireless control signal in response to a pressing operation on at least one key.

[0170] As an example, the remote control device generates a wireless control signal in response to a pressing operation on at least one key.

[0171] In step 304, the remote control device sends the wireless control signal to the TV device; the wireless control signal includes the number of presses and the press interval time of the pressing operation.

[0172] As an example, the remote control device sends the wireless control signal to the TV device; the wireless control signal includes the number of presses and the press interval time of the pressing operation.

[0173] In step 305, when the number of presses of the pressing operation is the same as the number of presses included in the configuration information, and the press interval time of the pressing operation is the same as the press interval time included in the configuration information, it is determined that the pressing operation is a screenshot trigger operation.

[0174] As an example, the TV device determines whether the number of presses of the pressing operation is the same as the number of presses included in the configuration information, and determines whether the press interval time of the pressing operation is the same as the press interval time included in the configuration information. When the number of presses of the pressing operation is the same as the number of presses included in the configuration information, and the press interval time of the pressing operation is the same as the press interval time included in the configuration information, the TV device determines that the pressing operation is a screenshot trigger operation.

[0175] In step 306, the TV device performs a screenshot process on the screen displayed by the TV device in response to the screenshot trigger operation implemented by the remote control device.

[0176] As an example, in response to a screen capture trigger operation performed by a remote control device, a television device performs a screen capture process on the screen displayed by the television device.

[0177] In step 307, the television device displays the screen capture image.

[0178] As an example, the screen capture image is displayed in the display screen of the television device.

[0179] In step 308, the television device sends the screen capture image to the mobile terminal.

[0180] As an example, the television device sends the screen capture image to the mobile terminal.

[0181] In step 309, the mobile terminal displays the screen capture image and the shooting preview screen of the mobile terminal in the image synthesis interface.

[0182] As an example, after receiving the screen capture image sent by the television device, the mobile terminal displays the screen capture image and the shooting preview screen of the mobile terminal in the image synthesis interface.

[0183] In step 310, in response to a shooting trigger operation, the mobile terminal synthesizes the shooting image generated based on the current shooting preview screen with the screen capture image, and displays the synthesized image in the synthesis interface.

[0184] As an example, in response to a shooting trigger operation, the mobile terminal synthesizes the shooting image generated based on the current shooting preview screen with the screen capture image, and displays the synthesized image in the synthesis interface.

[0185] Next, an exemplary application of the embodiments of the present application in an actual application scenario will be described.

[0186] In the embodiments of the present application, an entertainment function of taking a group photo with the wonderful picture of a television (i.e., the television device described above) can be provided for users. When the user sees a wonderful picture displayed on the television device, the user can quickly press the screen capture shortcut key on the remote control device. After the screen capture is completed, the group photo interface of the video APP will be automatically displayed on the mobile phone (i.e., the mobile terminal described above), and the screen capture picture will be transmitted to the mobile phone. After the transmission of the screen capture picture is completed, the front camera of the mobile phone will be turned on, and it is very convenient to combine one's own figure with the wonderful picture on the screen to form a group photo, which is then saved in the mobile phone. The saved picture can also be shared on social platforms, further increasing the fun and entertainment.

[0187] The screenshot technical solution in the related art can trigger the screenshot function on the mobile phone through the shortcut key (a combination of multiple keys), gesture action, or screenshot button on the status bar of the mobile phone. The screenshot path is short and convenient. On TV (i.e., the TV device mentioned above), most of the current TV devices do not have the screenshot function, and the screenshot image is also saved in the album on the TV device and cannot be shared to the mobile phone. The screenshot function of the mobile phone is relatively simple. If you want to integrate the person and the screenshot image together, you still need to use third-party image processing software to manually complete the image integration, which is a relatively complicated process.

[0188] The embodiment of the present application solves the three major problems of the prior art solutions by providing users with the function of taking photos with the wonderful pictures on TV: the TV has no screenshot function, the pictures after screenshot can only be saved locally and cannot be transferred to the mobile phone, and the pictures on the mobile phone cannot be taken with the user conveniently and quickly. The embodiment of the present application can further enhance the entertainment of the TV and achieve seamless linkage with the mobile phone, so that the pictures of the TV can be shared through the mobile phone in a timely manner.

[0189] In some embodiments, when a user turns on a TV device and uses the group photo function for the first time, a function usage guide interface will pop up, in which the method of using image processing will be displayed, wherein the method of taking a screenshot is: quickly and continuously pressing the down arrow key on the remote control twice. When a group photo is needed, the video APP of the mobile phone must first establish a local area network connection with the video APP of the TV, wherein the method of establishing the connection is: a connection QR code will be displayed on the display interface of the TV, and the connection QR code carries the IP address and service port of the TV. The mobile phone scans the connection QR code displayed on the TV. After the mobile phone scans, the mobile phone can obtain the IP address and service port of the TV from the connection QR code, and the mobile phone connects to the TV through the IP address and port. After responding to the screenshot operation, the user's settings will be judged first. If only screenshot is set, the screenshot will be displayed for three seconds after the screenshot is taken, and then the screenshot screen will be exited; if group photo is set and the TV has not established a connection with the mobile phone, a QR code will be displayed and the user will be prompted to connect. At this time, if the user presses the return button of the remote control device, the group photo will be abandoned and only the screenshot will be saved. If the user chooses to take a group photo, he will open the video APP on the mobile phone, scan the QR code displayed on the TV, and establish a connection with the TV video APP. After the connection is established, the TV video APP will display the group photo interface in the video APP on the mobile phone through the connection channel, and the group photo operation will be completed on the mobile phone.

[0190] In some embodiments, the group photo function can be turned on and off in the application settings, and can also be set to only take a screenshot, or take a group photo (take a photo with the screenshot on the mobile phone after taking the screenshot).

[0191] In some embodiments, see Figure 5A, Figure 5A It is a schematic diagram of the principle of the image processing method provided by the embodiments of the present application. The linkage service will establish an mDNS device discovery service and a WebSocket server on the TV. Its main responsibilities are to provide the device discovery service, as well as the connection channels between the mobile phone and the TV, and the data transmission channels.

[0192] In some embodiments, referring to Figure 5A , the video APP of the mobile phone 500 uses the mDNS discovery protocol and can search for the TV 600 that has enabled the mDNS service within the local area network. The mDNS service of the TV 600 will multicast a message to all the hosts within the local area network. The message carries the IP address of the TV 600 and the port of the linkage service. The mobile phone 500 can parse out the IP address of the TV and the linkage service port through the data packet returned by the mDNS protocol and establish a connection with the TV 600.

[0193] In some embodiments, referring to Figure 5F , Figure 5F It is a schematic diagram of the principle of the image processing method provided by the embodiments of the present application. For the connection between the mobile phone and the TV, in addition to relying on device discovery, it can also be established through a connection QR code, which solves the problem that it is impossible to establish a connection with the TV due to the inability to discover devices within the local area network. The connection QR code will carry the IP address of the TV and the port of the linkage service. After the video APP on the mobile phone scans the connection QR code, it will parse out the IP address and the linkage service port in the connection QR code and establish a connection with the TV. By refreshing the QR code, the IP address and the linkage service port in the QR code are extracted, and the IP address and the linkage service port are passed into the generation of the QR code.

[0194] In some embodiments, referring to Figure 5C , Figure 5C It is a schematic diagram of the principle of the image processing method provided by the embodiments of the present application. The triggering of the TV screenshot is mainly triggered by the buttons of the remote control (i.e., the remote control device mentioned above). The default triggering method is to click the down arrow key twice continuously, and the interval between the two clicks needs to be less than or equal to 250 milliseconds. Since there are many TV device brands, each TV device brand has its own unique functions, and these functions may conflict with the screenshot triggering method of the embodiments of the present application. Therefore, the key values for triggering need to be configurable in the background, and the time interval and the number of consecutive clicks also need to support background configuration, so as to flexibly avoid conflicts and can adjust the time interval and the number at any time to control the sensitivity. Referring to Figure 5C , Figure 5C It is mainly divided into two processing branches. The first branch is: initialize the mDNS service, package the mDNS protocol, and broadcast to other hosts; the second branch is: initialize WebSocket, bind the service port, and register the protocol parser.

[0195] In some embodiments, refer to Figure 5D , Figure 5D which is a schematic diagram of the principle of the image processing method provided by the embodiments of the present application. First, open the APP; then initialize the global configuration, pull the background configuration, save the background configuration, and read the configuration. Perform screenshot management initialization, read the screenshot button configuration, detect button events, respond to button events, and finally take a screenshot.

[0196] In some embodiments, refer to Figure 5B , for the implementation of screenshot, the embodiments of the present application adopt the screenshot interface provided by the system (i.e., the first screenshot interface described above), which can have higher compatibility and stability. If the screenshot fails, it will be retried. If the retry fails, it will attempt to call a more underlying interface (i.e., the second screenshot interface described above).

[0197] In some embodiments, refer to Figure 5E , Figure 5E which is a schematic diagram of the principle of the image processing method provided by the embodiments of the present application. After the screenshot is completed, the TV will initiate a group photo instruction through the connection channel. After receiving the group photo instruction, the video APP on the mobile phone will open the group photo interface, and then reply that the group photo interface has been opened (i.e., Figure 5E the reply instruction in

[0198] In some embodiments, the camera of the mobile phone is used for taking group photos. On the group photo interface, when the camera is turned on, the image captured by the camera is displayed on the interface, and the screenshot is synthesized into the image background. After the mobile phone receives the picture transmitted by the TV, the picture is displayed on the group photo interface as the background. At the same time, the camera interface of the system needs to be called to read the image data from the camera. After a complete frame of data is read, human recognition is performed on this data. The arrangement of the image data can be regarded as a two-dimensional array. Each coordinate in this two-dimensional array represents a pixel point, and the corresponding value is the RGB value. After the person is recognized, the data of the person corresponds to the specific coordinates of the two-dimensional array. Then, in this embodiment of the application, the coordinates of non-people are filled with transparent RGB values, so that a picture with only people can be formed. The non-person part of the picture is transparent and can be stacked on the background, so as to achieve the effect of taking a group photo. Before the user takes the final photo, this embodiment of the application updates a picture of the person every second, which can also achieve the effect of changes in the person's expression or movement.

[0199] This embodiment of the application can capture the screen picture of the TV on the mobile phone and easily add itself to the captured picture. The user can share the synthesized picture to the Moments, which increases the fun and entertainment, improves the user experience, and also provides a way for the user to share the wonderful TV picture with one key.

[0200] Next, the exemplary structure of the implementation of the image processing device 455 provided in this embodiment of the application as a software module will be continued. In some embodiments, as Figure 2A shown, the software module stored in the image processing device 455 in the memory 450 may include: a screenshot module, configured to perform screenshot processing on the picture displayed by the TV device in response to a screenshot trigger operation implemented based on a remote control device, and display the screenshot image; a sending module, configured to send the screenshot image to the mobile terminal; wherein, the screenshot image is used for synthesis with the captured image of the mobile terminal.

[0201] In this embodiment of the application, the above-mentioned image processing device further includes: a first display module, configured to display a graphic code when the automatic connection condition is met, and the graphic code is used for the mobile terminal to scan to establish a communication connection with the TV device; wherein, the automatic connection condition includes at least one of the following: the TV device is powered on, the TV device is awakened from the sleep mode, the TV device plays an interesting program, the TV device plays a program that has been screenshot.

[0202] In this embodiment of the application, when the screenshot image is displayed, the above-mentioned image processing device further includes: a second display module, configured to display a graphic code; wherein, the graphic code is used for the mobile terminal to scan to establish a communication connection with the TV device.

[0203] In an embodiment of the present application, the above image processing apparatus further includes: a third display module, configured to display the following candidate function items: a single screenshot function, a synthesis function, and a cancellation function; and in response to a selection operation for the group photo function, display a graphic code.

[0204] In an embodiment of the present application, the above image processing apparatus further includes: a saving module, configured to perform at least one of the following operations in response to a selection operation for the single screenshot function: save a screenshot image in a television device; send the screenshot image to a server for saving; send the screenshot image to a mobile terminal for saving. A stop display module, configured to stop displaying the screenshot image in response to a selection operation for the cancellation function.

[0205] In an embodiment of the present application, the above image processing apparatus further includes: a connection module, configured to, before receiving a screenshot trigger operation implemented based on a remote control device, or when displaying a graphic code, the television device broadcasts or multicasts connection information based on a device discovery service, where the connection information includes an address and a port of the television device in a local area network; and the connection information is used for a mobile terminal to resolve the address and the port and establish a communication connection with the television device in the local area network based on the address and the port.

[0206] In an embodiment of the present application, the above screenshot module is further configured to call a first screenshot interface of an operating system of the television device to perform screenshot processing on a screen displayed by the television device; where the first screenshot interface is an interface opened by the operating system to external applications; when the call to the first screenshot interface fails, perform reflection processing on a second screenshot interface of the operating system of the television device, and call the reflected second screenshot interface to perform screenshot processing on the screen displayed by the television device; where the second screenshot interface is an interface of the operating system of the television device that is not opened for external application calls.

[0207] In an embodiment of the present application, the above screenshot module is further configured to call a first screenshot interface of an operating system of the television device to perform screenshot processing on a screen displayed by the television device; where the first screenshot interface is an interface opened by the operating system to external applications; when the call to the first screenshot interface fails, perform reflection processing on a second screenshot interface of the operating system of the television device, and call the reflected second screenshot interface to perform screenshot processing on the screen displayed by the television device; where the second screenshot interface is an interface of the operating system of the television device that is not opened for external application calls.

[0208] In the embodiment of the present application, the above screenshot module is further configured to perform screenshot processing on the picture displayed by the TV device at the operation time point of the screenshot trigger operation; wherein, the screenshot area of the picture is the entire area of the picture or a partial area set by the screenshot operation.

[0209] In the embodiment of the present application, the above screenshot module is further configured to determine a screenshot time window according to the operation time point of the screenshot trigger operation, wherein the screenshot time window starts, has its midpoint, or ends at the operation time point; perform screenshot processing on multiple pictures displayed by the TV device within the screenshot time window, and correspondingly obtain multiple screenshot images. The image processing device further includes: a marking module, configured to mark the selected screenshot image among the multiple screenshot images as the screenshot image to be sent to the mobile terminal in response to a selection operation on the multiple screenshot images.

[0210] In the embodiment of the present application, the above screenshot module is further configured to determine a screenshot time window according to the operation time point of the screenshot trigger operation, wherein the screenshot time window starts, has its midpoint, or ends at the operation time point; perform screenshot processing on multiple pictures displayed by the TV device within the screenshot time window, and splice the correspondingly obtained multiple screenshot images into a complete screenshot image; wherein, the complete screenshot image is used to be sent to the mobile terminal.

[0211] In the embodiment of the present application, the above sending module is further configured to split the screenshot image into multiple data shards, sequentially send the multiple data shards to the mobile terminal through a communication connection, and display the transmission speeds of the multiple data shards; wherein, the multiple data shards are used for the mobile terminal to assemble and obtain the screenshot image.

[0212] In the embodiment of the present application, the image processing device further includes: an instruction sending module, configured to send a synthesis instruction to the mobile terminal; wherein, the synthesis instruction is used to trigger the mobile terminal to display a synthesis interface.

[0213] The following continues to describe the exemplary structure of the software module implementation of the image processing device 655 provided by the embodiment of the present application. In some embodiments, as Figure 2B shown, the software module in the image processing device 655 stored in the memory 650 may include: a receiving module, configured to receive the screenshot image sent by the TV device; wherein, the screenshot image is obtained by performing screenshot processing on the picture displayed by the TV device; a display module, configured to display the screenshot image and the shooting preview picture of the mobile terminal in an image synthesis interface; a generating module, configured to synthesize the shooting image generated based on the current shooting preview picture with the screenshot image in response to a shooting trigger operation, and display the synthesized image in the synthesis interface.

[0214] In an embodiment of the present application, the above display module is further configured to perform one of the following operations: display a screenshot image and a shooting preview screen of the mobile terminal in different regions of the image synthesis interface; perform a transparency processing on the background in the shooting preview screen, and superimpose the screenshot image on the shooting preview screen after the transparency processing; superimpose the screenshot image on the shooting preview screen with a set transparency applied.

[0215] In an embodiment of the present application, the above image processing device further includes: a refresh module, configured to periodically refresh the shooting preview screen of the mobile terminal.

[0216] In an embodiment of the present application, the above image processing device further includes: an instruction receiving module, configured to receive a synthesis instruction sent by a television device; a jump module, configured to, according to the synthesis instruction, transfer to execute the step of displaying a screenshot image and a shooting preview screen of the mobile terminal in the image synthesis interface.

[0217] In an embodiment of the present application, the above generating module is further configured to splice the screenshot image and the shooting image in the width direction or the height direction; or superimpose the foreground in the shooting image on the screenshot image. The generating module is further configured to identify the foreground in the shooting image, use the part other than the foreground in the shooting image as the background in the shooting image, and modify the color value of each pixel included in the background in the shooting image to a transparent color value; where the foreground includes at least one object; and superimpose the screenshot image on the modified shooting image.

[0218] An embodiment of the present application provides a computer program product or a computer program, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the above image processing method of the embodiment of the present application.

[0219] An embodiment of the present application provides a computer-readable storage medium storing executable instructions, where the executable instructions are stored, and when the executable instructions are executed by a processor, the processor will be caused to execute the image processing method provided by the embodiment of the present application. For example, Figure 3A the image processing method shown.

[0220] An embodiment of the present application provides a television device, including: a memory, configured to store executable instructions; a processor, configured to implement the image processing method provided by the embodiment of the present application when executing the executable instructions stored in the memory.

[0221] An embodiment of the present application provides a mobile terminal, including: a memory for storing executable instructions; and a processor for implementing the image processing method provided by the embodiment of the present application when executing the executable instructions stored in the memory.

[0222] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disc, or CD-ROM; or may be various devices including one or any combination of the above memories.

[0223] In some embodiments, the executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including being deployed as an independent program or being deployed as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0224] As an example, the executable instructions may or may not correspond to a file in the file system, may be stored as part of a file that stores other programs or data, for example, stored in one or more scripts in a Hyper Text Markup Language (HTML) document, stored in a single file dedicated to the program being discussed, or stored in multiple cooperating files (for example, files that store one or more modules, subroutines, or code portions).

[0225] As an example, the executable instructions may be deployed to execute on one electronic device, or on multiple electronic devices located at one location, or, on multiple electronic devices distributed at multiple locations and interconnected through a communication network. The electronic device described above may be the mobile terminal or the television device described above.

[0226] In summary, the embodiments of the present application have the following beneficial effects:

[0227] (1) By using a remote control device to perform a screenshot process on the screen displayed by the television device, and while displaying the screenshot image on the television device, sending the screenshot image to the mobile terminal to be synthesized with the captured image of the mobile terminal. In this way, the screenshot function is realized on the television device, and the screenshot image of the television device is synthesized with the screen in the mobile terminal, thereby significantly improving the processing effect and interactive experience of the screenshot function of the television device.

[0228] (2) By displaying candidate function items in the human-computer interaction interface of the television device, different user needs can be better met, and users can select corresponding function options according to their own needs, thereby significantly improving the user experience.

[0229] (3) When the connection condition is met, a graphic code for the mobile terminal to scan and establish a communication connection with the TV device is displayed, thus effectively avoiding the situation that the graphic code is always displayed in the human-computer interaction interface of the TV device and affecting the user's viewing experience.

[0230] (4) The screen capture process of the picture displayed in the TV device is completed through the remote control device, enabling the user to capture the video picture at any time according to their own needs when watching videos through the TV device, meeting the user's personalized needs for screen capture of the TV device.

[0231] (5) By sending the screen capture image to the server and the mobile terminal for storage respectively, it is convenient for the server and the mobile terminal to perform further processing on the screen capture image.

[0232] (6) By obtaining the sensitivity configuration information and then judging whether the pressing operation on at least one key of the remote control device is a screen capture trigger operation according to the sensitivity configuration information, it effectively avoids the misidentification of the pressing operation sent by the remote control device as a screen capture trigger operation, and effectively improves the accuracy of the screen capture trigger operation.

[0233] (7) By calling the interfaces opened by the operating system of the TV device to external applications and the interfaces not opened for external application calls to perform the screen capture process on the picture displayed by the TV device, it provides double guarantees for the normal progress of the screen capture process and ensures that the screen capture process can proceed normally.

[0234] (8) By performing the screen capture process on the picture displayed by the TV device at the operation time point according to the operation time point of the screen capture trigger operation, it ensures that the operation time point is consistent with the display time of the picture to be screen captured, ensuring that the picture to be screen captured is the picture that the user needs to screen capture, thus effectively avoiding the situation of inaccurate screen capture images caused by operation delay.

[0235] (9) Through the setting of the screen capture time window, continuous multiple screen capture processes can be performed on the picture displayed in the TV device to obtain multiple screen capture images, thus effectively avoiding the situation of missing screen capture images and effectively improving the effect of the screen capture process.

[0236] (10) Through the setting of the screen capture time window, continuous multiple screen capture processes can be performed on the picture displayed in the TV device to obtain multiple screen capture images, and then select from the multiple screen capture images to determine the screen capture image that the user himself needs, and send the selected screen capture image to the mobile terminal, thus effectively avoiding the situation of missing valid screen capture images.

[0237] (11) By stitching multiple screenshot images into a complete screenshot image, it is convenient to comprehensively and completely display the multiple screenshot images, effectively enhancing the user experience.

[0238] (12) By sending the screenshot image to the mobile terminal in a data sharding manner, the transmission speed can be displayed in real time on the display interface of the TV device, facilitating the user to grasp the transmission situation in real time and effectively enhancing the user experience.

[0239] (13) By synthesizing and displaying the captured image of the mobile terminal and the screenshot image in the synthesis interface of the mobile terminal, the user can complete the synthesis display of the screenshot image of the TV device in the synthesis interface of the mobile terminal, thereby significantly improving the user interaction of the TV device and enhancing the user experience.

[0240] (14) After making the background in the shooting preview screen transparent and then overlaying the screenshot image on the transparently processed shooting preview screen, the degree of fusion between the screenshot image and the shooting preview screen is effectively enhanced, enabling the screenshot image and the shooting preview screen to be more closely integrated.

[0241] As described above, the above are only embodiments of the present application and are not used to limit the protection scope of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. An image processing method, characterized in that, Applied to a television device, the method includes: In response to a screen capture trigger operation implemented based on a remote control device, performing a screen capture process on the screen displayed by the television device, displaying a screen capture image, and Sending a synthesis instruction to a mobile terminal; wherein, the synthesis instruction is used to trigger the mobile terminal to display an image synthesis interface; the image synthesis interface is used to display the screen capture image and the shooting preview screen of the mobile terminal; Sending the screen capture image to the mobile terminal; wherein, the screen capture image is used to be synthesized with a captured image generated by the mobile terminal based on the current shooting preview screen, and the mobile terminal synthesizes the captured image and the screen capture image through the following steps: splicing the screen capture image and the captured image in the width direction or the height direction.

2. The method according to claim 1, characterized in that, Before responding to the screen capture trigger operation implemented based on the remote control device, the method further includes: When an automatic connection condition is met, displaying a graphic code, which is used for the mobile terminal to scan and establish a communication connection with the television device; wherein, the automatic connection condition includes at least one of the following: the television device is powered on, the television device is awakened from the sleep mode, the television device plays an interesting program, the television device plays a program that has been screen captured.

3. The method according to claim 1, wherein When displaying the screen capture image, the method further includes: Displaying a graphic code; wherein, the graphic code is used for the mobile terminal to scan and establish a communication connection with the television device.

4. The method according to claim 3, wherein Before displaying the graphic code, the method further includes: Displaying the following candidate function items: single screen capture function, synthesis function, cancellation function; In response to a selection operation for the synthesis function, displaying a graphic code; In response to a selection operation for the single screen capture function, performing at least one of the following operations: saving the screen capture image in the television device; sending the screen capture image to the server for saving; sending the screen capture image to the mobile terminal for saving; In response to a selection operation for the cancellation function, stopping displaying the screen capture image.

5. The method according to claim 2 or 3, characterized in that The graphic code is used for the mobile terminal to scan and identify the address and port of the television device in the local area network, and establish a communication connection with the television device in the local area network according to the address and port.

6. The method according to claim 2 or 3, characterized in that, The method further includes: Before receiving the screen capture trigger operation implemented based on the remote control device, or when displaying the graphic code, the television device broadcasts or multicasts connection information based on the device discovery service, and the connection information includes the address and port of the television device in the local area network; Wherein, the connection information is used for the mobile terminal to resolve the address and port, and establish a communication connection with the television device in the local area network based on the address and port.

7. The method according to claim 1, characterized in that, Before responding to the screen capture trigger operation implemented based on the remote control device, the method further includes: Obtaining sensitivity configuration information of the screen capture function from a server; wherein, the sensitivity configuration information includes the number of presses and the press interval time of the screen capture trigger operation for at least one key. Receive the wireless control signal sent by the remote control device; wherein, the wireless control signal is sent by the remote control device in response to a pressing operation on the at least one key, and includes the number of presses and the pressing interval time of the pressing operation; When the number of presses of the pressing operation is the same as the number of presses included in the configuration information, and the pressing interval time of the pressing operation is the same as the pressing interval time included in the configuration information, determine that the pressing operation is the screen capture trigger operation.

8. The method according to claim 1, characterized in that, The screen capture processing of the screen displayed by the television device includes: Invoke a first screen capture interface of the operating system of the television device to perform screen capture processing on the screen displayed by the television device; wherein, the first screen capture interface is an interface opened by the operating system to external applications; When the invocation of the first screen capture interface fails, perform reflection processing on a second screen capture interface of the operating system of the television device, and invoke the reflected second screen capture interface to perform screen capture processing on the screen displayed by the television device; wherein, the second screen capture interface is an interface that the operating system of the television device does not open for external application invocation.

9. The method according to claim 1, wherein The screen capture processing of the screen displayed by the television device includes: Perform screen capture processing on the screen displayed by the television device at the operation time point according to the operation time point of the screen capture trigger operation; Wherein, the screen capture area of the screen is the entire area of the screen, or a partial area set by the screen capture trigger operation.

10. The method according to claim 1, wherein The screen capture processing of the screen displayed by the television device includes: Determine a screen capture time window according to the operation time point of the screen capture trigger operation, wherein the screen capture time window starts, ends or has its midpoint at the operation time point; Perform screen capture processing on multiple screens displayed by the television device within the screen capture time window, and correspondingly obtain multiple screen capture images; The method further includes: In response to a selection operation on the multiple screen capture images, mark the selected screen capture image among the multiple screen capture images as the screen capture image to be sent to the mobile terminal.

11. The method according to claim 10, wherein, The multiple screen capture images are the screens in all the screens displayed within the screen capture time window that meet the screening conditions, and the screening conditions include at least one of the following: The display ratio of the foreground to the background in the screen capture image is greater than the display ratio threshold; The number of objects in the foreground of the screen capture image is greater than the quantity threshold; The pose angle of the objects in the foreground of the screen capture image is within a set angle range.

12. The method according to claim 1, wherein The screen capture processing of the screen displayed by the television device includes: Determine a screen capture time window according to the operation time point of the screen capture trigger operation, wherein the screen capture time window starts, ends or has its midpoint at the operation time point; Perform screen capture processing on multiple screens displayed by the television device within the screen capture time window, and splice the correspondingly obtained multiple screen capture images into a complete screen capture image; wherein, the complete screen capture image is used to be sent to the mobile terminal.

13. The method according to claim 2, wherein Sending the screen capture image to the mobile terminal includes: Split the screenshot image into multiple data shards, and sequentially send the multiple data shards to the mobile terminal via the communication connection, and display the transmission speeds of the multiple data shards; wherein, the multiple data shards are used for the mobile terminal to assemble to obtain the screenshot image.

14. An image processing method, characterized in that, Applied to a mobile terminal, the method includes: Receiving a synthesis instruction and a screenshot image sent by a television device; wherein, the synthesis instruction is used to trigger the mobile terminal to display an image synthesis interface; the screenshot image is obtained by taking a screenshot of the screen displayed by the television device; displaying the screenshot image and the shooting preview screen of the mobile terminal in the image synthesis interface; In response to a shooting trigger operation, synthesize a shooting image generated based on the current shooting preview screen with the screenshot image, and display the synthesized image in the image synthesis interface; synthesizing the shooting image generated based on the current shooting preview screen with the screenshot image includes: splicing the screenshot image and the shooting image in the width direction or the height direction.

15. The method according to claim 14, wherein The displaying the screenshot image and the shooting preview screen of the mobile terminal in the image synthesis interface includes at least one of the following operations: Displaying the screenshot image and the shooting preview screen of the mobile terminal in different regions of the image synthesis interface; Performing a transparency processing on the background in the shooting preview screen, and superimposing the screenshot image on the shooting preview screen after the transparency processing; Superimposing the screenshot image on the shooting preview screen with a set transparency applied.

16. The method according to claim 14, wherein Before responding to the shooting trigger operation, the method further includes: Periodically refreshing the shooting preview screen of the mobile terminal.

17. The method according to claim 14, wherein Receiving the synthesis instruction sent by the television device; According to the synthesis instruction, transfer to execute the step of displaying the screenshot image and the shooting preview screen of the mobile terminal in the image synthesis interface.

18. An image processing apparatus, characterized in that, Applied to a television device, the apparatus includes: A screenshot module, configured to respond to a screenshot trigger operation implemented based on a remote control device, take a screenshot of the screen displayed by the television device, and display the screenshot image; A sending module, configured to send a synthesis instruction to a mobile terminal; wherein, the synthesis instruction is used to trigger the mobile terminal to display an image synthesis interface; the image synthesis interface is used to display the screenshot image and the shooting preview screen of the mobile terminal; sending the screenshot image to the mobile terminal; wherein, the screenshot image is used to be synthesized with a shooting image generated by the mobile terminal based on the current shooting preview screen, and the mobile terminal synthesizes the shooting image and the screenshot image through the following steps: splicing the screenshot image and the shooting image in the width direction or the height direction.

19. An image processing apparatus, characterized in that, Applied to a mobile terminal, the apparatus includes: A receiving module, configured to receive a screenshot image sent by a television device; wherein, the screenshot image is obtained by taking a screenshot of the screen displayed by the television device; Wherein, the mobile terminal is further configured to receive a synthesis instruction sent by the television device; the synthesis instruction is used to trigger the mobile terminal to display an image synthesis interface; A display module, configured to display the screenshot image and the shooting preview screen of the mobile terminal in the image synthesis interface; A generation module, configured to, in response to a shooting trigger operation, synthesize a shooting image generated based on the current shooting preview screen with the screenshot image, and display the synthesized image in the image synthesis interface; synthesizing the shooting image generated based on the current shooting preview screen with the screenshot image includes: splicing the screenshot image and the shooting image in the width direction or the height direction.

20. A television device, characterized in that, The television device includes: A memory, configured to store executable instructions; A processor, configured to implement the image processing method according to any one of claims 1 to 13 when executing the executable instructions stored in the memory.

21. A mobile terminal, characterized in that, The mobile terminal includes: A memory, configured to store executable instructions; A processor, configured to implement the image processing method according to any one of claims 14 to 17 when executing the executable instructions stored in the memory.

22. A computer-readable storage medium storing executable instructions or a computer program, characterized in that, The executable instructions, when executed by the processor, implement the image processing method according to any one of claims 1 to 17.

23. A computer program product, comprising a computer program or instructions, characterized in that, The computer program or instructions, when executed by the processor, implement the image processing method according to any one of claims 1 to 17.

Images