Display device and circling-based screenshot capture method

Abstract

The present application relates to a display device and a circling-based screenshot capture method. The display device comprises a display, a memory and at least one processor, wherein the at least one processor is configured to execute a computer program or an instruction that is stored in the memory, so as to cause the display device to: receive pointing parameters sent by a remote controller, and on the basis of the received pointing parameters, generate a trajectory line, wherein the pointing parameters change as the remote controller moves; on the basis of the trajectory line, determine a region having a preset shape, and use same as a circled region corresponding to the pointing parameters, wherein the region having a preset shape corresponds to a region represented by the trajectory line; capture, on the currently displayed page, a screenshot of a region corresponding to the circled region, so as to obtain a region image; and on the basis of the region image, perform an image search, so as to display object information of an object in the region image. By using the method, regardless of the shape of a trajectory line circled by a user, a region having a preset shape and including the entire trajectory line can be determined on the basis of the trajectory line, thereby ensuring the smooth implementation of a subsequent image search.

Description

Display devices and selection screenshot methods

[0001] Cross-references to related applications

[0002] This application claims priority to Chinese patent applications filed on March 20, 2025, application number 202510336560.2; filed on June 30, 2025, application number 202510894016.X; and filed on July 25, 2025, application number 202511039203.6, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of computer technology, and in particular to a display device and a method for selecting and taking screenshots. Background Technology

[0004] With the development of devices (such as smart TVs), some devices have integrated a selection screenshot (or partial screenshot) function to meet users' diverse information query needs for the interface displayed on the device. For example, the device can take a screenshot of a portion of the displayed content and perform a search based on the captured portion of the image to display search results for that portion of the image to the user.

[0005] However, in related technologies, the device can only recognize a single-shaped selected area. When the user selects other shaped areas, it cannot perform screenshot processing, which in turn prevents subsequent image search from being performed. Summary of the Invention

[0006] According to some embodiments of this application, a display device is provided, including: a display, a memory, and at least one processor; the memory is configured to store computer programs or instructions; the at least one processor is configured to execute the computer programs or instructions to cause the display device to: receive pointing parameters sent by a remote controller, and generate a trajectory line according to the received pointing parameters; wherein the pointing parameters change with the movement of the remote controller; determine a preset shape region according to the trajectory line as a selected region corresponding to the pointing parameters; wherein the preset shape region corresponds to the region represented by the trajectory line; take a screenshot of the region corresponding to the selected region in the currently displayed page to obtain a region image; and perform an image search based on the region image to display object information of objects in the region image.

[0007] According to some embodiments of this application, a method for selecting and capturing images on a display device is also provided, comprising: receiving pointing parameters sent by a remote controller, and generating a trajectory line based on the received pointing parameters; wherein the pointing parameters change with the movement of the remote controller; determining a preset shape region based on the trajectory line as the selected region corresponding to the pointing parameters; wherein the preset shape region corresponds to the region represented by the trajectory line; capturing a screenshot of the region corresponding to the selected region on the currently displayed page to obtain a region image; and performing an image search based on the region image to display object information of objects in the region image.

[0008] According to some embodiments of this application, a selection and screenshot device for a display device is also provided, comprising: a trajectory line generation module, configured to receive pointing parameters sent by a remote controller and generate a trajectory line based on the received pointing parameters; wherein the pointing parameters change with the movement of the remote controller; a region determination module, configured to determine a preset shape region based on the trajectory line as the selection region corresponding to the pointing parameters; wherein the preset shape region corresponds to the region represented by the trajectory line; a screenshot module, configured to take a screenshot of the region corresponding to the selection region on the currently displayed page to obtain a region image; and an image search module, configured to perform an image search based on the region image to display object information of objects in the region image.

[0009] According to some embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it performs the following steps: receiving pointing parameters sent by a remote control, and generating a trajectory line according to the received pointing parameters; wherein the pointing parameters change with the movement of the remote control; determining a preset shape region as the selected region corresponding to the pointing parameters according to the trajectory line; wherein the preset shape region corresponds to the region represented by the trajectory line; taking a screenshot of the region corresponding to the selected region on the currently displayed page to obtain a region image; and performing an image search based on the region image to display object information of objects in the region image.

[0010] According to some embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps: receiving pointing parameters sent by a remote control, and generating a trajectory line based on the received pointing parameters; wherein the pointing parameters change with the movement of the remote control; determining a preset shape region based on the trajectory line as a selected region corresponding to the pointing parameters; wherein the preset shape region corresponds to the region represented by the trajectory line; taking a screenshot of the region corresponding to the selected region on the currently displayed page to obtain a region image; and performing an image search based on the region image to display object information of objects in the region image. Attached Figure Description

[0011] Figure 1 is a schematic diagram of an operation scenario between a display device and a control device according to some embodiments of this application;

[0012] Figure 2 is a hardware configuration block diagram of a display device provided according to some embodiments of this application;

[0013] Figure 3 is a schematic diagram of the hardware configuration of a control device provided according to some embodiments of this application;

[0014] Figure 4 is a schematic diagram of the software configuration of a display device according to some embodiments of this application;

[0015] Figure 5 is a flowchart illustrating one of the selection screenshot methods provided according to some embodiments of this application;

[0016] Figure 6 is a schematic flowchart of generating trajectory lines according to some embodiments of this application;

[0017] Figure 7 is a schematic flowchart of generating trajectory lines according to some other embodiments of this application;

[0018] Figure 8 is a flowchart illustrating the process of determining a rectangular region according to some embodiments of this application;

[0019] Figure 9 is a flowchart illustrating the process of determining a rectangular region according to other embodiments of this application;

[0020] Figure 10 is a schematic diagram of determining the rectangular region corresponding to the regular trajectory line provided according to some embodiments of this application;

[0021] Figure 11 is a schematic diagram of determining the rectangular region corresponding to the irregular trajectory line according to some embodiments of this application;

[0022] Figure 12 is a schematic diagram of determining the rectangular region corresponding to an invalid trajectory line according to some embodiments of this application;

[0023] Figure 13 is a schematic flowchart of screenshot processing according to some embodiments of this application;

[0024] Figure 14 is one of the interactive diagrams between devices during the selection screenshot process according to other embodiments of this application;

[0025] Figure 15 is a second schematic flowchart of a selection screenshot method provided according to some embodiments of this application;

[0026] Figure 16 is a schematic diagram of the process for obtaining a status identifier according to some embodiments of this application;

[0027] Figure 17 is one of the schematic diagrams of an image search page provided according to some embodiments of this application;

[0028] Figure 18 is one of the schematic diagrams showing object information provided according to some embodiments of this application;

[0029] Figure 19 is a flowchart illustrating the process of determining a region image according to some embodiments of this application;

[0030] Figure 20 is a flowchart illustrating the process of determining a rectangular region according to some embodiments of this application;

[0031] Figure 21 is a flowchart illustrating the process of correcting the selected area according to some embodiments of this application;

[0032] Figure 22 is a second schematic diagram of the interaction between devices during the selection and screenshot process according to some other embodiments of this application;

[0033] Figure 23 is a third schematic flowchart of a selection screenshot method provided according to some embodiments of this application;

[0034] Figure 24 is a second schematic diagram of an object information display page provided according to some embodiments of this application;

[0035] Figure 25 is a third schematic diagram of an object information display page provided according to some embodiments of this application;

[0036] Figure 26 is a third schematic diagram of the interaction between devices during the selection and screenshot process according to some other embodiments of this application;

[0037] Figure 27 is a structural block diagram of a selection screenshot device provided according to some embodiments of this application. Detailed Implementation

[0038] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. Obviously, the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other related drawings based on these drawings without creative effort. The implementation methods described in the following embodiments do not represent all implementation methods consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application as detailed in the claims.

[0039] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0040] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate. The terms "comprising" and "having," and any variations thereof, are intended to be comprehensive but not exclusive; for example, a product or device comprising a series of components is not necessarily limited to all components explicitly listed, but may include other components not explicitly listed or inherent to such product or device. The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code capable of performing the functions associated with that element.

[0041] In some embodiments, the term "display device" refers to any device capable of displaying images and processing data. For example, display devices include, but are not limited to, televisions, mobile terminals, computers, monitors, advertising screens, wearable devices, virtual reality devices, and augmented reality devices.

[0042] Figure 1 is a schematic diagram of an operation scenario between a display device and a control device according to some embodiments of this application. As shown in Figure 1, a user can operate the display device 200 through touch operation, a mobile terminal 300, and a control device 100. For example, the control device 100 can be a remote control, a stylus, a gamepad, etc.

[0043] The mobile terminal 300 can serve as a control device for human-computer interaction between the user and the display device 200. The mobile terminal 300 can also serve as a communication device for establishing a communication connection with the display device 200 and exchanging data. In some embodiments, the mobile terminal 300 can install software applications with the display device 200 to establish a connection and communication via network communication protocols, achieving one-to-one control operations and data communication; it can also transmit audio and video content displayed on the mobile terminal 300 to the display device 200 to achieve synchronous display.

[0044] As also shown in Figure 1, the display device 200 communicates with the server 400 via various communication methods. The display device 200 can communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0045] Display device 200 can provide broadcast television reception function, and can also be equipped with intelligent network television function that provides computer support, including but not limited to network television, smart television, Internet Protocol Television (IPTV), etc.

[0046] Figure 2 is a hardware configuration block diagram of a display device according to some embodiments of this application. In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, at least one processor 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface 280.

[0047] In some embodiments, detector 230 is used to acquire signals from the external environment or to interact with the outside world. For example, detector 230 includes a light receiver, a sensor for acquiring ambient light intensity; or, detector 230 includes an image acquisition device, such as a camera, which can be used to acquire external environmental scenes, user attributes, or user interaction gestures; or, detector 230 includes a sound acquisition device, such as a microphone, for receiving external sounds.

[0048] In some embodiments, the display 260 includes display function components for presenting an image and driving components for driving the image display. The display 260 is used to receive and display image signals output from at least one processor 250. For example, the display 260 can be used to display video content, image content, menu control interface components, and user control UI interfaces, etc.

[0049] In some embodiments, the communication device 220 is a component used to communicate with external devices or the server 400 according to various communication protocol types. The display device 200 may have multiple communication devices 220 depending on the supported communication methods. For example, when the display device 200 supports wireless network communication, it may have a communication device 220 that includes the international wireless Fidelity (WiFi) standard. When the display device 200 supports Bluetooth connectivity, it needs to have a communication device 220 that includes Bluetooth functionality.

[0050] The communication device 220 enables the display device 200 to communicate with external devices or the server 400 via wireless or wired connections. Wired connections utilize data cables, interfaces, or other components to connect the display device 200 to external devices. Wireless connections utilize wireless signals or wireless networks. The display device 200 can directly establish a connection with external devices or indirectly through gateways, routers, or other connection devices.

[0051] In some embodiments, the processor may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and provide a first to an nth interface for input / output. At least one processor 250 controls the operation of the display device and responds to user operations via various software control programs stored in memory. At least one processor 250 controls the overall operation of the display device 200.

[0052] In some embodiments, at least one processor 250 and tuner 210 may be located in different separate devices, that is, tuner 210 may also be located in an external device of the main device where at least one processor 250 is located, such as an external set-top box.

[0053] In some embodiments, a user can input user commands through a graphical user interface (GUI) displayed on a display 260, and the user input interface 280 receives the user input commands through the graphical user interface (GUI).

[0054] In some embodiments, the audio output device 270 can be a built-in speaker of the display device 200 or an external audio output device connected to the display device 200. For the external audio output device connected to the display device 200, the display device 200 may also be provided with an external audio output terminal, through which the audio output device can be connected to the display device 200 to output sound from the display device 200.

[0055] In some embodiments, the user input interface 280 can be used to receive instructions from user input.

[0056] Figure 3 is a hardware configuration block diagram of a control device provided according to some embodiments of this application. As shown in Figure 3, the control device 100 may include: a control unit 110, a communication interface 130, a user input / output interface 140, a memory 190, and a power supply 180.

[0057] The control device 100 is configured to control the display device 200, and to receive user input operation commands and convert the operation commands into commands that the display device 200 can recognize and respond to, thus acting as an intermediary for interaction between the user and the display device 200.

[0058] In some embodiments, the control device 100 may be an intelligent device. For example, the control device 100 may be equipped with various applications for controlling the display device 200 according to user needs.

[0059] In some embodiments, as shown in FIG1, a mobile terminal 300 or other smart electronic device can perform similar functions to control device 100 after installing an application that controls display device 200.

[0060] The control device 110 includes a processor 112, a random access memory (RAM) 113, a read-only memory (ROM) 114, a communication interface 130, and a communication bus. The control device 110 is used to control the operation of the control device 100, as well as the communication and cooperation between internal components and the external and internal data processing functions.

[0061] Under the control of the control device 110, the communication interface 130 enables communication of control signals and data signals with the display device 200. The communication interface 130 may include at least one of other near-field communication modules such as WiFi chip 131, Bluetooth module 132, and Near Field Communication (NFC) module 133.

[0062] User input / output interface 140, wherein the input interface includes at least one of other input interfaces such as microphone 141, touchpad 142, sensor 143, button 144, etc.

[0063] In some embodiments, the control device 100 includes at least one of a communication interface 130 and a user input / output interface 140. The control device 100 is configured with the communication interface 130, such as a WiFi, Bluetooth, or NFC module, which can encode user input commands via WiFi, Bluetooth, or NFC protocols and send them to the display device 200.

[0064] The memory 190 is used to store various operating programs, data, and applications for driving and controlling the control device 100 under the control of the control device 110. The memory 190 can also store various control signal instructions input by the user.

[0065] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the control device 110.

[0066] In some embodiments, to enable user interaction, the display device 200 may run an operating system. The operating system is a computer program used to manage and control the hardware and software resources of the display device 200. The operating system can (control the display device) provide a user interface, allowing users to interact with the display device 200 and supporting the running of various applications.

[0067] It should be noted that the operating system can be a native operating system based on a specific operating platform, a third-party operating system that is deeply customized based on a specific operating platform, or an independent operating system specifically developed for display devices.

[0068] Operating systems can be divided into different modules or layers based on the functions they implement. For example, as shown in Figure 4, in some embodiments, the system is divided into four layers, from top to bottom: the Applications layer (referred to as the "Application Layer"), the Application Framework layer (referred to as the "Framework Layer"), the System Library layer, and the Kernel layer.

[0069] In some embodiments, the application layer provides services and interfaces for applications, enabling the display device 200 to run applications and interact with the user based on the applications. The application layer may contain at least one application, which may be a built-in Windows program, system settings program, or clock program of the operating system; or it may be an application developed by a third-party developer. In specific implementations, the application packages in the application layer are not limited to the examples above.

[0070] The framework layer provides application programming interfaces (APIs) and a programming framework for applications. The application framework layer includes predefined functions. It acts as a central processing unit, determining the actions taken by applications within the application layer. Through the API, applications can access system resources and obtain system services during execution.

[0071] As shown in Figure 4, in some embodiments, the application framework layer includes a view system, managers, and content providers. The view system designs and implements the application's interface and interactions, and includes lists, grids, text boxes, and buttons. Managers include at least one of the following modules: an Activity Manager for interacting with all running activities in the system; a Location Manager for providing system services or applications with access to system location services; a Package Manager for retrieving various information related to application packages currently installed on the device; a Notification Manager for controlling the display and clearing of notification messages; and a Window Manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0072] In some embodiments, the Activity Manager manages the lifecycle of individual applications and common navigation and back functions, such as controlling application exit, opening, and back actions. The Window Manager manages all window programs, such as obtaining the screen size, determining if a status bar is present, locking the screen, capturing the screen, and controlling changes to the display window, such as shrinking the display window, shaking the display, or distorting the display.

[0073] In some embodiments, the system runtime library layer can provide support for the framework layer. When the framework layer is used, the operating system runs the instruction library contained in the system runtime library layer, such as the C / C++ instruction library, to implement the functions to be performed by the framework layer.

[0074] In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. For example, as shown in Figure 4, the kernel layer can be configured with hardware drivers, which can be at least one of the following: audio driver, display driver, Bluetooth driver, camera driver, Wi-Fi driver, USB driver, HDMI driver, sensor driver (such as a fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0075] It should be noted that the above examples are merely a simple division of operating system functions and do not limit the specific form of the operating system of the display device 200 in some embodiments. Depending on the function of the display device, the type of operating system, and other factors, the number of layers and the specific type of the operating system may take other forms.

[0076] With the development of devices (such as smart TVs), some devices have integrated selection screenshot (or partial screenshot) functions to meet users' diverse information query needs for the displayed interface. For example, the device can take a screenshot of a portion of the displayed content and perform a search based on the captured image to display search results for that portion. However, in related technologies, the device can only recognize regularly shaped selection areas. If the user selects an irregular area, it cannot perform screenshot processing, thus preventing subsequent image searches.

[0077] Based on this, in some embodiments, a selection screenshot method is provided. This selection screenshot method can be implemented by a display device and a mobile terminal (such as a remote control). In one optional embodiment, taking the selection screenshot method applied to a display device and the mobile terminal as a remote control as an example, the display device includes a display, a memory, and at least one processor; the memory is configured to store computer programs or instructions; a communication connection exists between the display and at least one processor, which can be a wired or wireless communication connection; the memory is connected to at least one processor, and the at least one processor executes the computer program or instructions to cause the display device to perform the aforementioned selection screenshot method. As shown in Figure 5, the method specifically includes, but is not limited to, the following steps:

[0078] S510 receives the pointing parameters sent by the remote control and generates a trajectory line based on the received pointing parameters.

[0079] The so-called pointing parameter is used to characterize the direction in which the remote control points to the currently displayed page on the display device; furthermore, the pointing parameter changes with the movement of the remote control and may include, but is not limited to, the position information of the remote control.

[0080] Optionally, the remote control can be a pointing remote control, i.e., a remote control with an integrated pointing device. When the pointing device is activated, it can acquire the remote control's location information in real time, generate pointing parameters based on the remote control's location information, and send the pointing parameters to the display device in the form of an electrical signal using infrared / Bluetooth transmission at a preset parameter transmission frequency. After receiving the pointing parameters, the display device can convert the electrical signal into a digital signal and display the location indicated by the pointing parameters on the currently displayed page in the form of a cursor.

[0081] Alternatively, the remote control can also be an air mouse remote control, which integrates a gyroscope. Because the air mouse remote control is equipped with a gyroscope, it can sense changes in direction and speed when the user operates the remote control and generate pointing parameters based on these changes. These pointing parameters are then sent to the display device.

[0082] It's important to note that the location information contained in the pointing parameter is latitude and longitude in a geodetic coordinate system. Therefore, to more accurately display the location indicated by the pointing parameter on the current page, a two-dimensional planar coordinate system (or screen coordinate system) can be determined on the current page, using the lower left corner of the current page as the origin and based on the display device's resolution (mostly 1920*1080). Then, the latitude and longitude information in the geodetic coordinate system is converted into horizontal and vertical coordinates in the planar coordinate system. The so-called current page refers to the page (interface) displayed on the device at the current moment; it can be a playback page or a non-playback page (such as a homepage or search page).

[0083] The display device can simultaneously support multiple image processing functions, and different image processing functions correspond to different triggering methods. Specifically, it can include image processing functions such as screen recording, full-screen screenshot, and selected screenshot. For example, the image processing function used by the user can be determined based on the duration of the user pressing the screenshot control and / or the on / off state of the pointed device. The screenshot control is located in the remote control.

[0084] Optionally, if it is determined that the user is using the selection screenshot function based on the user's current triggering method, the display device can convert the position information in the pointing parameter into the coordinate information of the corresponding trajectory point on the current display page through a preset coordinate transformation method, and store the coordinate information of the trajectory point in the trajectory array associated with the current display page.

[0085] Then, based on the transmission order of each trajectory point in the trajectory array and the coordinate information corresponding to each trajectory point, the coordinate changes between each trajectory point are determined. Following this coordinate change, a trajectory line is generated on the currently displayed page. The trajectory line is the image selection track drawn by the user using a remote control. As a specific implementation method, the trajectory points can be connected according to their transmission order to generate the trajectory line.

[0086] Understandably, to avoid modifying the displayed page during the selection process, the generated trajectory lines are displayed in a layer above the video image on the monitor after the selection screenshot is taken. The area outside the trajectory lines in the upper layer is transparent. As a specific implementation, the display device can overlay a transparent canvas on the current display page and perform the trajectory line generation operation within the transparent canvas. In this case, the user can directly see the selection result on the current display page without modifying the current display page. Furthermore, if the user makes an incorrect selection, the trajectory lines on the transparent canvas can be cleared, allowing for a new selection screenshot.

[0087] In some embodiments, when a selection screenshot process is detected, a pen can be initialized, and when a new trajectory point is received, the pen is used to draw the position information corresponding to the trajectory point on a transparent canvas, thereby generating a trajectory line; when a selection stop event is detected, the current pen is destroyed, that is, the selection operation ends at this time.

[0088] S520 determines a preset shape area based on the trajectory line, which serves as the selection area corresponding to the pointing parameter.

[0089] The so-called preset shape area is the selection area that includes all the trajectory lines. The selection area is used to represent the area that the user wants to select. That is, the preset shape area corresponds to the area represented by the trajectory lines. In some embodiments, the preset shape area can be a rectangular area (including a square area, a rectangular area), a circular area, an elliptical area, a triangular area, etc.

[0090] Optionally, a preset shape region can be determined based on the horizontal and vertical coordinate parameters of each coordinate in the trajectory array associated with the trajectory line. It is worth noting that only one preset shape region can be configured to determine the preset shape region corresponding to the region represented by the trajectory line; alternatively, multiple candidate preset shape regions can be configured simultaneously. Before determining the preset shape region corresponding to the trajectory line, the target preset shape region that best fits the trajectory line can be selected from multiple candidate preset shape regions based on the trajectory line's selection trend, thereby determining the preset shape region corresponding to the region represented by the trajectory line.

[0091] For example, if the preset shape area is rectangular, the minimum and maximum values ​​of the horizontal and vertical coordinates of each trajectory point can be selected and combined to obtain the four vertices of the rectangular area, thus determining the rectangular area. For example, based on the minimum horizontal coordinate X-min, the maximum horizontal coordinate X-max, the minimum vertical coordinate Y-min, and the maximum vertical coordinate Y-max, the coordinate points (X-min, Y-min), (X-min, Y-max), (X-max, Y-max), and (X-max, Y-max) can be determined, and the rectangular area can be determined based on these coordinate points.

[0092] In some embodiments, when the preset shape region is circular, the maximum value of the difference between the horizontal and vertical coordinates can be determined based on the coordinates of each trajectory point. The larger of these two values ​​is then used as the diameter of the circular region. Next, the center point of the circular region is determined based on the mean of the horizontal and vertical coordinates of each trajectory point. At this point, an initial circular region can be determined based on the center point and diameter of the circular region.

[0093] Furthermore, it is determined whether the initial circular region completely contains all the trajectory points. If so, the initial circular region is directly used as the determined circular region; otherwise, the initial circular region is expanded based on the center until the initial circular region contains all the trajectory points, thereby determining the circular region.

[0094] Alternatively, the trajectory line can be input into a trained preset shape region determination model, which will then determine and output the preset shape region based on the trajectory line's position information and model parameters.

[0095] It's worth noting that regardless of the shape of the trajectory line selected by the user, a preset shape region encompassing all trajectory lines can be determined based on that trajectory line. Furthermore, since there are no restrictions on the shape or regularity of the trajectory lines, the above steps reduce the difficulty for users to select screenshots, improve the user experience, and ensure the smooth progress of subsequent image searches.

[0096] Understandably, to facilitate subsequent image search operations, the preset shape area can be optimized based on the current page type when determining it. In the case of a playback page, since it does not contain page controls, the above method can be used to determine the preset shape area solely based on the trajectory line.

[0097] When the current display page is a non-playback page, since non-playback pages contain various page controls (including image controls and text controls), these page controls can be used as auxiliary information for image search to improve accuracy. Specifically, based on the position information of the trajectory line, the trajectory-related controls associated with the trajectory line can be determined. Then, based on the coordinate information of the trajectory-related controls on the current display page and the position information of the trajectory line, a preset shape area containing both the trajectory line and the trajectory-related controls can be determined. It is not limited to only page controls located on or within the trajectory line being considered trajectory-related controls; alternatively, page controls associated with images within the trajectory line can also be considered trajectory-related controls.

[0098] S530: Take a screenshot of the selected area on the currently displayed page to obtain the area image.

[0099] The so-called region image refers to the image within the corresponding preset shape area on the currently displayed page.

[0100] Optionally, a screenshot can be taken of the area corresponding to the preset shape region on the currently displayed page based on the coordinate information of the preset shape region, thereby obtaining an area image. For example, if the preset shape region is a rectangle, a screenshot can be taken of the corresponding rectangular area on the currently displayed page based on the coordinate information of the rectangle, thereby obtaining an area image.

[0101] In some embodiments, when the preset shape area is a rectangular area, to ensure the adaptability between the image area and the display page, a screenshot of the image selected by the rectangular area in the currently displayed page can be taken to obtain the original image. Then, based on the aspect ratio of the display page, the original image can be optimized to obtain the area image. For example, when the aspect ratio of the display page is 16:9, the original image can be adjusted to an area image with an aspect ratio of 16:9 without modifying the image length; alternatively, the original image can be adjusted to an area image with an aspect ratio of 16:9 without modifying the image width.

[0102] S540 performs an image search based on the region image to display object information of objects in the region image.

[0103] The so-called object information refers to the relevant information of the objects contained in the region image. Furthermore, the above-mentioned objects include people and things, which can be plants and animals, characters played by actors, daily products, and other objects that can be displayed in the form of images.

[0104] Optionally, an image acquisition path corresponding to the region image can be generated based on the device identification information of the display device; then, the region image and its image acquisition path are simultaneously uploaded to the image recognition system in the cloud associated with the display device. To improve image upload efficiency, the region image can be compressed before uploading, and the compressed image and its image acquisition path can then be uploaded to the image recognition system.

[0105] After acquiring a region image, the image recognition system can identify the region image and, based on the image acquisition path, feed the recognition result back to the display device as object information of the objects in the region image. In some embodiments, the image recognition system can segment the region image based on image features to obtain the objects contained in each sub-image. Then, for each object contained in a sub-image, the system can use that object as index information to perform consistency matching among standard objects in a pre-determined image library, and use the object information corresponding to the standard object with the highest matching degree as the object information of that object. After obtaining the object information of the objects contained in each sub-image, the object information is simultaneously fed back to the display device.

[0106] Alternatively, you can directly perform an image search on the region image from the locally stored image information to obtain the object information of the objects in the region image.

[0107] Optionally, after obtaining object information, the display device can determine an information display overlay on the current display page and display the object information at a preset display position; alternatively, it can display the object information on the information display overlay based on the position information of each object in the display area image. In some embodiments, if there is only one object in the area image, the object information of that object is directly displayed on the information display overlay; if there are multiple objects in the area image, the object information of each object can be displayed sequentially in the information display overlay according to the positional arrangement of the objects.

[0108] The above scheme receives pointing parameters from a remote control that change with the remote control's movement and generates a trajectory line based on these parameters. Then, based on the trajectory line, a preset shape region containing any trajectory point is determined, and a screenshot of the corresponding selected area (i.e., the preset shape region) on the currently displayed page is taken to obtain a region image. Subsequently, an image search is performed based on the region image to display object information within the region image. Using this method, in the scenario of selecting and taking screenshots, regardless of the shape of the trajectory line selected by the user, a preset shape region containing all trajectory lines can be determined based on the trajectory line, and a screenshot of the currently displayed page can be taken based on this determined preset shape region. On the one hand, since there are no restrictions on the shape or regularity of the trajectory lines, the difficulty of selecting and taking screenshots is reduced, improving the user experience and ensuring the smooth progress of subsequent image searches, thus improving the efficiency of obtaining image search results. On the other hand, since the determined preset shape region contains all trajectory lines, it ensures that the region image obtained from the screenshot matches the user's actual query requirements, guaranteeing the accuracy of the region image acquisition and, consequently, the accuracy of the image search results displayed to the user.

[0109] Based on the above embodiments, in some embodiments, the process of generating the trajectory line is refined, as shown in Figure 6, and specifically includes the following steps:

[0110] S610, in response to a press event sent by the remote control for the screenshot control, performs a timing operation.

[0111] The so-called screenshot control is a control that enables the screenshot function on the display device; the so-called press event is an indication event sent to the display device when the user presses the screenshot control, which may include the timestamp of the user pressing the screenshot control.

[0112] Understandably, in order to help the display device distinguish the user's image processing needs, the corresponding function triggering method can be pre-configured for each image processing function based on the pressing duration of the screenshot control, so as to determine the corresponding image processing method based on the user's triggering method.

[0113] Optionally, when the remote control detects that the user has pressed the screenshot control on the remote, it can continuously send timing parameters to the display device. When the display device detects the timing parameters, it records the start timestamp of receiving the timing parameters and triggers the timing operation.

[0114] S620 takes a full-screen screenshot of the currently displayed page when it detects a lift event that indicates the screenshot control has been lifted, and the timeout period has not exceeded the preset duration.

[0115] The "release event" refers to the indicator event corresponding to when the user releases the screenshot control; the "timeout duration" is used to measure the duration of the press event. Furthermore, if the press event duration exceeds or equals the timeout duration, it is determined that the user has long-pressed the screenshot control; if the press event duration does not exceed the timeout duration, it is determined that the user has short-pressed the screenshot control.

[0116] Understandably, to improve the efficiency of full-screen screenshots, the trigger method can be configured to be a short press of the screenshot control. A full-screen screenshot is an image obtained by capturing the entire currently displayed page.

[0117] Optionally, the remote control can stop sending timing parameters to the display device when it detects that the user has released the screenshot control (i.e., the screenshot control is lifted). When the display device no longer detects the timing parameters, it can be considered to have listened to the lift event representing the lifting of the screenshot control, and at this time, the stop timestamp when receiving timing parameters is stopped will be recorded.

[0118] Alternatively, you can continuously monitor changes in the press parameters corresponding to press events, and consider any detected change in the press parameters as a detected release event indicating that the screenshot control has been released. For example, when the user presses the screenshot control, the press parameter is 100; when the user releases the screenshot control, the press parameter is 85. That is, if the received press parameter is 100, it proves that the user has continuously pressed the screenshot control; if the received press parameter changes to 85, a release event is detected.

[0119] In one alternative implementation, if a lift event indicating that the screenshot control has been lifted is detected and the timeout period has not exceeded the preset duration, it proves that the user has briefly pressed the screenshot control. At this time, the full-screen screenshot processing logic will be triggered to take a full-screen screenshot of the currently displayed page, thereby obtaining a full-screen image.

[0120] Furthermore, after obtaining the full-screen image, an image acquisition path for the full-screen image can be generated based on the device identification information of the display device. Then, the image acquisition paths of both the compressed full-screen image and the region image are simultaneously uploaded to the image recognition system associated with the display device. After acquiring the full-screen image, the image recognition system can recognize it and, based on the image acquisition path, feed the recognition result back to the display device as object information of objects within the full-screen image. Once the display device obtains the object information, it can determine an information display overlay on the current display page and display the object information at a preset display position.

[0121] S630 receives the pointing parameters sent by the remote control when the timing duration exceeds the preset duration and no lift event representing the lifting of the screenshot control is detected within the timing duration, and generates a trajectory line based on the received pointing parameters.

[0122] Understandably, since the time required to select and take a screenshot is relatively long, the trigger method for selecting and taking a screenshot can be configured to be pressed for a long period of time.

[0123] Optionally, if the timing exceeds the preset duration and no lift event indicating the screenshot control is lifted is detected within the timing duration, it proves that the user has long-pressed the screenshot control. At this time, the screenshot selection processing logic will be triggered, thereby receiving the pointing parameters sent by the remote control, and after converting the pointing parameters into trajectory points, determining the trajectory line based on the coordinate information of the trajectory points.

[0124] It is worth noting that, to ensure the accuracy of the trajectory line generated in the selected screenshot, the pointing parameters sent by the remote control are acquired and stored before the user's request for a selected screenshot is confirmed. After the selected screenshot processing logic is triggered, a trajectory line is generated based on the acquired pointing parameters and the real-time acquired pointing parameters. Then, referring to steps S520 and S530 above, a preset shape region is determined based on the coordinate information of each trajectory point in the trajectory line. This preset shape region is then used as the selected area corresponding to the pointing parameters. A screenshot of the corresponding selected area (i.e., the preset shape region) on the currently displayed page can then be taken to obtain a region image. Afterwards, the image search processing logic based on the region image is executed.

[0125] The above solution, on the one hand, equips the remote control with a screenshot control, providing users with the convenience to take screenshots of the pages displayed on the device; on the other hand, based on the duration of the user's press on the screenshot control, it provides two screenshot methods: a short press scenario (the timer duration exceeds the preset duration, and a release event indicating the screenshot control is released is heard within the timer duration) to take a full-screen screenshot of the currently displayed page, and a long press scenario (the timer duration exceeds the preset duration, and no release event indicating the screenshot control is released is heard within the timer duration) to take a selected screenshot of the currently displayed page. This not only broadens the screenshot scenarios of the display device but also further enhances the user experience.

[0126] Based on the above embodiments, in some embodiments, the process of generating trajectory lines is further refined, as shown in Figure 7, specifically including the following steps:

[0127] S710 reads from the local source the status parameters representing the switching state of the remote control's pointing switch.

[0128] The so-called pointing switch is a switch on the remote control that controls the transmission of pointing parameters. Furthermore, when the pointing device is on, the remote control can send pointing parameters to the display device; when the pointing device is off, the remote control cannot send pointing parameters to the display device. The so-called status parameter is a string that represents the state of the switch; for example, a status parameter of 1 indicates that the pointing switch is on, and a status parameter of 0 indicates that the pointing switch is off.

[0129] Optionally, the user can modify the on / off state of the directional switch by triggering the directional switch in the remote control. When the on / off state of the directional switch changes, the remote control sends a status update request containing the current status parameters of the directional switch to the display device. After receiving the status update request, the display device stores the current status parameters locally.

[0130] It is understandable that whether a remote control can send pointing parameters to a display device is related to the image processing method triggered by the display device. A display device can simultaneously support multiple image processing functions, and different image processing functions correspond to different triggering methods. Specifically, this may include image processing functions such as screen recording, full-screen screenshot, and selected screenshot. Therefore, in response to a press event sent by the remote control targeting the screenshot control, the status parameters representing the on / off state of the remote control's pointing switch can be obtained from locally stored information.

[0131] It is worth noting that when the pointing switch is turned on, the remote control can send pointing parameters to the display device regardless of whether the user presses the screenshot control. However, the pointing parameters will not form a trajectory line at this time, but will be presented in the form of moving the cursor.

[0132] S720, when the timing duration exceeds the preset duration, no lift event indicating the screenshot control is lifted is detected within the timing duration, and the state parameter indicating the pointing switch is in the on state, receives the pointing parameter sent by the remote control and generates a trajectory line based on the received pointing parameter.

[0133] Specifically, when the pointing switch is in the on state, the remote control sends pointing parameters.

[0134] It is understandable that when the pointing switch is in the on state, the remote control sends pointing parameters. At this time, the display device can execute the processing logic of selecting and taking screenshots based on the pointing parameters.

[0135] Optionally, if the timing exceeds a preset duration, no lift-up event indicating the screenshot control has been detected within the timing duration, and the state parameter indicating the pointing switch is in the on state, the pointing parameter sent by the remote control is received, and a trajectory line is generated based on the trajectory points corresponding to the pointing parameter. Then, referring to steps S520 and S530 above, a preset shape area is determined based on the coordinate information of each trajectory point in the trajectory line. After this preset shape area is used as the selected area corresponding to the pointing parameter, a screenshot can be taken of the area corresponding to the selected area, i.e., the preset shape area, on the currently displayed page to obtain the area image.

[0136] Based on the device identification information of the display device, an image acquisition path for the region image is generated. Then, the compressed region image and its acquisition path are simultaneously uploaded to the image recognition system associated with the display device. After acquiring the region image, the image recognition system can identify the region image and, based on the acquisition path, feed back the recognition result as object information of the objects in the region image to the display device. Once the display device receives the object information, it can define an information display overlay on the current display page and display the object information at a preset location.

[0137] S730 will record the screen of the currently displayed page if the timer exceeds the preset time, no lift event indicating the screenshot control is lifted is detected within the timer, and the status parameter indicates the switch is in the off state.

[0138] Understandably, when the pointing switch is off, the remote control will not send pointing parameters to the display device. This means the display device cannot execute the selection / screenshot processing logic. In this case, screen recording logic can be configured for the display device. Specifically, if the timer exceeds a preset duration, no release event indicating the screenshot control has been detected within the timer, and the pointing switch is off, the display device will trigger screen recording logic to record the currently displayed page. The recording result will then be stored locally for subsequent processing by the user, such as sending the recording.

[0139] The above solution, on the one hand, provides a prerequisite for the smooth operation of the selection screenshot function by configuring a directional switch in the remote control; on the other hand, based on the on / off state of the directional switch, it provides two processing methods: screen recording and selection screenshot, which not only broadens the usage scenarios of the display device, but also improves the user experience.

[0140] Furthermore, based on the above embodiments, the image processing logic can also be determined solely based on the state parameters of the switch state. Specifically, the state parameters representing the on / off state of the remote control's pointing switch are read locally; if the state parameters indicate that the pointing switch is in the off state, the current display page is recorded; if the state parameters indicate that the pointing switch is in the on state, the pointing parameters sent by the remote control are received, and a trajectory line is generated based on the received pointing parameters.

[0141] To ensure the accuracy of the aforementioned predetermined shape region determination, based on the above embodiments, the predetermined shape region determination process is refined in some embodiments, as shown in Figure 8, specifically including the following steps:

[0142] S810 compares the width value of the trajectory line in the screen width direction and the length value in the screen length direction with the first threshold respectively.

[0143] The width value is the maximum difference in the horizontal coordinates between points on the trajectory line, and the length value is the maximum difference in the vertical coordinates between points on the trajectory line. For example, if the coordinate unit on the current display page is pixels, for horizontal coordinates x1, x2, and x3, the difference between x1 and x2 is 5 pixels, the difference between x1 and x3 is 10 pixels, and the difference between x2 and x3 is 5 pixels, so the width value is 10 pixels; for vertical coordinates y1, y2, and y3, the difference between y1 and y2 is 4 pixels, the difference between y1 and y3 is 10 pixels, and the difference between y2 and y3 is 6 pixels, so the length value is 10 pixels.

[0144] The first threshold is a numerical value used to determine whether a trajectory line is valid. For example, the first threshold can be 100 pixels. In some embodiments, the first threshold can be set based on the experience of those skilled in the art or determined based on a large amount of experimental data, and there is no limitation thereto.

[0145] It is understandable that in practical applications, there may be issues such as users making non-standard selections, resulting in a shorter selection length in a certain direction. This would lead to a smaller predetermined shape area, thus failing to guarantee the accuracy of image search results.

[0146] Based on this, in order to ensure the rationality of the determination of the preset shape region, the width value of the trajectory line in the screen width direction and the length value in the screen length direction can be compared with the first threshold respectively, and the subsequent preset shape region determination process can be performed according to the comparison result.

[0147] S820, when the width value is greater than or equal to the first threshold, or the length value is greater than or equal to the first threshold, determines a preset shape region based on the trajectory line, which is used as the selection region corresponding to the pointing parameter.

[0148] It is understandable that a width value greater than or equal to the first threshold, or a length value greater than or equal to the first threshold, can include both the width and length values ​​being greater than or equal to the first threshold, and only one of the width and length values ​​being greater than or equal to the first threshold. In both of these cases, the area selected by the user is valid.

[0149] Optionally, if the width and length of the trajectory line are both greater than or equal to the first threshold, it proves that the area selected by the user is valid in both the length and width directions. In this case, the preset shape area can be determined directly based on the coordinate information of each trajectory point in the trajectory line.

[0150] If either the width or length value of the trajectory line is less than a first threshold, the validity of the length / width value less than the first threshold can be further verified. If the length / width value less than the first threshold is valid, the preset shape region can still be determined based on the trajectory line; if the length / width value less than the first threshold is invalid, it needs to be optimized to determine the preset shape region. For example, a preset valid length can be used to replace the length / width value less than the first threshold to determine the preset shape region.

[0151] S830 outputs a screenshot failure message if both the width and length values ​​are less than the first threshold.

[0152] The so-called screenshot failure information refers to the prompt information displayed by the device when the selected screenshot fails, which may include, but is not limited to, information about the reason for the screenshot failure.

[0153] Optionally, if both the width and length of the trajectory line are less than the first threshold, it indicates that the area selected by the user is too small. In this case, the selected area is invalid, and the display device can define a floating layer on the current display page and output a screenshot failure message in the floating layer in the form of text. For example, the floating layer can output text such as "Selected area is too small, please select again".

[0154] The above solution introduces a first threshold to verify the width and length of the trajectory line along the screen width and length, respectively, to determine its validity. Only when the trajectory line is deemed valid can the operation of determining a preset shape region based on it be performed. This ensures that the region image obtained based on the preset shape region contains valid image information, thereby improving the accuracy of image search results. Furthermore, if the trajectory line is invalid, a screenshot failure message can be output to inform the user, further enhancing the user experience.

[0155] When the preset shape area is a rectangular area, in order to ensure the accuracy of the rectangular area determination, based on the above embodiments, the rectangular area determination process is further refined in some embodiments, as shown in Figure 9, specifically including the following steps:

[0156] S910, if the first value is greater than the first threshold and the second value is greater than the second threshold, select the minimum value of the horizontal coordinate, the maximum value of the horizontal coordinate, the minimum value of the vertical coordinate, and the maximum value of the vertical coordinate from the coordinate data of the trajectory points on the trajectory line.

[0157] In this system, the first value is one of the length and width values, and the second value is the other of the length and width values; the first threshold is greater than the second threshold. Further, the second threshold is the value used to determine whether the length / width value of the trajectory line is valid. For example, the second threshold can be 20 pixels. In some embodiments, the second threshold can be set based on the experience of those skilled in the art, or it can be determined based on a large amount of experimental data; there is no limitation on this.

[0158] In this case, the first value is greater than the second value. For example, when the length value is greater than the width value, the length value is the first value and the width value is the second value; when the width value is greater than the length value, the width value is the first value and the length value is the second value.

[0159] Optionally, if the first value of the length and width is greater than the first threshold and the second value is greater than the second threshold, it proves that both the length and width values ​​are valid. In this case, the minimum horizontal coordinate, maximum horizontal coordinate, minimum vertical coordinate, and maximum vertical coordinate can be filtered from the coordinate data of each trajectory point in the trajectory line.

[0160] It is worth noting that the second value in the length and width values ​​being greater than the second threshold also includes the second value in the length and width values ​​being greater than the first threshold. Therefore, in both cases where the length and width values ​​are both greater than the first threshold, and where the first value in the length and width values ​​is greater than the first threshold and the second value is between the first and second thresholds, step S920 can be performed.

[0161] S920, by combining the minimum and maximum x-coordinates, minimum and maximum y-coordinates, at least two vertex coordinates are obtained.

[0162] Among them, at least two vertex coordinates are used to represent the rectangular region.

[0163] Optionally, by combining the minimum and maximum x-coordinates, as well as the minimum and maximum y-coordinates, at least two vertex coordinates can be obtained. Subsequently, a rectangular region is determined based on the obtained vertex coordinates. For example, vertex coordinates (X-min, Y-min), (X-min, Y-max), (X-max, Y-max), and (X-max, Y-max) can be determined based on the minimum x-coordinate X-min, maximum x-coordinate X-max, minimum y-coordinate Y-min, and maximum y-coordinate Y-max, thereby determining a rectangular region based on these vertex coordinates.

[0164] Understandably, when only two vertex coordinates are determined, the two vertex coordinates must be diagonal coordinates. For example, when only two vertex coordinates are determined based on the minimum x-coordinate X-min, the maximum x-coordinate X-max, the minimum y-coordinate Y-min, and the maximum y-coordinate Y-max, the two vertex coordinates can be (X-min, Y-min) and (X-max, Y-max), or they can be (X-min, Y-max) and (X-max, Y-max).

[0165] For example, refer to the schematic diagram of the rectangular region corresponding to the regular trajectory line shown in Figure 10. When the width and length values ​​of trajectory line 1 are both greater than the first threshold, the coordinate points (X-min, Y-min) and (X-max, Y-max) can be directly determined based on the minimum horizontal coordinate X-min, maximum horizontal coordinate X-max, minimum vertical coordinate Y-min, and maximum vertical coordinate Y-max in trajectory line 1, thereby determining the rectangular region 2 based on the above coordinate points.

[0166] Understandably, if the width and length values ​​of the trajectory line are both greater than a first threshold and a second threshold, and the trajectory line is an intersecting irregular curve, the rectangular region can be determined based on the extreme values ​​of the horizontal and vertical coordinates of the trajectory points. For example, refer to Figure 11 for a schematic diagram of the rectangular region determination corresponding to an irregular trajectory line. When both the width and length values ​​of trajectory line 3 are greater than the first threshold, the coordinate points (X-min, Y-min) and (X-max, Y-max) can be directly determined based on the minimum horizontal coordinate X-min, maximum horizontal coordinate X-max, minimum vertical coordinate Y-min, and maximum vertical coordinate Y-max of trajectory line 3, thereby determining the rectangular region 4 based on these coordinate points.

[0167] The above solution introduces a second threshold to validate length / width values ​​smaller than the first threshold. If a length / width value smaller than the first threshold is valid, the vertex coordinates of the rectangular region are determined directly based on the minimum and maximum x-coordinates, minimum and maximum y-coordinates of the trajectory points. This provides an alternative method for quickly determining rectangular regions while ensuring that the subsequently acquired region image contains valid image information. Furthermore, this solution presents the region determined by the trajectory line as a rectangle, a presentation method that adapts to the way the display device presents the page, thus making the captured region image more reasonable.

[0168] In the case where the preset shape area is a rectangular area, based on the above embodiments, in some embodiments, another optional method for determining vertex coordinates is provided. Specifically, when the first value is greater than the first threshold and the second value is less than the second threshold, at least two vertex coordinates are determined according to the first value, the trajectory point on the trajectory line corresponding to the first value, and the screen aspect ratio of the display device.

[0169] Here, the first value is one of the length and width values, and the second value is the other. The screen aspect ratio is the ratio between the screen length and the screen width, which can be the ratio between the length and width values ​​of the currently displayed page. The aspect ratio of the rectangular area represented by at least two vertex coordinates is the same as the screen aspect ratio, and the first threshold is greater than the second threshold. If the second value of the length and width is less than the second threshold, it proves that the length / width value of the trajectory line that is less than the second threshold is invalid, that is, the length / width of the selected area is invalid.

[0170] In one alternative implementation, to generate a valid rectangular area when the trajectory line is invalid (either the length or width value is invalid), a target value can be calculated based on the valid length and width values ​​and the aspect ratio of the display device's screen. As a specific implementation, when the length value of the trajectory line is invalid, an optimized length value can be calculated based on the width value and the screen aspect ratio. For example, if the trajectory line length is 10 pixels, the width is 90 pixels, and the screen aspect ratio is 16:9, the optimized length value can be calculated as 160 pixels. Similarly, if the width value of the trajectory line is invalid, an optimized width value can be calculated based on the length value and the screen aspect ratio. For example, if the trajectory line width is 10 pixels, the length is 160 pixels, and the screen aspect ratio is 16:9, the optimized width value can be calculated as 90 pixels.

[0171] Next, based on the selection direction of the trajectory line, the corresponding expansion direction is determined, and the selected area is expanded according to the corresponding expansion direction to obtain the expanded selected area. When the trajectory line is a straight line, the expansion direction cannot be determined. In this case, the expansion direction can be chosen based on the image richness on both sides of the trajectory line, using the side with higher image richness as the expansion direction.

[0172] For example, when the width value is valid but the length value is invalid, an initial selection area can be determined based on the current width and length values. Then, based on the selection direction of the trajectory line, the length expansion direction of the initial selection area is determined, and the length of the initial selection area is expanded to the target value in the length expansion direction, thus obtaining the expanded selection area. Alternatively, when the length value is valid but the width value is invalid, an initial selection area can be determined based on the current width and length values. Then, based on the selection direction of the trajectory line, the width expansion direction of the initial selection area is determined, and the width of the initial selection area is expanded to the target value in the width expansion direction, thus obtaining the expanded selection area.

[0173] Optionally, after determining the expanded selection area, the minimum and maximum values ​​of the horizontal coordinate, the minimum and maximum values ​​of the vertical coordinate in the expanded selection area can be combined horizontally and vertically to obtain at least two vertex coordinates.

[0174] For example, refer to the schematic diagram of determining the rectangular region corresponding to the invalid width trajectory line shown in Figure 12. An initial rectangle 6 that completely contains trajectory line 5 can be determined first; at this point, the width value of the initial rectangle 6 is invalid. Based on the selection direction of trajectory line 5, the expansion direction of the initial rectangle 6 can be determined to be downward. Then, based on the length value of the initial rectangle 6 and the screen aspect ratio, the optimized width value can be calculated, and the width of the initial rectangle 6 can be extended downwards to the optimized width value, thus obtaining the rectangular region 7.

[0175] In another alternative implementation, a standard rectangular area can be determined based on the screen aspect ratio, and two coordinate points corresponding to the first value can be determined based on the coordinate values ​​of each trajectory coordinate in the trajectory line; then, based on the two coordinate points, the standard rectangular area is scaled so that the length of one side of the scaled standard rectangular area is the first value, and the scaled standard rectangular area contains the trajectory line, thereby obtaining the rectangular area.

[0176] The above solution, by incorporating the screen aspect ratio of the display device, eliminates the need for manual modification when the trajectory line's value in any direction of the screen is small (i.e., invalid). Instead, it determines the rectangular area based on the screen aspect ratio and the trajectory line's value in the effective screen direction. This improves the ease of user selection and provides an alternative method for quickly determining rectangular areas while ensuring that the subsequently acquired area image contains valid image information. Furthermore, the solution presents the area determined by the trajectory line as a rectangle, adapting to the way the display device presents the page, thus making the captured area image more reasonable.

[0177] When the preset shape area is a rectangular area, based on the above embodiments, in some embodiments, the screenshot processing process is further refined, as shown in Figure 13, specifically including the following steps:

[0178] S1310 corrects the rectangular area according to the screen aspect ratio when the aspect ratio of the rectangular area is different from that of the display device.

[0179] The aspect ratio of the modified rectangular area is the same as that of the screen.

[0180] Optionally, after determining the rectangular area, you can first determine the aspect ratio of the rectangular area and compare it with the screen aspect ratio. If the aspect ratio of the rectangular area is the same as the screen aspect ratio, you can directly take a screenshot of the corresponding rectangular area on the currently displayed page to obtain the area image; if the aspect ratio of the rectangular area is different from the screen aspect ratio, you need to correct the rectangular area according to the screen aspect ratio.

[0181] For example, when the aspect ratio of the rectangular area is greater than that of the screen, the length of the rectangular area can be shortened based on the center point of the rectangular area, so that the aspect ratio of the modified rectangular area is the same as that of the screen; when the aspect ratio of the rectangular area is less than that of the screen, the width of the rectangular area can be shortened based on the center point of the rectangular area, so that the aspect ratio of the modified rectangular area is the same as that of the screen.

[0182] For example, if the aspect ratio of the rectangular area is 16:10, the screen aspect ratio is 16:9, and the width of the rectangular area is 100 pixels, then the width of the rectangular area needs to be reduced. That is, without changing the length of the rectangular area, the width needs to be reduced by 10 pixels. In this case, we can use the center point of the rectangular area as a reference and shrink both sides inward by 5 pixels simultaneously to obtain the modified rectangular area.

[0183] S1320: Take a screenshot of the area within the corresponding corrected rectangular area on the currently displayed page to obtain the area image.

[0184] Optionally, a screenshot can be taken of the area within the corresponding corrected rectangular region on the currently displayed page to obtain an image of the area with the same aspect ratio as the screen.

[0185] The above solution corrects the rectangular area according to the screen aspect ratio to obtain an area image with the same aspect ratio as the screen, ensuring the adaptability between the area image and the current display page, and thus making the captured area image more reasonable.

[0186] Based on the solutions of the above embodiments, an optional embodiment is provided when the display device includes a television system and functional applications, and the preset shape area is a rectangular area. Referring to Figure 14, which shows a schematic diagram of the interaction between devices during the selection and screenshot process, it includes:

[0187] S1401: When the remote control detects a user pressing the screenshot area, it sends a press event for the screenshot control to the TV system.

[0188] Specifically, before step S1401, the user presses the screenshot control on the remote control (corresponding to S1401' in Figure 14).

[0189] S1402, after receiving the press event, the TV system sends a start command to the screenshot application to instruct the screenshot application to perform a timing operation.

[0190] S1403, the screenshot application performs a timing operation.

[0191] S1404: If the screenshot application sends a full-screen screenshot command to the TV system within the time limit period and the time limit period is not exceeded, and a lift event indicating that the screenshot control has been lifted is detected within the time limit period.

[0192] S1405, the TV system takes a full-screen screenshot of the currently displayed page, obtains the target screenshot, and executes S1418.

[0193] S1406, if the screenshot application sends a request to the TV system to obtain the on / off status when the timer duration exceeds the preset duration and no lift event indicating that the screenshot control has been lifted is detected within the timer duration.

[0194] S1407, the television system reads from the local source the status parameters representing the on / off state of the remote control's pointing switch.

[0195] S1408, the TV system sends status parameters back to the screenshot application.

[0196] S1409, the screenshot application determines whether the directional switch is turned on based on the status parameters. If not, it executes S1410; if so, it executes S1414.

[0197] S1410, the screenshot application sends a screen recording command to the TV system, instructing the TV system to record the currently displayed page.

[0198] S1411, the television system performs screen recording.

[0199] S1412, The remote control continuously sends pointing parameters to the TV system to instruct the TV system to send the pointing parameters to the screenshot application.

[0200] S1413, the TV system sends a pointer parameter to the screenshot application.

[0201] S1414, the screenshot application receives the pointing parameters sent by the remote control, and generates a trajectory line based on the received pointing parameters when a lift event is detected.

[0202] Specifically, before step S1414, the user releases the screenshot control (corresponding to S1414' in Figure 14).

[0203] S1415, the screenshot application determines a rectangular area based on the trajectory line, which serves as the selected area corresponding to the pointing parameter.

[0204] S1416, The screenshot application sends a selection screenshot command containing the coordinate information of a rectangular area to the television system.

[0205] S1417, the television system takes a screenshot of the currently displayed page based on the coordinate information of the rectangular area to obtain the target screenshot.

[0206] S1418, the TV system sends the target screenshot and the screenshot acquisition path of the target screenshot to the screenshot application.

[0207] S1419, The screenshot application uploads the target screenshot and the screenshot acquisition path to the image recognition system.

[0208] S1420, the image recognition system performs an image search based on the target screenshot to determine the object information of the object in the target screenshot.

[0209] S1421, the image recognition system, based on the screenshot acquisition path, feeds back the object information of the object in the target screenshot to the screenshot application.

[0210] S1422, The screenshot application sends the object information of the object in the target screenshot to the television system so that the object information can be displayed on the television system.

[0211] S1423, The television system displays object information.

[0212] It should be noted that when the remote control's pointing switch is on, the remote control can periodically send pointing parameters to the television system. That is, when the user presses the screenshot control on the remote control, if the pointing switch is on, the remote control can carry pointing parameters in S1401 when sending a press event for the screenshot control to the television system. The above embodiment describes the remote control sending pointing parameters to the television system in S1412, but this does not mean that the remote control has not sent pointing parameters to the television system before S1412; it is merely an example.

[0213] Furthermore, the specific processes of S1401-S1423 described above can be found in the description of the above method embodiments, and their implementation principles and technical effects are similar, so they will not be repeated here.

[0214] In practical applications, when taking a screenshot of a dynamic scene, it takes a certain amount of time for the user to draw the selection area with the cursor. During the selection process, the screen keeps changing, and by the time the selection area is completed, the playing image has already changed. The actual image recognition uses a screenshot taken at the moment the user triggers the selection (the starting moment of drawing the selection area). To ensure that the screen the user sees when the selection is complete is consistent with the screen displayed after the selection is completed (the screenshot taken at the starting moment of drawing the selection area), in some embodiments, as shown in Figure 15, the display device can also perform the following steps to improve the accuracy of the selected screenshot:

[0215] S1510, in response to the received preset press command, takes a screenshot of the display interface to obtain a screen image.

[0216] The screen image refers to the interface image on which the user intends to perform the selection operation. The following explanation uses pointing to a remote control as an example, but it should be noted that this solution is not limited to pointing to a remote control; it can also include the aforementioned air mouse remote control, etc., which will not be elaborated further. The preset press command indicates that a preset button on the remote control is pressed. The preset button can be a screenshot control on the remote control; further, a screenshot control is a control that enables the screenshot function on the display device.

[0217] Optionally, when a user needs to select a specific area for a screenshot, a preset press command can be sent to the display device by pressing a preset button on the remote control. Correspondingly, in one optional implementation, after receiving the preset press command, the display device can launch a screenshot application, which then uses the screenshot function provided by the display device's system to capture the currently displayed interface, obtaining a screen image. This screen image can then be cached in local storage.

[0218] It is worth noting that due to a certain time delay between the user's screenshot request moment (the moment the user presses the remote control) and the moment the display device executes the screenshot, the screen image acquired by the display device may be inconsistent with the image the user wants to capture. Therefore, in another optional implementation, previously played historical images (i.e., the interface already displayed on the screen) can be cached locally while the dynamic video is playing. After receiving a preset press command, the display device can determine the user's actual screenshot request moment based on the current moment and the preset image acquisition delay; then, it can retrieve the screen image from the local cache based on the screenshot request moment.

[0219] The image acquisition latency can be determined by pre-collecting behavioral data (including image data of the user pressing a preset button) of each user associated with the display device when they are selecting a screenshot. Then, based on this behavioral data, the image acquisition latency for that user is analyzed (e.g., the time difference between the moment the user presses the preset button and the moment they send a preset press command to the remote control is used as the image acquisition latency). This latency is then associated with the user's identity information and stored locally. Furthermore, when any user is detected sending a preset press command to the remote control, an image acquisition device (e.g., a camera) can capture the user's facial image to confirm their identity. Finally, based on the user's identity information, the corresponding image acquisition latency is retrieved from the local cache.

[0220] For example, if the image acquisition delay for user A is 1 second, and a preset press command sent by user A to the remote control is received at the current moment, then the user's actual screenshot request time is determined to be 1 second before the current moment. In this case, the historical image corresponding to the 1 second before the current moment stored in the cache can be used as the screen image.

[0221] S1520: When the current interface displays a dynamic image, control the monitor to display the screen image, and take a screenshot of the screen image based on the selected area to obtain the area image.

[0222] The "current interface" refers to the currently displayed screen (page) on the monitor, which can show either dynamic or static images. Dynamic images are those whose screen content changes in real time, such as a video playback screen; static images are those whose screen content remains unchanged, such as a homepage or search page (not a playback page). When the current interface displays a dynamic image, the "region image" refers to the image corresponding to the selected area on the screen.

[0223] The selected area in this embodiment is determined based on the pointing parameters sent by the remote control, and is used to represent the area that the user wants to select. The pointing parameters have been introduced in the previous embodiments, and will not be repeated here. It should be noted that the current interface in this embodiment is the aforementioned current display page.

[0224] In some embodiments, the pointing parameters include the horizontal and vertical coordinates of the pointer on the display device in the screen coordinate system after coordinate conversion. The display device can directly determine the display position of the cursor based on the received horizontal and vertical coordinate information.

[0225] In one alternative approach, the determination of whether the current interface displays a dynamic image can be based on the operating state of the display device. In some embodiments, the operating state may include the playback state of the multimedia player and / or the application state of the current interactive application, thereby determining whether the current interface displays a dynamic image based on the playback state of the multimedia player and / or the application state of the current interactive application. For example, if the multimedia player is in playback mode and / or the current interactive application is playing media content, it is confirmed that the current interface displays a dynamic image.

[0226] In another alternative approach, the decision to display a dynamic image can be based on the changes between locally cached historical frames. For example, if there are changes between consecutive cached historical frames, the current interface is determined to display a dynamic image.

[0227] When the current interface displays a dynamic image, to ensure the accuracy of the user's selection, the monitor can be controlled to display a screen image. Using a preset coordinate transformation method, the position information in the obtained pointing parameters is converted into the coordinate information corresponding to the selection cursor on the current interface. Based on the coordinate changes of the selection cursor, the selection area is determined in the displayed screen image; alternatively, the selection area can be directly determined in the displayed screen image based on the horizontal and vertical coordinate information contained in the pointing parameters. Then, a screenshot of the area corresponding to the selected area in the screen image is taken to obtain the area image. Next, step S1540 is executed.

[0228] S1530: When the current interface displays a static image, delete the screen image and take a screenshot of the static image based on the selected area to obtain the area image.

[0229] In the case of a static screen on the current interface, the so-called region image is the image corresponding to the selected area in the static screen.

[0230] In some embodiments, when it is determined that the current interface displays a static image, since the image displayed on the monitor has not changed, the saved screen image can be deleted, and a selection screenshot can be taken directly from the static image. Specifically, the position information in the obtained pointing parameters can be converted into the coordinate information corresponding to the selection cursor in the current interface using a preset coordinate transformation method. Based on the coordinate changes of the selection cursor, the selection area can be determined in the static image; alternatively, the selection area can be determined directly based on the horizontal and vertical coordinate information contained in the pointing parameters. Further, a screenshot of the area corresponding to the selection area in the static image is taken to obtain the area image. Then, step S1540 is executed.

[0231] S1540 performs image recognition based on the region image to display object information of objects in the region image.

[0232] For an explanation of step S1540, please refer to the explanation of step S540 above, which will not be repeated here.

[0233] In some embodiments, taking a person as an example, when performing step S1540, the display device can perform image recognition based on the person in the area image to display the object information of the person in the area image.

[0234] The above solution, on the one hand, captures a screenshot of the display interface upon receiving a preset press command, obtaining a screen image. When the current interface displays a dynamic scene, the solution controls the display to show this image. Since the screen image shown to the user is a still image captured when the user points to a preset button on the remote control, the screenshot is taken from the screen image based on the pointing parameters sent by the remote control. The resulting area image matches the user's actual needs, achieving the effect that when the display shows a dynamic scene, the image seen by the user after the selection is completed is consistent with the displayed interface, thus ensuring the accuracy of the selection screenshot. On the other hand, when the current interface displays a static scene, the screenshot is taken directly from the static scene displayed on the display, resulting in a high-resolution area image shown to the user. Furthermore, this solution can select the corresponding selection screenshot method based on the type of screen displayed (dynamic / static), not only expanding the application scenarios of the display device's screenshot function but also improving the user experience.

[0235] Based on the above embodiments, in some embodiments of this application, an optional method is provided to determine whether the current interface displays a dynamic or static image. Specifically, a status identifier of the multimedia player is obtained, and the type of image displayed on the current interface is determined based on the status identifier. When the status identifier indicates that the multimedia player is in a playback state, the current interface displays a dynamic / video image; when the status identifier indicates that the multimedia player is in a non-playback state, the current interface displays a static image.

[0236] The so-called status identifier is an identifier used to represent the state of the multimedia player, such as MEDIA PLAYER, and its value can be represented by a numerical value. For example, when the status identifier is 1, it is determined that the multimedia player is in playback state, that is, the current interface displays dynamic images / video images; when the status identifier is 0, it is determined that the multimedia player is in non-playback state, that is, the current interface displays static images.

[0237] Optionally, upon receiving a preset press command, the system can query the current status identifier of the multimedia player in the display device's overall system based on the multimedia player's device identifier.

[0238] As an alternative method for processing screen images, when the status identifier indicates that the multimedia player is in a playback state, the display device can control the monitor to display the screen image; when the status identifier indicates that the multimedia player is not in a playback state, the screen image is deleted.

[0239] Optionally, after obtaining the status identifier, read the value of the status identifier. If the value is 1, it indicates that the multimedia player is in playback mode, and the screen image can be displayed on the monitor. If the value is 0, it indicates that the multimedia player is in non-playback mode, and the screen image can be deleted.

[0240] The above solution introduces a status identifier for the multimedia player. By obtaining the status identifier of the multimedia player at the current moment, it can be determined whether the multimedia player is in a playback state. Then, based on the determination result, the corresponding screen image processing method is selected, which can ensure the rationality of screen image processing.

[0241] Based on the above embodiments, in some embodiments of this application, the process of obtaining the status identifier of the multimedia player is refined, as shown in FIG16, and specifically includes, but is not limited to, the following steps:

[0242] S1610, upon receiving a preset press command, reads from the local machine the status parameter representing the switch state of the pointer switch pointing to the remote control.

[0243] In some embodiments, when performing step S1610, the display device may, in response to a preset press command sent by the remote controller, obtain a status parameter representing the on / off state of the pointing switch pointing to the remote controller from the locally stored current device information. It should be noted that details regarding the aforementioned pointing switch, the status parameter of the pointing switch's on / off state, and the status update of the status parameter are provided in the description of step S710 above, and will not be repeated here.

[0244] S1620: When the state parameter indicates that the switch is in the on state, obtain the state identifier of the multimedia player.

[0245] Optionally, when the state parameter indicating the pointing switch is in the off state, the pointing remote control will not send pointing parameters, meaning the display device cannot execute the selection screenshot processing logic. In this case, the image processing method triggered by the display device can be screen recording / full-screen screenshot. When the state parameter indicating the pointing switch is in the on state, the pointing remote control can send pointing parameters, and the image processing method triggered by the display device can be selection screenshot.

[0246] Subsequently, if it is determined that the image processing method triggered by the display device is a selection screenshot, the status identifier of the multimedia player can be obtained to identify the scene of the current interface, and then the subsequent selection screenshot processing can be executed.

[0247] In some alternative embodiments, to facilitate the display device in identifying the user's image processing needs, a corresponding function triggering method can be pre-configured for each image processing function based on the press duration of the screenshot control. This allows the corresponding image processing method to be determined based on the user's triggering method. For example, when the remote control detects that the user has pressed the screenshot control, it can continuously send timing parameters to the display device. Upon detecting the timing parameters, the display device records the start timestamp of the received timing parameters and triggers a timing operation.

[0248] Specifically, the trigger for full-screen screenshots can be configured to be triggered by pressing the screenshot control within a short period of time. That is, upon receiving a stop screenshot command and if the timer duration does not exceed a preset duration, the image processing method is determined to be a full-screen screenshot. A full-screen screenshot is an image obtained by capturing the entire current interface. The stop screenshot command can be a command sent when a preset control on the remote control is released.

[0249] The trigger for circling and capturing screenshots can be configured to require a prolonged press of the screenshot control while the indicator switch is on. Specifically, if the timer exceeds a preset duration and the indicator switch remains on, the image processing method is determined to be circling and capturing screenshots. For example, the press parameter might be 100 when the user presses the preset control and 85 when the user releases it. A received press parameter of 100 indicates the user has been continuously pressing the preset control, while a received press parameter of 85 indicates a stop screenshot command has been detected.

[0250] Optionally, if the timeout period exceeds the preset duration and the status parameter indicating the switch is in the off state, the display device will trigger screen recording processing logic to record the current interface and obtain the recording result. The recording result will then be stored locally for subsequent processing such as sending the recording to the user.

[0251] Based on the above embodiments, in some embodiments of this application, another optional method is provided to determine whether the current interface displays a dynamic or static image. Specifically, the display device can obtain the application status of the home application and determine whether the current interface displays a dynamic or static image based on the application status. That is, when the application status indicates that the home application is playing media asset content, the current interface displays a dynamic / video image; when the application status indicates that the home application is not playing media asset content, the current interface displays a static image.

[0252] The home application is the application on the display device that currently provides visual interaction with the user; it is essentially a desktop application. A desktop application can be an application that starts by default when the display device is opened, and it can be used to support the display device in playing media data. In some embodiments, the desktop application is a pre-built application that can serve as a signal source for the display device. The application state refers to the current running state of the home application.

[0253] Optionally, the application status of the homepage application can be obtained from the overall system of the display device based on the application identifier of the homepage application.

[0254] In some embodiments of this application, an optional method for processing screen images is provided, specifically, when the application state indicates that the home application is playing media content, the display device can control the display to show the screen image; when the application state indicates that the home application is not playing media content, the screen image is deleted.

[0255] Optionally, when the application status indicates that the home application is playing media content, it proves that the current interface is displaying dynamic images. At this time, the display screen image can be controlled to be displayed. When the application status indicates that the home application is not playing media content, it proves that the current interface is displaying static images. At this time, the screen image can be deleted.

[0256] For example, if the analysis shows that the application state of the home page application is playing video B, the monitor can be controlled to display a screen image B' pre-acquired from video B, and a screenshot operation can be performed on screen image B'; if the analysis shows that the application state of the home page application is displaying a static screen of image selection interface C, the pre-acquired screen image C' can be deleted, and a screenshot operation can be performed directly on image selection interface C.

[0257] The above solution introduces the application status of the homepage application. By determining whether the homepage application is playing media content based on its application status, the solution selects the appropriate screen image processing method based on the determination result, thus ensuring the rationality of the screen image processing.

[0258] Based on the above embodiments, in some embodiments of this application, the process of controlling the display to show the screen image is refined, specifically, maintaining the playback of dynamic images and displaying the screen image on the layer above the dynamic images.

[0259] The layer above the dynamic image is the display layer that overlays the dynamic image.

[0260] It's worth noting that when selecting and capturing a screenshot of a moving image, to avoid interrupting its playback, a screen image can be displayed in a layer above the moving image while it continues playing. In this case, if the moving image contains audio, the user can hear the audio.

[0261] The above solution introduces a layer above the dynamic image. By maintaining the playback of the dynamic image during the selection and screenshot process and displaying the screen image in the layer above the dynamic image, the selection and screenshot process can be completed without affecting the playback of the dynamic image, thus improving the convenience of selection and screenshot.

[0262] In some embodiments, after the user selects and captures a screenshot of the screen image displayed in the layer above the dynamic image to obtain a region image, an information display method is also provided. Specifically, the display device can cancel the display of the screen image and display the screen image and search instruction information in a preset display area of ​​the layer above the dynamic image.

[0263] The preset display area refers to the area where the screen image and search instructions are displayed during the image search process. For example, the upper left corner of the layer above the dynamic image might be used as the preset display area. Furthermore, the area of ​​the preset display area is smaller than the display area of ​​the dynamic image, and the area outside the preset display area in the upper layer is transparent, meaning that the dynamic image played outside the preset display area can be viewed normally. The search instructions are used to inform the user that an image search is being performed on the screen image; for example, it could be text messages such as "Selected content is being recognized" or "The following results have been identified for you."

[0264] Optionally, after acquiring the region image, to avoid affecting the display of the dynamic image, the full-screen display of the screen image can be canceled; then, the screen image and the search instruction information generated when performing image recognition on objects in the region image can be displayed in the preset display area of ​​the layer above the dynamic image. For example, referring to the image search page diagram shown in Figure 17, when the upper left corner of the layer above the dynamic image is used as the preset display area, the screen image 8 and the search instruction information 9 "Recognizing selected content..." can be displayed in the upper left corner of the layer above the dynamic image.

[0265] It's worth noting that, in addition to setting a fixed area as the preset display area, to ensure the rationality of the preset display area, it can also be determined based on the location information of preset elements that cannot be obscured in the current dynamic scene. For example, a preset display area can be selected from other areas outside the area where the preset elements that cannot be obscured are located.

[0266] Optionally, after identifying the object information of the object in the region image, the object information can be displayed in a preset display area in the layer above the dynamic screen. For example, referring to the schematic diagram of object information display shown in Figure 18, after identifying the object information of the object in the region image by using the upper left corner of the layer above the dynamic screen as the preset display area, the object information 10 and the search instruction information 11 "The following results were identified for you" can be displayed when the upper left corner of the layer above the dynamic screen is used as the preset display area.

[0267] The above solution introduces a layer above the dynamic screen. By canceling the display of the screen image during the image search process, users can continue to watch the dynamic screen. Furthermore, displaying the screen image and search instructions in a preset display area of ​​the layer above the dynamic screen allows users to intuitively understand the image search progress and improves the user experience.

[0268] Based on the above embodiments, steps S1520 and S1530 are refined in some embodiments of this application. As shown in FIG19, the specific steps include, but are not limited to, the following:

[0269] S1910: Based on the pointing parameters sent by the pointing remote control, generate a trajectory line, and determine a preset shape area based on the trajectory line as the selected area corresponding to the pointing parameters.

[0270] The preset shape region corresponds to the region represented by the trajectory line, and can be a selected area containing all the trajectory lines. Optionally, the preset shape region can be a rectangular region (including square regions, rectangular regions), a circular region, an elliptical region, a triangular region, etc.

[0271] In some embodiments, when the position information contained in the pointing parameter is the position information relative to the display device in the geodetic coordinate system, and when it is determined that the user is using the selection screenshot function based on the user's current triggering method, the display device can convert the position information in the pointing parameter sent by the pointing remote control into the coordinate information of the corresponding trajectory point in the current interface through a preset coordinate transformation method, and store the coordinate information of the trajectory point in the trajectory array associated with the current interface.

[0272] In other embodiments, when the pointing parameters include the horizontal and vertical coordinates of the display device in the screen coordinate system after coordinate conversion by the pointing remote control, and it is determined that the user is using the selection screenshot function based on the user's current triggering method, the display device can use the position information in the pointing parameters sent by the pointing remote control as the coordinate information of the corresponding trajectory point in the current interface, and store the coordinate information of the trajectory point in the trajectory array associated with the current interface.

[0273] Then, based on the sending order of each trajectory point in the trajectory array and the coordinate information corresponding to each trajectory point, the coordinate changes between each trajectory point are determined. Then, based on the coordinate changes between each trajectory point, a trajectory line is generated in the currently displayed interface. As a specific implementation method, the trajectory points can be connected according to their sending order to generate a trajectory line.

[0274] It is understandable that when performing the above-described selection screenshot process, the display device generates a trajectory line within a transparent canvas overlaid on the screen image. Specifically, when the selection screenshot process is detected, the display device can refer to the aforementioned processing flow description, which will not be repeated here.

[0275] In some embodiments, the display device may refer to the description of the aforementioned step S520 to determine a preset shape area based on the generated trajectory line, wherein the aforementioned currently displayed page is the current interface in this embodiment.

[0276] S1920: Take a screenshot of the area corresponding to the circled region in the target image to obtain the region image.

[0277] The target image is either a screen image or a static image. That is, when the current interface displays a dynamic image, the target image is a screen image; when the current interface displays a static image, the target image is a static image.

[0278] Optionally, a region in the target image corresponding to the preset shape region can be cropped based on the coordinate information of the preset shape region to obtain a region image. For example, if the preset shape region is a rectangular region, a region in the target image corresponding to the rectangular region can be cropped based on the coordinate information of the rectangular region to obtain a region image.

[0279] In some embodiments, when the preset shape area is a rectangular area, to ensure the adaptability between the image area and the display page, a screenshot of the image selected by the rectangular area in the target image can be taken to obtain the original image. Then, based on the aspect ratio of the display page, the original image can be optimized to obtain the area image. For example, when the aspect ratio of the display page is 16:9, the original image can be adjusted to an area image with an aspect ratio of 16:9 without modifying the image length; alternatively, the original image can be adjusted to an area image with an aspect ratio of 16:9 without modifying the image width.

[0280] In the scenario described above, regardless of the shape of the trajectory line selected by the user, a preset shape region containing all trajectory lines can be determined based on the trajectory lines. The target image is then captured based on this predetermined shape region. On one hand, since there are no restrictions on the shape or regularity of the trajectory lines, the difficulty of selecting and capturing images is reduced, improving the user experience and ensuring the smooth progress of subsequent image searches, thus increasing the efficiency of obtaining image search results. On the other hand, because the predetermined shape region contains all trajectory lines, it ensures that the image region obtained from the screenshot matches the user's actual query requirements, thereby guaranteeing the accuracy of the image region acquisition.

[0281] Based on the above embodiments, in some embodiments of this application, the preset shape region is a rectangular region, and step S1920 is refined. As shown in Figure 20, the specific steps include, but are not limited to, the following:

[0282] S2010: Select the minimum, maximum, minimum, and maximum values ​​of the horizontal coordinate, vertical coordinate, and ordinate from the coordinate data of the trajectory points on the trajectory line.

[0283] The minimum horizontal coordinate value is the smallest horizontal coordinate value in the trajectory line; the maximum horizontal coordinate value is the largest horizontal coordinate value in the trajectory line; the minimum vertical coordinate value is the smallest vertical coordinate value in the trajectory line; and the maximum vertical coordinate value is the largest vertical coordinate value in the trajectory line.

[0284] Understandably, in practical applications, there may be issues such as users selecting areas incorrectly, resulting in shorter selection lengths in certain directions. This leads to a smaller determined preset shape area, compromising the accuracy of image search results. To address this, before executing step S2010, the display device can compare the width and length values ​​of the trajectory line in the screen width direction with a first threshold. If either the width or length value is greater than or equal to the first threshold, the trajectory line is deemed valid. In this case, the minimum, maximum, minimum, and maximum horizontal coordinates of the trajectory points can be selected from the coordinate data. Furthermore, if both the width and length values ​​are less than the first threshold, a screenshot failure message is output. The specific process for determining the validity of the trajectory line can be found in the flowchart shown in Figure 8 above, and will not be repeated here.

[0285] S2020 combines the minimum and maximum x-coordinates, minimum and maximum y-coordinates to obtain at least two vertex coordinates.

[0286] For an explanation of step S2020, please refer to the aforementioned step S920, which will not be repeated here.

[0287] Based on the above embodiments, in some embodiments of this application, the preset shape region is a rectangular region, and step S1920 is further refined. As shown in Figure 21, the specific steps include, but are not limited to, the following:

[0288] S2110: When the aspect ratio of the selected area differs from that of the display device's screen, the selected area is corrected according to the screen's aspect ratio.

[0289] For an explanation of step S2110, please refer to step S1310 above, and it will not be repeated here.

[0290] S2120, take a screenshot of the area within the corrected selected area in the target image to obtain the area image.

[0291] For an explanation of step S2120, please refer to the aforementioned step S1320, which will not be repeated here.

[0292] Based on the solutions of the above embodiments, taking a television device as an example, the television device includes a television system and a screenshot application. The television system is the entire system of the television device. Through the cooperation of the television system and the screenshot application, a screenshot function for the display interface is provided. Furthermore, when the display interface is a dynamic image, an optional embodiment is provided. Referring to Figure 22, which shows a schematic diagram of the interaction between devices during the selection and screenshot process, including but not limited to:

[0293] S2201: When the remote control detects a user pressing a preset control, it sends a preset pressing command to the TV system.

[0294] Specifically, before S2201, the user presses a preset control in the remote control (corresponding to S2201' in Figure 22), which is the aforementioned screenshot control.

[0295] S2202: After receiving the preset press command, the TV system sends a start command to the screenshot application.

[0296] S2203: After the screenshot application is launched, it calls the screenshot method provided by the TV system to take a screenshot of the display interface and obtain the screen image.

[0297] Specifically, the screenshot application sends a screenshot command to the TV system, which then takes a screenshot of the display screen and sends the captured screen image back to the screenshot application.

[0298] S2204, if the screenshot application detects that the pressing duration of the preset control exceeds the preset duration, it sends a status query command for the remote control's pointing switch to the TV system.

[0299] S2205, the TV system feeds back status parameters to the screenshot application, representing the on / off state of the remote control's pointing switch.

[0300] S2206, when the state parameter indicates that the switch is in the on state, the screenshot application obtains the state identifier of the multimedia player.

[0301] S2207, when the status identifier indicates that the multimedia player is in playback state, the screenshot application maintains the playback of the dynamic image and controls the display to show the screen image in the layer above the dynamic image.

[0302] S2208, the remote control continuously sends pointing parameters to the TV system.

[0303] S2209, the TV system sends a pointer parameter to the screenshot application.

[0304] S2210, the screenshot application generates a trajectory line based on the pointing parameters sent by the remote control, and determines a rectangular area based on the trajectory line as the selected area corresponding to the pointing parameters.

[0305] Specifically, before step S2210, the user releases the screenshot control (corresponding to S2210' in Figure 22).

[0306] S2211, The screenshot application sends a screenshot command containing the coordinates of the selected area to the television system.

[0307] S2212, the television system captures a screenshot of the screen image based on the coordinate information of the rectangular area to obtain the area image.

[0308] S2213, The television system sends a region image and the screenshot acquisition path of the region image to the screenshot application.

[0309] S2214, The screenshot application uploads the region image and the screenshot acquisition path to the image recognition system.

[0310] S2215, the screenshot application cancels the display of the screen image and displays the screen image and search instructions in the preset display area of ​​the layer above the display interface.

[0311] S2216, The image recognition system performs image search based on the regional image to determine the object information of the object in the regional image.

[0312] S2217, The image recognition system obtains the path from the screenshot and feeds back the object information of the object in the area image to the screenshot application.

[0313] S2218, Take a screenshot of the object information of an object in the preset display area image of the layer above the display interface.

[0314] In this embodiment, when the status identifier indicates that the multimedia player is in a non-playback state, the screenshot application deletes the screen image and, after determining the selected area based on the pointing parameters sent by the remote control, sends a selection screenshot command containing the coordinate information of the selected area to the television system. The television system takes a screenshot of the display interface based on the rectangular area coordinate information, obtains the area image, and sends the area image and the screenshot acquisition path of the area image to the screenshot application. Then, step S2214 is executed.

[0315] Furthermore, the specific processes of S2201-S2218 described above can be found in the description of the above method embodiments, and their implementation principles and technical effects are similar, so they will not be repeated here.

[0316] In some embodiments, as shown in Figure 23, the display device may further perform the following steps to simultaneously select and capture multiple frames of images displayed on the screen. Accordingly, the following explanation will only use pointing to a remote control as an example. However, it should be noted that this solution is not limited to pointing to a remote control; it can also be the aforementioned air mouse remote control, etc., which will not be elaborated further.

[0317] S2310, during video stream playback, in response to a received preset press command, stores the first image corresponding to the screen display.

[0318] The so-called first image is the screen displayed on the monitor when the user performs the selection operation.

[0319] Optionally, when a user needs to select a specific area for a screenshot, a preset press command can be sent to the display device by pressing a preset button on the remote control. Correspondingly, in one optional implementation, after the display device receives the preset press command, it can launch a screenshot application, which then uses the screenshot function provided by the display device's system to capture the currently displayed interface, obtaining a first image. This first image can then be cached in local storage.

[0320] In another alternative implementation, during video stream playback, previously played historical images (i.e., the interface already displayed on the screen) can be cached in local space. After the display device receives a preset press command, it can determine the user's actual screenshot request time based on the current time and the preset image acquisition delay; then, the first image is retrieved from the local cache based on the screenshot request time.

[0321] The method for determining the image acquisition delay can be found in the aforementioned method and will not be repeated here. For example, if the image acquisition delay for user A is 1 second, and a preset press command sent by user A to the remote control is received at the current moment, then the user's actual screenshot request time is determined to be 1 second before the current moment. In this case, the historical image corresponding to the 1 second before the current moment stored in the cache can be used as the aforementioned first image.

[0322] S2320 stores the second image corresponding to the display screen during the process of receiving the pointing parameters sent by the pointing remote control and generating the trajectory line according to the received pointing parameters.

[0323] The pointer parameters are detailed in the foregoing embodiments and will not be repeated here.

[0324] The so-called second image is the image displayed on the screen during the generation of the trajectory line. Furthermore, the second image can be one frame or multiple frames.

[0325] In some embodiments, after receiving the pointing parameters sent by the pointing remote control, the pointing parameters can be processed to generate a trajectory line for selection.

[0326] In one optional implementation, when the position information contained in the pointing parameter is the position information relative to the display device in the geodetic coordinate system, the display device can convert the position information in the pointing parameter sent by the pointing remote control into the coordinate information of the corresponding trajectory point in the screen coordinate system on the display device through a preset coordinate transformation method, and store the coordinate information of the trajectory point in the trajectory array.

[0327] In another alternative implementation, if the pointing parameters include the horizontal and vertical coordinates of the display device in the screen coordinate system after coordinate conversion by the pointing remote control, the display device can directly use the position information in the pointing parameters sent by the pointing remote control as the coordinate information of the trajectory point and store the coordinate information of the trajectory point in the trajectory array.

[0328] Then, based on the sending order of each trajectory point in the trajectory array and the coordinate information corresponding to each trajectory point, the coordinate changes between each trajectory point are determined. Finally, based on the coordinate changes between each trajectory point, a trajectory line is generated.

[0329] Meanwhile, to ensure the reliability of image selection, a second image can be determined and stored based on the screen displayed during the trajectory line generation process. In some embodiments, each frame displayed on the screen can be used as the second image; alternatively, a subset of frames can be extracted from the screen displayed on the screen as the second image. For example, an image that differs significantly from adjacent frames can be used as the second image.

[0330] S2330: Based on the trajectory line, determine the preset shape region as the selected region corresponding to the pointing parameter, and take screenshots of the first image and the second image according to the selected region to obtain the region image set.

[0331] The so-called selected area is used to represent the area that the user wants to select. The so-called region image is the image corresponding to the selected area, such as the image corresponding to the selected area in the first image. Furthermore, the region image set is a collection containing the images of each region.

[0332] In some embodiments, the selected area can be determined based on the horizontal and vertical coordinate parameters of each trajectory point in the trajectory array associated with the trajectory line. This selected area can be a region of the aforementioned preset shape, as detailed above, and will not be repeated here.

[0333] Optionally, after determining the selected area, the corresponding areas of the selected area in the first and second images can be captured sequentially according to the image acquisition order to obtain the corresponding area images for each image. Then, the corresponding area images for each image are sorted and stored sequentially according to the image acquisition order to obtain the area image set. Specifically, capturing the corresponding area of ​​the selected region in the first image can be done by capturing the corresponding area in the first image based on the coordinate information of the selected area. It should be noted that the process of capturing the second image is the same as that of the first image, and will not be repeated here.

[0334] In some embodiments, when the preset shape area is a rectangular area, to ensure the adaptability between the area image and the display page, a screenshot can be taken of the image selected by the rectangular area in the first and second images to obtain the original image. Then, based on the aspect ratio of the display page, the original image is optimized to obtain the area image. For example, when the aspect ratio of the display page is 16:9, the original image can be adjusted to an area image with an aspect ratio of 16:9 without modifying the image length; alternatively, the original image can be adjusted to an area image with an aspect ratio of 16:9 without modifying the image width.

[0335] S2340, perform image recognition based on the region image set to display object information of the recognized objects.

[0336] The term "object" can include people and things, such as plants and animals, characters portrayed by actors, everyday products, and other objects that can be displayed in image form. Taking a person as an example, the object information is the information corresponding to the person, such as a brief introduction of the person.

[0337] In some optional implementations, an image acquisition path corresponding to the regional image set can be generated based on the device identification information of the display device; then, the regional image set and its image acquisition path are simultaneously uploaded to the server. To improve image upload efficiency, the regional image set can be compressed before uploading, and the compressed image set and its image acquisition path can then be uploaded to the server.

[0338] The server integrates an image recognition system. After acquiring a set of regional images, the server can invoke the image recognition system to sequentially recognize each regional image. Based on the image acquisition path, the recognition results are fed back to the display device as object information of objects within the regional image set. In some embodiments, for each regional image in the regional image set, the image recognition system can segment the regional image based on image features to obtain the objects contained in each sub-image. Then, for each object contained in a sub-image, the object can be used as index information to perform consistency matching among standard objects in a pre-determined image library. The object information corresponding to the standard object with the highest matching degree is used as the object information of that object. After obtaining the object information of objects contained in each sub-image, this object information is simultaneously fed back to the display device.

[0339] In some alternative implementations, the object information of the objects in each region image set can be obtained by directly searching the object information stored locally in the region image set in turn.

[0340] Optionally, after acquiring the object information, the display device can determine an information display overlay on the current display interface and display the object information corresponding to each area image at a preset display position within the information display overlay. In some embodiments, the object information corresponding to each area image can be displayed sequentially at the preset display position within the information display overlay according to the order of the area images and at a preset display time interval. Alternatively, the object information corresponding to each area image can be displayed simultaneously at the preset display position within the information display overlay according to the order of the area images. For example, referring to the information display page shown in Figure 24, after recognizing each area image, actor M and actor N are identified sequentially. At this time, the object information corresponding to actor M and actor N can be directly displayed on the information display overlay.

[0341] The above solution, during video playback, on the one hand, stores the first image corresponding to the screen display upon receiving a preset press command, and stores the second image corresponding to the screen display during the process of generating a trajectory line based on the received pointing parameters; then, based on the selection area determined by the trajectory line, screenshots are taken from the first and second images to obtain a set of region images. This allows for simultaneous selection and screenshotting of multiple frames displayed on the screen during the selection operation, ensuring the integrity and accuracy of the selection operation. On the other hand, by simultaneously performing image recognition on each region image in the set of region images and displaying the recognized object information, the comprehensiveness of the displayed object information is improved, making the displayed object information more in line with the user's actual needs.

[0342] Based on the above embodiments, in some embodiments of this application, the process of acquiring the second image is refined. One optional implementation involves receiving pointing parameters sent by the remote control and generating a trajectory line based on the received pointing parameters, taking screenshots of the video screen displayed on the monitor at set time intervals to obtain the second image, and storing the second image. The set time interval is a pre-set time interval for taking screenshots.

[0343] Optionally, the set time interval can be determined based on the playback speed of the video stream. For example, the set time interval can be appropriately shortened when the playback speed is faster (e.g., 2x or 3x speed), and appropriately increased when the playback speed is slower (e.g., 0.5x speed).

[0344] Optionally, during the process of receiving pointing parameters from the remote control and generating a trajectory line based on the received pointing parameters, a screenshot of the video displayed on the monitor can be taken directly at a set time interval adapted to the playback speed of the video stream to obtain a second image. The captured second image is then stored. For example, if the time interval is set to 1 second, and the process of generating the trajectory line based on the received pointing parameters takes 10 seconds, then 10 frames can be captured as the second image.

[0345] The above solution provides a simple and quick alternative for obtaining a second image by taking screenshots of the video screen displayed on the monitor at set time intervals.

[0346] Another optional implementation involves detecting the playing video frames during the process of receiving pointing parameters sent by the remote controller and generating a trajectory line based on the received pointing parameters. If a video frame is detected as a keyframe, it is stored as a second image. A keyframe is a video frame in the video stream that undergoes significant changes. For example, a video frame experiencing drastic / large-area changes can be used as a keyframe.

[0347] Optionally, during the process of receiving the pointing parameters sent by the remote controller and generating the trajectory line based on the received pointing parameters, for each video frame in the video stream, the video frame can be detected based on the degree of change between the previous video frame and the current video frame, and if the video frame is detected to be a key frame, the key frame can be stored as a second image.

[0348] In some embodiments, the process of detecting whether a video frame is a keyframe can be as follows: A screenshot application controls the home application to detect each frame of the video stream being played, and if a keyframe is detected, the keyframe is stored as a second image. Alternatively, the home application on the display device can send a screenshot instruction to the screenshot application when a keyframe is identified, allowing the screenshot application to capture a screenshot of the video displayed on the screen and store the screenshot as a second image. Here, the home application is the application on the display device that currently provides visual interaction with the user.

[0349] The above scheme, by directly using keyframes from the video stream as the second image, provides an optional method for obtaining the second image. On the other hand, since keyframes are important video frames in the video stream and contain richer object information, the reliability of the second image is guaranteed, resulting in richer and more accurate object information in the final image.

[0350] Furthermore, another optional implementation method can be provided, specifically, a monitoring model capable of monitoring image richness can be pre-configured in the display device. Accordingly, during the process of receiving pointing parameters sent by the remote control and generating a trajectory line based on the received pointing parameters, the detection model can be invoked to detect the image richness of the playing video frames, and video frames with image richness greater than a preset richness threshold can be stored as second images.

[0351] Based on the above embodiments, in some embodiments of this application, the process of displaying object information is refined, specifically, displaying object information of the identified object and image download information.

[0352] The image download information includes the image to be downloaded and the download entry; the image to be downloaded is the image data related to the selection and screenshot operation. Optionally, the download entry can be a control or link for downloading the image. The image to be downloaded includes at least one of the following: a region image set, a target video, and an animation generated based on the region image set.

[0353] The target video is obtained by cropping the screen video based on the selected area. The screen video is recorded during the process of generating the trajectory line on the monitor. In essence, to allow users to intuitively understand the entire process of selecting and capturing the image, the screen recording function can be activated in response to a received preset press command to record the monitor's display interface. Furthermore, screen recording continues while receiving pointing parameters from the remote control and generating the trajectory line based on these parameters, resulting in the screen video. Subsequently, the screen video can be cropped based on the coordinate information corresponding to the selected area to obtain the target video.

[0354] Optionally, after obtaining the regional image set, each regional image in the regional image set can be processed sequentially based on a preset frame duration to obtain the animation corresponding to the regional image set.

[0355] Optionally, while performing image recognition on the regional image set, screen video obtained by recording the display interface can be acquired, and the screen video can be cropped according to the selected area to obtain the target video. Furthermore, the images of each region in the regional image set can be processed to obtain image animation. Further, after recognizing the object information, the object information, the image to be downloaded, and the corresponding download entry can be displayed in an information display overlay on the current display interface.

[0356] In some embodiments, after acquiring the regional image set, the target video, and the animation generated based on the regional image set, the images to be downloaded of each type can be uploaded to the corresponding storage location on the server. After storage is completed, the server will send the corresponding storage location information back to the display device. The display device can generate corresponding download entries based on the storage location information of each type of image to be downloaded, and associate and display each type of image to be downloaded with its corresponding download entry.

[0357] For example, when the image to be downloaded simultaneously contains a set of regional images, a target video, and an animation generated based on the regional images, referring to the information display page shown in Figure 25, the download entry A' of the regional image set A, the download entry B' of the target video B, and the download entry C' of the animation C can be displayed below the object information 12.

[0358] The above solution, on the one hand, introduces a download entry corresponding to the image to be downloaded, so that when users select and take screenshots, they can not only obtain the object information in the image, but also download the image to be downloaded, thus enriching the selection and screenshot function; on the other hand, the image to be downloaded includes the video related to the entire selection and screenshot process, i.e., the target video, which makes it easier for users to intuitively understand the entire selection and screenshot process, further improving the user selection and screenshot experience.

[0359] Based on the above embodiments, in some embodiments of this application, the display device can upload a set of regional images to a server, where an image recognition system sequentially recognizes each regional image to obtain object information corresponding to each regional image. Then, the object information for each object is deduplicated, and the deduplicated object information is fed back to the display device. That is, the object information of the recognized objects displayed by the display device is the result of the server sequentially recognizing each regional image in the regional image set and deduplicating the object information of the recognized objects in each regional image.

[0360] In one optional embodiment, when the same object is identified from multiple region images, the server can retain only the region image where the object was first identified, based on the identification order. For example, if actor P is identified in both region images S1 and S2, if the server identifies actor P first from region image S1, it retains only the image information of actor P identified in region image S1 and feeds back the image information of actor P in region image S1, along with actor P's object information, to the display device. If the server identifies actor P first from region image S2, it retains only the image information of actor P identified in region image S2 and feeds back the image information of actor P in region image S2, along with actor P's object information, to the display device.

[0361] The above solution, by deduplicating duplicate object information, can ensure the simplicity of the object information displayed to the user; on the other hand, by deleting duplicate object information, it can also reduce the storage space occupied by the display device.

[0362] Furthermore, based on the above embodiments, in this application embodiment, when the server identifies the object corresponding to the region image, it can simultaneously determine the confidence level of the identified object. The confidence level is used to characterize the degree of similarity between the object in the region image and the corresponding standard object template.

[0363] Accordingly, when multiple regions of images identify the same object, the object information can be processed based on the confidence level of each region of images that identify the same object. Specifically, if the confidence levels of the regions of images that identify the same object are the same, the duplicate object information can be directly deduplicated; if the confidence levels of the regions of images that identify the same object are different, only the object information identified by the region of image with the highest confidence level can be retained. That is, the object information of the identified object displayed on the display device is the result of the server deduplicating the object information of the same object with the same confidence level and removing object information of the same object with different confidence levels, except for the one with the highest confidence level.

[0364] For example, if actor Q is identified in all three regions S3, S4 and S5, and the confidence level of actor Q identification in region S3 is 80%, the confidence level of actor Q identification in region S4 is 40%, and the confidence level of actor Q identification in region S5 is 70%, then only the image information of actor Q identified in region S3 is retained, and the image information of actor Q in region S3, as well as the object information of actor Q, are fed back to the display device.

[0365] The above solution, by deduplicating and removing object information based on the confidence level of the identified objects, can ensure the reliability and simplicity of the object information display on the one hand; on the other hand, by deleting related object information, it can also reduce the storage space occupied by the display device.

[0366] Based on the above embodiments, in some embodiments of this application, the process of generating the trajectory line can refer to the foregoing description, that is, the generated trajectory line is displayed in a layer above the video screen displayed on the monitor, and the area outside the trajectory line in the upper layer is a transparent area. In this way, by overlaying a transparent layer above the video screen when performing the selection screenshot process, the drawn trajectory line can be displayed without affecting video playback.

[0367] Furthermore, based on the above embodiments, the information display process is further refined in some embodiments of this application. Specifically, the target image and search instruction information are displayed in a preset display area above the video screen on the display. The preset display area and search instruction information are detailed above and will not be repeated here. The target image is any regional image in a set of regional images, or a set of regional images.

[0368] Optionally, after obtaining the regional image set, in order not to affect the display of the video screen, any regional image in the regional image set, or the regional image set, and the search instruction information generated when recognizing the regional images can be displayed in the preset display area of ​​the layer above the video screen.

[0369] For example, referring to the image search page diagram shown in Figure 17, if the upper left corner of the layer above the video screen is used as the preset display area, any area image 8 in the area image set can be displayed in the upper left corner of the layer above the video screen, along with the search instruction information 9 "Selected content recognition in progress...".

[0370] It's worth noting that, in addition to setting a fixed area as the preset display area, to ensure the rationality of the preset display area, it can also be determined based on the location information of preset elements that cannot be obstructed in the current video frame. For example, a preset display area can be selected from other areas outside the area where the preset elements that cannot be obstructed are located.

[0371] Optionally, after identifying the object information of the object in the region image, the object information can be displayed in a preset display area in the layer above the video screen. For example, referring to the object information display diagram shown in Figure 18, after identifying the object information of the object in the region image by using the upper left corner of the layer above the video screen as the preset display area, the object information 10 and the search instruction information 11 "The following results were identified for you" can be displayed when the upper left corner of the layer above the video screen is used as the preset display area.

[0372] The above solution, by displaying the target image and search instructions in a preset display area on the layer above the video screen, allows users to intuitively understand the image search progress and improves the user experience.

[0373] Based on the above embodiments, in this application embodiment, when the selected area is a rectangular area, the process of determining the selected area is refined. For details, please refer to the flowchart shown in Figure 20, which will not be repeated here. The above scheme determines the vertex coordinates of the selected area based on the minimum and maximum horizontal coordinates, minimum and maximum vertical coordinates of the trajectory points. This provides an optional method for quickly determining the selected area while ensuring that the subsequently acquired area image contains valid image information. Furthermore, this scheme presents the area determined by the trajectory line as a rectangle, which is compatible with the way the display device presents the page, thus making the captured area image more reasonable.

[0374] Based on the above embodiments, in some embodiments of this application, to ensure the rationality of the selected screenshot, the type of screen displayed on the current interface can also be detected. Optionally, when the current interface displays a video screen, during the process of generating a trajectory line according to the received pointing parameters, the second image corresponding to the screen displayed on the display is stored. Then, according to the determined selected area, the first image and the second image are captured to obtain a set of area images. When the current interface displays a static screen, the first image is deleted, and according to the determined selected area, a screenshot is taken in the static screen to obtain an area image. Here, the so-called current interface refers to the current display interface (page) of the display, which can display video screens or static screens. The so-called video screen refers to a screen image that changes in real time, such as a video playback screen; the so-called static screen refers to an image whose screen image does not change, such as a homepage or query page, or other non-playback pages.

[0375] In one alternative approach, the type of screen currently displayed can be determined based on the status identifier of the multimedia player.

[0376] In another alternative approach, the type of screen currently displayed can be determined based on the application status of the home page application.

[0377] Optionally, if the current screen is determined to be a video screen based on the status identifier of the multimedia player / application status of the home page application, then refer to steps S2310-S2330 above to select and capture the first image and the second image to obtain a set of regional images.

[0378] If the current screen is determined to be static based on the multimedia player's status identifier / the application's status, then to improve the resolution of the area image, the pre-saved first image can be deleted, and a screenshot of the currently displayed interface can be taken directly to obtain the area image. This area image is then uploaded to the server, where an image recognition system identifies the area image, obtains the corresponding object information, and provides feedback.

[0379] The above solution, by selecting the corresponding selection screenshot method based on the type of screen displayed on the current interface (dynamic / static), not only broadens the application scenarios of the screenshot function of the display device, but also improves the user experience.

[0380] In some embodiments, to better distinguish the image processing function to be triggered by the user, the image processing function used by the user can be determined from a variety of preset functions based on the on / off state of the pointing switch on the remote control. Specifically, in response to a preset press command sent by the pointing remote control, a status parameter representing the on / off state of the pointing switch on the remote control can be obtained from the locally stored current device information. It is worth noting that when the pointing switch is on, the remote control can send pointing parameters to the display device regardless of whether the user presses the screenshot control; however, the pointing parameters at this time will not form a selection area, but will be presented in the form of moving the cursor, etc.

[0381] Optionally, when the state parameter indicating the pointing switch is in the off state, the pointing remote control will not send pointing parameters, meaning the display device cannot execute the selection screenshot processing logic. In this case, the image processing method triggered by the display device can be screen recording / full-screen screenshot. When the state parameter indicating the pointing switch is in the on state, the pointing remote control can send pointing parameters, and the image processing method triggered by the display device can be selection screenshot.

[0382] Furthermore, the image processing function used by the user can be determined from a variety of preset functions by combining the duration of the user's press on the screenshot control. Specifically, the trigger method for full-screen screenshot can be configured to be a short press on the screenshot control. The trigger method for selection screenshot can be configured to be a long press on the screenshot control with the indicator switch on. That is, if the timer exceeds the preset duration and the indicator switch is on, the image processing method is determined to be selection screenshot. Optionally, if the timer exceeds the preset duration and the indicator switch is off, the display device will trigger screen recording logic to record the current interface and obtain the screen recording result. The screen recording result is then stored locally for subsequent processing such as sending the screen recording to the user.

[0383] Based on the above embodiments, taking a television device as an example, the television device includes a television system and a screenshot application. The television system is the entire system of the television device. Through the cooperation of the television system and the screenshot application, a screenshot function for the display interface is provided. Furthermore, in the case of displaying a video image, an optional embodiment is provided. Referring to Figure 26, which shows a schematic diagram of the interaction between devices during the selection and screenshot process, it includes:

[0384] S2601 indicates that when the remote control detects a user pressing a preset control, it sends a preset pressing command to the television system.

[0385] Specifically, before step S2601, the user presses a preset control in the remote control, corresponding to S2601' in Figure 26. The preset control is the aforementioned screenshot control.

[0386] S2602, after receiving the preset press command, the TV system sends a start command to the screenshot application.

[0387] S2603: After the screenshot application is launched, it calls the screenshot method provided by the TV system to take a screenshot of the display interface and obtain the first image.

[0388] Specifically, the screenshot application sends a screenshot command to the TV system, which then takes a screenshot of the display screen and sends the captured first image back to the screenshot application.

[0389] S2604, if the screenshot application detects that the pressing duration of the preset control exceeds the preset duration, it sends a status query command for the remote control's pointing switch to the TV system.

[0390] S2605, the TV system feeds back status parameters to the screenshot application, representing the on / off state of the remote control's pointing switch.

[0391] S2606, the remote control continuously sends pointing parameters to the TV system.

[0392] S2607, the TV system sends a pointer parameter to the screenshot application.

[0393] S2608, when the state parameter indicates that the pointing switch is in the on state and the state identifier of the multimedia player is in the playback state, the screenshot application takes screenshots of the video screen displayed on the monitor at set time intervals during the process of receiving the pointing parameter sent by the remote control and generating the trajectory line according to the received pointing parameter, to obtain the second image.

[0394] Specifically, the screenshot application can also control the home application to detect video frames being played, and if a video frame is detected as a keyframe, the keyframe will be stored as a second image.

[0395] S2609, the screenshot application determines a rectangular area based on the trajectory line, which serves as the selected area corresponding to the pointing parameter.

[0396] S2610, the screenshot application sends a screenshot command containing the coordinates of the selected area to the television system.

[0397] Specifically, before step S2609, the user releases the screenshot control, corresponding to S2609' in Figure 26.

[0398] S2611, the television system takes screenshots of the first and second images based on the coordinate information of the rectangular area to obtain a set of regional images.

[0399] S2612, The television system sends a region image set and the screenshot acquisition path of the region image set to the screenshot application.

[0400] S2613, The screenshot application uploads the region image set and the screenshot acquisition path to the image recognition system.

[0401] S2614, The screenshot is applied to the preset display area of ​​the layer above the display interface to display any region image / region image set in the region image set, as well as search instruction information.

[0402] S2615, the image recognition system performs image search based on the regional image set to determine the object information of objects in the regional image set, and performs deduplication processing on the object information of objects in the regional image set.

[0403] S2616, The image recognition system obtains the path from the screenshot and feeds back the object information of the deduplicated object to the screenshot application.

[0404] S2617, the screenshot application displays object information of objects in each region of the image after deduplication in the preset display of the layer above the display interface.

[0405] Furthermore, the specific processes of S2601-S2617 described above can be found in the description of the above method embodiments, and their implementation principles and technical effects are similar, so they will not be repeated here.

[0406] Based on the same inventive concept, some embodiments also provide a selection and screenshot device for implementing the above-described selection and screenshot method. The solution provided by this device is similar to the solution described in the above-described method. Therefore, the specific limitations of one or more selection and screenshot device embodiments provided below can be found in the limitations of the selection and screenshot method described above, and will not be repeated here.

[0407] In some possible embodiments, as shown in FIG27(a), a selection and screenshot device is provided, applied to a display device, including: a trajectory line generation module 2710, a region determination module 2720, a screenshot module 2730, and an image search module 2740, wherein: the trajectory line generation module 2710 is used to receive pointing parameters sent by a remote control and generate a trajectory line according to the received pointing parameters; wherein the pointing parameters change with the movement of the remote control; the region determination module 2720 is used to determine a preset shape region as the selection region corresponding to the pointing parameters according to the trajectory line; wherein the preset shape region corresponds to the region represented by the trajectory line; the screenshot module 2730 is used to take a screenshot of the region corresponding to the selection region in the currently displayed page to obtain a region image; the image search module 2740 is used to perform an image search based on the region image to display object information of objects in the region image.

[0408] Based on the same inventive concept, some embodiments also provide another selection and screenshot device for implementing the above-described selection and screenshot method. The solution provided by this device is similar to the solution described in the above-described method. Therefore, the specific limitations of one or more selection and screenshot device embodiments provided below can be found in the limitations of the selection and screenshot method above, and will not be repeated here.

[0409] In some possible embodiments, as shown in FIG27(b), a selection screenshot device is provided, applied to a display device, including: an image acquisition module 2710', a first screenshot module 2720', a second screenshot module 2730', and an image recognition module 2740', wherein: the image acquisition module 2710' is used to take a screenshot of the currently displayed interface of the display in response to a received preset press command, and obtain a screen image; wherein the preset press command indicates that a preset button in the remote control is pressed; the first screenshot module 2720' is used to control the display to display a screen image when the current interface displays a dynamic picture, and to take a screenshot of the screen image according to the selected area, and obtain a region image; wherein the selected area is determined according to the pointing parameters sent by the remote control, and the pointing parameters change with the movement of the remote control; the second screenshot module 2730' is used to delete the screen image when the current interface displays a static picture, and to take a screenshot of the static picture according to the selected area, and obtain a region image; the image recognition module 2740' is used to perform image recognition based on the region image to display object information of objects in the region image.

[0410] Based on the same inventive concept, some embodiments also provide another screening device for implementing the screening method described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more screening device embodiments provided below can be found in the limitations of the screening method above, and will not be repeated here.

[0411] In some possible embodiments, as shown in FIG27(c), a selection and screenshot device is provided, applied to a display device, including: a first storage module 2710”, a second storage module 2720”, a screenshot module 27”, and an object recognition module 2740”, wherein: the first storage module 2710” is used to store a first image corresponding to the display screen in response to a received preset press command during video stream playback; wherein the preset press command indicates that a preset button in the remote control is pressed; the second storage module 2720” is used to store a second image corresponding to the display screen during the process of receiving pointing parameters sent by the remote control and generating a trajectory line according to the received pointing parameters; the screenshot module 2730” is used to determine a preset shape area as the selection area corresponding to the pointing parameters according to the trajectory line, and to take screenshots of the first image and the second image according to the selection area to obtain a set of area images; wherein the pointing parameters change with the movement of the remote control; the object recognition module 2740” is used to perform image recognition based on the set of area images to display the object information of the recognized object.

[0412] Each module in any of the above-mentioned selection and screenshot devices can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer as software.

[0413] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0414] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0415] It should be noted that the data involved in this application (including but not limited to image data) is all data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0416] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application can be general-purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0417] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0418] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A display device, comprising: Display, memory, and at least one processor; wherein, The memory is configured to store computer programs or instructions; The at least one processor is configured to execute the computer program or instructions to cause the display device to: The system receives pointing parameters sent by the remote controller and generates a trajectory line based on the received pointing parameters; wherein the pointing parameters change with the movement of the remote controller. Based on the trajectory line, a preset shape region is determined as the selected region corresponding to the pointing parameter; wherein, the preset shape region corresponds to the region represented by the trajectory line; Take a screenshot of the area corresponding to the selected region on the currently displayed page to obtain an image of the region; An image search is performed based on the region image to display object information of objects in the region image.

2. The display device according to claim 1, wherein when the at least one processor executes the function of receiving pointing parameters sent by a remote controller and generating a trajectory line based on the received pointing parameters, it is configured to execute the computer program or instructions to cause the display device to: In response to a press event on the screenshot control sent by the remote control, a timing operation is performed; If a lift event indicating that the screenshot control has been lifted is detected, and the timeout period has not exceeded the preset duration, a full-screen screenshot of the currently displayed page is taken. If the timing duration exceeds the preset duration and no lift event indicating that the screenshot control has been lifted is detected within the timing duration, the remote control sends a pointing parameter, and a trajectory line is generated based on the received pointing parameter.

3. The display device according to claim 2, wherein when the at least one processor receives a pointing parameter sent by the remote control and generates a trajectory line according to the received pointing parameter, in the case that the timing duration exceeds a preset duration and no lift-up event representing the lifting of the screenshot control is detected within the timing duration, the processor is configured to execute the computer program or instructions to cause the display device to: Read the status parameters representing the switching state of the pointer switch of the remote control from the local source; If the timeout period exceeds a preset duration, and no lift-up event indicating the screenshot control has been detected within the timeout period, and the status parameter indicates the pointing switch is in the on state, then the system receives pointing parameters sent by the remote control and generates a trajectory line based on the received pointing parameters; wherein, The pointing switch is a switch on the remote control that controls the sending of pointing parameters. When the pointing switch is in the on state, the remote control sends the pointing parameters. If the timeout period exceeds the preset timeout period, and no lift event indicating that the screenshot control has been lifted is detected within the timeout period, and the status parameter indicates that the pointing switch is in the off state, the screen recording of the currently displayed page will be performed; wherein, when the pointing switch is in the off state, the remote control does not send pointing parameters.

4. The display device according to claim 1, wherein the at least one processor is further configured to execute the computer program or instructions to cause the display device to: In response to a received preset press command, a screenshot is taken of the display interface to obtain a screen image; wherein, The preset press command indicates that a preset button in the remote control is pressed; When the current interface displays a dynamic image, the display is controlled to show the screen image, and a screenshot is taken from the screen image according to the selected area to obtain the area image; wherein, the selected area is determined according to the pointing parameters sent by the remote control; When the current interface displays a static image, delete the screen image and take a screenshot of the static image based on the selected area to obtain the area image; Image recognition is performed based on the region image to display object information of objects in the region image.

5. The display device according to claim 4, wherein before the at least one processor executes the computer program or instructions to control the display to show the screen image when the current interface displays a dynamic picture, the at least one processor is further configured to execute the computer program or instructions to cause the display device to: Obtain the status identifier of the multimedia player; where... When the status identifier indicates that the multimedia player is in playback state, the current interface displays dynamic images; When the status identifier indicates that the multimedia player is in a non-playback state, the current interface displays a static image; When the at least one processor controls the display to show the screen image while the current interface is displaying a dynamic image, it is configured to execute the computer program or instructions to cause the display device to: When the status identifier indicates that the multimedia player is in a playback state, the display is controlled to show the screen image; When the at least one processor deletes the screen image while the current interface is displaying a static image, it is configured to execute the computer program or instructions to cause the display device to: When the status identifier indicates that the multimedia player is in a non-playing state, the screen image is deleted.

6. The display device according to claim 5, wherein when the at least one processor executes the process of acquiring the status identifier of the multimedia player, it is configured to execute the computer program or instructions to cause the display device to: Upon receiving a preset press command, read from the local machine the status parameter representing the switch state of the remote control's pointing switch; When the state parameter indicates that the pointer switch is in the on state, the state identifier of the multimedia player is obtained; wherein... The pointing switch is a switch on the remote control that controls the sending of pointing parameters. When the pointing switch is in the on state, the remote control sends pointing parameters; when the pointing switch is in the off state, the remote control does not send pointing parameters.

7. The display device according to claim 4, wherein the at least one processor is further configured to execute the computer program or instructions to cause the display device to: (i.e., before controlling the display to display the screen image when a dynamic image is displayed on the current interface) Get the application status of the homepage application; where, The homepage application is the application on the display device that currently engages in visual interaction with the user; when the application state indicates that the homepage application is playing media asset content, the current interface displays dynamic images; When the application state indicates that the home application is not playing media content, the current interface displays a static image; When the at least one processor controls the display to show the screen image while the current interface is displaying a dynamic image, it is configured to execute the computer program or instructions to cause the display device to: When the application state indicates that the home application is playing media content, control the display to show the screen image; When the at least one processor deletes the screen image while the current interface is displaying a static image, it is configured to execute the computer program or instructions to cause the display device to: When the application state indicates that the home application is not playing media content, the screen image is deleted.

8. The display device according to claim 4, wherein when the at least one processor executes the command to control the display to show the screen image, it is configured to execute the computer program or instructions to cause the display device to: The playback of the dynamic image is maintained, and the screen image is displayed in a layer above the dynamic image.

9. The display device according to claim 8, wherein when the current scene is a dynamic scene, the at least one processor, after obtaining the region image, is further configured to execute the computer program or instructions to cause the display device to: Cancel the display of the screen image, and display the screen image and search instruction information in a preset display area of ​​the layer above the dynamic screen; wherein, The area of ​​the preset display area is smaller than the display area of ​​the dynamic image. The area outside the preset display area in the upper layer is a transparent area. The search indication information is used to inform the user that an image search is being performed on the screen image.

10. The display device according to claim 4, wherein when the at least one processor performs a screenshot of a target image based on a selected area to obtain a region image, it is configured to execute the computer program or instructions to cause the display device to: Based on the pointing parameters received from the remote control, a trajectory line is generated, and based on the trajectory line, a preset shape region is determined as the selection area corresponding to the pointing parameters; wherein, The preset shape region corresponds to the region represented by the trajectory line; A screenshot is taken of the area corresponding to the selected region in the target image to obtain a region image; wherein, the target image is the screen image or the static image.

11. The display device according to claim 10, characterized in that, The selected area is a rectangular area; when the at least one processor performs a screenshot of the area corresponding to the selected area in the target image to obtain an area image, it is configured to execute the computer program or instructions to cause the display device to: If the aspect ratio of the selected area is different from the aspect ratio of the display device screen, the selected area is corrected according to the screen aspect ratio; wherein, the aspect ratio of the modified selected area is the same as the screen aspect ratio. A screenshot is taken of the area within the corrected selected region in the target image to obtain the region image.

12. The display device of claim 1, wherein the at least one processor is further configured to execute the computer program or instructions to cause the display device to: During video playback, in response to a received preset press command, a first image corresponding to the screen display is stored; wherein, The preset press command indicates that a preset button in the remote control is pressed; During the process of receiving the pointing parameters sent by the remote control and generating a trajectory line based on the received pointing parameters, the second image corresponding to the display screen is stored. Based on the trajectory line, a preset shape region is determined as the selected region corresponding to the pointing parameter, and based on the selected region, the first image and the second image are captured to obtain a set of region images; Image recognition is performed based on the region image set to display object information of the recognized objects.

13. The display device according to claim 12, wherein when the at least one processor stores the second image corresponding to the display screen during the process of receiving pointing parameters sent by the remote controller and generating a trajectory line according to the received pointing parameters, it is configured to execute the computer program or instructions to cause the display device to: During the process of receiving the pointing parameters sent by the remote control and generating a trajectory line based on the received pointing parameters, the video screen displayed on the display is captured at set time intervals to obtain a second image, and the second image is stored.

14. The display device according to claim 12, wherein when the at least one processor stores the second image corresponding to the display screen during the process of receiving pointing parameters sent by the remote controller and generating a trajectory line according to the received pointing parameters, it is configured to execute the computer program or instructions to cause the display device to: During the process of receiving the pointing parameters sent by the remote control and generating a trajectory line based on the received pointing parameters, the video frames being played are detected, and if the video frame is detected to be a keyframe, the keyframe is stored as a second image.

15. The display device of claim 12, wherein when the at least one processor executes object information for displaying the identified object, it is configured to execute the computer program or instructions to cause the display device to: Displays object information of the identified objects, as well as image download information; among which, The image download information includes the image to be downloaded and the download entry; the image to be downloaded includes at least one of the region image set, the target video, and the animation generated based on the region image set; the target video is obtained by cropping the screen video according to the selected region, and the screen video is obtained by recording the display interface of the monitor during the process of generating the trajectory line.

16. The display device according to any one of claims 12-15, wherein the object information of the identified object is fed back by the server sequentially identifying each region image in the region image set and deduplicating the object information of the identified object in each region image.

17. The display device according to claim 16, wherein the object information of the identified object is fed back by the server after deduplicating the object information of the same object with the same confidence level and removing object information of the same object with different confidence levels except for the highest confidence level.

18. The display device of claim 12, wherein the at least one processor, in the process of receiving pointing parameters sent by the remote controller and generating a trajectory line according to the received pointing parameters, is further configured to execute the computer program or instructions to cause the display device to: The generated trajectory lines are displayed in a layer above the video image shown on the monitor; wherein, The area outside the trajectory line in the upper layer is a transparent area.

19. The display device of claim 12, wherein after obtaining the regional image set, the controller is further configured to execute the computer program or instructions to cause the display device to: The target image and search instruction information are displayed in a preset display area above the video image on the monitor; wherein, The area of ​​the preset display area is smaller than the display area of ​​the video screen. The area outside the preset display area in the upper layer is a transparent area. The search indication information is used to inform the user that an image search is being performed. The target image is any image in the image set or the image set itself.

20. The display device according to any one of claims 1-3, 10, and 12, wherein when the at least one processor executes the function of determining a preset shape region based on the trajectory line, it is configured to execute the computer program or instructions to cause the display device to: The width value of the trajectory line in the screen width direction and the length value in the screen length direction are compared with a first threshold value, respectively; wherein, The width value is the maximum horizontal coordinate difference between the horizontal coordinate differences of the trajectory points on the trajectory line, and the length value is the maximum vertical coordinate difference between the vertical coordinate differences of the trajectory points on the trajectory line. If the width value is greater than or equal to the first threshold, or if the length value is greater than or equal to the first threshold, a preset shape region is determined based on the trajectory line as the selected region corresponding to the pointing parameter.

21. The display device of claim 20, wherein the at least one processor is further configured to execute the computer program or instructions to cause the display device to: If both the width and length values ​​are less than the first threshold, a screenshot failure message is output.

22. The display device according to claim 20, wherein the preset shape region is a rectangular region; when the at least one processor executes the process of determining the preset shape region as the selected region corresponding to the pointing parameter based on the trajectory line, it is configured to execute the computer program or instructions to cause the display device to: If a first value is greater than the first threshold and a second value is greater than the second threshold, then the minimum horizontal coordinate, maximum horizontal coordinate, minimum vertical coordinate, and maximum vertical coordinate are selected from the coordinate data of the trajectory points on the trajectory line; wherein, The first threshold is greater than the second threshold; the first value is one of the length value and the width value, and the second value is the other of the length value and the width value; The minimum and maximum values ​​of the horizontal coordinates, the minimum and maximum values ​​of the vertical coordinates are combined to obtain at least two vertex coordinates; wherein the rectangular area represented by the at least two vertex coordinates is the selected area.

23. The display device according to claim 20, wherein the preset shape region is a rectangular region; when the at least one processor executes the process of determining the preset shape region as the selected region corresponding to the pointing parameter based on the trajectory line, it is configured to execute the computer program or instructions to cause the display device to: When a first value is greater than a first threshold and a second value is less than a second threshold, at least two vertex coordinates are determined based on the first value, the trajectory point on the trajectory line corresponding to the first value, and the screen aspect ratio of the display device; wherein, The rectangular region represented by the coordinates of at least two vertices is the selected region. The aspect ratio of the rectangular region represented by the coordinates of at least two vertices is the same as the aspect ratio of the screen. The first threshold is greater than the second threshold. The first value is one of the length value and the width value, and the second value is the other of the length value and the width value.

24. The display device according to any one of claims 1-4 and 12, wherein the selected area is a rectangular area; when the at least one processor performs a screenshot of the area corresponding to the selected area in the currently displayed page to obtain an image of the area, it is configured to execute the computer program or instructions to cause the display device to: If the aspect ratio of the rectangular region differs from the aspect ratio of the display device screen, the rectangular region is corrected according to the screen aspect ratio; wherein, The aspect ratio of the modified rectangular area is the same as that of the screen. Take a screenshot of the area within the corrected rectangular region on the currently displayed page to obtain the area image.

25. A method for taking a screenshot of a selection on a display device, comprising: The system receives pointing parameters sent by the remote controller and generates a trajectory line based on the received pointing parameters; wherein the pointing parameters change with the movement of the remote controller. Based on the trajectory line, a preset shape region is determined as the selected region corresponding to the pointing parameter; wherein, the preset shape region corresponds to the region represented by the trajectory line; Take a screenshot of the area corresponding to the selected region on the currently displayed page to obtain an image of the region; An image search is performed based on the region image to display object information of objects in the region image.

Images