Display device and image recognition method

Abstract

The application provides a display device and an image recognition method. The display device comprises a display and a controller. The controller is configured to acquire all layers in the display, detect whether the content to be displayed in the layers at a first time is the same as the content to be displayed in the layers at a second time, if the content is the same, the layers are in a static state, if the content is not the same, the layers are in a dynamic change state, when the layers are in the static state, the content to be displayed in the layers is subjected to static recognition, and when the layers are in the dynamic change state, the content to be displayed in the layers is subjected to dynamic recognition. The display device can directly recognize the image content, and does not need the user to capture the image and then recognize the image. The degree of intelligence is high, and the efficiency of image recognition is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of display devices, and in particular to a display device and an image recognition method. BACKGROUND

[0002] With the rapid development of intelligent display devices, the functions of display devices will be more and more rich, and the performance will be more and more powerful, and bidirectional man-machine interaction function can be realized, and audio, video, entertainment, data and other functions are integrated to meet the diversified and personalized needs of users. With the increasing demand of users for the functions or services provided by display devices, image recognition has gradually become a function frequently used by users, which can identify the content in the image and provide it to the user.

[0003] For some display devices, when performing image recognition, the user needs to capture the image currently played by the display device, and the display device performs image recognition on the image captured by the user. When the captured image is a static image, a static recognition model can be used to identify the content in the image; when the captured image is a dynamic image, a dynamic recognition model can be used to identify the target object in the image.

[0004] Since the existing display device uses different recognition models for different types of images when performing image recognition, the user needs to capture the image independently and select the corresponding recognition mode to complete the image recognition, which results in poor user experience. SUMMARY

[0005] The present application provides a display device and an image recognition method. To solve the problem of low efficiency in the existing display device when performing image recognition.

[0006] In a first aspect, the present application provides a display device, which includes a display and a controller. The controller is configured to perform the following steps:

[0007] All layers in the display are obtained, and the all layers include at least one of a video layer and a picture layer; it is detected whether the content to be displayed in the layer at a first time is the same as the content to be displayed in the layer at a second time; if the same, the layer is in a static state, and if not the same, the layer is in a dynamic change state; when the layer is in the static state, the content to be displayed in the layer is statically identified; and when the layer is in the dynamic change state, the content to be displayed in the layer is dynamically identified.

[0008] In some implementations, the controller is further configured to, in the step of detecting whether the content to be displayed in the layer at the first time is the same as the content to be displayed in the layer at the second time,

[0009] performing image subtraction processing on the content to be displayed in the layer at the first time instant and the content to be displayed in the layer at the second time instant; if the image subtraction processing result is 0, the content to be displayed in the layer at the first time instant is the same as the content to be displayed in the layer at the second time instant; if the image subtraction processing result is not 0, the content to be displayed in the layer at the first time instant is different from the content to be displayed in the layer at the second time instant.

[0010] In some implementations, the controller is further configured to, in the step of performing static identification on the content to be displayed in the layer when the layer is in a static state, perform text identification processing on the content to be displayed in the layer to obtain text information, the text information including text content to be displayed in the layer and position information of the text content in the layer.

[0011] In some implementations, the controller is further configured to, in the step of performing dynamic identification on the content to be displayed in the layer when the layer is in a dynamic change state,

[0012] obtaining an image subtraction processing result, the image subtraction processing result including a layer difference region and position information of the layer difference region in the layer; the layer difference region being a change region of a target object in the layer; inputting the content to be displayed in the layer at the first time instant, the content to be displayed in the layer at the second time instant, and the image subtraction processing result into a dynamic identification model to obtain a dynamic identification result; the dynamic identification result including an identification result of the target object.

[0013] In a second aspect, the present application also provides a display device, the display device including a display and a controller. The controller is configured to perform the following steps:

[0014] obtaining all layers in the display, the all layers including a first layer and a second layer, the first layer and the second layer being used to generate a target image; detecting whether the content to be displayed at a first time instant and the content to be displayed at a second time instant in any one of the first layer and the second layer are different; if yes, the target image is in a dynamic change state, and if not, the target image is in a static state; when the target image is in the static state, performing static identification on the target image; when the target image is in the dynamic change state, performing dynamic identification on the target image.

[0015] In some implementations, the controller is further configured to, before performing the step of detecting whether the content to be displayed at a first time instant and the content to be displayed at a second time instant in any one of the first layer and the second layer are different,

[0016] obtaining image data, the image data being the content to be displayed in all layers.

[0017] In some implementations, the display device further comprises an image collector configured to collect the first image data.

[0018] In some implementations, the controller is further configured to, in the step of collecting the image data, receive the first image data sent by the image collector, and / or collect the second image data from a network signal source.

[0019] In some implementations, the controller is further configured to, in the step of collecting the image data,

[0020] When the image data of the first time is collected, the image data of the first time is stored in a preset layer cache queue; the image data of the first time comprises content to be displayed in all layers at the first time; when the image data of the second time is collected, the image data of the second time is stored in the preset layer cache queue, and the image data is arranged in time sequence; the image data of the second time comprises content to be displayed in all layers at the second time; the first time and the second time are different by a preset period.

[0021] In some implementations, the controller is further configured to collect, from the preset layer cache queue, the image data of the first time comprising content to be displayed in all layers at the first time; perform superimposition processing on the content to be displayed in all layers at the first time to obtain a target image at the first time; and control the display to display the target image at the first time.

[0022] In a third aspect, the present application provides an image recognition method applied to a display device, the method comprising:

[0023] collecting all layers in the display, the all layers comprising at least one of a video layer and a picture layer; detecting whether content to be displayed in the layers at a first time is the same as content to be displayed in the layers at a second time; if the same, the layers are in a static state, and if not the same, the layers are in a dynamic change state; when the layers are in the static state, performing static recognition on the content to be displayed in the layers; and when the layers are in the dynamic change state, performing dynamic recognition on the content to be displayed in the layers.

[0024] It can be seen from the above technical solutions that the display device and the image recognition method provided by the application can obtain all layers in the display. Whether the content to be displayed in the layer at the first moment is the same as the content to be displayed in the layer at the second moment is detected again; if the same, the layer is in a static state, if not the same, the layer is in a dynamic change state. When the layer is in a static state, the content to be displayed in the layer is statically recognized; when the layer is in a dynamic change state, the content to be displayed in the layer is dynamically recognized. The display device can directly recognize the image content, without the need for the user to capture the image and then recognize it. The degree of intelligence is high, and the efficiency of image recognition is improved. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the application, the drawings needed in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0026] Figure 1 A use scenario of a display device according to some embodiments is shown;

[0027] Figure 2 A hardware configuration block diagram of the control device 100 according to some embodiments is shown;

[0028] Figure 3 A hardware configuration block diagram of the display device 200 according to some embodiments is shown;

[0029] Figure 4 A software configuration diagram in the display device 200 according to some embodiments is shown;

[0030] Figure 5 A schematic diagram of a display device displaying a frame of image in a feasible embodiment is shown;

[0031] Figure 6 A schematic diagram of a frame of image displayed in a display device in a feasible embodiment is shown;

[0032] Figure 7 A schematic diagram of a frame of image displayed in a display device in a feasible embodiment is shown;

[0033] Figure 8a A schematic diagram of image content of a first layer at a first moment in a feasible embodiment is shown;

[0034] Figure 8b A schematic diagram of image content of a first layer at a second moment in a feasible embodiment is shown;

[0035] Figure 8c A schematic diagram of a layer difference area in a feasible embodiment is shown;

[0036] Figure 9 An interaction flow chart of components of a display device in some embodiments is shown.

[0037] Figure 10 A flowchart of an embodiment of an image recognition method is shown. DETAILED DESCRIPTION

[0038] For the purpose of making the purposes and embodiments of the present application more clear, the following will combine the drawings in the exemplary embodiments of the present application to clearly and completely describe the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application.

[0039] It should be noted that the brief description of the terms in the present application is only for the convenience of understanding the following described embodiments, and is not intended to limit the embodiments of the present application. Unless otherwise specified, these terms should be understood according to their ordinary and general meanings.

[0040] The terms "first", "second", "third", and the like in the specification and claims of the present application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit the specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchanged under appropriate circumstances.

[0041] The terms "include" and "have" and any variations thereof are intended to cover but not exclusive inclusion, for example, a product or device including a series of components does not necessarily limit to all components clearly listed, but can include other components not clearly listed or inherent to these products or devices.

[0042] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware or / and software code capable of performing a function associated with that element.

[0043] Figure 1 A schematic diagram of the use scenario of the display device in accordance with the embodiments is shown. As shown in Figure 1 The display device 200 also communicates data with the server 400, and the user can operate the display device 200 through the smart device 300 or the control device 100.

[0044] In some embodiments, the control device 100 can be a remote controller, and the communication between the remote controller and the display device includes at least one of infrared protocol communication or Bluetooth protocol communication, and other short distance communication methods, to control the display device 200 by wireless or wired means. The user can input user instructions through at least one of the keys on the remote controller, voice input, control panel input, etc. to control the display device 200.

[0045] In some embodiments, the smart device 300 may include any one of a mobile terminal, tablet computer, computer, laptop computer, AR / VR device, etc.

[0046] In some embodiments, a smart device 300 may also be used to control the display device 200. For example, an application running on the smart device may be used to control the display device 200.

[0047] In some embodiments, the smart device 300 and the display device may also be used for data communication.

[0048] In some embodiments, the display device 200 can also be controlled in ways other than the control device 100 and the smart device 300. For example, it can be controlled by directly receiving the user's voice commands through a module configured inside the display device 200 for acquiring voice commands, or it can be controlled by receiving the user's voice commands through a voice control device set outside the display device 200.

[0049] In some embodiments, the display device 200 also communicates with the server 400. The display device 200 may communicate via a local area network (LAN), wireless local area network (WLAN), and other networks. The server 400 may provide various content and interactive features to the display device 200. The server 400 may be a cluster or multiple clusters, and may include one or more types of servers.

[0050] In some embodiments, software steps executed by one execution entity can be migrated to another execution entity with which it communicates data, as needed. For example, software steps executed by a server can be migrated to a display device with which it communicates data, and vice versa.

[0051] Figure 2 An exemplary block diagram of the configuration of the control device 100 according to an exemplary embodiment is shown. Figure 2 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 can receive user input operation commands and convert the operation commands into commands that the display device 200 can recognize and respond to, thus acting as an intermediary for interaction between the user and the display device 200.

[0052] In some embodiments, the communication interface 130 is used for external communication and includes at least one of a WIFI chip, a Bluetooth module, an NFC module, or an alternative module.

[0053] In some embodiments, the user input / output interface 140 includes at least one of a microphone, a touchpad, a sensor, a button, or an alternative module.

[0054] Figure 3 A hardware configuration block diagram of the display device 200 according to an exemplary embodiment is shown.

[0055] In some embodiments, the display device 200 includes at least one of a tuner and demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, a user interface.

[0056] In some embodiments, the controller includes a central processing unit, a video processor, an audio processor, a graphic processor, a RAM, a ROM, a first interface to an n-th interface for input / output.

[0057] In some embodiments, the display 260 includes a display screen component for presenting a picture, and a driving component for driving the image display, a component for receiving an image signal originated from the controller output, and displaying video content, image content, and a menu operation interface, and a user operation UI interface, etc.

[0058] In some embodiments, the display 260 can be at least one of a liquid crystal display, an OLED display, and a projection display, and can also be a projection device and a projection screen.

[0059] In some embodiments, the tuner and demodulator 210 receives a broadcast television signal through a wired or wireless receiving manner, and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

[0060] In some embodiments, the communicator 220 is a component for communicating with an external device or a server according to various communication protocol types. For example, the communicator can include at least one of a Wifi module, a Bluetooth module, a wired Ethernet module, other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display device 200 can establish transmission and reception of control signals and data signals with the control device 100 or the server 400 through the communicator 220.

[0061] In some embodiments, the detector 230 is used to collect signals of an external environment or interaction with the outside. For example, the detector 230 includes a light receiver for collecting ambient light intensity, or an image collector such as a camera for collecting external environment scenes, user attributes, or user interaction gestures, or a sound collector such as a microphone for receiving external sounds.

[0062] In some embodiments, the external device interface 240 can include, but is not limited to, any one or more of a High-Definition Multimedia Interface (HDMI), an analog or digital High-Definition Component Input Interface (Component), a Composite Video Input Interface (CVBS), a USB Input Interface (USB), an RGB port, etc. It can also be a composite input / output interface formed by a plurality of the above interfaces.

[0063] In some embodiments, the controller 250 and the tuner demodulator 210 can be located in different separate devices, i.e., the tuner demodulator 210 can also be located in an external device of the main device where the controller 250 is located, such as an external set-top box, etc.

[0064] In some embodiments, the controller 250 controls the operation of the display device and responds to user operations by storing various software control programs in the memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command for selecting a UI object displayed on the display 260, the controller 250 can perform an operation related to the object selected by the user command.

[0065] In some embodiments, the object can be any one of selectable objects, such as a hyperlink, an icon, or other operable control. The operation related to the selected object can be an operation of displaying a page connected to a hyperlink, a document, an image, etc., or an operation of executing a program corresponding to the icon.

[0066] In some embodiments, the controller includes at least one of a Central Processing Unit (CPU), a video processor, an audio processor, a Graphics Processing Unit (GPU), a RAM (Random Access Memory), a ROM (Read-Only Memory), a first to n-th interface for input / output, a communication bus, etc.

[0067] The CPU processor is used to execute operating system and application program instructions stored in the memory, and to execute various application programs, data and content according to various interaction instructions received from external input, so as to finally display and play various audio and video content. The CPU processor can include a plurality of processors. For example, it can include a main processor and one or more sub-processors.

[0068] In some embodiments, the graphics processor is configured to generate various graphical objects, such as at least one of an icon, an operation menu, and a display pattern of a user input instruction. The graphics processor includes an operation unit configured to perform an operation by receiving various interaction instructions from a user, and display various objects according to display attributes; and a renderer configured to perform rendering on the various objects obtained based on the operation unit, and the rendered objects are used for display on a display.

[0069] In some embodiments, the video processor is configured to receive an external video signal, and perform at least one of decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis on the input signal according to a standard codec protocol, to obtain a signal directly displayable on the display device 200.

[0070] In some embodiments, the video processor includes at least one of a demultiplexing module, a video decoding module, an image synthesizer, a frame rate conversion module, and a display formatting module. The demultiplexing module is configured to perform demultiplexing processing on an input audio / video data stream. The video decoding module is configured to perform processing, such as decoding and scaling, on the demultiplexed video signal. The image synthesizer is configured to perform superimposition and mixing processing on a video image processed by the scaling on the GUI signal generated by the graphics generator according to a user input or self-generation, to generate an image signal displayable. The frame rate conversion module is configured to convert the frame rate of an input video. The display formatting module is configured to change a signal, such as an output RGB data signal, to a signal conforming to a display format, on the basis of the video signal output after the frame rate conversion.

[0071] In some embodiments, the audio processor is configured to receive an external audio signal, and perform at least one of decompression and decoding, and processing, such as noise reduction, digital-to-analog conversion, and amplification, on the input signal according to a standard codec protocol, to obtain a sound signal playable on a speaker.

[0072] In some embodiments, the user can input a user command through a graphical user interface (GUI) displayable on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user can input a user command through input of a specific sound or gesture, and the user input interface receives the user input command by recognizing the sound or gesture through a sensor.

[0073] In some embodiments, a "user interface" is a medium interface between an application program or an operating system and a user for interaction and information exchange, which realizes the conversion between the internal form of information and the form that the user can accept. The commonly used form of the user interface is a graphic user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be an icon, a window, a control, etc. displayed on the display screen of an electronic device, wherein the control can include at least one of the following visual interface elements: an icon, a button, a menu, a tab, a text box, a dialog box, a status bar, a navigation bar, a widget, etc.

[0074] In some embodiments, the user interface 280 is an interface (e.g., a physical button on the body of the display device, or the like) that can be used to receive a control input.

[0075] In some embodiments, the system of the display device can include a kernel, a shell, a file system, and an application program. The kernel, the shell, and the file system together form a basic operating system structure, which allows a user to manage files, run programs, and use the system. After power-on, the kernel is started, the kernel space is activated, the hardware is abstracted, the hardware parameters are initialized, the virtual memory is run and maintained, the scheduler, the signal, and the inter-process communication (IPC) are maintained. After the kernel is started, the shell and the user application program are loaded. The application program is compiled into machine code after being started, forming a process.

[0076] Referring to Figure 4 In some embodiments, the system is divided into four layers from top to bottom, namely, an application layer (referred to as "application layer" for short), an application framework layer (referred to as "framework layer" for short), an Android runtime and system library layer (referred to as "system runtime library layer" for short), and a kernel layer.

[0077] In some embodiments, at least one application program is run in the application layer, which can be a window (Window) program, a system setting program, or a clock program, etc. provided by the operating system; or an application program developed by a third-party developer. In a specific implementation, the application program package in the application layer is not limited to the above examples.

[0078] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer includes some pre-defined functions. The application framework layer is equivalent to a processing center that decides which application in the application layer to act. The application accesses resources in the system and obtains services of the system through the API interface during execution.

[0079] As shown in Figure 4 In the embodiments of the present application, the application framework layer includes managers (Managers) and content providers (Content Provider), wherein the managers include at least one of the following modules: an activity manager (ActivityManager) for interacting with all activities running in the system; a location manager (Location Manager) for providing access to system location services for system services or applications; a package manager (Package Manager) for retrieving various information related to application packages currently installed on the device; a notification manager (NotificationManager) for controlling the display and clearing of notification messages; and a window manager (Window Manager) for managing icons, windows, toolbars, wallpapers, and desktop components on the user interface.

[0080] In some embodiments, the activity manager is used to manage the life cycle of each application and the general navigation back function, such as controlling the exit, opening, back, etc. of the application. The window manager is used to manage all window programs, such as obtaining the size of the display screen, determining whether there is a status bar, locking the screen, intercepting the screen, controlling the display window change (such as reducing the display window, shaking the display, twisting the display, etc.), etc.

[0081] In some embodiments, the system runtime library layer provides support for the upper layer, i.e. the framework layer. When the framework layer is used, the Android operating system runs the C / C++ library contained in the system runtime library layer to realize the functions of the framework layer.

[0082] In some embodiments, the kernel layer is a layer between hardware and software. As shown in Figure 4 The kernel layer includes at least one of the following drivers: an audio driver, a display driver, a Bluetooth driver, a camera driver, a WIFI driver, a USB driver, an HDMI driver, a sensor driver (such as a fingerprint sensor, a temperature sensor, a pressure sensor, etc.), and a power supply driver, etc.

[0083] For some display devices, when performing image recognition, a user needs to capture the image currently played by the display device, and the display device performs image recognition on the image captured by the user. When the image is in a static state, a static recognition model, such as an OCR text model, can be used to recognize the text content in the image, such as subtitles or prompt words. When the image is in a dynamic state, a dynamic recognition model can be used to recognize the target object in the image, such as each person in the image. Since the existing display device needs to capture the image by the user when performing image recognition, the intelligence is low, and the user experience is poor. At the same time, the process of capturing the image by the user needs to spend a certain amount of time, which leads to a long time of image recognition and low efficiency.

[0084] The present application provides a display device, comprising a display and a controller.

[0085] In some embodiments, the display can play media resources in the network, such as video, etc. When playing the media, the controller can obtain image data from the network signal source and process the image data to obtain a series of images and display them in the display.

[0086] For the display, when displaying the picture, the display can include a plurality of layers surface, each layer can correspond to a type, for example: the display can include a video layer and a picture layer, etc. The image data obtained from the network signal source is the content to be displayed by all layers in the display. The display can display one frame of image at each moment, and the image data obtained by the display device at time t is the content to be displayed by all layers at time t.

[0087] The image data can include a plurality of image contents, each image content is the content to be displayed in a layer at a certain moment.

[0088] It should be noted that each layer in the display has its own layer attribute, such as layer coordinates and layer size. The image data obtained by the display device includes a plurality of image contents, each image content has its own attribute information, which can be used to determine which layer each image content corresponds to. For example, the first layer in the display is a rectangle with a length of A and a width of B. The coordinates of the first layer are the area between points X1, X2, X3 and X4. For all the obtained image contents, the size and coordinates of each image content can be matched. When the size and coordinates of the image content are the same as those of the first layer, the image content is the content to be displayed in the first layer.

[0089] Therefore, the layer attribute can be used to determine which layer the image content belongs to.

[0090] In some embodiments, the display can include multiple layers, such as a video layer, a text layer, and a picture layer. The video layer can include video content, such as a movie or a TV series. The text layer can include text content, such as an e-book or media information. The picture layer can include picture content, such as a menu UI of the display device or various pictures.

[0091] Each layer can further include multiple sub-layers. For example, the display can display a first window and a second window, and the two windows can play two media resources. In this case, the display includes two video layers.

[0092] In some embodiments, the display can play the media in full screen. In this case, the image data obtained by the controller can only include the video content corresponding to the media. Meanwhile, the display can display the video layer in full screen.

[0093] In some embodiments, the display can also play the media in a windowed manner. In this case, the image data obtained by the controller can include the image content of the video layer, i.e., the video content corresponding to the media. The image data can also include the image content of the text layer, such as the work information of the media, including the information of the leading actors, the director, the scriptwriter, the region, the film and television type, and the release time. The image data can further include the image content of the picture layer, such as the icon of the media.

[0094] Figure 5 FIG. 1 shows a schematic diagram of a frame of image displayed by the display in an example. The display includes three layers. The image content of the video layer is the media resource being played. The image content of the text layer is the work information of the media resource. The image content of the picture layer is the icon of the media resource. The image content of the three layers at the same time can be obtained by superimposition processing to obtain a frame of image.

[0095] In some embodiments, the controller can continuously obtain the image data. When the image data is obtained, the image data can be stored in a preset layer cache queue, so that the layer cache queue can include all the image data received by the controller, i.e., the image content of all the layers. When the display device needs to use the image content, such as generating a frame of image using the image content, or other functions requiring the use of the image content, the image content can be directly obtained from the layer cache queue, without the need to obtain from the network signal source, thereby improving the efficiency.

[0096] In some embodiments, when the new image data K1 is acquired, the controller can store the image data K1 into the preset layer cache queue and record the current time t1. When the image data K2 of the next time is acquired, the controller can store the image data K2 into the preset layer cache queue and record the time t2 corresponding to the image data K2. Meanwhile, all the image data in the layer cache queue is arranged in time sequence. Table 1 shows the layer cache queue in a feasible embodiment.

[0097] Table 1 Layer cache queue

[0098] Image data Layer A Layer B Layer C Time t1 A1 B1 C1 Time t2 A2 B2 C2

[0099] The display includes three types of layers, A, B and C. The first frame of image data acquired by the controller includes three image contents, A1, B1 and C1, and the time is t1. The second frame of image data acquired by the controller includes three image contents, A2, B2 and C2, and the time is t2. When the next frame of image data is acquired, the new image data is continuously added to the layer cache queue in time sequence, and the time is recorded.

[0100] In some embodiments, the controller can generate an image by using the image content. Specifically, the controller can generate an image by using the internal image generation module. The image generation module stores the execution unit of the image generation algorithm, which can generate an image by using the image content, for example, by superimposing the image content to obtain the image.

[0101] Specifically, the controller can acquire the image data of a certain time from the preset layer cache queue, superimpose all the image contents in the image data of the time, obtain a frame of image, that is, the target image of the time, and display the target image in the display.

[0102] The controller can sequentially superimpose each image data of a certain time in the order of the layer cache queue, thereby obtaining each frame of target image to be played in the display. Meanwhile, each frame of target image can be played in the display in time sequence, thereby realizing the playing of the corresponding media or video, etc.

[0103] In some embodiments, the display device further includes an image collector, for example, a camera. The camera can be built-in as a detector in the display device or externally connected to the display device as an external device. For the camera externally connected to the display device, the camera can be connected to the external device interface of the display device and accessed to the display device. The user can use the camera to complete the photographing or shooting function on the display device, thereby displaying the data collected by the camera to the display for the user to watch.

[0104] The camera can further include a lens assembly in which a photosensitive element and a lens are arranged. The lens can refract light through a plurality of lens pieces, so that the light of the image of the scene can be irradiated on the photosensitive element. The photosensitive element can be selected according to the specification of the camera based on the detection principle of CCD (Charge-coupled Device) or CMOS (Complementary Metal Oxide Semiconductor), and the photosensitive material can convert the optical signal into an electrical signal, and output the converted electrical signal into image data. The camera can also acquire image data frame by frame according to the set sampling frequency, so as to form video stream data according to the image data.

[0105] In some embodiments, the user can send a control instruction to start the camera to the display device to turn on the camera.

[0106] The user can send a camera start instruction to the display device by operating a specified key of the remote controller. The correspondence between the camera start instruction and the remote controller key is bound in advance in the actual application. For example, a camera start key is set on the remote controller, when the user touches the key, the remote controller sends the camera start instruction to the controller, and the controller controls the camera to start at this time. When the user touches the key again, the controller can control the camera to close.

[0107] The camera start option can also be set in the UI interface of the display device, when the user clicks the option, the display device can be controlled to start the camera.

[0108] In some embodiments, when the camera is in the start state, the image data can be acquired in real time through the lens assembly, and the acquired image data is sent to the display device. The controller can receive the image data sent by the camera. And generate a target image according to the image data of the camera, so as to be displayed in the display for the user to watch.

[0109] The image data of the camera includes an image content, that is, the picture taken by the camera, which is used to be displayed in the corresponding layer of the camera. The controller can control the display to directly display the picture taken by the camera for the user to watch.

[0110] In some embodiments, the display device can simultaneously display the corresponding picture captured by the camera and the corresponding picture obtained from the network signal source in the display. At this time, the display device needs to simultaneously obtain the first image data sent by the camera and the second image data in the network signal source. For t1 moment, the first image data includes the picture captured by the camera, which is used for display in the camera corresponding layer. The second image data includes several image contents in the network signal source. Then the first image data and the second image data at t1 moment are superimposed to obtain a frame of image corresponding to this moment, and then each frame of image to be played in the display is obtained.

[0111] The controller can control the display to display the first window and the second window, wherein the first window displays the picture corresponding to the camera, and the second window displays the media resource. Figure 6 A schematic diagram of the picture displayed in a feasible embodiment is shown. The first window includes a layer for displaying the camera picture. The first image data sent by the camera includes an image content, which is the content to be displayed in the camera layer. The second window includes three layers for displaying the media picture. Among them, the image content of the media resource is the video layer, the image content of the work information of the media resource is the text layer, and the image content of the icon of the media resource is the picture layer. Therefore, the second image data in the network signal source includes three kinds of image contents. The controller obtains a frame of image currently displayed in the display by superimposing the four kinds of image contents at the same time.

[0112] In some embodiments, the user can use the display device for video chat, and at this time the controller can control the display to display the first window and the second window. Among them, the user's situation is photographed by the camera and displayed in the first window. The situation of the chat object is obtained through the network signal source and displayed in the second window. Figure 7 A schematic diagram of the picture displayed in a feasible embodiment is shown. The first window is the camera picture, which shows the user's situation. The camera picture is a layer, and at this time the first image data sent by the camera includes an image content. The second window is the chat object picture, and the camera picture of the chat object can be obtained from the network signal source, and at this time the second image data in the network signal source also includes an image content. The controller obtains a frame of image currently displayed in the display by superimposing the two kinds of image contents.

[0113] The display device can also directly identify the image content while generating the target image according to the image content.

[0114] In some embodiments, the display device has an image recognition function, and can recognize the image content of all layers in the display, i.e., the content to be displayed in all layers. The user can input an image recognition instruction to the display device, and the controller can control the display device to enter an image recognition mode when the image recognition instruction is received. At this time, the controller can recognize the image content of all layers.

[0115] In some embodiments, the user can send an image recognition instruction to the display device by operating a specified key of the remote controller. The correspondence between the image recognition instruction and the key of the remote controller can be bound in advance in actual application. For example, an image recognition key is provided on the remote controller, and when the user touches the key, the remote controller sends an image recognition instruction to the controller, and at this time, the controller controls the display device to enter an image recognition mode and recognize the image content of all layers in the display. When the user touches the key again, the controller can control the display device to exit the image recognition mode.

[0116] In some embodiments, the correspondence between the image recognition instruction and multiple keys of the remote controller can also be bound in advance, and the remote controller sends the image recognition instruction when the user touches multiple keys bound with the image recognition instruction. In a feasible embodiment, the keys bound with the image recognition instruction are direction keys (left, down, left, down) in sequence, i.e., the remote controller sends the image recognition instruction to the controller only when the user continuously touches the keys (left, down, left, down) within a preset time. The above binding method can avoid that the image recognition instruction is sent due to user's misoperation.

[0117] The embodiments of the present application only exemplarily provide several binding relationships between the image recognition instruction and the keys, and the binding relationship between the image recognition instruction and the keys can be set according to the user's habits in actual application, which is not limited herein.

[0118] In some embodiments, the user can send an image recognition instruction to the display device when using a smart device to control the display device, for example, using a mobile phone. A control can be provided in the mobile phone in actual application, and the control can be used to select whether to enter an image recognition mode, so as to send an image recognition instruction to the controller, and at this time, the controller controls the display device to enter the image recognition mode.

[0119] In some embodiments, when the user uses the mobile phone to control the display device, the mobile phone can be sent a continuous clicking instruction. The continuous clicking instruction refers to that the user clicks the same area of the touch screen of the mobile phone more than a preset threshold number of times within a preset period. For example, when the user continuously clicks the same area of the touch screen of the mobile phone 3 times within 1 second, it is considered as one continuous clicking instruction. After receiving the continuous clicking instruction, the mobile phone can send an image recognition instruction to the display device, so that the controller controls the display device to enter the image recognition mode.

[0120] In some embodiments, when the user uses the mobile phone to control the display device, the mobile phone can also be set to send an image recognition instruction to the display device when it is detected that the user's touch pressure value on a certain area of the touch screen of the mobile phone exceeds a preset pressure threshold.

[0121] The image recognition option can also be set in the UI interface of the display device. When the user clicks the option, the display device can be controlled to enter or exit the image recognition mode.

[0122] In some embodiments, when the display device is in the image recognition mode, the controller can recognize the image content. Assuming that the display includes two layers, a first layer and a second layer, the image data includes two types of image content, and each layer corresponds to one type of image content, at this time, the image content of the two layers can be recognized respectively.

[0123] When recognizing the image content of the first layer, i.e., the content to be displayed in the first layer, the controller can detect the change state of the first layer. If the first layer changes within a specified time, it can be considered that the first layer is in a dynamic change state, such as displaying a video or a dynamically changing picture, etc. If the first layer does not change within a specified time, it can be considered that the first layer is in a static state, such as displaying a still picture or text, etc.

[0124] In some embodiments, the controller can obtain the image content of the first layer at a first time and the image content of the first layer at a second time from the layer cache queue, and detect whether the two image contents are the same. The first time and the second time are different by a preset period. The first time can be the time when the display device enters the image recognition mode.

[0125] The preset period can be a time corresponding to receiving one frame of image data. For example, if the image data obtained by the controller is 60 frames, the time for the controller to receive one frame of image data is 1 / 60 second, and the preset period is set to 1 / 60 second. Therefore, the image content of the first layer at the first time and the image content of the first layer at the second time are two consecutive image contents received by the controller. At this time, the controller can detect whether the two consecutive image contents are the same. If the two image contents are the same, the first layer is in a static state, and if the two image contents are different, the first layer is in a dynamic change state.

[0126] In some embodiments, the controller can compare the new image content with the previous image content every time it receives the image content corresponding to the first layer, thereby detecting the change state of the first layer in real time.

[0127] In some embodiments, considering that the degree of change in one frame of time is not very obvious, the preset period can also be set to a time corresponding to receiving multiple frames of image data. For example, the time corresponding to one frame of image data is T1, and the preset period can be 3T1. That is, the controller can detect the image content of the first frame and the fourth frame, and determine whether the two image contents are the same.

[0128] In some embodiments, when detecting whether the image content of the first layer at the first time and the image content of the first layer at the second time are the same, the controller can perform image subtraction (diff) processing on the two image contents, for example, inputting the two image contents into a DiffImg tool for processing.

[0129] If the image subtraction processing result is 0, it means that the two image contents have no difference and are the same. At this time, the first layer is in a static state.

[0130] If the image subtraction processing result is not 0, it means that there is a difference between the two image contents and they are different. At this time, the first layer is in a dynamic change state.

[0131] It should be noted that the image content can include a plurality of target objects, and when the two image contents are different, the target objects and the position information will also change. The image subtraction processing result of the two image contents can include the change area of the target objects, i.e., the layer difference area. It can also include the position information of the layer difference area.

[0132] Figure 8a A schematic diagram of the image content of the first layer at the first time in a feasible embodiment is shown, which includes two target objects P1 and P2. Figure 8bFig. 2 shows a schematic diagram of the image content of the first layer at the second time point in an embodiment, in which the positions of the target objects P1 and P2 have changed. After image subtraction processing is performed on the two image contents, an image subtraction processing result can be obtained. Figure 8c Fig. 3 shows a schematic diagram of the layer difference region in an embodiment, in which Q1 is the changed region of the target object P1 and Q2 is the changed region of the target object P2.

[0133] In some embodiments, when it is detected that the first layer is in a static state, the controller can perform static recognition on the image content of the first layer, which can be text recognition processing to obtain text information. For example, the first layer is input into an OCR text recognition model, and text content in the image content of the first layer and position information of the text content in the first layer can be obtained.

[0134] In some embodiments, when it is detected that the first layer is in a dynamic change state, the controller can perform dynamic recognition processing on the image content of the first layer. For example, the image content of the first layer at the first time point, the image content of the first layer at the second time point, and the image subtraction processing result are input into a dynamic recognition model to obtain a dynamic recognition result. The dynamic recognition result can include the recognition result of each target object, and the dynamic recognition model can be a target detection model or a target tracking model.

[0135] In some embodiments, the controller can detect the change state of each layer in real time, and then recognize the image content of each layer according to the detection result, so as to recognize the content in all layers. Figure 9 Fig. 4 shows an interaction flowchart of the components of the display device in some embodiments.

[0136] The embodiments of the present application can directly recognize the image content by using the layer cache queue, so that the image content recognition can be performed at the same time when the image is generated, without the need to wait until the image is generated and then perform recognition, thereby improving the efficiency, and all the work is completed by the display device, without the need for the user to intercept the image, and the degree of intelligence is high.

[0137] In some embodiments, an intelligent recognition module can be arranged in the controller, and the intelligent recognition module internally stores algorithms corresponding to the above-mentioned methods for recognizing the layer content. Therefore, the intelligent recognition module can directly obtain the image content from the layer cache queue, judge the change state of the layer, and further recognize the image content.

[0138] In some embodiments, a monitoring module can be arranged in the controller, which can detect image data in the layer cache queue in real time, and other modules can register their identities in the monitoring module. When the other modules need to use the image data in the layer cache queue, they can communicate with the monitoring module. The monitoring module detects whether the module has registered its identity, and if so, the monitoring module sends the image data in the layer cache queue to the module. For example, when the display device enters the image recognition mode, the intelligent recognition module can communicate with the monitoring module. After the monitoring module confirms the identity, it can send the image data in the layer cache queue to the intelligent recognition module. At the same time, whenever new image data is added to the layer cache queue, the monitoring module can send the new image data to the intelligent recognition module.

[0139] In some embodiments, when the controller obtains image data from the layer cache queue, it can detect whether there is a layer in a dynamic change state, and confirm the change state of the target image to be displayed in the display.

[0140] For example, the display includes two layers, a first layer and a second layer, and the image data of the first layer and the image data of the second layer can generate a target image to be displayed in the display. The controller can detect whether the content to be displayed at a first time and the content to be displayed at a second time of any one of the first layer and the second layer are different. If so, it is considered that the target image to be displayed in the display is in a dynamic change state, and if not, it is considered that the target image to be displayed in the display is in a static state.

[0141] Specifically, when it is detected that the first layer or the second layer or both layers are in a dynamic change state, it is considered that the target image corresponding to the image data is in a dynamic change state. When it is detected that both layers are in a static state, it is considered that the target image corresponding to the image data is in a static state.

[0142] The controller can obtain the target image according to all the layers, and then perform content recognition on the target image. When the target image is in a dynamic change state, dynamic recognition processing is performed on the target image to identify the target object in the target image. When the target image is in a static state, OCR text recognition processing is performed on the target image to obtain text information.

[0143] It can be seen that the difference between the present embodiment and the above-mentioned embodiments is that when the image data is obtained, the change state of the image is determined according to the change state of the layer, and further content recognition is performed on the image.

[0144] The present application also provides an image recognition method, which is applied to a display device, as shown in Figure 10 The method comprises the following steps:

[0145] S101, acquire all layers in the display, the all layers including at least one of a video layer and a picture layer;

[0146] S102, detect whether the content to be displayed in the layer at the first time is same as the content to be displayed in the layer at the second time; if yes, the layer is in a static state, if not, the layer is in a dynamic change state;

[0147] S103, when the layer is in the static state, perform static identification on the content to be displayed in the layer; when the layer is in the dynamic change state, perform dynamic identification on the content to be displayed in the layer.

[0148] The same and similar parts among various embodiments in the specification can be referred to each other, and will not be described here.

[0149] Those skilled in the art can clearly understand that the technology in the embodiments of the present application can be realized by means of software and necessary general hardware platforms. Based on such understanding, the technical solutions in the embodiments of the present application can be embodied in the form of a software product, which can be stored in a storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the method of various embodiments or some parts of the embodiments of the present application.

[0150] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

[0151] In order to facilitate explanation, the above description has been made in combination with specific embodiments. However, the above exemplary discussion is not intended to exhaust or limit the embodiments to the specific forms disclosed above. Various modifications and variations can be derived according to the above teachings. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different modified embodiments suitable for specific use considerations.

Claims

1. A display device, characterized by comprising: The application relates to a display device and a display method. The display device comprises a display and a controller configured to: acquire all layers in the display, the all layers comprising at least one of a video layer and a picture layer; perform image subtraction processing on content to be displayed in the layer at a first time and content to be displayed in the layer at a second time; detect, based on the image subtraction processing result, whether the content to be displayed in the layer at the first time and the content to be displayed in the layer at the second time are the same; if yes, the layer is in a static state, and if no, the layer is in a dynamic change state; when the layer is in the static state, perform static identification on the content to be displayed in the layer; when the layer is in the dynamic change state, perform dynamic identification on the content to be displayed in the layer based on a layer difference region obtained after the image subtraction processing and position information of the layer difference region in the layer, wherein the layer difference region is a change region of a target object in the layer. The controller is further configured to:

2. The display device of claim 1, wherein, in the step of detecting, based on the image subtraction processing result, whether the content to be displayed in the layer at the first time and the content to be displayed in the layer at the second time are the same, if the image subtraction processing result is 0, the content to be displayed in the layer at the first time and the content to be displayed in the layer at the second time are the same; and if the image subtraction processing result is not 0, the content to be displayed in the layer at the first time and the content to be displayed in the layer at the second time are different. The controller is further configured to:

3. The display device of claim 1, wherein, in the step of performing, when the layer is in the static state, static identification on the content to be displayed in the layer, perform text identification processing on the content to be displayed in the layer to obtain text information, the text information comprising text content to be displayed in the layer and position information of the text content in the layer. The controller is further configured to:

4. The display device of claim 2, wherein, in the step of performing, when the layer is in the dynamic change state, dynamic identification on the content to be displayed in the layer, input the content to be displayed in the layer at the first time, the content to be displayed in the layer at the second time, the layer difference region and the position information of the layer difference region in the layer into a dynamic identification model to obtain a dynamic identification result, wherein the dynamic identification result comprises an identification result of a target object. The application relates to a display device and a display method.

5. A display device, characterized by The display device comprises a display and a controller configured to: acquire all layers in the display, the all layers comprising a first layer and a second layer, the first layer and the second layer being used to generate a target image; perform image subtraction processing on content to be displayed in a target layer at a first time and content to be displayed in the target layer at a second time; wherein the target layer is any one of the first layer and the second layer; detect, based on the image subtraction processing result, whether the content to be displayed at the first time and the content to be displayed at the second time are different; if yes, the target image is in a dynamic change state, and if no, the target image is in a static state; when the target image is in the static state, perform static identification on the target image. ​ ​ When the target image is in a dynamic change state, the target image is dynamically recognized based on a layer difference region obtained through image subtraction processing and position information of the layer difference region in the layer; the layer difference region is a change region of a target object in the layer.

6. The display device of claim 5, wherein, The controller is further configured to: Before the step of detecting whether the content to be displayed at the first time and the content to be displayed at the second time are different based on the image subtraction processing result, acquire image data, the image data being the content to be displayed in all layers.

7. The display device of claim 6, wherein, The display device further comprises: an image collector configured to collect first image data; the controller is further configured to: In the step of acquiring image data, receive the first image data sent by the image collector, and / or acquire second image data from a network signal source.

8. The display device of claim 6, wherein, The controller is further configured to: In the step of acquiring image data, when the image data at the first time is acquired, store the image data at the first time into a preset layer cache queue; the image data at the first time includes the content to be displayed in all layers at the first time; when the image data at the second time is acquired, store the image data at the second time into the preset layer cache queue and arrange the image data in chronological order; the image data at the second time includes the content to be displayed in all layers at the second time; the first time and the second time are different by a preset period.

9. The display device of claim 8, wherein, The controller is further configured to: acquire the image data at the first time from the preset layer cache queue, the image data at the first time including the content to be displayed in all layers at the first time; superimpose the content to be displayed in all layers at the first time to obtain a target image at the first time, and control the display to display the target image at the first time.

10. An image recognition method applied to a display device, characterized in that, The method comprises: acquire all layers in the display, the all layers including at least one of a video layer and a picture layer; perform image subtraction processing on the content to be displayed in the layer at the first time and the content to be displayed in the layer at the second time; detect whether the content to be displayed in the layer at the first time and the content to be displayed in the layer at the second time are the same based on the image subtraction processing result; if they are the same, the layer is in a static state, and if they are not the same, the layer is in a dynamic change state; when the layer is in a static state, perform static recognition on the content to be displayed in the layer; when the layer is in a dynamic change state, perform dynamic recognition on the content to be displayed in the layer based on a layer difference region obtained through image subtraction processing and position information of the layer difference region in the layer; the layer difference region is a change region of a target object in the layer.

Images