Display device and interaction-based display device control method

Abstract

The invention relates to the technical field of display equipment, in particular to display equipment and an interaction-based display equipment control method. The method comprises the following steps: constructing an operation instruction according to demand description information input by a user, and controlling the display equipment to perform autonomous operation based on the operation instruction; controlling the display to display the autonomous operation result, and receiving a judgment result of the user for the autonomous operation result; under the condition that the judgment result is unqualified, receiving modification information input by the user; and updating the operation instruction according to a modification operation demand corresponding to the modification information, and returning to the step of controlling the display equipment to perform autonomous operation based on the operation instruction and continuing to execute until the qualification judgment result of the user for the autonomous operation result is qualified. In the process of controlling the display equipment to perform autonomous operation, the operation instruction is updated in combination with the modification information fed back by the user for the second time, so that the updated operation instruction better meets the actual demand of the user.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and an interactive display device control method. Background Art

[0002] Display devices are usually used to display multimedia content. Currently, in order to improve the convenience of users in using display devices, operation guidance for display devices has emerged through artificial intelligence.

[0003] However, as the functions of display devices become increasingly rich, diversified applications and constantly updated functions often bring operational problems to users, especially specific groups such as the elderly and children. In addition, with the explosive growth of content played by display devices, the artificial intelligence operation guidance in traditional technologies can no longer meet the users' personalized guidance needs.

[0004] Therefore, how to meet the user's personalized guidance needs so that the user can quickly master the various functions of the display device is a problem that needs to be solved urgently. Summary of the invention

[0005] The present application provides a method for solving the problem that the artificial intelligence operation guidance in traditional technology can no longer meet the personalized guidance needs of users.

[0006] In a first aspect, some embodiments provide a display device, a display, a display, configured to display content from a broadcast system or network and / or a user interface;

[0007] and at least one controller connected to the display and configured to:

[0008] According to the requirement description information input by the user, construct an operation instruction for completing the corresponding operation process, and control the display device to perform autonomous operation based on the operation instruction;

[0009] Controlling the display to present the autonomous operation result, and receiving the user's judgment result on the autonomous operation result;

[0010] If the judgment result is unqualified, receiving modification information input by the user;

[0011] The operation instruction is updated according to the modification operation requirement corresponding to the modification information, and the step of controlling the display device to perform autonomous operation based on the operation instruction is returned and continued to be executed until it is determined that the user's qualified judgment result for the autonomous operation result is qualified.

[0012] Based on the demand description information input by the user, an operation instruction for completing the corresponding operation process is constructed, and the display device is controlled to perform autonomous operation based on the operation instruction, and the user's judgment result on the autonomous operation result is received. When the judgment result is unqualified, the operation instruction is updated according to the modification information input by the user, and the step of controlling the display device to perform autonomous operation based on the operation instruction is returned and continued to be executed until it is determined that the user's judgment result on the autonomous operation result is qualified. According to the above content, it can be seen that in the process of responding to the demand description information input by the user, the present application will not only construct operation instructions and control the display device to perform autonomous operation based on the operation instructions, but also receive the modification information input by the user when the user judges that the autonomous operation result is unqualified, and update the operation instructions based on the modification operation requirements corresponding to the modification information, so as to ensure that the autonomous operation result with the judgment result as qualified can be obtained according to the updated operation instructions; therefore, in the process of controlling the display device to perform autonomous operation, the present application does not only perform autonomous operation based on the demand description information input by the user for the first time, but also updates the operation instructions based on the modification information fed back by the user for the second time, so that the updated operation instructions are more in line with the actual needs of the user, thereby ensuring that the autonomous operation of the display device can meet the personalized guidance needs of the user, so that the user can quickly master the relevant functional processes of the display device according to the above process, thereby improving the convenience and flexibility of the user in using the display, and reducing the difficulty of operating the display device.

[0013] In one of the embodiments, when the controller executes updating of the operation instruction according to the modification operation requirement corresponding to the modification information, the controller is further configured to:

[0014] According to the modification information, determine the operation steps to be adjusted in the operation process, as well as the reference operation steps after the operation steps to be adjusted are modified according to the modification description requirements;

[0015] The instructions to be adjusted corresponding to the operation steps to be adjusted in the operation instructions are updated based on the reference operation steps.

[0016] By updating the instructions to be adjusted corresponding to the operation steps to be adjusted, the adjusted operation instructions are made more in line with the actual needs of the users, ensuring that the user's demand description information can be fully met, and optimizing the user's operating experience for the display device.

[0017] In one embodiment, when the controller executes updating of the to-be-adjusted instruction corresponding to the to-be-adjusted operation step in the operation instruction based on the reference operation step, the controller is further configured to:

[0018] Replace the instructions to be adjusted corresponding to the operation steps to be adjusted in the operation instructions with the reference instructions corresponding to the reference operation steps.

[0019] By replacing the instructions to be adjusted with reference instructions, the specific update processing of the operation instructions is achieved, which ensures the smooth progress of subsequent processes and optimizes the user's operating experience for the display device.

[0020] In one embodiment, when the controller executes the demand description information input by the user to construct an operation instruction for completing the corresponding operation process, it is also configured to:

[0021] Determine the user's device operation requirements for the display device according to the requirement description information input by the user;

[0022] According to the equipment operation requirements, construct operation instructions for completing the corresponding operation process.

[0023] By determining the device operation requirements, operation instructions are constructed according to the device operation requirements, so that the constructed operation instructions fully meet each device operation requirement contained in the requirement description information, prevent the omission of user device operation requirements, and ensure that the user experience is not affected.

[0024] In one embodiment, when the controller executes the operation instruction for completing the corresponding operation process according to the device operation requirements, it is also configured to:

[0025] According to the device operation requirement, a target operation step is selected from the candidate operation steps included in the display device;

[0026] According to the execution order corresponding to the target operation steps, the target operation steps are combined to obtain the target operation process corresponding to the user;

[0027] Construct operation instructions according to the target operation process.

[0028] The construction of operation instructions for the target operation process is realized, which ensures the smooth execution of the operation instructions with full reference to the target operation steps during the construction process, and prevents abnormal equipment operation when executing the operation instructions.

[0029] In one of the embodiments, when the controller selects a target operation step from candidate operation steps included in the display device according to the device operation requirement, the controller is further configured to:

[0030] Obtain the user's historical operation records on the display device;

[0031] Selecting a preferred operation step from the candidate operation steps included in the display device; wherein the preferred operation step is an operation step whose triggering times are greater than a threshold in the user's historical operation records on the display device;

[0032] According to the device operation requirement, an intermediate operation step is selected from the candidate operation steps included in the display device;

[0033] Verify whether the intermediate operation steps contain replaceable operation steps with the same effect as the preferred operation steps;

[0034] If so, replace the replaceable operation steps in the intermediate operation steps with the corresponding preferred operation steps to obtain the target operation steps.

[0035] By introducing intermediate operation steps, it is ensured that the user's historical operation habits can be combined in the process of determining the target operation steps, so that when controlling the display device for autonomous operation, it can comply with the user's usage habits, further improving the user's operation experience of the display device.

[0036] In one of the embodiments, when the controller determines the device operation requirement of the user for the display device according to the requirement description information input by the user, the controller is further configured to:

[0037] Based on the artificial intelligence model, the device operation requirements corresponding to the requirement description information input by the user are obtained.

[0038] Achieve efficient acquisition of device operation requirements corresponding to the requirement description information input by the user.

[0039] In one embodiment, the artificial intelligence model includes an artificial intelligence extraction model and an artificial intelligence analysis model;

[0040] Based on the artificial intelligence model, when the controller obtains the device operation requirements corresponding to the requirement description information input by the user, it is also configured as follows:

[0041] Based on the artificial intelligence extraction model, key information is extracted from the demand description information input by the user to obtain the target description information;

[0042] Based on the artificial intelligence analysis model, the target description information is subjected to information semantic analysis to obtain the equipment operation requirements.

[0043] The key information extraction of the demand description information input by the user and the information semantic analysis of the target description information are respectively realized, ensuring that the device operation requirements corresponding to the demand description information input by the user are smoothly obtained according to the artificial intelligence model.

[0044] In one of the embodiments, the device operation requirement includes at least one of a remote control operation requirement, a subtitle reading requirement, a network connection requirement, an application download requirement, a display device setting requirement, and an erroneous operation recovery requirement.

[0045] Describes the types of requirements included in the equipment operation requirements.

[0046] In a second aspect, some embodiments further provide a display device control method based on interaction, the method is applied to a controller in a display device, the controller is used to receive a control signal of the control device, and control a display in the display device to display according to the control signal, the method comprising:

[0047] According to the requirement description information input by the user, construct an operation instruction for completing the corresponding operation process, and control the display device to perform autonomous operation based on the operation instruction;

[0048] Controlling the display to present the autonomous operation result, and receiving the user's judgment result on the autonomous operation result;

[0049] When the judgment result is unqualified, receiving modification information input by the user;

[0050] The operation instruction is updated according to the modification operation requirement corresponding to the modification information, and the step of controlling the display device to perform autonomous operation based on the operation instruction is returned and continued to be executed until it is determined that the user's qualified judgment result for the autonomous operation result is qualified.

[0051] Some embodiments provide an interactive display device control method, which constructs operation instructions for completing the corresponding operation process according to the demand description information input by the user, controls the display device to perform autonomous operation based on the operation instructions, receives the user's judgment result on the autonomous operation result, and when the judgment result is unqualified, updates the operation instructions according to the modification information input by the user, and returns to the step of controlling the display device to perform autonomous operation based on the operation instructions and continues to execute until it is determined that the user's judgment result on the autonomous operation result is qualified. According to the above content, it can be seen that in the process of responding to the demand description information input by the user, the present application will not only construct operation instructions and control the display device to perform autonomous operation based on the operation instructions, but also receive the modification information input by the user when the user judges that the autonomous operation result is unqualified, and update the operation instructions based on the modification operation requirements corresponding to the modification information, so as to ensure that the autonomous operation result with the judgment result as qualified can be obtained according to the updated operation instructions; therefore, in the process of controlling the display device to perform autonomous operation, the present application does not only perform autonomous operation based on the demand description information input by the user for the first time, but also updates the operation instructions based on the modification information fed back by the user for the second time, so that the updated operation instructions are more in line with the actual needs of the user, thereby ensuring that the autonomous operation of the display device can meet the personalized guidance needs of the user, so that the user can quickly master the relevant functional processes of the display device according to the above process, thereby improving the convenience and flexibility of the user in using the display, and reducing the difficulty of operating the display device. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0053] FIG1 is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0054] FIG2 is a schematic diagram of a hardware configuration of a display device provided in some embodiments of the present application;

[0055] FIG3 is a schematic diagram of a hardware configuration of a control device provided in some embodiments of the present application;

[0056] FIG4 is a schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0057] FIG5 is a flowchart of the steps of a first interactive display device control method provided in some embodiments of the present application;

[0058] FIG6 is a flowchart of the steps of a second interactive display device control method provided in some embodiments of the present application;

[0059] FIG7 is a flowchart of the steps of a third interactive display device control method provided in some embodiments of the present application;

[0060] FIG8 is a flowchart of the steps of a fourth interactive display device control method provided in some embodiments of the present application;

[0061] Figure 9 is a timing diagram of an interactive display device control method provided in some embodiments of the present application. DETAILED DESCRIPTION

[0062] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0063] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0064] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0065] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0066] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0067] In the embodiment of the present application, the display device 200 generally refers to a device with image display and data processing capabilities. For example, the display device 200 includes but is not limited to a smart TV, a mobile terminal, a computer, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.

[0068] Figure 1 is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. As shown in Figure 1, a user can operate a display device 200 through touch operation, a mobile terminal 300, and a control device 100. For example, the control device 100 can be a remote controller, a stylus, a handle, etc.

[0069] The mobile terminal 300 can be used as a control device for performing human-computer interaction between the user and the display device 200. The mobile terminal 300 can also be used as a communication device for establishing a communication connection with the display device 200 and performing data interaction. In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0070] 1 , the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN) and other networks.

[0071] The display device 200 may provide a broadcast receiving television function, and may also additionally provide an intelligent network television function of a computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), etc.

[0072] FIG. 2 is a hardware configuration block diagram of the display device 200 in FIG. 1 provided in some embodiments of the present application.

[0073] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0074] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0075] In some embodiments, the display 260 includes a display function component for presenting a picture, and a driving component for driving an image display. The display 260 is used to receive an image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, and components of a menu control interface and a user control UI interface.

[0076] In some embodiments, the communication device 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 may be provided with a plurality of communication devices 220 according to different supported communication modes. For example, when the display device 200 supports wireless network communication, the display device 200 may be provided with a communication device 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function.

[0077] The communication device 220 can enable the display device 200 to communicate with the external device or server 400 by wireless or wired connection. Among them, the wired connection can connect the display device 200 with the external device through components such as data cables and interfaces. The wireless connection can connect the display device 200 with the external device through wireless signals or wireless networks. The display device 200 can establish a connection relationship with the external device directly, or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0078] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0079] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0080] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0081] In some embodiments, the audio output device 270 may be a local speaker of the display device 200, or may be an external audio output device of the display device 200. In particular, for the external audio output device of the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device may be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0082] In some embodiments, the user input interface 280 may be used to receive instructions from a user.

[0083] Fig. 3 is a hardware configuration block diagram of the control device in Fig. 1 provided in some embodiments of the present application. As shown in Fig. 3, the control device 100 may include: a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0084] The control device 100 is configured to control the display device 200 , and can receive user input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200 .

[0085] In some embodiments, the control device 100 may be a smart device, for example, the control device 100 may be installed with various applications for controlling the display device 200 according to user needs.

[0086] In some embodiments, as shown in FIG. 1 , the mobile terminal 300 or other intelligent electronic device can perform similar functions as the control device 100 after installing an application for operating the display device 200 .

[0087] The controller 110 includes a processor 112, a RAM 113, a ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between the internal components and the external and internal data processing functions.

[0088] The communication interface 130 implements communication of control signals and data signals with the display device 200 under the control of the controller 110. The communication interface 130 may include at least one of other near field communication modules such as a WiFi chip 131, a Bluetooth module 132, and an NFC module 133.

[0089] The user input / output interface 140 , wherein the input interface includes at least one of other input interfaces such as a microphone 141 , a touch panel 142 , a sensor 143 , and a button 144 .

[0090] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with a communication interface 130, such as a WiFi, Bluetooth, NFC or other module, and can encode the user input command through the WiFi protocol, Bluetooth protocol, or NFC protocol and send it to the display device 200.

[0091] The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0092] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0093] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources in the display device 200. The operating system may (control the display device) provide a user interface, allow the user to interact with the display device 200, and support the running of various application programs.

[0094] It should be noted that the operating system can be a native operating system based on a specific operating platform, a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device.

[0095] The operating system can be divided into different modules or levels according to the functions implemented. For example, as shown in Figure 4, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications layer) (referred to as "application layer"), the application framework layer (Application Framework layer) (referred to as "framework layer"), the system library layer and the kernel layer.

[0096] In some embodiments, the application layer is used to provide services and interfaces for applications so that the display device 200 can run applications and interact with users based on applications. At least one application can be run in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0097] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions of applications in the application layer. Applications can access system resources and obtain system services during execution through the API interface.

[0098] As shown in FIG4 , the application framework layer in the embodiment of the present application includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager is used to interact with all activities running in the system; a location manager is used to provide system services or applications with access to system location services; a package manager is used to retrieve various information related to the application package currently installed on the device; a notification manager is used to control the display and clearing of notification messages; and a window manager is used to manage icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0099] In some embodiments, the activity manager is used to manage the life cycle of each application and the usual navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes, for example, reducing the display window, shaking the display, distorting the display, etc.

[0100] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions to be implemented by the framework layer.

[0101] In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. For example, as shown in FIG4 , a hardware driver can be configured in the kernel layer, and the driver included in the kernel layer can be at least one of the following drivers: an audio driver, a display driver, a Bluetooth driver, a camera driver, a WIFI driver, a USB driver, an HDMI driver, a sensor driver (such as a fingerprint sensor, a temperature sensor, a pressure sensor, etc.), and a power driver, etc.

[0102] It should be noted that the above example is only a simple division of the operating system functions and does not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

[0103] The above embodiments illustrate the hardware / software architecture and functional implementation of a display device 200. In some embodiments, the display device 200 is configured with a user interaction application, and the display in the display device 200 can display the interactive interface of the user interaction application; the controller in the display device can receive the control signal of the control device 100, and control the display in the display device to display the user interaction application according to the control signal; the user can control the display device 200 through the control device 100 to perform user interaction operations in the user interaction application; the user's voice can also be received through the sound collector in the detector 230 to interact with the user to complete the user interaction operation in the user interaction application.

[0104] As the functions of display devices become increasingly rich, the diverse applications and constantly updated functions often bring operational problems to users, especially specific groups such as the elderly and children, and it is difficult to meet their needs for personalized guidance and instant help. The rich functions of display devices were originally intended to bring convenience to users, but for some users, the operation of display devices has become increasingly difficult, and with the explosive growth of content played by display devices, the artificial intelligence operation guidance in traditional technologies can no longer meet users' needs for personalized guidance.

[0105] Based on this, in some embodiments, the present application provides a display device, which includes a display and a controller; wherein the display is configured to display content from a broadcast system or network and / or a user interface; and at least one controller, which is connected to the display and is configured to: construct operation instructions for completing a corresponding operation process based on demand description information input by a user, and control the display device to perform autonomous operation based on the operation instructions; control the display to present the autonomous operation results, and receive the user's judgment result on the autonomous operation results; if the judgment result is unqualified, receive modification information input by the user; update the operation instructions according to the modification operation requirements corresponding to the modification information, and return to the step of controlling the display device to perform autonomous operation based on the operation instructions and continue to execute until it is determined that the user's qualified judgment result on the autonomous operation result is qualified.

[0106] Optionally, referring to FIG5 , the interactive display device control method includes the following steps:

[0107] S501, constructing an operation instruction for completing a corresponding operation process according to the demand description information input by the user, and controlling the display device to perform autonomous operation based on the operation instruction.

[0108] It should be noted that there are many ways for users to input demand description information. For example, the demand description information can be input through voice input, or through text input, or by displaying pre-set candidate description information to the user so that the user can select from the candidate description information, and the candidate description information selected by the user is used as the demand description information input by the user. The following will explain the above three ways of obtaining demand description information:

[0109] In one embodiment of the present application, the display device can use the voice information output by the user through the sound collector in the detector, and perform text content recognition on the voice information, and the recognized text content is the demand description information input by the user; further, in order to improve the accuracy of obtaining the demand description information input by the user and prevent the construction of subsequent operation instructions from being affected by invalid information, the display device can pre-set a wake-up word, and receive the voice information output by the user within a preset time period after the sound collector collects the wake-up word, and perform text content recognition on the voice information within the preset time period, and use the recognized text content as the demand description information input by the user.

[0110] In another embodiment of the present application, when the user needs to input demand description information, the user can start the user interaction application in the display through the control device, and trigger the input window in the application interface corresponding to the user interaction application. Text can be input in the input window through the control device, and the input text is the demand description information entered by the user.

[0111] In another embodiment of the present application, when the user needs to input requirement description information, the user can start a user interaction application on the display through a control device. At least one candidate description information can be displayed in the application interface of the user interaction application. The user can select candidate description information that meets the user's needs from the candidate description information through the control device. The candidate description information is the requirement description information.

[0112] It is further explained that after obtaining the requirement description information, the operation process required for the display device to implement the requirement description information can be determined, and then, the operation instructions can be constructed according to the operation process corresponding to the requirement description information; specifically, the operation process corresponding to the requirement description information can be at least one operation step arranged in sequence; therefore, when constructing the operation instructions, candidate instructions corresponding to different candidate steps can be pre-set, and after determining at least one operation step arranged in sequence, the candidate step that is the same as the operation step is determined from each candidate step according to the mapping relationship between each candidate step and each candidate instruction, and the candidate instruction corresponding to the candidate step is used as the operation instruction corresponding to each operation step, and the operation instructions corresponding to each operation step are arranged in sequence according to the operation order of each operation step to obtain the operation instructions that complete the corresponding operation process.

[0113] In one embodiment of the present application, in order to improve the operational efficiency of the display device in performing autonomous operations, when there are multiple operation processes that can enable the display device to implement the required description information, the number of operation steps included in each operation process can be counted separately, and then the operation process with the least number of operation steps can be used as the operation process actually required for the display device to implement the required description information.

[0114] In another embodiment of the present application, in order to make the display device comply with the user's historical operating habits when operating autonomously, when there are multiple operating processes that can enable the display device to implement the required description information, the degree of overlap between the operating steps included in each operating process and the historical steps included in the user's historical operation records of the display device can be verified separately, and the operating process with the highest overlap is used as the operating process actually required for the display device to implement the required description information.

[0115] It should be noted that in order to ensure that the controller can accurately obtain the actual needs of users, a semantic understanding system can be built. The semantic understanding system obtains the user's voice text and performs semantic analysis, business distribution, business parsing and text generation on the voice text to obtain the corresponding slot information, user intentions and business scenarios. Specifically, the pre-trained speech understanding model in the semantic understanding system can perform lexical, syntactic and semantic analysis on the input user voice text, understand the user's intentions, and send the business processing results to the display for display. Through deep learning models, the user's true intentions are understood, including context understanding, emotional tendencies, etc., to ensure that user needs are accurately grasped.

[0116] S502, controlling the display to present the autonomous operation result, and receiving the user's judgment result on the autonomous operation result.

[0117] It should be noted that in order to ensure that the user's judgment results on the autonomous operation results can be received smoothly, after the control display presents the autonomous operation results, the user can be asked about the operation results, so that the user can input the judgment results of the autonomous operation results to the display device based on the operation.

[0118] Among them, the judgment result is used to characterize whether the autonomous operation result meets the user's demand description information. Specifically, the judgment result may include two situations, namely: the autonomous operation result meets the user's demand description information and is qualified, and the autonomous operation result does not meet the user's demand description information and is unqualified.

[0119] In one embodiment of the present application, after the controller controls the display device to perform autonomous operation based on the operation instruction, the application interface of the user interactive application is displayed through the display, and the user can be asked about the operation result in text form in the application interface. The inquiry content can be: Please explain whether the autonomous operation result meets the requirement description information you input; (the inquiry content is not limited and can be adjusted according to actual conditions and actual needs), so that after seeing the inquiry content, the user can input the judgment result of the autonomous operation result to the display device, so that the controller completes the operation of receiving the user's judgment result of the autonomous operation result.

[0120] In another embodiment of the present application, after the controller controls the display device to perform autonomous operation based on the operation instruction, a voice inquiry is initiated to the user through the sound player of the display. The audio data played by the voice inquiry can be obtained by the controller after voice conversion of the inquiry content in text form, so that after hearing the voice inquiry, the user can input the judgment result of the autonomous operation result to the display device, so that the controller completes the operation of receiving the user's judgment result of the autonomous operation result.

[0121] S503: When the judgment result is unqualified, receiving modification information input by the user.

[0122] It should be noted that if the judgment result is unqualified, it means that the autonomous operation result of the controller controlling the display device to perform autonomous operation cannot meet the user's demand description information. At this time, the modification information entered by the user can be received, and the modification information includes the reason why the autonomous operation result cannot meet the user's demand description information.

[0123] To further explain, when the judgment result is unqualified, the controller can start the user interaction application and present the application interface of the user interaction application on the display (if the user interaction application is in the started state, the display priority of the application interface of the user interaction application on the display is modulated to the highest level), and open the modification information receiving window in the application interface; the user can complete the operation of receiving the modification information input by the user in the modification information receiving window by controlling the device.

[0124] In one embodiment of the present application, when it is necessary to receive modification information input by the user, an application interface of the user interaction application can be presented on the display, and a modification information receiving window can be opened in the application interface, so that the user can input text through the modification information receiving window in the display of the control device to input the modification information to the display device, so that the controller completes the step of receiving the modification information input by the user.

[0125] In another embodiment of the present application, when it is necessary to receive modification information input by the user, an application interface of the user interaction application can be presented on the display, and a modification information presentation window can be opened in the application interface, and the sound player of the display can be triggered to receive audio data of the modification information input by the user, and the text data obtained after the audio data is converted into text can be displayed in the modification information presentation window. After the user confirms that the text data presented in the modification information presentation window is correct, the user triggers the confirmation button in the application interface through the control device to input the modification information to the display device, so that the controller completes the step of receiving the modification information input by the user.

[0126] S504, updating the operation instruction according to the modification operation requirement corresponding to the modification information, and returning to the step of controlling the display device to perform autonomous operation based on the operation instruction and continuing to execute until it is determined that the user's qualification judgment result for the autonomous operation result is qualified.

[0127] It should be noted that when the operation instructions are updated according to the modification operation requirements corresponding to the modification information, the purpose is to modify some instructions in the operation instructions that do not conform to the requirement description information input by the user, or to add new instructions that conform to the requirement description information input by the user. Therefore, when the operation instructions are updated according to the modification operation requirements corresponding to the modification information, it is necessary to determine the abnormal instructions in the operation instructions that do not conform to the requirement description information input by the user, or the part of the requirement description information that is missing in the operation instructions; furthermore, the operation instructions can be updated according to the modification operation requirements corresponding to the modification information by modifying the abnormal instructions, or completing the operation instructions of the missing part of the requirement description information.

[0128] As an example, if the user is an elderly person with poor eyesight and is not good at using a remote control (a control device for a display device), when the user wants to watch the live broadcast of TV station A, he or she can input demand description information to the display device in the form of voice input. The demand description information is used to characterize that the user wants to watch the live broadcast of TV station A. Then, the controller constructs operation instructions for completing the corresponding operation process, and controls the display device to perform autonomous operation based on the operation instructions. If the user's judgment on the result of the autonomous operation is qualified, the operation is stopped.

[0129] As another example, if the user is a younger user and wants to watch movie B, the controller constructs operation instructions for completing the corresponding operation process, and controls the display device to perform autonomous operation based on the operation instructions. However, movie B can only be watched in a certain video playback application, but the video playback application is not installed in the display device, then the actual situation of movie B is displayed on the display as the autonomous operation result. For example, the autonomous operation result may be: "Movie B is available on XX application, but your display device has not installed this application. I will download and install it for you now. Is it okay for you to watch it?" If the user's judgment result on the autonomous operation result is unqualified at this time, and the modification information input by the user is "Allow download", the controller updates the operation instructions according to the modification operation requirements corresponding to the modification information, so that the display device downloads and installs the corresponding video playback application according to the new operation instructions, and plays movie B through the video playback application.

[0130] As another example, if the user is a younger user and wants to watch C, the controller constructs operation instructions for completing the corresponding operation process, and controls the display device to perform autonomous operation based on the operation instructions. The autonomous operation result is that the display device plays TV series C on the display. If the user's judgment result on the autonomous operation result is unqualified at this time, and the modification information input by the user is "I want to watch movie C, not TV series C", the controller updates the operation instructions according to the modification operation requirements corresponding to the modification information, so that the display device executes the steps of playing movie C according to the new operation instructions.

[0131] The above-mentioned interactive display device control method constructs operation instructions for completing the corresponding operation process according to the demand description information input by the user, and controls the display device to perform autonomous operation based on the operation instructions, receives the user's judgment result on the autonomous operation result, and when the judgment result is unqualified, updates the operation instructions according to the modification information input by the user, and returns to the step of controlling the display device to perform autonomous operation based on the operation instructions and continues to execute until it is determined that the user's qualified judgment result on the autonomous operation result is qualified. According to the above content, it can be seen that in the process of responding to the demand description information input by the user, the present application will not only construct operation instructions and control the display device to perform autonomous operation based on the operation instructions, but also receive the modification information input by the user when the user judges that the autonomous operation result is unqualified, and update the operation instructions based on the modification operation requirements corresponding to the modification information, so as to ensure that the autonomous operation result with the judgment result as qualified can be obtained according to the updated operation instructions; therefore, in the process of controlling the display device to perform autonomous operation, the present application does not only perform autonomous operation based on the demand description information input by the user for the first time, but also updates the operation instructions based on the modification information fed back by the user for the second time, so that the updated operation instructions are more in line with the actual needs of the user, thereby ensuring that the autonomous operation of the display device can meet the personalized guidance needs of the user, so that the user can quickly master the relevant functional processes of the display device according to the above process, thereby improving the convenience and flexibility of the user in using the display, and reducing the difficulty of operating the display device.

[0132] In one embodiment, as shown in FIG6 , when the controller executes updating of the operation instruction according to the modification operation requirement corresponding to the modification information, the following contents may be specifically included:

[0133] S601, determining, according to the modification information, the operation steps to be adjusted in the operation flow, and the reference operation steps after the operation steps to be adjusted are modified according to the modification description requirements.

[0134] It should be noted that the operation steps to be adjusted refer to the abnormal steps in the operation process that do not conform to the requirement description information, or the new steps that need to be added in the operation process to conform to the requirement description information; therefore, the reference operation steps refer to the correct steps that conform to the requirement description information after the step content of the abnormal steps is modified, or the new steps that need to be added in the operation process to conform to the requirement description information.

[0135] Since the modification information can be used to characterize abnormal steps in the operation process that do not conform to the requirement description information, as well as new steps that need to be added in the operation process that conform to the requirement description information, after receiving the modification information, based on the content recorded in the modification information, the operation steps to be adjusted are selected from at least one step in the operation process and based on actual problems of the operation steps to be adjusted, and a reference operation step after the operation step to be adjusted is obtained after the modification description requirement is met.

[0136] For example, the operation process is: open the video playback application - play TV series C, but the modification information shows that what is played is movie C, not TV series C. According to the modification information, it is determined that the "play TV series C" step does not match the requirement description information, and the "play TV series C" step is modified to obtain the modified "play movie C" step. "Play movie C" is the reference operation step after the operation step to be adjusted is modified according to the modification description requirement.

[0137] S602: Update the instruction to be adjusted corresponding to the operation step to be adjusted in the operation instruction based on the reference operation step.

[0138] It should be noted that after determining the reference operation steps, a reference instruction corresponding to the reference operation steps can be constructed, and the instruction to be adjusted corresponding to the operation step to be adjusted during the operation execution can be located, and then, based on the reference instruction and the instruction to be adjusted, the operation instruction update operation can be implemented.

[0139] Specifically, the instruction to be adjusted corresponding to the operation step to be adjusted in the operation instruction is replaced with the reference instruction corresponding to the reference operation step.

[0140] To summarize the above, this application determines the operation steps to be adjusted in the operation process, and the reference operation steps after the operation steps to be adjusted are modified according to the modification description requirements, so as to replace the instructions to be adjusted corresponding to the operation steps to be adjusted in the operation instructions with the reference instructions corresponding to the reference operation steps, thereby completing the update of the operation instructions, ensuring that the updated operation instructions are more in line with the actual needs of users and optimizing the user experience of the display device.

[0141] In one embodiment, as shown in FIG. 7 , when the controller executes the operation instructions for completing the corresponding operation flow according to the requirement description information input by the user, the following contents may also be included:

[0142] S701, determining the device operation requirement of the user for the display device according to the requirement description information input by the user.

[0143] It should be noted that in order to ensure the simplicity and accuracy of the subsequent construction of operation instructions, after receiving the requirement description information entered by the user, the operation keywords contained in the requirement description information can be extracted to determine the user's device operation requirements for the display device based on the operation keywords.

[0144] Among them, the device operation requirements include at least one of remote control operation requirements, subtitle reading requirements, network connection requirements, application download requirements, display device setting requirements and error operation recovery requirements.

[0145] In one embodiment of the present application, a keyword range corresponding to each device operation requirement can be pre-constructed, and then, after receiving the requirement description information input by the user, the operation keywords contained in the requirement description information are extracted, and the keyword range to which each operation keyword belongs is determined. Then, based on the device operation requirements corresponding to each keyword range, the device operation requirements corresponding to the requirement description information are determined, that is, the user's device operation requirements for the display device are determined.

[0146] In another embodiment of the present application, an operation requirement judgment model can be pre-trained. After receiving the requirement description information input by the user and extracting the operation keywords contained in the requirement description information, the extracted operation keywords are input into the pre-trained operation requirement judgment model, and the output result of the operation requirement judgment model is obtained. The output result is the user's device operation requirement for the display device.

[0147] Among them, the training process of the operation requirement judgment model includes obtaining at least one sample keyword and labeling each sample keyword with a sample operation requirement, and then training the initial judgment model based on the sample keywords labeled with sample operation requirements to obtain a trained operation requirement judgment model.

[0148] S702: constructing operation instructions for completing corresponding operation procedures according to the equipment operation requirements.

[0149] It should be noted that when constructing operation instructions for completing the corresponding operation process, the following contents may be specifically included: according to the device operation requirements, the target operation steps are selected from the candidate operation steps contained in the display device; according to the execution order corresponding to the target operation steps, the target operation steps are combined to obtain the target operation process corresponding to the user; and the operation instructions are constructed according to the target operation process.

[0150] Further explanation: when the controller selects the target operation step from the candidate operation steps included in the display device according to the device operation requirements, the following contents may be included: obtaining the user's historical operation records for the display device; selecting the preferred operation step from the candidate operation steps included in the display device; wherein the preferred operation step is an operation step that is triggered more than a threshold number of times in the user's historical operation records for the display device; selecting an intermediate operation step from the candidate operation steps included in the display device according to the device operation requirements; verifying whether the intermediate operation step contains replaceable operation steps with the same step effect as the preferred operation step; if so, replacing the replaceable operation steps in the intermediate operation steps with the corresponding preferred operation steps to obtain the target operation step.

[0151] The number threshold can be set or adjusted according to actual conditions, and the value range of the number threshold is not limited here.

[0152] In summary, the present application determines the user's device operation requirements for the display device, and then constructs operation instructions for completing the corresponding operation process based on the device operation requirements, thereby ensuring the smooth determination of the operation instructions, so that the display device can complete the corresponding operation according to the operation instructions and the execution order corresponding to each target operation step, thereby ensuring that the user's needs are met and that the user can become familiar with the operation process of the display device based on the operation process of the display device.

[0153] In one embodiment, when the controller determines the device operation requirement of the user for the display device according to the requirement description information input by the user, the following contents may also be included:

[0154] Specifically, based on the artificial intelligence model, the device operation requirements corresponding to the requirement description information input by the user are obtained.

[0155] In one embodiment of the present application, the artificial intelligence model may be a corresponding intelligent model in an LLM-based Agent (agent system based on a large language model) system; wherein the LLM-based Agent system is a system that uses artificial intelligence technology to perform specific tasks or achieve specific goals, and the system can perceive the environment, make decisions and take actions. It outputs an executable code from the LLM brain, like a semantic analyzer. The system can understand the meaning of the demand description information input by the user, and then construct an operation instruction for completing the corresponding operation process, and combine with the controller to realize the process of controlling the display device to perform autonomous operation based on the operation instruction. Its characteristics are: Autonomy: without direct human intervention, it can carry out a certain degree of autonomous exploration and decision-making, and has a certain degree of control over its actions and internal states; Reactivity: The system can perceive changes in the surrounding environment and take appropriate actions quickly; Pro-activeness: The system can actively take goal-oriented actions, reason, make plans and take proactive measures in the action to achieve specific goals or adapt to environmental changes; Social Ability: The system has the ability to interact with other agents (including humans) through natural language.

[0156] Furthermore, the LLM-based Agent system may include but is not limited to: Perceptors: Perceptors are used to receive environmental information, which may be in the form of images, sounds, text, etc. Processors: Processors are used to process the received information, which usually involves some complex algorithms, such as machine learning, deep learning, natural language processing, etc. Actuators: Actuators are used to influence the environment to achieve their goals. Actuators may be robotic arms, speech synthesizers, text generators, etc. Learning Modules: Learning modules can continuously learn and optimize system behavior, which usually involves reinforcement learning, supervised learning and other technologies.

[0157] In one embodiment of the present application, the artificial intelligence model can be further subdivided into an artificial intelligence extraction model and an artificial intelligence analysis model. Therefore, based on the artificial intelligence model, the process of obtaining the device operation requirements corresponding to the demand description information input by the user can also include the following contents: based on the artificial intelligence extraction model, key information is extracted from the demand description information input by the user to obtain the target description information; based on the artificial intelligence analysis model, information semantic analysis is performed on the target description information to obtain the device operation requirements.

[0158] The target description information may be an operation keyword; when the target description information is subjected to information semantic analysis, the device operation requirements to which the operation keyword belongs are analyzed.

[0159] To summarize the above, this application is based on an artificial intelligence model to obtain the device operation requirements corresponding to the demand description information input by the user, realizes efficient analysis of device operation requirements, and ensures the subsequent smooth construction of operation instructions for completing the corresponding operation process based on the device operation requirements.

[0160] In one embodiment, as shown in FIG8 , when the controller needs to control the display device to perform autonomous operations that meet the requirement description information, the following may be included:

[0161] S801, based on the artificial intelligence model, obtaining the device operation requirements corresponding to the requirement description information input by the user.

[0162] S802: Select a target operation step from candidate operation steps included in the display device according to the device operation requirement.

[0163] S803, combining the target operation steps according to the execution order corresponding to the target operation steps to obtain the target operation flow corresponding to the user.

[0164] S804, constructing an operation instruction according to the target operation process.

[0165] S805, controlling the display device to perform autonomous operation based on the operation instruction.

[0166] S806, controlling the display to present the autonomous operation result, and receiving the user's judgment result on the autonomous operation result.

[0167] S807: When the judgment result is unqualified, receiving modification information input by the user.

[0168] S808, determining the operation steps to be adjusted in the operation flow according to the modification information, and the reference operation steps after the operation steps to be adjusted are modified according to the modification description requirements.

[0169] S809, replacing the instruction to be adjusted corresponding to the operation step to be adjusted in the operation instruction with the reference instruction corresponding to the reference operation step.

[0170] S810, returning to the step of controlling the display device to perform autonomous operation based on the operation instruction and continuing to execute until it is determined that the user's qualification judgment result for the autonomous operation result is qualified.

[0171] In one embodiment of the present application, the present application may include an LLM-based Agent module, a user interface module, a television content management module and a data storage and analysis module; the LLM-based Agent module also includes: an instruction chain construction and execution unit, a context understanding and dynamic adjustment unit, an error handling and user guidance unit and a personalized experience optimization unit; wherein the LLM-based Agent module can realize the construction of operation instructions for completing the corresponding operation process according to the demand description information input by the user; specifically, the instruction chain construction and execution module can select the target operation steps from the candidate operation steps included in the display device according to the device operation requirements, and automatically generate and execute the intermediate instructions of the target operation steps; the context understanding and dynamic adjustment unit can realize the update of the operation instructions according to the modification operation requirements corresponding to the modification information, and the error handling and user guidance unit can try to understand the user's intention in the face of unclear or directly executable instructions, and output the target operation steps; the personalized experience optimization unit is used to select the target operation steps according to the preferred operation steps in the user's historical operation records for the display device. The user interface module is used to control the display to present autonomous operation results; the TV content management module stores, manages and schedules the content of complex display devices; the data storage and analysis module is responsible for storing user data and performing in-depth analysis to optimize the performance of the above modules and user experience.

[0172] To summarize the above, based on the demand description information input by the user, operation instructions for completing the corresponding operation process are constructed, and the display device is controlled to perform autonomous operation based on the operation instructions, and the user's judgment result on the autonomous operation result is received. If the judgment result is unqualified, the operation instruction is updated according to the modification information input by the user, and the step of controlling the display device to perform autonomous operation based on the operation instruction is returned and continued to be executed until it is determined that the user's judgment result on the autonomous operation result is qualified. According to the above content, it can be seen that in the process of responding to the demand description information input by the user, the present application will not only construct operation instructions and control the display device to perform autonomous operation based on the operation instructions, but also receive the modification information input by the user when the user judges that the autonomous operation result is unqualified, and update the operation instructions based on the modification operation requirements corresponding to the modification information, so as to ensure that the autonomous operation result with the judgment result as qualified can be obtained according to the updated operation instructions; therefore, in the process of controlling the display device to perform autonomous operation, the present application does not only perform autonomous operation based on the demand description information input by the user for the first time, but also updates the operation instructions based on the modification information fed back by the user for the second time, so that the updated operation instructions are more in line with the actual needs of the user, thereby ensuring that the autonomous operation of the display device can meet the personalized guidance needs of the user, so that the user can quickly master the relevant functional processes of the display device according to the above process, thereby improving the convenience and flexibility of the user in using the display, and reducing the difficulty of operating the display device.

[0173] In some exemplary embodiments, referring to FIG. 9 , FIG. 9 provides a timing diagram of a display device control method based on interaction, and the specific implementation process is as follows:

[0174] 1. The user controls the device to trigger the user interaction application, or triggers the command to open the user interaction application by voice inputting "open content user interaction application".

[0175] 2. The user interaction application responds to the opening content generation instruction and synchronizes the instruction to enter the user interaction application to the controller.

[0176] 3. Based on the instruction to enter the user interaction application, the controller controls the display to display the application interface of the user interaction application.

[0177] 4. The display presents the application page of the user interactive application.

[0178] 5. The user interaction application responds to the input requirement description information operation and synchronizes the user input requirement description information to the controller.

[0179] 6. The controller constructs operation instructions for completing the corresponding operation process according to the demand description information input by the user, and controls the display device to perform autonomous operation based on the operation instructions.

[0180] 7. The display shows the results of autonomous operation.

[0181] 8. The user interaction application responds to the modification information input by the user when the judgment result is unqualified; and synchronizes the modification information to the controller.

[0182] 9. The controller updates the operation instruction according to the modification operation requirement corresponding to the modification information, and returns to the step of controlling the display device to perform autonomous operation based on the operation instruction and continues to execute until it is determined that the user's qualified judgment result for the autonomous operation result is qualified.

[0183] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0184] Based on the same inventive concept, the embodiment of the present application also provides an interactive display device control device for implementing the interactive display device control method mentioned above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method. The specific limitations can be found in the interactive display device control process above, which will not be repeated here.

[0185] In one embodiment, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned interactive display device control method are implemented.

[0186] In one embodiment, the present application further provides a computer program product, including a computer program, which implements the steps of the above-mentioned interaction-based display device control method when executed by a processor.

[0187] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0188] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0189] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A display device, characterized in that: include: a display configured to display content from a broadcast system or network and / or a user interface; and at least one controller connected to the display and configured to: Constructing operation instructions for completing the corresponding operation process according to the demand description information input by the user, and controlling the display device to perform autonomous operation based on the operation instructions; Controlling the display to present the autonomous operation result, and receiving the user's judgment result on the autonomous operation result; In the case where the judgment result is unqualified, receiving modification information input by the user; The operation instruction is updated according to the modification operation requirement corresponding to the modification information, and the step of controlling the display device to perform autonomous operation based on the operation instruction is returned and continued to be executed until it is determined that the user's qualified judgment result for the autonomous operation result is qualified.

2. The display device according to claim 1, characterized in that When the controller executes updating of the operation instruction according to the modification operation requirement corresponding to the modification information, the controller is further configured to: Determine, according to the modification information, the operation steps to be adjusted in the operation process, and the reference operation steps after the operation steps to be adjusted are modified according to the modification description requirements; The instruction to be adjusted corresponding to the operation step to be adjusted in the operation instruction is updated based on the reference operation step.

3. The display device according to claim 2, characterized in that When the controller executes updating of the instruction to be adjusted corresponding to the operation step to be adjusted in the operation instruction based on the reference operation step, the controller is further configured to: The instruction to be adjusted corresponding to the operation step to be adjusted in the operation instruction is replaced with the reference instruction corresponding to the reference operation step.

4. The display device according to claim 1, characterized in that When the controller executes the demand description information input by the user to construct an operation instruction for completing the corresponding operation process, it is also configured to: Determine the user's device operation requirements for the display device according to the requirement description information input by the user; According to the equipment operation requirements, construct operation instructions for completing the corresponding operation process.

5. The display device according to claim 4, characterized in that When the controller executes the operation instruction constructed according to the device operation requirements to complete the corresponding operation process, it is also configured to: According to the device operation requirement, selecting a target operation step from the candidate operation steps included in the display device; Combining the target operation steps according to the execution order corresponding to the target operation steps to obtain the target operation process corresponding to the user; An operation instruction is constructed according to the target operation process.

6. The display device according to claim 5, characterized in that When the controller selects a target operation step from the candidate operation steps included in the display device according to the device operation requirement, the controller is further configured to: Obtaining historical operation records of the user on the display device; Selecting a preferred operation step from the candidate operation steps included in the display device; wherein the preferred operation step is an operation step whose triggering times are greater than a threshold in the historical operation records of the user on the display device; According to the device operation requirement, selecting an intermediate operation step from the candidate operation steps included in the display device; Verifying whether the intermediate operation steps include replaceable operation steps with the same effect as the preferred operation steps; If so, the replaceable operation steps in the intermediate operation steps are replaced with the corresponding preferred operation steps to obtain the target operation steps.

7. The display device according to claim 4, characterized in that When the controller determines the device operation requirement of the user for the display device according to the requirement description information input by the user, the controller is further configured to: Based on the artificial intelligence model, the device operation requirements corresponding to the requirement description information input by the user are obtained.

8. The display device according to claim 7, characterized in that The artificial intelligence model includes an artificial intelligence extraction model and an artificial intelligence analysis model; When the controller obtains the device operation requirement corresponding to the requirement description information input by the user based on the artificial intelligence model, it is also configured to: Based on the artificial intelligence extraction model, key information is extracted from the demand description information input by the user to obtain target description information; Based on the artificial intelligence analysis model, information semantic analysis is performed on the target description information to obtain equipment operation requirements.

9. The display device according to claim 4, characterized in that: The device operation requirement includes at least one of a remote control operation requirement, a subtitle reading requirement, a network connection requirement, an application download requirement, a display device setting requirement, and an error operation recovery requirement.

10. A display device control method based on interaction, characterized in that: The method is applied to a controller in a display device, the controller is used to receive a control signal of a control device, and control a display in the display device to display according to the control signal, and the method includes: Constructing operation instructions for completing the corresponding operation process according to the demand description information input by the user, and controlling the display device to perform autonomous operation based on the operation instructions; Controlling the display to present the autonomous operation result, and receiving the user's judgment result on the autonomous operation result; In the case where the judgment result is unqualified, receiving modification information input by the user; The operation instruction is updated according to the modification operation requirement corresponding to the modification information, and the step of controlling the display device to perform autonomous operation based on the operation instruction is returned and continued to be executed until it is determined that the user's qualified judgment result for the autonomous operation result is qualified.

Images