Operation method and device of display equipment and related equipment

Abstract

The invention provides a display equipment operation method and device and related equipment.According to the technical scheme, in the process that a user uses head-mounted display equipment, an operation auxiliary instruction input by the user on the display equipment is obtained, so that data information of a currently-running application program is obtained; the data information is analyzed by means of the large language model, the operation prompt information is generated, the current operation method is prompted to the user from the application program, and compared with the technical scheme that only the application program situation can be broadcasted for the user through voice in the prior art, the method can better help the user to use the application program of the display device, and the user experience is improved. And the experience effect of using the display equipment by the user is improved.

Description

Technical Field

[0001] The present invention relates to the field of communication technologies, and in particular, to an operation method, device, and related equipment for a display device. Background Art

[0002] At the present stage, with the development of technologies, when experiencing using Virtual Reality (VR) devices, the eyes are prone to visual fatigue, and it is just that the eyes are very uncomfortable. However, the body is still very energetic, and it is also hoped to continue experiencing the game with eyes closed.

[0003] In the prior art, when the user closes their eyes while using a VR device, only the current situation of the application program can be voice-reported to the user, and the situation where the user can continue to use the VR device cannot be achieved, resulting in a problem of poor experience effect when the user uses the VR device. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide an operation method, device, and related equipment for a display device, which can solve the problem of poor experience effect when the user uses a VR device in the prior art.

[0005] To solve the above technical problem, the embodiments of the present invention provide the following technical solutions:

[0006] In a first aspect, an embodiment of the present invention provides an operation method for a display device, which is applied to the display device. The method includes:

[0007] Obtain an operation assistance instruction input by a user on the display device, where the operation assistance instruction is used to request to display an operation prompt on the display device;

[0008] Based on the operation assistance instruction, obtain data information of the application program currently running on the display device, where the data information is used to indicate the usage situation of the user in the application program;

[0009] Input the data information into a large language model for analysis to obtain an operation prompt information, where the operation prompt information is used to prompt the user about the operation method in the application program;

[0010] Push the operation prompt information to the user in the application program.

[0011] Optionally, the obtaining the data information of the application program currently running on the display device based on the operation assistance instruction includes:

[0012] Obtain the resolution information corresponding to the screen of the display device;

[0013] Establish a coordinate system for the screen of the display device according to the resolution information to obtain a target coordinate system, where the horizontal axis of the target coordinate system corresponds to the screen length of the display device, and the vertical axis of the target coordinate system corresponds to the screen width of the display device;

[0014] Determine the data information of the application currently running on the display device according to the target coordinate system.

[0015] Optionally, the inputting the data information into a large language model for analysis to obtain operation prompt information includes:

[0016] Determine first coordinate information and second coordinate information according to the data information, where the first coordinate information is the coordinate of the game character corresponding to the user in the target coordinate system, and the second coordinate information is the coordinate of the game character corresponding to other users in the target coordinate system;

[0017] Input the first coordinate information and the second coordinate information into the large language model for analysis to obtain the operation prompt information.

[0018] Optionally, when the distance between the first coordinate information and the second coordinate information is less than or equal to a preset distance, the operation prompt information is used to prompt the user to perform a target operation in the application, and the target operation is used to control the interaction between the user's corresponding character and the characters corresponding to other users;

[0019] When the distance between the first coordinate information and the second coordinate information is greater than the preset distance, the operation prompt information is used to prompt the user to maintain the current state in the application.

[0020] Optionally, the establishing a coordinate system for the screen of the display device according to the resolution information to obtain a target coordinate system includes:

[0021] Obtain a preset calculation unit corresponding to the display device, and the preset calculation unit is used to determine the proportional relationship between the target coordinate system and the resolution information;

[0022] Establish a coordinate system for the screen of the display device according to the preset calculation unit and the resolution information to obtain the target coordinate system;

[0023] Or

[0024] Receive a target calculation unit input by the user, and the target calculation unit is used to determine the proportional relationship between the target coordinate system and the resolution information;

[0025] Establish a coordinate system for the screen of the display device according to the target calculation unit and the resolution information to obtain the target coordinate system.

[0026] Optionally, before inputting the data information into the large language model for analysis to obtain operation prompt information, it includes:

[0027] Obtain the fixation state of the user's two eyes on the display device;

[0028] In the case where the fixation state indicates that the user's two eyes do not fixate on the display device within a target time period, push model prompt information to the user, and the model prompt information is used to prompt the user to start the large language model;

[0029] Based on the model prompt information, obtain a model start instruction input by the user on the display device;

[0030] Run the large language model according to the model start instruction.

[0031] Optionally, after running the large language model according to the model start instruction, the method further includes:

[0032] Obtain a model stop instruction input by the user on the display device;

[0033] Stop running the large language model according to the model stop instruction.

[0034] Optionally, the operation assistance instruction includes at least one of the following: gesture action instruction, voice instruction, and shortcut key press instruction.

[0035] In a second aspect, an operation device of a display device provided by an embodiment of the present invention is characterized in that it is applied to a display device, and the device includes:

[0036] A first acquisition module, configured to acquire an operation assistance instruction input by a user on the display device, where the operation assistance instruction is used to request to display an operation prompt on the display device;

[0037] A second acquisition module, configured to acquire data information of an application program currently running on the display device based on the operation assistance instruction, where the data information is used to indicate the usage situation of the user in the application program;

[0038] An analysis module, configured to input the data information into a large language model for analysis to obtain operation prompt information, where the operation prompt information is used to prompt the user about the operation method in the application program;

[0039] A push module, configured to push the operation prompt information to the user in the application program.

[0040] In a third aspect, the present invention further provides an electronic device, including a processor, a memory, and a computer program stored on the memory and executable on the processor. When the computer program is executed by the processor, the steps in the method described in the first aspect above are implemented.

[0041] In a fourth aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the method described in the first aspect above are implemented.

[0042] In a fifth aspect, the present invention further provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the steps in the method described in the first aspect above are implemented.

[0043] The present invention provides a method, device, and related equipment for operating a display device, which is applied to the display device. The method includes: obtaining an operation assistance instruction input by a user on the display device, where the operation assistance instruction is used to request to display an operation prompt on the display device; based on the operation assistance instruction, obtaining data information of the application program currently running on the display device, where the data information is used to indicate the usage situation of the user in the application program; inputting the data information into a large language model for analysis to obtain an operation prompt information, where the operation prompt information is used to prompt the user about the operation method in the application program; and pushing the operation prompt information to the user in the application program.

[0044] The embodiments of the present invention have the following beneficial effects:

[0045] In the technical solution of the present invention, during the process of the user using the head-mounted display device, by obtaining the operation assistance instruction input by the user on the display device, the data information of the currently running application program is obtained, and the large language model is relied on to analyze the data information to generate the operation prompt information, so as to prompt the user about the current operation method in the application program. Compared with the prior art solution that can only perform voice broadcast of the application program situation for the user, the present application can better help the user use the application program of the display device and improve the user experience effect of using the display device. Description of the Drawings

[0046] Figure 1 It is a schematic flowchart of the operation method of the display device in the embodiment of the present invention;

[0047] Figure 2 It is a schematic diagram of the coordinate system of the display device in the embodiment of the present invention;

[0048] Figure 3 It is one of the schematic flowcharts in the embodiment of the present invention;

[0049] Figure 4 This is the second process schematic diagram in the embodiments of the present invention;

[0050] Figure 5 This is the structural schematic diagram of the operating device of the display device in the embodiments of the present invention;

[0051] Figure 6 This is the structural schematic diagram of the electronic device in the embodiments of the present invention. Detailed implementation manners

[0052] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0053] The terms "first", "second", etc. in the embodiments of the present invention are used to distinguish similar objects and do not necessarily need to describe a specific order or sequence. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those clearly listed steps or units, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices. In addition, in this application, the use of "and / or" means at least one of the connected objects. For example, A and / or B and / or C means including seven cases of A alone, B alone, C alone, A and B existing, B and C existing, A and C existing, and A, B, and C existing.

[0054] The embodiments of the present invention address the problem of poor user experience when using VR devices in the prior art and provide an operating method for a display device. As Figure 1 shown, the steps of this method include:

[0055] Step 101, obtain an operation assistance instruction input by the user on the display device, where the operation assistance instruction is used to request to display an operation prompt on the display device.

[0056] In this embodiment, the display device is generally a head-mounted display device, such as a VR device. Generally, a VR device consists of a head-mounted display, sensors, controllers, computing devices, and audio devices, and is a device that creates a virtual environment through computer technology and enables users to immerse themselves in this environment.

[0057] When a user uses a VR device, the user can input an operation assistance instruction on the VR device to request assistance in using an application. Optionally, the operation assistance instruction includes at least one of the following: a gesture instruction, a voice instruction, and a shortcut key instruction. Specifically, the gesture instruction includes arm sliding, finger sliding, and so on. The voice instruction is that the user directly inputs voice content in the display device, such as "Enable operation assistance". The shortcut key instruction is the content directly input by the user by pressing a key on the display device. For example, the user presses a preset shortcut key to trigger the operation assistance instruction. Exemplarily, a camera sensor can identify the operation assistance instruction.

[0058] It should be noted that the operation assistance instruction is used to display an operation prompt on the display device, where the operation prompt is used to help the user better use the application. For example, it prompts the user about the operation method on the current page in the application and so on.

[0059] Step 102: Based on the operation assistance instruction, obtain data information of the application currently running on the display device, where the data information is used to indicate the user's usage situation in the application.

[0060] In this embodiment, according to the operation assistance instruction, data information of the currently running application is obtained in the display device, where the data information includes the user's current state in the application. For example, when the application is a game program, the data information may include the current time point of the game, the location of the user's character, the locations of other user characters, and so on. Thus, an operation prompt for the user's current state can be generated based on the data information.

[0061] Step 103: Input the data information into a large language model for analysis to obtain operation prompt information, where the operation prompt information is used to prompt the user about the operation method in the application.

[0062] In this embodiment, the large language model (LLM) is a natural language processing model based on deep learning technology, aiming to understand and generate natural language text. Such models usually have a very large parameter scale and can learn and capture complex patterns and structures in human language.

[0063] It should be noted that in this embodiment, the large language model used is a trained large language model. After inputting the data information, the large language model can analyze the data information and generate operation prompt information, where the operation prompt information is used to prompt the user about the operation method in the application. Exemplarily, if the user is currently playing a game, the operation prompt information can prompt the user about how to operate currently. For example, the user currently needs to move or attack, and so on.

[0064] Step 104: Push the operation prompt information to the user in the application program.

[0065] In this embodiment, after generating the operation prompt information, the operation prompt information is pushed to the user. The pushing methods may include: screen vibration, voice broadcast, handle vibration, etc., to prompt the user to operate according to the operation prompt information. Compared with the prior art, the present application realizes various interaction forms between the display device and the user, improving the user experience.

[0066] In the process of the user using the head-mounted display device, the technical solution of the present invention obtains the data information of the currently running application program by obtaining the operation assistance instruction input by the user on the display device, analyzes the data information by relying on the large language model, generates the operation prompt information, and prompts the user about the current operation method in the application program. Compared with the prior art solution that can only perform voice broadcast of the application program situation for the user, the present application can better help the user use the application program of the display device, improving the experience effect of the user using the display device.

[0067] In other feasible embodiments, optionally, obtaining the data information of the currently running application program of the display device based on the operation assistance instruction includes:

[0068] Obtain the resolution information corresponding to the screen of the display device;

[0069] Establish a coordinate system for the screen of the display device according to the resolution information to obtain a target coordinate system, where the horizontal axis of the target coordinate system corresponds to the screen length of the display device, and the vertical axis of the target coordinate system corresponds to the screen width of the display device;

[0070] Determine the data information of the currently running application program of the display device according to the target coordinate system.

[0071] In this embodiment, it is necessary to obtain the resolution information corresponding to the screen of the display device. Specifically, when the resolution of the display device is 1920x1080, the display device includes approximately 2.07 million pixels; when the resolution of the display device is 3840x2160, the display device includes approximately 8.29 million pixels; when the resolution of the display device is 7680x4320, the display device includes approximately 33.18 million pixels.

[0072] As Figure 2 shown, Figure 2A coordinate schematic diagram of a display device with a resolution of 1920x1080. A coordinate system is established for the screen of the display device according to the resolution information to obtain a target coordinate system. Among them, the horizontal axis of the target coordinate system corresponds to the screen length of the display device, and the vertical axis of the target coordinate system corresponds to the screen width of the display device.

[0073] When the application is a game, as Figure 2 shown, the roles of the user, another user A, and another user B can all be represented in coordinate form, thereby generating data information corresponding to the roles of the user, another user A, and another user B.

[0074] By establishing a coordinate system for the screen, the specific position of the user in the application can be better determined, thereby achieving better positioning and the generation of data information.

[0075] Optionally, inputting the data information into a large language model for analysis to obtain operation prompt information includes:

[0076] Determining first coordinate information and second coordinate information according to the data information. The first coordinate information is the coordinate of the game role corresponding to the user in the target coordinate system, and the second coordinate information is the coordinate of the game role corresponding to another user in the target coordinate system;

[0077] Inputting the first coordinate information and the second coordinate information into the large language model for analysis to obtain the operation prompt information.

[0078] In this embodiment, the coordinate information corresponding to each user is determined according to the determined data information. Specifically, the first coordinate information is the coordinate of the game role corresponding to the user in the target coordinate system, and the second coordinate information is the coordinate of the game role corresponding to another user in the target coordinate system.

[0079] According to the determined first coordinate information and second coordinate information, rely on the large language model for analysis to judge whether the distance between the first coordinate information and the second coordinate information exceeds a preset distance, and thus generate different operation prompt information according to different situations to achieve operation prompts for different application program scenarios.

[0080] Optionally, when the distance between the first coordinate information and the second coordinate information is less than or equal to the preset distance, the operation prompt information is used to prompt the user to perform a target operation in the application program, and the target operation is used to control the interaction between the role corresponding to the user and the role corresponding to another user;

[0081] When the distance between the first coordinate information and the second coordinate information is greater than a preset distance, the operation prompt information is used to prompt the user to maintain the current state in the application.

[0082] In this embodiment, specifically, when the distance between the first coordinate information and the second coordinate information is less than or equal to the preset distance, that is, the distance between the game character corresponding to the user and the game character corresponding to other users is relatively close. At this time, the target operation can be an interactive operation, such as controlling the game character corresponding to the user to attack the game character corresponding to other users, or controlling the game character corresponding to the user to talk to the game character corresponding to other users, etc.

[0083] When the distance between the first coordinate information and the second coordinate information is greater than the preset distance, that is, the distance between the game character corresponding to the user and the game character corresponding to other users is relatively far. At this time, the user can maintain the current state without performing other operations.

[0084] See Figure 3 , Figure 3 is one of the process schematic diagrams in this embodiment. Among them, first, the AI large model process is started. After entering the process of starting the AI large model (large language model), does the camera sensor recognize whether the eyes are closed? If not, prompt the user once to close their eyes to experience the AI large model guiding operation, and then continue to start the AI large model. If so, directly start the AI large model, establish a two-dimensional coordinate system based on the screen information. By default, the width of the calculation unit of the AI large model is one-tenth of the resolution pixels of the screen width, and the height of the calculation unit is one-tenth of the resolution pixels of the screen height, which is called a calculation unit. One calculation unit is one cell. In this way, the screen is divided into a two-dimensional coordinate system of 100 cells in 10 rows and 10 columns. Analyze the game character coordinates. Taking the pixel coordinate point (x0, y0) of our hero as the center and △n pixels as the radius, analyze whether there are game characters nearby? If so, every few seconds, according to the game character coordinate relationship, voice prompt the user about the straight-line distance between the pixel coordinate point of our hero and the pixel coordinate point of the enemy hero or between our hero and the enemy hero, as well as the direction at a certain time and the distance in pixels. For example, voice prompt "There is an enemy target 5 pixels away in the 3 o'clock direction" or the abbreviated expression "3 o'clock, 5 grids". Then, determine whether the enemy hero is within the attack range of our hero? If so, vibrate the handle to prompt the user that they can attack.

[0085] Optionally, the establishing a coordinate system for the screen of the display device according to the resolution information to obtain a target coordinate system includes:

[0086] Obtain a preset calculation unit corresponding to the display device, where the preset calculation unit is used to determine the proportional relationship between the target coordinate system and the resolution information;

[0087] Establish a coordinate system for the screen of the display device according to the preset calculation unit and the resolution information to obtain the target coordinate system;

[0088] Or

[0089] Receive the target calculation unit input by the user, where the target calculation unit is used to determine the proportional relationship between the target coordinate system and the resolution information;

[0090] Establish a coordinate system for the screen of the display device according to the target calculation unit and the resolution information to obtain the target coordinate system.

[0091] In this embodiment, the preset calculation unit is the default calculation unit of the large language model. Among them, the width of the preset calculation unit is one-tenth of the width pixels of the screen, and the height of the calculation unit is one-tenth of the height pixels of the screen. One calculation unit is one cell. In this way, the screen is divided into a two-dimensional coordinate system of 100 cells in 10 rows and 10 columns. The calculation unit can be modified according to needs. The smallest calculation unit is a cell with a width of 1 pixel and a height of 1 pixel.

[0092] The target calculation unit is the calculation unit input by the user. Among them, the user can open the interface for setting the width and height of the AI calculation unit, and set the number of pixels corresponding to the width and height of one AI calculation unit on this interface. It is used to conveniently prompt the user by voice. In the case where the user does not make settings, by default, the width of one calculation unit, that is, one cell, is 1 / 10 of the resolution pixels of the screen width, and the height of the cell is 1 / 10 of the resolution pixels of the screen height. In this way, the screen is divided into a two-dimensional coordinate system of 10 rows and 10 columns, with 10 cells in each row and 10 cells in each column.

[0093] By using the target calculation unit input by the user or the default preset calculation unit, the target coordinate system can be established more quickly, thereby improving the efficiency of the large language model in recognizing coordinate information.

[0094] Optionally, before inputting the data information into the large language model for analysis to obtain the operation prompt information, it includes:

[0095] Obtain the fixation state of the user's two eyes on the display device;

[0096] When the fixation state indicates that the user's two eyes do not fixate on the display device within the target time period, push the model prompt information to the user, where the model prompt information is used to prompt the user to start the large language model;

[0097] Based on the model prompt information, obtain the model start instruction input by the user on the display device;

[0098] Run the large language model according to the model startup instruction.

[0099] In this embodiment, when the display device is a VR device, this embodiment is mainly applicable to the situation after the user closes both eyes. After the user closes both eyes, auxiliary prompts are given to the user to help the user still be able to operate after closing both eyes, so as to avoid the decline in vision caused by teenagers using their eyes for a long time to play games.

[0100] Specifically, at regular time intervals, the VR device will obtain the gaze state of the user's two eyes on the VR device. When the gaze state indicates that the user's two eyes have not gazed at the display device during the target time period, model prompt information is pushed to the user to prompt the user to start the large language model.

[0101] Refer to Figure 4 , Figure 4 This is the second flow diagram in this embodiment. Among them, the camera sensor is used to identify the user's eyes. Has the user's eyes been closed for more than a certain number of seconds? If so, continue to judge whether the AI large model has been started? If so, do nothing. If not, does the voice prompt start the operation to guide the start of the AI large model? If so, enter the process of starting the AI large model. If not, exit the voice prompt and do nothing.

[0102] Optionally, after running the large language model according to the model startup instruction, the method further includes:

[0103] Obtain a model stop instruction input by the user on the display device;

[0104] Stop running the large language model according to the model stop instruction.

[0105] In this embodiment, the user can input a model stop instruction to stop running the large language model. Among them, the model stop instruction is the same as the operation assistance instruction, and can include: gesture action instructions, voice instructions, and shortcut key press instructions. Specifically, the gesture action instructions include arm sliding, finger sliding, etc. The voice instruction is the voice content directly input by the user in the display device, such as "turn on operation assistance". The shortcut key press instruction is the content directly input by the user pressing a key on the display device. For example, the user presses a preset shortcut key to trigger the operation assistance instruction.

[0106] In the technical solution of the present invention, during the user's use of the head-mounted display device, by obtaining the operation assistance instruction input by the user on the display device, the data information of the currently running application program is obtained, and the large language model is relied on to analyze the data information to generate an operation prompt message, so as to prompt the user of the current operation method in the application program. Compared with the prior art solution that can only perform voice broadcast of the application program situation for the user, this application can better help the user use the application program of the display device and improve the user experience effect of using the display device.

[0107] See Figure 5 , Figure 5 is the structural diagram of the operation device of the display device provided by the embodiment of the present invention. As Figure 5 shown, the operation device 500 of the display device includes:

[0108] The first acquisition module 510 is used to acquire the operation assistance instruction input by the user on the display device, and the operation assistance instruction is used to request to display an operation prompt on the display device;

[0109] The second acquisition module 520 is used to acquire the data information of the currently running application program of the display device based on the operation assistance instruction, and the data information is used to indicate the usage situation of the user in the application program;

[0110] The analysis module 530 is used to input the data information into a large language model for analysis to obtain an operation prompt message, and the operation prompt message is used to prompt the user of the operation method in the application program;

[0111] The push module 540 is used to push the operation prompt message to the user in the application program.

[0112] Optionally, the second acquisition module 520 includes:

[0113] An acquisition sub-module, used to acquire the resolution information corresponding to the screen of the display device;

[0114] A establishment sub-module, used to establish a coordinate system for the screen of the display device according to the resolution information to obtain a target coordinate system, the horizontal axis of the target coordinate system corresponds to the screen length of the display device, and the vertical axis of the target coordinate system corresponds to the screen width of the display device;

[0115] A first determination sub-module, used to determine the data information of the currently running application program of the display device according to the target coordinate system.

[0116] Optionally, the analysis module 530 includes:

[0117] A second determination sub-module, configured to determine first coordinate information and second coordinate information according to the data information, where the first coordinate information is the coordinate of the game character corresponding to the user in the target coordinate system, and the second coordinate information is the coordinate of the game character corresponding to another user in the target coordinate system;

[0118] An analysis sub-module, configured to input the first coordinate information and the second coordinate information into the large language model for analysis to obtain the operation prompt information.

[0119] Optionally, when the distance between the first coordinate information and the second coordinate information is less than or equal to a preset distance, the operation prompt information is used to prompt the user to perform a target operation in the application program, and the target operation is used to control the interaction between the character corresponding to the user and the character corresponding to another user;

[0120] When the distance between the first coordinate information and the second coordinate information is greater than the preset distance, the operation prompt information is used to prompt the user to maintain the current state in the application program.

[0121] Optionally, the establishment sub-module includes:

[0122] An acquisition unit, configured to acquire a preset calculation unit corresponding to the display device, and the preset calculation unit is used to determine the proportional relationship between the target coordinate system and the resolution information;

[0123] A first establishment unit, configured to establish a coordinate system for the screen of the display device according to the preset calculation unit and the resolution information to obtain the target coordinate system;

[0124] Or

[0125] A reception unit, configured to receive a target calculation unit input by the user, and the target calculation unit is used to determine the proportional relationship between the target coordinate system and the resolution information;

[0126] A second establishment unit, configured to establish a coordinate system for the screen of the display device according to the target calculation unit and the resolution information to obtain the target coordinate system.

[0127] Optionally, it further includes:

[0128] A third acquisition module, configured to acquire the gaze state of the user's two eyes on the display device;

[0129] An information push module, configured to push a model prompt information to the user when the gaze state indicates that the user's two eyes do not gaze at the display device within a target time period, and the model prompt information is used to prompt the user to start the large language model;

[0130] A fourth acquisition module, configured to acquire a model startup instruction input by a user on the display device based on the model prompt information;

[0131] An operation module, configured to run the large language model according to the model startup instruction.

[0132] Optionally, it further includes:

[0133] A fifth acquisition module, configured to acquire a model stop instruction input by a user on the display device;

[0134] A stop module, configured to stop running the large language model according to the model stop instruction.

[0135] Optionally, the operation assistance instruction includes at least one of the following: a gesture action instruction, a voice instruction, and a shortcut key press instruction.

[0136] In the technical solution of the present invention, during the process of a user using a head-mounted display device, by acquiring an operation assistance instruction input by the user on the display device, the data information of the currently running application is obtained, and the large language model is relied on to analyze the data information to generate an operation prompt information, so as to prompt the user about the current operation method in the application. Compared with the prior art solution that can only perform voice announcements of the application situation for the user, this application can better help the user use the application of the display device and improve the user experience effect of using the display device.

[0137] Figure 6 The following is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. As Figure 6 shown, the electronic device 600 includes a memory 610 and a processor 620. The number of processors 620 in the electronic device 600 can be one or more. Figure 6 Taking one processor 620 as an example; the memory 610 and the processor 620 in the server can be connected through a bus or other means. Figure 6 Taking the connection through the bus as an example.

[0138] The memory 610, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the title generation method in the embodiment of the present invention. The processor 620 executes various functional applications and data processing of the server / terminal / server by running the software programs, instructions, and modules stored in the memory 610, that is, implements the above-mentioned operation method of the display device.

[0139] Among them, the processor 620 is used to run the computer program stored in the memory 610 to implement the following steps:

[0140] Obtain the operation assistance instruction input by the user on the display device, where the operation assistance instruction is used to request the display of operation prompts on the display device;

[0141] Based on the operation assistance instruction, obtain the data information of the application currently running on the display device, where the data information is used to indicate the user's usage situation in the application;

[0142] Input the data information into a large language model for analysis to obtain operation prompt information, where the operation prompt information is used to prompt the user about the operation method in the application;

[0143] In the application, push the operation prompt information to the user.

[0144] Optionally, the obtaining the data information of the application currently running on the display device based on the operation assistance instruction includes:

[0145] Obtain the resolution information corresponding to the screen of the display device;

[0146] Establish a coordinate system for the screen of the display device according to the resolution information to obtain a target coordinate system, where the horizontal axis of the target coordinate system corresponds to the length of the screen of the display device, and the vertical axis of the target coordinate system corresponds to the width of the screen of the display device;

[0147] Determine the data information of the application currently running on the display device according to the target coordinate system.

[0148] Optionally, the inputting the data information into a large language model for analysis to obtain operation prompt information includes:

[0149] Determine first coordinate information and second coordinate information according to the data information, where the first coordinate information is the coordinate of the user's corresponding game character in the target coordinate system, and the second coordinate information is the coordinate of the other user's corresponding game character in the target coordinate system;

[0150] Input the first coordinate information and the second coordinate information into the large language model for analysis to obtain the operation prompt information.

[0151] Optionally, when the distance between the first coordinate information and the second coordinate information is less than or equal to a preset distance, the operation prompt information is used to prompt the user to perform a target operation in the application, and the target operation is used to control the interaction between the user's corresponding character and the other user's corresponding character;

[0152] When the distance between the first coordinate information and the second coordinate information is greater than a preset distance, the operation prompt information is used to prompt the user to maintain the current state in the application.

[0153] Optionally, the establishing a coordinate system for the screen of the display device according to the resolution information to obtain a target coordinate system includes:

[0154] Obtaining a preset calculation unit corresponding to the display device, where the preset calculation unit is used to determine the proportional relationship between the target coordinate system and the resolution information;

[0155] Establishing a coordinate system for the screen of the display device according to the preset calculation unit and the resolution information to obtain the target coordinate system;

[0156] Or

[0157] Receiving a target calculation unit input by the user, where the target calculation unit is used to determine the proportional relationship between the target coordinate system and the resolution information;

[0158] Establishing a coordinate system for the screen of the display device according to the target calculation unit and the resolution information to obtain the target coordinate system.

[0159] Optionally, before inputting the data information into the large language model for analysis to obtain operation prompt information, it includes:

[0160] Obtaining the fixation state of the user's two eyes on the display device;

[0161] When the fixation state indicates that the user's two eyes do not fixate on the display device within a target time period, pushing model prompt information to the user, where the model prompt information is used to prompt the user to start the large language model;

[0162] Based on the model prompt information, obtaining a model start instruction input by the user on the display device;

[0163] Running the large language model according to the model start instruction.

[0164] Optionally, after running the large language model according to the model start instruction, the method further includes:

[0165] Obtaining a model stop instruction input by the user on the display device;

[0166] Stopping the running of the large language model according to the model stop instruction.

[0167] Optionally, the operation assistance instruction includes at least one of the following: gesture action instruction, voice instruction, and shortcut key pressing instruction.

[0168] In the process of the user using the head-mounted display device, the technical solution of the present invention obtains the data information of the currently running application program by obtaining the operation assistance instructions input by the user on the display device, analyzes the data information by relying on the large language model, generates operation prompt information, and prompts the user with the current operation method in the application program. Compared with the prior art technical solution that can only perform voice broadcast of the application program situation for the user, this application can better help the user use the application program of the display device and improve the user experience effect of using the display device.

[0169] Those of ordinary skill in the art can understand that all or part of the steps of implementing the method of the above embodiments can be completed by hardware related to program instructions, and the program can be stored in a readable medium.

[0170] The embodiments of the present disclosure also provide a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it can implement any of the steps in the corresponding method embodiments described above, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here. Figure 1 The embodiments of the present disclosure also provide a readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it can implement any of the steps in the corresponding method embodiments described above, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0171] The computer-readable storage medium of the embodiments of the present disclosure can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device.

[0172] The computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device.

[0173] The program code contained on the storage medium can be transmitted by any suitable medium, including but not limited to wireless, wire, optical fiber cable, RF, etc., or any suitable combination of the above.

[0174] Computer program code for performing the operations of the present disclosure can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or terminal. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0175] The above is the preferred implementation mode of the embodiments of the present disclosure. It should be noted that for those of ordinary skill in the art, without departing from the principles described in the present disclosure, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present disclosure.

Claims

1. A method for operating a display device, characterized in that, Applied to a display device, the method includes: Obtain an operation assistance instruction input by a user on the display device, where the operation assistance instruction is used to request the display of an operation prompt on the display device; Based on the operation assistance instruction, obtain data information of the application currently running on the display device, where the data information is used to indicate the usage situation of the user in the application; Input the data information into a large language model for analysis to obtain operation prompt information, where the operation prompt information is used to prompt the user about the operation method in the application; In the application, push the operation prompt information to the user.

2. The method according to claim 1, wherein The obtaining the data information of the application currently running on the display device based on the operation assistance instruction includes: Obtain the resolution information corresponding to the screen of the display device; Establish a coordinate system for the screen of the display device according to the resolution information to obtain a target coordinate system, where the horizontal axis of the target coordinate system corresponds to the length of the screen of the display device, and the vertical axis of the target coordinate system corresponds to the width of the screen of the display device; Determine the data information of the application currently running on the display device according to the target coordinate system.

3. The method according to claim 2, characterized in that The inputting the data information into a large language model for analysis to obtain operation prompt information includes: Determine first coordinate information and second coordinate information according to the data information, where the first coordinate information is the coordinate of the game character corresponding to the user in the target coordinate system, and the second coordinate information is the coordinate of the game character corresponding to other users in the target coordinate system; Input the first coordinate information and the second coordinate information into the large language model for analysis to obtain the operation prompt information.

4. The method according to claim 3, characterized in that, When the distance between the first coordinate information and the second coordinate information is less than or equal to a preset distance, the operation prompt information is used to prompt the user to perform a target operation in the application, and the target operation is used to control the interaction between the character corresponding to the user and the character corresponding to other users; When the distance between the first coordinate information and the second coordinate information is greater than the preset distance, the operation prompt information is used to prompt the user to maintain the current state in the application.

5. The method according to claim 2, wherein The establishing a coordinate system for the screen of the display device according to the resolution information to obtain a target coordinate system includes: Obtain a preset calculation unit corresponding to the display device, where the preset calculation unit is used to determine the proportional relationship between the target coordinate system and the resolution information; Establish a coordinate system for the screen of the display device according to the preset calculation unit and the resolution information to obtain the target coordinate system; Or Receive a target calculation unit input by the user, where the target calculation unit is used to determine the proportional relationship between the target coordinate system and the resolution information; Establish a coordinate system for the screen of the display device according to the target calculation unit and the resolution information to obtain the target coordinate system.

6. The method according to claim 1, wherein Before the inputting the data information into a large language model for analysis to obtain operation prompt information, it includes: Obtain the gaze state of the user's two eyes on the display device; When the gaze state indicates that the user's binoculars do not fixate on the display device within the target time period, push model prompt information to the user, where the model prompt information is used to prompt the user to start the large language model; Based on the model prompt information, obtain a model startup instruction input by the user on the display device; Run the large language model according to the model startup instruction.

7. The method according to claim 6, wherein After running the large language model according to the model startup instruction, the method further includes: Obtain a model stop instruction input by the user on the display device; Stop running the large language model according to the model stop instruction.

8. The method according to any one of claims 1-7, characterized in that, The operation assistance instruction includes at least one of the following: a gesture action instruction, a voice instruction, and a shortcut key press instruction.

9. An operating device for a display device, characterized in that, Applied to a display device, the device includes: A first acquisition module, configured to acquire an operation assistance instruction input by the user on the display device, where the operation assistance instruction is used to request to display an operation prompt on the display device; A second acquisition module, configured to acquire data information of the application currently running on the display device based on the operation assistance instruction, where the data information is used to indicate the user's usage situation in the application; An analysis module, configured to input the data information into a large language model for analysis to obtain operation prompt information, where the operation prompt information is used to prompt the user about the operation method in the application; A push module, configured to push the operation prompt information to the user in the application.

10. An electronic device, characterized in that, Includes a processor, a memory, and a program or instruction stored on the memory and executable on the processor, where when the program or instruction is executed by the processor, it implements the steps of the operation method of the display device according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, A program or instruction is stored on the readable storage medium, and when the program or instruction is executed by the processor, it implements the steps of the operation method of the display device according to any one of claims 1 to 8.

12. A computer program product, characterized in that, Includes computer instructions, and when the computer instructions are executed by the processor, it implements the steps of the operation method of the display device according to any one of claims 1 to 8.

Images