Display method, device, electronic equipment and readable storage medium

Abstract

The application discloses a display method, a display device, an electronic device and a readable storage medium, and belongs to the technical field of display. The display method is executed by a wearable display device, the wearable display device is in communication connection with an input device, and the display method comprises the following steps: displaying a target image, the target image comprising at least two image objects, the image objects being used for displaying image information; in the case that a first input is received through the input device, determining a target image object from the at least two image objects, the target image object being an image object that is gazed at by a user; setting a control object of the input device as the target image object; and in response to the first input, controlling the first image object to display first image information corresponding to the first input.

Description

Technical Field

[0001] This application belongs to the field of display technology, specifically relating to a display method, apparatus, electronic device, and readable storage medium. Background Technology

[0002] Among related technologies, Augmented Reality (AR) display, Virtual Reality (VR) display, and Mediated Reality (MR) can display virtual images and present information to users through virtual image objects.

[0003] In scenarios where AR devices are connected to personal computers, mobile phones, and other electronic devices, there is an application where users can simultaneously see the physical display screen of the electronic device and the virtual display screen of the AR device. The physical and virtual displays generally require two different input devices for operation. For example, users can use a physical keyboard connected to a personal computer to operate the physical display screen, or a virtual keyboard to operate the virtual display screen in AR. If users need to control the content in different display screens, they need to switch input methods repeatedly, making the operation steps complicated. Summary of the Invention

[0004] The purpose of this application is to provide a display method, apparatus, electronic device, and readable storage medium that can solve the problem that users need to repeatedly switch input methods to operate content on different display screens when using AR devices.

[0005] In a first aspect, embodiments of this application provide a display method, which is executed by a wearable display and communicatively connected to an input device. The display method includes:

[0006] Display the target image, which includes at least two image objects, which are used to display image information;

[0007] Upon receiving a first input via an input device, a target image object is determined among at least two image objects, the target image object being the image object the user is looking at;

[0008] Set the control object of the input device to the target image object; and

[0009] In response to the first input, the target image information corresponding to the first input is displayed through the target image object.

[0010] Secondly, embodiments of this application provide a display device, which is performed by a wearable display and is communicatively connected to an input device. The display device includes:

[0011] The display module is used to display the target image, which includes at least two image objects, and the image objects are used to display image information.

[0012] The determination module is configured to, upon receiving a first input via an input device, determine a target image object among at least two image objects, wherein the target image object is the image object being viewed by the user;

[0013] The settings module is used to set the control object of the input device as the target image object;

[0014] The display module is used to respond to the first input by displaying the target image information corresponding to the first input through the target image object.

[0015] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory, wherein the memory stores a program or instructions that can run on the processor, and when the program or instructions are executed by the processor, they implement the steps of the method as described in the first aspect.

[0016] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method as described in the first aspect.

[0017] Fifthly, embodiments of this application provide a chip including a processor and a communication interface coupled to the processor, the processor being used to run programs or instructions to implement the steps of the method as described in the first aspect.

[0018] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method as described in the first aspect.

[0019] In this embodiment, when displaying an image through a wearable display, such as an AR headset, if multiple image objects that can display content information based on user input exist simultaneously on the screen, the system detects which image object the user is currently looking at. When the user operates an input device, such as a physical keyboard, the system displays the result matching the user's input based on the detected image object the user is looking at. This allows the user to control both the physical and virtual display screens simultaneously through a single input device, without the need for the user to manually switch between different input devices or manually switch the association between the input device and the display screen. This simplifies the input process when displaying images using a wearable display. Attached Figure Description

[0020] Figure 1 A flowchart illustrating a display method of some embodiments of this application is shown;

[0021] Figure 2 A schematic diagram illustrating images displayed by a wearable display according to some embodiments of this application is shown;

[0022] Figure 3 A schematic diagram illustrating images displayed by a wearable display according to some embodiments of this application is shown;

[0023] Figure 4 Structural block diagrams of display devices according to some embodiments of this application are shown;

[0024] Figure 5 A structural block diagram of an electronic device according to an embodiment of this application is shown;

[0025] Figure 6 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0027] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0028] The display method, apparatus, electronic device, and readable storage medium provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0029] In some embodiments of this application, a display method is provided, which is performed by a wearable display and is communicatively connected to an input device. Figure 1 Flowcharts illustrating display methods of some embodiments of this application are shown, such as... Figure 1 As shown, the display methods include:

[0030] Step 102: Display the target image, which includes at least two image objects used to display image information.

[0031] In the embodiments of this application, Figure 2 Schematic diagrams illustrating wearable display images of some embodiments of this application are shown, such as... Figure 2 As shown, the user views the target image 200 through the wearable display 20. The target image 200 includes an image object of the physical display screen 202 and an image object of the virtual display screen 204. The wearable display 20 is connected to the physical display screen 202 via a network connection. At the same time, the wearable display 20 is also connected to the input device 30. For example, the input device 30 includes peripheral input devices such as a mouse and a keyboard.

[0032] The physical display screen 202 displays the screen output by electronic devices such as personal computers, mobile phones, and tablets. For example, Figure 2 As shown, the physical display screen 202 displays the desktop 2022 of an electronic device. The virtual display screen 204 is a virtual image object generated by AR, VR, or other devices. For example, the virtual display screen 204 displays a web page interface 2042.

[0033] In some implementations, there are two image objects, one of which is a physical display screen and the other is a virtual display screen.

[0034] In other embodiments, the number of image objects is two or more, including one or more virtual displays. For example, the number of image objects is three, where the first image object is a physical display, the second image object is a first virtual display, and the third image object is a second virtual display.

[0035] In other implementations, the two image objects can be different image objects displayed on the same display screen, such as different display windows on the same display screen. For example, a virtual display screen may show a software window for a text editing software and a software window for a web browsing software, where the software window for the text editing software is one image object and the software window for the web browsing software is another image object.

[0036] Step 104: Upon receiving the first input via the input device, determine the target image object among at least two image objects, wherein the target image object is the image object being viewed by the user.

[0037] In this embodiment, the first input is input made by the user through an input device, such as typing on a keyboard, moving or clicking a mouse, or sliding a touchpad. After detecting the user's first input, the input device sends the signal corresponding to the first input to the wearable display and an electronic device communicatively connected to the wearable display.

[0038] After detecting the user's first input, the wearable display determines which image object the user is currently looking at. For example, the target image displayed by the wearable display includes image objects from a physical display screen and image objects from a virtual display screen. After detecting the first input, the wearable display determines whether the user is currently looking at a physical display screen or a virtual display screen.

[0039] Step 106: Set the control object of the input device to the target image object.

[0040] In this embodiment, the control object of the input device is specifically the object that responds to the user's input operation on the input device. For example, if the user moves the mouse and the control object of the input device is a physical display screen, then the mouse pointer displayed on the physical display screen is moved accordingly based on the user's mouse movement. Similarly, if the user types on the keyboard and the control object of the input device is a virtual display screen, then the virtual display screen displays the characters entered by the user's keyboard typing.

[0041] Step 108: In response to the first input, control the target image object to display the target image information corresponding to the first input.

[0042] In this embodiment, after determining the image object that the user is currently looking at, the target image object that the user is looking at is determined as the image object responding to the first input. Specifically, taking the image objects of the physical display screen and the image objects of the virtual display screen currently displayed on the wearable display as an example, when it is detected that the user is currently looking at the physical display screen, the image object of the physical display screen is determined as the target image object, and it is determined that the first input made by the user through the input device is an input to the content on the physical display screen. Therefore, the image information corresponding to the input result of the first input, that is, the target image information, is displayed through the physical display screen.

[0043] When it is detected that the user is looking at the virtual display screen, the image object on the virtual display screen is identified as the target image object. It is also determined that the first input made by the user through the input device is an input to the content on the virtual display screen. Therefore, the image information corresponding to the input result of the first input, i.e. the target image information, is displayed on the virtual display screen.

[0044] For example, the user is currently looking at a physical display screen that shows the desktop of an electronic device. When the user moves the mouse, the mouse pointer on the desktop displayed on the physical display screen moves according to the user's mouse movement.

[0045] For example, the user is currently looking at a virtual display screen that shows a web page interface. When the user types on the keyboard, the input boxes in the web page interface displayed on the virtual display screen show the characters entered by the user through typing.

[0046] For example, the physical display screen shows a text editing interface and a web page interface. The user is currently looking at the text editing interface. When the user types on the keyboard, the input box in the text editing interface displayed on the physical display screen shows the characters entered by the user through typing on the keyboard.

[0047] For example, the virtual display screen shows a text editing interface and a web page interface. The user is currently looking at the web page interface. When the user types on the keyboard, the input box in the web page interface displayed on the virtual display screen shows the characters entered by the user through typing.

[0048] This application embodiment displays the result matching the user's input by detecting the image object the user is looking at, thereby allowing the user to control both the physical display screen and the virtual display screen simultaneously through a single input device, without the need for the user to manually switch between different input devices or manually switch the association between the input device and the display screen, thus simplifying the operation steps for inputting images when using a wearable display.

[0049] In some embodiments of this application, the wearable display includes an eye-tracking device for detecting human eye information;

[0050] Determine the target image object among at least two image objects, including:

[0051] Based on human eye information, determine the region of focus of the human eye in the target image;

[0052] If the area of ​​human eye gaze is located within the display area of ​​the first image object among at least two image objects, the first image object is identified as the target image object.

[0053] In this embodiment of the application, the wearable display includes an eye-tracking device. For example, the eye-tracking device may be an image sensor or an eye tracker, etc. It determines the corner point of the human eye by detecting changes in the position of the human eye pupil, the dilation and contraction of the human eye pupil, etc., and determines the human eye gaze area in the currently displayed target image through an eye-tracking algorithm. The human eye gaze area is also the area currently viewed by the user's vision.

[0054] After determining the area of ​​human eye gaze, the system further determines the correspondence between the area of ​​human eye gaze and at least two image objects in the current target image. Specifically, if the current area of ​​human eye gaze is within the display area of ​​a certain image object, for example, if the area of ​​human eye gaze is within the display area of ​​a first image object, then that first image object is determined as the target image object that the user is currently gazing at.

[0055] It is understandable that if part of the area of ​​human eye gaze is within the display area of ​​one image object, and another part of the area of ​​human eye gaze is within the display area of ​​another image object, then the image object with the larger overlapping area with the area of ​​human eye gaze will be identified as the target image object that the user is currently looking at.

[0056] For example, the target image includes image objects from a physical display screen and image objects from a virtual display screen. When it is detected that the area of ​​human eye gaze is within the display area of ​​the physical display screen, the image object of the physical display screen is determined as the target image object that the user is currently gazing at.

[0057] For example, the target image includes an image object of a virtual display screen and an image object of a virtual display screen. When it is detected that the area of ​​human eye gaze is within the display area of ​​the virtual display screen, the image object of the virtual display screen is determined as the target image object that the user is currently gazing at.

[0058] This application embodiment uses a wearable display to track the user's eyeballs. Based on the detection results, the area of ​​the user's gaze is determined in the image displayed on the wearable display. According to the correspondence between the area of ​​the user's gaze and the display areas of different image objects, the image object that the user is paying attention to is identified. The user's input through the input device at this time is determined as the input for the image object of attention. Therefore, it is possible to control multiple image objects through one input device, such as controlling a physical display screen and a virtual display screen at the same time. The user does not need to switch input devices repeatedly, which simplifies the operation steps.

[0059] In some embodiments of this application, the wearable display further includes an image sensor for acquiring environmental images;

[0060] The target image includes a virtual image region and a real image region. The virtual image region is used to display virtual image objects, and the real image region is used to display the environment image, which includes the image from the input device.

[0061] In this embodiment, the wearable display can be an AR device, a VR device, or a MR device. The wearable display is equipped with an image sensor capable of acquiring environmental images of the current surroundings; exemplarily, the image sensor is a camera.

[0062] The target image displayed by the wearable display includes a virtual image area for displaying virtual objects. The virtual image objects displayed in this virtual image area can be virtual spatial backgrounds, such as virtual forests, virtual starry skies, virtual rooms, etc., or they can include virtual objects, such as virtual characters, virtual furniture, or virtual animals, plants, etc.

[0063] The target image also includes a real-world image region for displaying environmental images acquired by the image sensor. This real-world image region displays images of the actual environment in which the current user is located, including images from the input device.

[0064] Specifically, when using AR, VR, or MR to view virtual scenes, the virtual background and objects will fill the user's field of vision. If the user needs to input through physical input devices such as a keyboard, the user may not be able to find the keyboard because they cannot see the surrounding environment.

[0065] Therefore, this application embodiment acquires environmental images through an image sensor and identifies input devices such as keyboards from them. While displaying virtual scene images, it also displays environmental images including images of input devices such as keyboards in the real image area. This allows users to grasp information in the real environment to a certain extent and see images of input devices without affecting their viewing of virtual scene image content. This enables users to find input devices as soon as they need to use them.

[0066] For example, Figure 3 Schematic diagrams illustrating wearable display images of some embodiments of this application are shown, such as... Figure 3 As shown, the user displays a target image 302 through the wearable display 20. The target image 302 includes a virtual image area 3022 and a real image area 3024. The virtual image area 3022 displays virtual image objects, including virtual characters, animals, tables, screens, and virtual windows. The real image area 3024 displays images of the real environment, including images from the input device 30.

[0067] This application embodiment displays virtual image objects and real environment images in different display areas of the wearable display, enabling users to quickly locate physical input devices through real environment images, and facilitating users to operate the content displayed on the wearable display through physical input devices.

[0068] In some embodiments of this application, the wearable display further includes an image sensor for acquiring image information from the input device;

[0069] At least two image objects include a second image object, which is a virtual image object generated based on the image information of the input device, and the display position of the second image object in the target image corresponds to the position information of the input device.

[0070] Wearable displays can be AR devices, VR devices, or MR devices. Wearable displays are equipped with image sensors that can acquire environmental images of the current surroundings; for example, the image sensor is a camera.

[0071] After acquiring environmental images using an image sensor, including image information of input devices such as a keyboard that are communicatively connected to the wearable display, the image information of the input devices is identified from the environmental images. Based on this image information, the shape, size, and spatial location of the input devices are determined, and a second image object corresponding to the input devices is generated.

[0072] The second image object appears in the target image in the same shape and size as the input device, and its display position in the target image is the same as the spatial position of the input device, which means that the real input device is "projected" into the virtual space.

[0073] The target image displayed by the wearable display includes virtual image objects generated through 3D modeling, computer graphics algorithms, etc. These virtual image objects can be virtual spatial backgrounds, such as virtual forests, virtual starry skies, virtual rooms, etc., or they can include virtual objects, such as virtual people, virtual furniture, or virtual animals and plants.

[0074] The target image also includes a second image object generated based on information such as the shape, size, and spatial location of the input device. Since the appearance and location of the second image object are the same as those of the input device in the real world, users can use the second image object to know the location of the input device in the real world environment, so that users can find the input device as soon as they need to use it.

[0075] This application embodiment acquires an image of the input device and generates a second image object corresponding to the shape and position of the input device based on the image. This allows the user to quickly locate the physical input device through the second image object, facilitating the user to operate the content displayed on the wearable display through the physical input device.

[0076] The display method provided in this application can be executed by a display device. This application uses a display device executing the display method as an example to illustrate the display device provided in this application.

[0077] In some embodiments of this application, a display device is provided, which is communicatively connected to an input device. Figure 4 Structural block diagrams of display devices according to some embodiments of this application are shown, such as Figure 4 As shown, the display device 400 includes:

[0078] Display module 402 is used to display a target image, the target image including at least two image objects, the image objects being used to display image information;

[0079] The determination module 404 is configured to determine a target image object among at least two image objects upon receiving a first input via an input device, wherein the target image object is the image object being viewed by the user.

[0080] Setting module 406 is used to set the control object of the input device as the target image object;

[0081] The display module 408 is used to respond to the first input and control the target image object to display the target image information corresponding to the first input.

[0082] This application embodiment displays the result matching the user's input by detecting the image object the user is looking at, thereby allowing the user to control both the physical display screen and the virtual display screen simultaneously through a single input device, without the need for the user to manually switch between different input devices or manually switch the association between the input device and the display screen, thus simplifying the operation steps for inputting images when using a wearable display.

[0083] In some embodiments of this application, the display device includes an eye-tracking device for detecting human eye information;

[0084] The determination module is also used to determine the human eye gaze region in the target image based on human eye information; and to determine the first image object as the target image object if the human eye gaze region is located within the display area of ​​the first image object among at least two image objects.

[0085] This application embodiment uses a wearable display to track the user's eyeballs. Based on the detection results, the area of ​​the user's gaze is determined in the image displayed on the wearable display. According to the correspondence between the area of ​​the user's gaze and the display areas of different image objects, the image object that the user is paying attention to is identified. The user's input through the input device at this time is determined as the input for the image object of attention. Therefore, it is possible to control multiple image objects through one input device, such as controlling a physical display screen and a virtual display screen at the same time. The user does not need to switch input devices repeatedly, which simplifies the operation steps.

[0086] In some embodiments of this application, the display device further includes an image sensor for acquiring an environmental image;

[0087] The target image includes a virtual image area and a real image area. The virtual image area is used to display virtual image objects, and the real image area is used to display the environment image, which includes the image from the input device and the image from the physical display screen.

[0088] This application embodiment displays virtual image objects and real environment images in different display areas of the wearable display, enabling users to quickly locate physical input devices through real environment images, and facilitating users to operate the content displayed on the wearable display through physical input devices.

[0089] In some embodiments of this application, the display device further includes an image sensor for acquiring image information from the input device;

[0090] At least two image objects include a second image object, which is a virtual image object generated based on the image information of the input device, and the display position of the second image object in the target image corresponds to the position information of the input device.

[0091] This application embodiment acquires an image of the input device and generates a second image object corresponding to the shape and position of the input device based on the image. This allows the user to quickly locate the physical input device through the second image object, facilitating the user to operate the content displayed on the wearable display through the physical input device.

[0092] The display device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television set (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the scope of the device.

[0093] The display device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system used.

[0094] The display device provided in this application embodiment can implement the various processes implemented in the above method embodiments, and will not be described again here to avoid repetition.

[0095] Optionally, embodiments of this application also provide an electronic device. Figure 5 A structural block diagram of an electronic device according to an embodiment of this application is shown, such as... Figure 5 As shown, the electronic device 500 includes a processor 502, a memory 504, and a program or instructions stored in the memory 504 and executable on the processor 502. When the program or instructions are executed by the processor 502, they implement the various processes of the above method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0096] It should be noted that the electronic devices in the embodiments of this application include the aforementioned mobile electronic devices and non-mobile electronic devices.

[0097] Figure 6 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0098] The electronic device 600 includes, but is not limited to, components such as: radio frequency unit 601, network module 602, audio output unit 603, input unit 604, sensor 605, display unit 606, user input unit 607, interface unit 608, memory 609, and processor 610.

[0099] Those skilled in the art will understand that the electronic device 600 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 610 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 6 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0100] The display unit 606 is used to display a target image, which includes at least two image objects and the image objects are used to display image information.

[0101] The processor 610 is configured to, upon receiving a first input via an input device, determine a target image object among at least two image objects, the target image object being the image object being viewed by the user; and, in response to the first input, control the target image object to display target image information corresponding to the first input.

[0102] This application embodiment displays the result matching the user's input by detecting the image object the user is looking at, thereby allowing the user to control both the physical display screen and the virtual display screen simultaneously through a single input device, without the need for the user to manually switch between different input devices or manually switch the association between the input device and the display screen, thus simplifying the operation steps for inputting images when using a wearable display.

[0103] Optionally, the processor 610 is further configured to determine the human eye gaze region in the target image based on human eye information; and to determine the first image object as the target image object if the human eye gaze region is located within the display area of ​​the first image object among at least two image objects.

[0104] This application embodiment uses a wearable display to track the user's eyeballs. Based on the detection results, the area of ​​the user's gaze is determined in the image displayed on the wearable display. According to the correspondence between the area of ​​the user's gaze and the display areas of different image objects, the image object that the user is paying attention to is identified. The user's input through the input device at this time is determined as the input for the image object of attention. Therefore, it is possible to control multiple image objects through one input device, such as controlling a physical display screen and a virtual display screen at the same time. The user does not need to switch input devices repeatedly, which simplifies the operation steps.

[0105] Optionally, the target image includes a virtual image region and a real image region. The virtual image region is used to display virtual image objects, and the real image region is used to display an environmental image, which includes an image from the input device.

[0106] This application embodiment displays virtual image objects and real environment images in different display areas of the wearable display, enabling users to quickly locate physical input devices through real environment images, and facilitating users to operate the content displayed on the wearable display through physical input devices.

[0107] Optionally, at least two image objects include a second image object, which is a virtual image object generated based on the image information of the input device, and the display position of the second image object in the target image corresponds to the position information of the input device.

[0108] This application embodiment acquires an image of the input device and generates a second image object corresponding to the shape and position of the input device based on the image. This allows the user to quickly locate the physical input device through the second image object, facilitating the user to operate the content displayed on the wearable display through the physical input device.

[0109] It should be understood that, in this embodiment, the input unit 604 may include a graphics processing unit (GPU) 6041 and a microphone 6042. The GPU 6041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 606 may include a display panel 6061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 607 includes at least one of a touch panel 6071 and other input devices 6072. The touch panel 6071 is also called a touch screen. The touch panel 6071 may include two parts: a touch detection device and a touch controller. Other input devices 6072 may include, but are not limited to, a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, and a joystick, which will not be described in detail here.

[0110] The memory 609 can be used to store software programs and various data. The memory 609 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 609 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 609 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0111] Processor 610 may include one or more processing units; optionally, processor 610 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 610.

[0112] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0113] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0114] This application also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0115] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0116] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above method embodiments and achieve the same technical effects. To avoid repetition, it will not be described again here.

[0117] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0118] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0119] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A display method, characterized in that, The display method is performed by a wearable display, which is communicatively connected to an input device, and the display method includes: Display a target image, the target image comprising at least two image objects, the image objects being used to display image information; Upon receiving a first input via the input device, a target image object is determined among at least two image objects, the target image object being the image object being viewed by the user. Set the control object of the input device to the target image object; In response to the first input, the target image object is controlled to display target image information corresponding to the first input; The wearable display also includes an image sensor for acquiring environmental images; The target image includes a virtual image region and a real image region. The virtual image region is used to display virtual image objects, and the real image region is used to display the environment image. The environment image includes the image of the input device and the image of the physical display screen. The image sensor is also used to acquire image information from the input device; At least two of the image objects include a second image object, which is a virtual image object generated based on the image information of the input device, and the display position of the second image object in the target image corresponds to the position information of the input device.

2. The display method according to claim 1, characterized in that, The wearable display includes an eye-tracking device for detecting human eye information; Determining the target image object among at least two of the image objects includes: Based on the human eye information, determine the human eye's gaze region in the target image; If the area of ​​human eye gaze is located within the display area of ​​the first image object among at least two image objects, the first image object is identified as the target image object.

3. A display device, characterized in that, The display device is communicatively connected to the input device, and the display device includes: A display module is used to display a target image, the target image including at least two image objects, the image objects being used to display image information; A determining module is configured to, upon receiving a first input via the input device, determine a target image object among at least two image objects, wherein the target image object is the image object being viewed by the user. The setting module is used to set the control object of the input device to the target image object; The display module is used to respond to the first input and control the target image object to display target image information corresponding to the first input; The display device further includes an image sensor, which is used to acquire environmental images; The target image includes a virtual image region and a real image region. The virtual image region is used to display virtual image objects, and the real image region is used to display the environment image. The environment image includes the image of the input device and the image of the physical display screen. The image sensor is also used to acquire image information from the input device; At least two of the image objects include a second image object, which is a virtual image object generated based on the image information of the input device, and the display position of the second image object in the target image corresponds to the position information of the input device.

4. The display device according to claim 3, characterized in that, The display device includes an eye-tracking device for detecting human eye information; The determining module is further configured to determine the human eye gaze region in the target image based on the human eye information; as well as If the area of ​​human eye gaze is located within the display area of ​​the first image object among at least two image objects, the first image object is identified as the target image object.

5. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the method as described in claim 1 or 2.

6. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the method as described in claim 1 or 2.

Images