Article display method, head-mounted device, storage medium, and program product

Abstract

The application discloses an article display method, a head-mounted device, a storage medium and a program product, relates to the technical field of head-mounted devices, and discloses an article display method applied to a head-mounted device configured with a projection unit, the article display method comprising the following steps: in response to a selection instruction of a current user, selecting a target article, and projecting the target article through the projection unit of the head-mounted device; in response to a locking instruction of the current user, acquiring a target projection position in a current real environment; and locking the projected target article to the target projection position. The application solves the technical problem that the existing display method is difficult to meet the needs of users for article display.

Description

Technical Field

[0001] This application relates to the field of head-mounted device technology, and more particularly to a method for displaying an item, a head-mounted device, a storage medium, and a program product. Background Technology

[0002] With the continuous advancement of technology, people have placed higher demands on the ways they acquire and interact with information in their lives and work.

[0003] In both daily life and work, there is a widespread need for product display and demonstration. However, in existing scenarios such as tourism and online shopping, when a product needs to be displayed or demonstrated, the physical object may not be available on-site. In such cases, relying solely on simple textual descriptions is insufficient to showcase the product's details and its effect in a real-world setting. In other words, existing display methods are inadequate to meet users' needs for product presentation.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of this application is to provide an invention title that aims to solve the technical problem that existing display methods are unable to meet users' needs for displaying items.

[0006] To achieve the above objectives, this application proposes a method for displaying an object, applied to a head-mounted device equipped with a projection unit, the method comprising:

[0007] In response to the current user's selection command, a target item is selected, and the target item is projected through the projection unit of the head-mounted device;

[0008] In response to the current user's lock command, obtain the target projection position in the current real environment;

[0009] Lock the projected target object to the target projection position.

[0010] In one embodiment, the step of projecting the target item through the projection unit of the head-mounted device includes:

[0011] Eye tracking is performed on the current user to obtain the target gaze point of the current user in the current real environment;

[0012] The target object is projected onto the target gaze point using the projection unit.

[0013] In one embodiment, the step of obtaining the target projection position in the current real environment in response to the current user's locking command includes:

[0014] Obtain the duration of the current user's continuous gaze at the target gaze point;

[0015] After the duration of continuous gaze exceeds a preset gaze duration threshold, a locking command is triggered, and the position of the target gaze point in the current real environment is taken as the target projection position.

[0016] In one embodiment, the step of locking the projected target object to the target projection position includes:

[0017] Obtain the reference object under the locking command, and the relative positional relationship between the reference object and the target projection position;

[0018] Based on the real-time reference position of the reference object and the relative positional relationship, the new target projection position of the target item is calculated.

[0019] The target object is projected according to the new target projection position so that the relative position of the projected target object and the reference object remains unchanged.

[0020] In one embodiment, after the step of adjusting the projection position of the target item, the method further includes:

[0021] The target object to be projected is captured to obtain an image of the projected object;

[0022] Based on the projected image of the object, target adjustment parameters are determined, and the projection unit is adjusted based on the target adjustment parameters.

[0023] In one embodiment, the step of determining the target adjustment parameters based on the projected object image includes:

[0024] The sharpness of the object's edge in the projected object image is obtained by performing sharpness detection on the object image.

[0025] After the sharpness of the object image is less than the preset lower limit of sharpness, the projection focal length of the projection unit is increased, and the increased projection focal length is used as the target adjustment parameter.

[0026] After the sharpness of the object image exceeds the preset sharpness upper limit, the projection focal length of the projection unit is reduced, and the reduced projection focal length is used as the target adjustment parameter.

[0027] In one embodiment, the step of determining the target adjustment parameters based on the projected object image further includes:

[0028] The real-time reference brightness of the target projection position is obtained, and a real-time brightness threshold is determined based on the real-time reference brightness, wherein the real-time brightness threshold is greater than the real-time reference brightness;

[0029] Identify the brightness of the target object in the projected object image to obtain the object projection brightness;

[0030] After the projected brightness of the object is less than the real-time brightness threshold, the projection power of the projection unit is increased, and the increased projection power is used as the target adjustment parameter.

[0031] After the projected brightness of the object exceeds the real-time brightness threshold, the projection power of the projection unit is reduced, and the reduced projection power is used as the target adjustment parameter.

[0032] In addition, to achieve the above objectives, this application also proposes a head-mounted device, which includes a projection unit and a control unit. The control unit includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The computer program is configured to implement the steps of the item display method described above.

[0033] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the item display method described above.

[0034] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the item display method described above.

[0035] One or more technical solutions proposed in this application have at least the following technical effects:

[0036] This application applies to a head-mounted device equipped with a projection unit. Responding to a selection command from the current user, a target item is selected, and the target item is projected through the projection unit of the head-mounted device, thereby projecting the desired target item. Then, responding to a locking command from the current user, the target projection position in the current real-world environment is obtained; the projected target item is locked at the target projection position. Thus, the projected target item is "fixed" in the current real-world environment. Therefore, this application projects the desired target item to the real environment through a head-mounted device. This allows for a more accurate and clear display of the target item even when the physical object is not physically present, meeting the user's need for item display. Attached Figure Description

[0037] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0038] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a flowchart illustrating an embodiment of the article display method of this application.

[0040] Figure 2 This is a scene diagram illustrating the projection target item involved in the embodiments of this application;

[0041] Figure 3 This is a flowchart illustrating Embodiment 2 of the article display method of this application;

[0042] Figure 4 This is a schematic diagram of a scenario involving updating the target projection position according to an embodiment of this application;

[0043] Figure 5 This is a flowchart illustrating Embodiment 2 of the article display method of this application;

[0044] Figure 6 This is a schematic diagram of the device structure of the head-mounted device in the embodiments of this application;

[0045] Figure 7 This is a schematic diagram of the outer side of the head-mounted device involved in the embodiments of this application;

[0046] Figure 8 This is a schematic diagram of the inner side of the head-mounted device involved in the embodiments of this application.

[0047] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0048] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0049] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0050] The main solution of this application embodiment is: in response to the current user's selection instruction, select a target item and project the target item through the projection unit of the head-mounted device; in response to the current user's locking instruction, obtain the target projection position in the current real environment; and lock the projected target item to the target projection position.

[0051] In this embodiment, for ease of description, the following description uses a head-mounted device as the execution subject.

[0052] The need for product display and demonstration is widespread in both daily life and work. However, in current scenarios such as tourism and online shopping, when a product needs to be displayed or demonstrated, the physical object may not be present. In such cases, simple textual descriptions are insufficient to showcase the product's details and its effect in a real-world setting. For example, in tourism, tour guides may mention cultural relics (such as porcelain or paintings) or scenic spots (such as a mountain or stone carving), but if these relics or spots are not physically present, the guide can only describe them verbally. In online shopping, people see clothes or jewelry online and want to buy them for family or themselves, hoping to see if they fit in person. Many similar scenarios exist, such as wanting to buy a painting and asking for suggestions on its placement and size on a wall. In short, existing display methods are insufficient to meet users' needs for product display.

[0053] This application provides a solution for a head-mounted device equipped with a projection unit. Responding to a user's selection command, a target item is selected, and the target item is projected through the projection unit of the head-mounted device, thereby projecting the desired target item. Then, responding to a user's locking command, the target projection position in the current real-world environment is obtained, and the projected target item is locked at that position. Thus, the projected target item is "fixed" in the current real-world environment. Therefore, this application projects the desired target item to the real environment using a head-mounted device, enabling a more accurate and clear display of the target item even when the physical object is absent, thus meeting the user's need for item display.

[0054] Based on this, the embodiments of this application provide a method for displaying items, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the article display method of this application.

[0055] In this embodiment, the item display method is applied to a head-mounted device equipped with a projection unit, and the item display method includes steps S10 to S40:

[0056] Step S10: In response to the current user's selection command, select the target item and project the target item through the projection unit of the head-mounted device;

[0057] It should be noted that the selection instruction is an instruction to select a target item, and the selection instruction can be issued through at least one of the user's gesture interaction, touch interaction, voice interaction, and eye-tracking interaction.

[0058] Additionally, it should be noted that the target item can be a two-dimensional virtual item (such as movie footage, game footage, photographs, or partial content within a scene) or a three-dimensional virtual item (such as a three-dimensional model of a real object like a puppy, a jar, a fan, or a potted plant). It is understood that the target item is essentially image or model data stored locally on the head-mounted device or in the cloud.

[0059] This embodiment can identify the selection command corresponding to the user's first interaction operation after receiving the first interaction operation. Then, in response to the selection command, the target item is selected. The target item can then be projected through the projection unit of the head-mounted device. For example, this embodiment can project the target item at a preset projection position, which is a pre-set initial projection position, such as a position relative to the front, lower right corner, or lower left corner of the head-mounted device. Alternatively, it can be the current user's target gaze point in the current real-world environment, and then the target item can be projected onto that target gaze point through the projection unit. Thus, the projected target item can move with the current user's target gaze point, thereby providing the current user with a real-time preview of the scene when the target item is projected onto the target gaze point.

[0060] In one feasible implementation, step S10 may include steps S11 to S12:

[0061] Step S11: Perform eye tracking on the current user to obtain the target gaze point of the current user in the current real environment;

[0062] Step S12: Project the target object onto the target gaze point using the projection unit.

[0063] Since the projected target object needs to be integrated with the current real-world environment, this embodiment uses eye tracking to obtain the current user's target gaze point in the current real-world environment to ensure accuracy. This method is more accurate than approximating the gaze point using head posture or facial orientation. Therefore, this embodiment can project the target object onto the target gaze point. The projected target object moves with the current user's target gaze point, providing a real-time preview of the scene when the target object is projected onto the target gaze point, facilitating subsequent locking operations.

[0064] Step S20: In response to the current user's locking command, obtain the target projection position in the current real environment;

[0065] It should be noted that the locking command is an instruction to lock the target object's projection position in the current real environment. The locking command can be issued through at least one of the user's interactive actions: gesture interaction, touch interaction, voice interaction, and eye-tracking interaction. The target projection position is the position where the current user expects the target object to be projected in the current real environment, and the target projection position can be determined based on the locking command.

[0066] This embodiment can identify the locking command corresponding to the second interaction operation after receiving it from the user. Then, in response to the locking command, it obtains the target projection position in the current real environment pointed to by the locking command. For example, the locking command may include the target projection position. This embodiment can use the intersection of the current user's gesture direction and the current real environment as the target projection position, or it can use the intersection of the current user's line of sight and the current real environment as the target projection position, etc.

[0067] In one feasible implementation, step S20 may include steps S21 to S22:

[0068] Step S21: Obtain the duration of the current user's continuous gaze at the target gaze point;

[0069] Step S22: After the continuous gaze duration exceeds the preset gaze duration threshold, a locking command is triggered, and the position of the target gaze point in the current real environment is taken as the target projection position.

[0070] In this embodiment, the duration of the current user's continuous gaze at the target gaze point can be obtained. Once the duration exceeds a preset gaze duration threshold, a locking command is triggered, and the location of the target gaze point in the current real-world environment is used as the target projection location. Furthermore, to avoid false locking, this embodiment can also display a locking query message after the duration exceeds the preset gaze duration threshold. Upon receiving confirmation from the current user regarding the locking query message, a locking command can be triggered. This avoids false locking of the target projection location.

[0071] Step S30: Lock the projected target object to the target projection position.

[0072] See Figure 2 , Figure 2 This is a scene diagram of the projection target object involved in the embodiments of this application. Figure 2The potted plant formed by the dashed lines is the target object of the projection.

[0073] As an example, this embodiment can convert the target projection position into target projection coordinates in the world coordinate system, then obtain the real-time user coordinates of the head-mounted device in the world coordinate system, and then project the target item to the target projection coordinates according to the real-time relative position between the real-time user coordinates and the target projection coordinates, so as to lock the projected target item to the target projection position.

[0074] As another example, considering situations such as the head-mounted device having no positioning signal, a weak positioning signal, or low positioning accuracy, this embodiment can also acquire a reference object under the locking command, and the relative positional relationship between the reference object and the target projection position; based on the real-time reference position of the reference object and the relative positional relationship, a new target projection position of the target item is calculated; the target item is projected according to the new target projection position so that the relative position between the projected target item and the reference object remains unchanged. Therefore, this embodiment can still maintain a good locking effect between the projected target item and the current real environment even in situations with no positioning signal, a weak positioning signal, or low positioning accuracy, ensuring the effective integration of the projected item and the real environment.

[0075] This embodiment provides a method for displaying an object, applied to a head-mounted device equipped with a projection unit. Responding to a selection command from the current user, a target object is selected, and the target object is projected through the projection unit of the head-mounted device, thereby projecting the desired target object. Then, responding to a locking command from the current user, the target projection position in the current real-world environment is obtained; the projected target object is locked to the target projection position. Thus, the projected target object is "fixed" in the current real-world environment. Therefore, this embodiment projects the desired target object to be displayed or demonstrated into the real environment through a head-mounted device. This allows for a more accurate and clear display of the target object even when the physical object is not physically present, meeting the user's need for object display.

[0076] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 Step S30 includes steps S31 to S33:

[0077] Step S31: Obtain the reference object under the locking command, and the relative positional relationship between the reference object and the target projection position;

[0078] Step S32: Calculate the new target projection position of the target object based on the real-time reference position of the reference object and the relative positional relationship;

[0079] Step S33: Project the target object according to the new target projection position so that the relative position of the projected target object and the reference object remains unchanged.

[0080] It should be noted that the real-time reference position of the reference object is the real-time relative position between the reference object and the head-mounted device. The reference object under the lock command can be the object closest to the target projection position, or any object within the field of view of the head-mounted device. For example, when the target projection position indicated by the lock command is on the current user's right hand, then the current user's right hand is the reference object in the current real environment. When the target projection position indicated by the lock command is on the desktop in the current real environment, then the desktop is the reference object in the current real environment.

[0081] See Figure 4 , Figure 4 This is a schematic diagram of a scene involving updating the target projection position according to an embodiment of this application. Therefore, this embodiment can obtain the reference object (i.e., ...) under the locking command. Figure 4 The person in the middle), and the reference object and the target projection position (i.e. Figure 4 The relative positional relationship between the reference object and the target projection position is determined by the relative positional relationship between the reference object and the target projection position. Furthermore, this embodiment can acquire the real-time reference position of the reference object using devices such as a camera and a distance sensor, thereby obtaining the real-time changing position of the reference object. Since the relative positional relationship describes the relative position between the reference object and the target projection position, the new target projection position of the target object (i.e., the target projection position) can be calculated based on the real-time reference position of the reference object and the relative positional relationship. Figure 4 (Circle A' in the image). Then, the target object can be projected according to the new target projection position, so that the relative position of the projected target object and the reference object remains unchanged.

[0082] In the second embodiment of this application, the new target projection position of the target item is locked by using a reference object. On the one hand, this can avoid the influence of situations such as no positioning signal, weak positioning signal, and low positioning accuracy. On the other hand, it can keep the relative position between the target item and the reference object unchanged. This is beneficial because when the target item is projected onto a moving reference object (such as an active hand or a moving person), the movement of the reference object will not cause the target item to fail to be closely integrated with the current real environment.

[0083] Based on the first embodiment of this application, in the third embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 5 After step S30, the method further includes steps S40 to S50:

[0084] Step S40: Acquire an image of the projected target object to obtain an image of the projected object;

[0085] Step S50: Determine target adjustment parameters based on the projected object image, and adjust the projection unit based on the target adjustment parameters.

[0086] It should be noted that the target adjustment parameters are the parameters for projecting the target item in a specified display state, such as projection focal length, projection power, image contrast, etc.

[0087] In actual projection, the projected target object may be blurry, distorted, or too dim to be clearly displayed, severely affecting the display effect. Therefore, this embodiment can acquire an image of the projected target object. Then, image analysis can be performed on the projected image to obtain analysis results, thereby determining the differences between the projected target object and the specified display state. Based on the analysis results, target adjustment parameters are determined, and the projection unit is adjusted according to these parameters.

[0088] As an example, this embodiment can perform sharpness detection on the edges of the target object in the projected image to obtain the image sharpness. If the image sharpness is less than a preset lower sharpness limit, the projection focal length of the projection unit is increased, and the increased focal length is used as the target adjustment parameter. If the image sharpness is greater than a preset upper sharpness limit, the projection focal length of the projection unit is decreased, and the decreased focal length is used as the target adjustment parameter. Therefore, this embodiment adjusts the projection focal length of the projection unit to ensure the clarity of the projected target object, avoiding blurring due to insufficient sharpness and distortion or jagged edges due to excessive sharpness.

[0089] In another example, this embodiment can obtain the real-time reference brightness of the target projection position and determine a real-time brightness threshold based on the real-time reference brightness, wherein the real-time brightness threshold is greater than the real-time reference brightness; identify the brightness of the target object in the projected object image to obtain the object projection brightness; after the object projection brightness is less than the real-time brightness threshold, increase the projection power of the projection unit and use the increased projection power as the target adjustment parameter; after the object projection brightness is greater than the real-time brightness threshold, decrease the projection power of the projection unit and use the decreased projection power as the target adjustment parameter. Thus, this embodiment adjusts the projection power of the projection unit to ensure that the brightness of the projected target object matches the brightness of the target projection position, avoiding situations where the target object's brightness is too low or too high, resulting in the target object not being clearly displayed or lacking detail.

[0090] In some embodiments, step S50 includes steps S51 to S53:

[0091] Step S51: Detect the sharpness of the edges of the target object in the projected object image to obtain the sharpness of the object image;

[0092] Step S52: After the sharpness of the object image is less than the preset lower limit of sharpness, the projection focal length of the projection unit is increased, and the increased projection focal length is used as the target adjustment parameter.

[0093] Step S53: After the sharpness of the object image is greater than the preset sharpness upper limit, the projection focal length of the projection unit is reduced, and the reduced projection focal length is used as the target adjustment parameter.

[0094] It should be noted that the sharpness detection refers to the operation of detecting the sharpness of the edges of the target image.

[0095] Additionally, it should be noted that the preset lower sharpness limit is a preset lower limit value for the sharpness of the target object image in the projected object image, and the preset upper sharpness limit is a preset upper limit value for the sharpness of the target object image in the projected object image.

[0096] This embodiment first identifies the edges of the target object in the projected image using edge detection algorithms such as gradient operators, Laplacian operators, Canny edge detectors, and neural network models. Then, it measures parameters such as gradient changes and grayscale changes of the target object's edges using sharpness evaluation algorithms such as 10-90% rise distance, modulation transfer function, and bevel algorithm to generate the object image sharpness. If the object image sharpness is less than a preset lower sharpness limit, it indicates that the sharpness of the projected target object is too low, and the projection focal length of the projection unit can be increased. It can be understood that this embodiment can determine the amount of increase in the projection focal length based on the difference between the object image sharpness and the preset lower sharpness limit, and increase the projection focal length of the projection unit accordingly. The increased projection focal length is used as a target adjustment parameter to improve the sharpness of the projected target object. If the object image sharpness is greater than a preset upper sharpness limit, it indicates that the sharpness of the projected target object is too high, and the projection focal length of the projection unit can be decreased. It is understood that in this embodiment, the amount of reduction in the projection focal length can be determined based on the difference between the sharpness of the object image and the preset upper limit of sharpness, and the projection focal length of the projection unit can be increased based on the amount of reduction. The reduced projection focal length is then used as a target adjustment parameter to reduce the sharpness of the projected target object.

[0097] This embodiment adjusts the projection focal length of the projection unit to ensure the clarity of the projected target object and avoid blurring due to low sharpness or distortion and jagged edges due to high sharpness.

[0098] In some embodiments, step S50 includes steps S54 to S57:

[0099] Step S54: Obtain the real-time reference brightness of the target projection position, and determine the real-time brightness threshold based on the real-time reference brightness, wherein the real-time brightness threshold is greater than the real-time reference brightness;

[0100] Step S55: Identify the brightness of the target object in the projected object image to obtain the object projection brightness;

[0101] Step S56: After the projected brightness of the object is less than the real-time brightness threshold, increase the projection power of the projection unit and use the increased projection power as the target adjustment parameter.

[0102] Step S57: After the projected brightness of the object is greater than the real-time brightness threshold, reduce the projection power of the projection unit and use the reduced projection power as the target adjustment parameter.

[0103] It should be noted that the real-time reference brightness can be the brightness at the target projection position in the current real environment, or the brightness of the reference object in the current real environment.

[0104] It should also be noted that the real-time brightness threshold is greater than the real-time reference brightness, and the real-time brightness threshold can be the sum of the difference (positive value) between the real-time reference brightness and the preset brightness.

[0105] Since different locations in the current real-world environment may have varying brightness, this embodiment aims to prevent the projected target object from being too dim or too bright, thus ensuring its clarity and detail. This embodiment obtains the real-time reference brightness of the target projection location and determines a real-time brightness threshold based on this reference brightness, where the real-time brightness threshold is greater than the reference brightness. Furthermore, this embodiment can identify the brightness of the target object in the projected image based on the grayscale values ​​of the area where the target object is located (e.g., by taking the average of the grayscale values ​​or performing histogram analysis on the grayscale values), thus obtaining the object's projected brightness. If the object's projected brightness is less than the real-time brightness threshold, it indicates that the projected target object is dimmer than the target projection location. In this case, the projection power of the projection unit can be increased, and the increased projection power can be used as a target adjustment parameter to improve the brightness of the projected target object. Conversely, if the object's projected brightness is greater than the real-time brightness threshold, it indicates that the projected target object is dimmer than the target projection location. In this case, the projection power of the projection unit can be decreased, and the decreased projection power can be used as a target adjustment parameter to reduce the brightness of the projected target object.

[0106] This embodiment adjusts the projection power of the projection unit to ensure that the brightness of the projected target object matches the brightness of the target projection position, thus avoiding situations where the target object cannot be clearly displayed or details cannot be displayed due to excessively low or high brightness.

[0107] The third embodiment of this application acquires an image of the projected target object. Based on this image, the projection status of the target object can be identified, and target adjustment parameters can be determined. The projection unit is then adjusted based on these parameters. This ensures that the projected target object achieves a good display effect.

[0108] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the method of displaying the items in this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0109] This application provides a head-mounted device, a projection unit for the head-mounted device, and a control unit. The control unit includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the item display method in Embodiment 1 above.

[0110] The following is for reference. Figure 6 The diagram illustrates a structure suitable for implementing a head-mounted device according to the embodiments of this application. The head-mounted device in the embodiments of this application may include virtual reality glasses, virtual reality helmets, augmented reality glasses, augmented reality helmets, mixed reality helmets, and other head-mounted terminal devices. Figure 6 The head-mounted device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of this application.

[0111] like Figure 6 As shown, the head-mounted device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the head-mounted device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, magnetic tape, a hard disk, etc.; and communication devices 1009. Communication device 1009 allows the head-mounted device to communicate wirelessly or wiredly with other devices to exchange data. Projection unit 1010 is used to project two-dimensional images or three-dimensional objects. Although head-mounted devices with various systems are shown in the figures, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented alternatively.

[0112] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0113] The head-mounted device provided in this application, employing the item display method described in the above embodiments, can solve the technical problem that existing display methods are unable to meet users' needs for item display. Compared with the prior art, the beneficial effects of the head-mounted device provided in this application are the same as those of the item display method provided in the above embodiments, and other technical features of the head-mounted device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0114] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0115] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0116] Taking the head-mounted device as virtual reality glasses as an example, see [link / reference]. Figure 7 and Figure 8 , Figure 7 This is a schematic diagram of the outer side of the head-mounted device involved in the embodiments of this application; Figure 8 This is a schematic diagram of the inner side of the head-mounted device according to an embodiment of this application. Figure 7 As shown, the outer side of the head-mounted device (i.e., the side facing away from the user) can be equipped with a projection unit, a left camera, a right camera, and a proximity sensor. Figure 8As shown, the inner side of the head-mounted device (the side facing the user) can house the lens barrel and eye-tracking camera. The projection unit projects the target object onto the real physical environment, typically within a 180° range in front. The left and right cameras capture the space in front and track user gestures. A distance sensor collects distance information to assist positioning in specific scenarios (e.g., when there's a wall in front, the camera cannot accurately determine the distance from the wall using the surrounding environment). The lens barrel projects the display content onto the user's eyes. The eye-tracking camera tracks the user's eyes and identifies the gaze point.

[0117] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the item display method in the above embodiments.

[0118] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0119] The aforementioned computer-readable storage medium may be included in the head-mounted device; or it may exist independently and not assembled into the head-mounted device.

[0120] The aforementioned computer-readable storage medium carries one or more programs that, when executed by the head-mounted device, cause the head-mounted device to: select a target item in response to a selection command from the current user, and project the target item through the projection unit of the head-mounted device; obtain the target projection position in the current real environment in response to a locking command from the current user; and lock the projected target item to the target projection position.

[0121] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the current user's computer, partially on the current user's computer, as a standalone software package, partially on the current user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the current user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0122] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0123] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0124] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described item display method, thereby solving the technical problem that existing display methods are unable to meet users' needs for item display. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the item display method provided in the above embodiments, and will not be repeated here.

[0125] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the article display method described above.

[0126] The computer program product provided in this application can solve the technical problem that existing display methods are unable to meet users' needs for displaying items. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the item display methods provided in the above embodiments, and will not be repeated here.

[0127] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for displaying items, characterized in that, The method for displaying an item, applicable to a head-mounted device equipped with a projection unit, includes: In response to the current user's selection command, a target item is selected, and the target item is projected through the projection unit of the head-mounted device; In response to the current user's lock command, obtain the target projection position in the current real environment; Lock the projected target object to the target projection position; The step of locking the projected target object to the target projection position includes: Obtain the reference object under the locking command, and the relative positional relationship between the reference object and the target projection position; Based on the real-time reference position of the reference object and the relative positional relationship, the new target projection position of the target item is calculated, wherein the real-time reference position of the reference object is the real-time relative position between the reference object and the head-mounted device. The target object is projected according to the new target projection position so that the relative position of the projected target object and the reference object remains unchanged.

2. The method as described in claim 1, characterized in that, The step of projecting the target item through the projection unit of the head-mounted device includes: Eye tracking is performed on the current user to obtain the target gaze point of the current user in the current real environment; The target object is projected onto the target gaze point using the projection unit.

3. The method as described in claim 2, characterized in that, The step of obtaining the target projection position in the current real environment in response to the current user's locking command includes: Obtain the duration of the current user's continuous gaze at the target gaze point; After the duration of continuous gaze exceeds a preset gaze duration threshold, a locking command is triggered, and the position of the target gaze point in the current real environment is taken as the target projection position.

4. The method according to any one of claims 1 to 3, characterized in that, After the step of adjusting the projection position of the target item, the method further includes: The target object to be projected is captured to obtain an image of the projected object; Based on the projected image of the object, target adjustment parameters are determined, and the projection unit is adjusted based on the target adjustment parameters.

5. The method as described in claim 4, characterized in that, The step of determining the target adjustment parameters based on the projected object image includes: The sharpness of the object's edge in the projected object image is obtained by performing sharpness detection on the object image. After the sharpness of the object image is less than the preset lower limit of sharpness, the projection focal length of the projection unit is increased, and the increased projection focal length is used as the target adjustment parameter. After the sharpness of the object image exceeds the preset sharpness upper limit, the projection focal length of the projection unit is reduced, and the reduced projection focal length is used as the target adjustment parameter.

6. The method as described in claim 5, characterized in that, The step of determining the target adjustment parameters based on the projected object image further includes: The real-time reference brightness of the target projection position is obtained, and a real-time brightness threshold is determined based on the real-time reference brightness, wherein the real-time brightness threshold is greater than the real-time reference brightness; Identify the brightness of the target object in the projected object image to obtain the object projection brightness; After the projected brightness of the object is less than the real-time brightness threshold, the projection power of the projection unit is increased, and the increased projection power is used as the target adjustment parameter. After the projected brightness of the object exceeds the real-time brightness threshold, the projection power of the projection unit is reduced, and the reduced projection power is used as the target adjustment parameter.

7. A head-mounted device, characterized in that, The head-mounted device includes a projection unit and a control unit, the control unit including: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the article display method as described in any one of claims 1 to 6.

8. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the article display method as described in any one of claims 1 to 6.

9. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the article display method as described in any one of claims 1 to 6.

Images