Display method, device, equipment and storage medium of near-eye display device

Abstract

The application relates to the interactive technical field and provides a display method and device of a near-eye display equipment, equipment and a storage medium, which comprises the following steps: obtaining a target image obtained by shooting a target object, the target object being display content or a real object of a terminal equipment other than the near-eye display equipment, the target image comprising a first image and a second image, the first image and the second image being different images of the same target object; performing modeling processing on the target object according to the first image to obtain a three-dimensional model of the target object; performing matching degree evaluation on the second image and the three-dimensional model of the target object to obtain a matching degree evaluation result; when the matching degree evaluation result is greater than a preset similarity threshold, obtaining preset environment three-dimensional size data through the near-eye display equipment, displaying the three-dimensional model of the target object in the preset environment in the near-eye display equipment based on the preset environment three-dimensional size data, and improving the intuitive nature and accuracy of the adaptability evaluation of the near-eye display equipment on the target object and the preset environment.

Description

Technical Field

[0001] This application relates to the field of interactive technology, and in particular to a display method, apparatus, device and storage medium for a near-eye display device. Background Technology

[0002] Currently, with economic development, people's needs for goods are becoming more and more diverse. When purchasing goods, users often only see the rendered pictures or sample layouts provided by merchants for some non-wearable home items, but they do not know whether the size and style of the items are suitable for their home environment. Therefore, it is easy to encounter the inconvenience of buying unsuitable products and having to return and refund them. Summary of the Invention

[0003] The main objective of this application is to provide a display method, apparatus, device, and storage medium for a near-eye display device, aiming to solve the technical problem that the near-eye display device cannot assess whether the size and style of an object are suitable for one's home environment, resulting in poor intuitiveness in assessing the adaptability of the target object to the preset environment.

[0004] In a first aspect, this application provides a display method for a near-eye display device, comprising:

[0005] Acquire a target image captured by a target object, wherein the target object is the display content or physical object of a terminal device other than the near-eye display device, and the target image includes a first image and a second image, wherein the first image and the second image are different images of the same target object;

[0006] Based on the first image, the target object is modeled to obtain a three-dimensional model of the target object;

[0007] A matching degree evaluation is performed on the second image and the 3D model of the target object to obtain the matching degree evaluation result between the 3D model of the target object and the target object.

[0008] When the matching degree evaluation result is greater than or equal to a preset similarity threshold, the preset three-dimensional size data of the environment is obtained through the near-eye display device, and the three-dimensional model of the target object in the preset environment is displayed in the near-eye display device based on the preset three-dimensional size data of the environment.

[0009] Secondly, this application provides a display device for a near-eye display device, the display device comprising:

[0010] The image acquisition module is used to acquire a target image captured by a target object, wherein the target object is the display content or physical object of a terminal device other than the near-eye display device, and the target image includes a first image and a second image, wherein the first image and the second image are different images of the same target object;

[0011] The modeling module is used to perform modeling processing on the target object based on the first image to obtain a three-dimensional model of the target object;

[0012] The matching degree evaluation module is used to evaluate the matching degree between the second image and the three-dimensional model of the target object, and obtain the matching degree evaluation result between the three-dimensional model of the target object and the target object.

[0013] The model display module is used to acquire preset environmental three-dimensional size data through the near-eye display device when the matching degree evaluation result is greater than or equal to a preset similarity threshold, and display the three-dimensional model of the target object in the preset environment in the near-eye display device based on the preset environmental three-dimensional size data.

[0014] Thirdly, this application provides a near-eye display device, which includes a memory and a processor;

[0015] The memory is used to store computer programs;

[0016] The processor is configured to execute the computer program and, when executing the computer program, implement the steps of the display method for the near-eye display device as described above.

[0017] Fourthly, this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the display method for a near-eye display device as described above.

[0018] This application provides a display method, apparatus, device, and storage medium for a near-eye display device. The display method includes: acquiring a target image captured by a photograph of a target object, wherein the target object is the display content or physical object of a terminal device other than the near-eye display device, and the target image includes a first image and a second image, wherein the first image and the second image are different images of the same target object; performing modeling processing on the target object based on the first image to obtain a three-dimensional model of the target object; performing a matching degree evaluation on the second image and the three-dimensional model of the target object to obtain a matching degree evaluation result between the three-dimensional model of the target object and the target object; and when the matching degree evaluation result is greater than or equal to a preset similarity threshold, acquiring preset environmental three-dimensional size data through the near-eye display device, and displaying the three-dimensional model of the target object in the preset environment on the near-eye display device based on the preset environmental three-dimensional size data.

[0019] When a target image of a target object is acquired by a near-eye display device, since the target image includes a first image and a second image, which are different images of the same target object, the near-eye display device can determine the 3D model of the target object based on the first image. The near-eye display device can also determine the matching degree evaluation result between the 3D model of the target object and the target object based on the second image, to assess whether the 3D model of the target object matches the target object. When the matching degree evaluation result is greater than or equal to a preset similarity threshold, it can be determined that the 3D model of the target object matches the entity of the target object. If the 3D model of the target object matches the target object, the near-eye display device can display the 3D model of the target object in the preset environment based on the acquired 3D dimension data of the preset environment. The 3D model of the target object in the preset environment displayed by the near-eye display device is interactive for users to simulate trying out the target object in the preset environment, which helps improve the intuitiveness of the near-eye display device's assessment of the adaptability between the target object and the preset environment. Correspondingly, when the matching degree evaluation result is greater than or equal to the preset similarity threshold, it can be ensured that the virtual trial experience when the user virtually tries out the 3D model of the target object through the near-eye display device is similar to or the same as the user's actual trial experience when trying out the target object. This can reduce the situation where users return and refund the target object after purchasing it because the target object is not compatible with the preset environment, thereby improving the accuracy of the near-eye display device's assessment of the compatibility between the target object and the preset environment. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic flowchart of a display method for a near-eye display device provided in an embodiment of this application;

[0022] Figure 2 This is a schematic block diagram of a display device for a near-eye display apparatus provided in an embodiment of this application;

[0023] Figure 3 This is a schematic block diagram of a near-eye display device provided in an embodiment of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0026] This application provides a display method, apparatus, device, and storage medium for a near-eye display device. The display method for this near-eye display device can be applied to near-eye display devices. Near-eye display devices may include augmented reality (AR) glasses, mixed reality (MR) glasses, AR helmets, MR helmets, etc., and are not limited thereto. The display method for this near-eye display device can also be applied to a server, which can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0027] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0028] Please see Figure 1 , Figure 1 This is a schematic flowchart illustrating a display method for a near-eye display device according to an embodiment of this application. It should be noted that the display method for a near-eye display device provided in this embodiment can be used in a near-eye display device or a server, and is not limited thereto.

[0029] like Figure 1 As shown, the display method of the near-eye display device includes steps S101 to S104.

[0030] S101. Acquire a target image captured by a camera. The target object is the display content or physical object of a terminal device other than a near-eye display device. The target image includes a first image and a second image, which are different images of the same target object.

[0031] For example, the near-eye display device has a display function. The near-eye display device can display visually visible content by invoking relevant components. While wearing the near-eye display device, the user can not only view the displayed content but also see the corresponding preset environment. In this application, the preset environment corresponding to the near-eye display device refers to the three-dimensional environment in which the user is located when wearing the near-eye display device, including the geometric structure, topological relationships, and spatial layout of the three-dimensional space.

[0032] During the use of a near-eye display device, if the user has a virtual trial requirement for an object outside of a preset environment, the near-eye display device can identify that object as the target object. In this application, the target object includes the display content or physical object of a terminal device other than the near-eye display device.

[0033] In some implementations, the target object can be the display content of a terminal device other than a near-eye display device. Terminal devices other than near-eye display devices include, for example, mobile phones and laptops, and are not limited thereto. The near-eye display device can determine the target object based on the display content of the terminal device. For example, the display content of the terminal device can include web pages, application interfaces, social media platforms, video platforms, etc. Taking a web page as an example, the web page can display different product images. If the near-eye display device determines that a user has a virtual trial need for a product indicated in a certain product image displayed on the terminal device, then the near-eye display device can determine the product indicated by that product image as the target object. It should be noted that the web page in this application includes not only computer-based web pages, but also application-type apps, aggregated software applications, and information aggregation applications, etc.

[0034] In other embodiments, the objects in the preset environment corresponding to the near-eye display device may include physical objects in a real-world scene. These physical objects include, for example, sofas, televisions, washing machines, etc., and are not limited thereto. The near-eye display device can determine the target object based on the displayed content of the physical objects in the preset environment.

[0035] For example, when a user is shopping in a physical store, if it is necessary to determine whether the size, shape, and color scheme of the product match the preset environment, the near-eye display device can identify the product as the target object.

[0036] For example, during the use of a near-eye display device, if the user has a virtual trial requirement for a certain object, the user can output a first operation command to the near-eye display device. The first operation command may include a first gesture command, a first head movement command, a first eye movement command, a first voice command, a first button command, a first touch command, etc., and is not limited thereto. The first operation command can be pre-set or user-defined, and is not limited thereto. Upon detecting the user's first operation command, the near-eye display device can identify the object indicated by the first operation command as the target object.

[0037] Taking an example where the object includes a webpage or a physical object, and the first operation instruction is a first gesture instruction, the webpage might display a product image of product A, and the physical object might be a physical product displayed in a shopping mall. When the near-eye display device detects that the user has outputted a first gesture instruction, and the object indicated by the first gesture instruction is product A, it can determine that the user has a virtual trial need for product A, and thus product A can be identified as the target object. Correspondingly, if the object indicated by the first gesture instruction is a physical object, the physical object can be identified as the target object.

[0038] In another embodiment of the application, the object includes a webpage or a physical object, and the first operation instruction is a first eye-tracking instruction. The product image on the webpage may include, for example, a product image of product A, and the physical object may be a physical product displayed in a shopping mall. When the near-eye display device detects that the user outputs a first eye-tracking instruction to the product image of product A, it can identify product A as the target object. Correspondingly, when the near-eye display device detects that the user outputs a first eye-tracking instruction to a physical object, it can identify the physical object as the target object. The first eye-tracking instruction can be determined based on the user's first eye-tracking operation on an object within a preset environment corresponding to the near-eye display device. The first eye-tracking operation may include, for example, gazing, blinking, and gazing followed by blinking. Gazing refers to focusing on a single target object's image or physical object for more than a preset time threshold, or multiple gazing actions on the same target object's image or physical object within a preset time range; this is not limited here.

[0039] Of course, there are other ways to determine the target object, and we will not limit them here.

[0040] Once a target object is identified, the near-eye display device can acquire a target image captured by the target object. The target image can be obtained by directly capturing the target object using the near-eye display device; it can also be acquired by the near-eye display device from other electronic devices with imaging capabilities; or it can be obtained by the near-eye display device identifying the target object to obtain key information, and then acquiring the target object image via a network using that key information.

[0041] In some implementations, the near-eye display device can capture images of the displayed content of the terminal device or real objects in a real-world scene to obtain a target image of the target object; alternatively, the near-eye display device can obtain key information and web page links by capturing images with a camera or scanning QR codes, thereby connecting to the internet to obtain the target object image.

[0042] For example, when a target object is identified, the near-eye display device can, in response to a second operation command from the user to the near-eye display device or the object, capture images of the displayed content or the object to obtain a target image of the target object. The target images of the target object can be different images of the same target object.

[0043] Taking the capture of the display content of a terminal device as an example, the target image may include at least one of the following: images related to the target object and text information related to the target object. Images related to the target object may include images from different perspectives such as the left view, front view, and top view of the target object, as well as images of the target object in its usage scenario, etc., without limitation. Text information related to the target object may include the size information, material information, and color scheme information of the target object, etc., without limitation.

[0044] Furthermore, taking the example of a product displayed on an application or webpage on a terminal device as the target object, once the target object is determined (e.g., the near-eye display device determines the target object in response to a user's first operation command), the display content of the terminal device can change from the application or webpage to the product's detailed description interface. For example, the near-eye display device and the terminal device can establish a connection. Upon receiving the first operation command, the near-eye display device can control the terminal device to change its display content based on the connection. Alternatively, the user's first operation command can be determined based on the user's operation on the terminal device; in this case, the terminal device can change its display content in response to the user's operation, without limitation. If the user has a need to browse the product's detailed description interface, the user may perform a browsing operation on the interface. The near-eye display device can detect whether a second operation command triggered by the user exists based on this browsing operation. Upon detecting a second operation command, the near-eye display device can respond by capturing or extracting text information related to the target object from the product's detailed description interface to obtain the target image of the product. The target image for a product can include the product's detailed description interface, such as images related to the product, text information related to the product, etc. There are no restrictions on this.

[0045] Taking the photographing of real objects in a real-world scene as an example, the target image can include a stereoscopic view of the target object, pictures from different perspectives such as left view, right view, and front view, and the text information related to the target object can include the target object's size information, the target object's material information, and the sign information containing the target object's color scheme information.

[0046] The second operation instructions involved in determining the target image of the target object may include second gesture instructions, second head movement instructions, second eye movement instructions, second voice instructions, second button instructions, second touch instructions, etc., and are not limited here. The second operation instructions may be preset or set by the user, and are not limited here. The second operation instructions may be different from or the same as the first operation instructions, and are not limited here.

[0047] In one embodiment of this application, the second operation instruction can be empty. The near-eye display device can determine the target object in response to the user's first operation instruction to the near-eye display device, and then capture the display content of the terminal device or the real object in the real scene to obtain the target image of the target object.

[0048] In one embodiment of this application, when a target object is identified, the near-eye display device can further utilize web search and / or artificial intelligence (AI) search functions to obtain a detailed description interface of the target object, which can then be displayed. Correspondingly, the near-eye display device can, based on the displayed detailed description interface, remind the user to take a picture of the target object, or automatically take a picture of the target object or capture text information related to the target object, thereby obtaining a target image of the target object.

[0049] Taking a physical object as an example, the near-eye display device can respond to a first operation command by capturing a product image or product description image of the physical object, where the product description image contains text information related to the physical object. The near-eye display device can then utilize web search and / or AI search functions to retrieve the product's detailed description interface based on the product image or product description image. If the detailed description interface is retrieved, the near-eye display device can determine the target image of the physical object based on it. For example, the near-eye display device can automatically acquire the product's detailed description interface. Alternatively, the near-eye display device can trigger a second operation command when it detects a user browsing the product's detailed description interface, and then respond to the second operation command to acquire the product's detailed description interface and obtain the target image of the physical object. Of course, this is not a limitation and is not specified here.

[0050] Once the target image of the target object is determined, the target image of the target object can be used by the near-eye display device to perform subsequent modeling processing on the target object to obtain a three-dimensional model of the target object.

[0051] In some implementations, to ensure that the target images of the target objects are the same target object, the near-eye display device can use computer vision algorithms to extract and match feature points in different target images to verify whether there is geometric consistency between the different target images. For example, the near-eye display device can use a deep learning model, such as a pre-trained convolutional neural network (CNN), to extract the global feature vectors of each of the different target images, and calculate the similarity between the different target images based on their respective global feature vectors. If the similarity between the different target images is greater than or equal to a preset similarity threshold, the near-eye display device can determine that the different target images have geometric consistency, and thus retain the corresponding target image. If the similarity between the different target images is less than the preset similarity threshold, the near-eye display device can determine that the different target images do not have geometric consistency, and thus retain the corresponding target image.

[0052] In one exemplary embodiment, the near-eye display device may divide the retained target image into a first image and a second image, the first image and the second image may include different images of the same target object.

[0053] In another exemplary embodiment, the near-eye display device can display a reserved target image to prompt the user to select at least one of a first image and a second image from the target image. The near-eye display device can determine the first image based on the user's selection of the first image. The near-eye display device can also determine the second image based on the user's selection of the second image. Accordingly, if the near-eye display device determines the first image based on the user's selection of the first image, it can also determine a target image other than the first image as the second image. And so on.

[0054] In yet another exemplary embodiment, the near-eye display device can display all target images. The near-eye display device can use a user interaction and feedback mechanism to indicate the similarity between each target image and other target images, thereby instructing the user to select a suitable target image as at least one of the first image and the second image.

[0055] When acquiring a target image of a target object captured by a near-eye display device, the first image in the target image can be used by the near-eye display device to determine the 3D model of the target object, and the second image in the target image can be used by the near-eye display device to verify or adjust at least one of the 3D model of the target object. This facilitates improving the ease of modeling, verification, and adjustment of the target object. Correspondingly, if the first and second images are different images of the same target object, it can be ensured that all target images acquired by the near-eye display device involve the target object. Therefore, when determining, verifying, or adjusting the 3D model of the target object based on the target images, it helps improve the accuracy of modeling, verification, and adjustment of the target object.

[0056] S102. Based on the first image, perform modeling processing on the target object to obtain a three-dimensional model of the target object.

[0057] For example, the first image can be used to model the target object to determine a 3D model of the target object. The number of first images may include multiple images, and there is no limitation on this.

[0058] In some implementations, the first image may include an image related to the target object. For example, the first image may include a left view, a front view, and a top view of the target object. The near-eye display device can determine the positions of key points of the target object in the left view, front view, and top view, respectively, based on these views. Correspondingly, the near-eye display device can determine the geometric relationships of the key points in the left view, front view, and top view based on their positions. Furthermore, the near-eye display device can determine the structural features of the target object based on these geometric relationships. The near-eye display device can then model the target object based on its structural features to obtain a three-dimensional model of the target object. Of course, the first image is not limited to the left view, front view, and top view of the target object, and this is not a limitation here.

[0059] Taking a sofa as an example, where the target object includes a sofa, and the first image includes the sofa's left view, front view, and top view, the near-eye display device can determine the sofa's structural features based on these views. Correspondingly, the near-eye display device can model the sofa based on its structural features to obtain a 3D model of the target object. Of course, the target object and the first image are not limited to this; no restrictions are imposed here.

[0060] During the process of modeling the target object based on the first image, as the number of first images increases, such as the number of images of the target object from different perspectives, the near-eye display device can capture more detailed information of the target object from the first image from different perspectives. This reduces the adverse effects on the accuracy of modeling the target object due to missing or occluded perspectives, and thus helps to improve the accuracy of modeling the target object.

[0061] In some implementations, the first image may include an image related to the target object and text information related to the target object. For example, if the first image includes a front view of the target object, and the text information included in the first image can be used to indicate the size information of the target object, the near-eye display device can perform modeling processing on the target object based on the front view and the size information of the target object to obtain a three-dimensional model of the target object. For instance, the near-eye display device can determine the two-dimensional structural feature information of the target object based on the front view. Correspondingly, the near-eye display device can determine the three-dimensional structural feature information of the target object based on the two-dimensional structural feature information and the size information of the target object. The near-eye display device determines the three-dimensional structural feature information of the target object as the structural features of the target object, and then performs modeling processing on the target object based on the structural features of the target object to obtain a three-dimensional model of the target object. Of course, the first image is not limited to this, and no limitation is made here.

[0062] Taking a sofa as an example, where the target object includes a front view of the sofa, and the first image of the target object also includes the sofa's dimensions. The sofa's dimensions can include its overall length, width, and height, as well as its local length, width, and height. The near-eye display device can determine the sofa's two-dimensional structural features based on the front view. Simultaneously, based on the sofa's dimensions and its two-dimensional structural features, it can obtain the sofa's three-dimensional structural features. With the sofa's three-dimensional structural features determined, the near-eye display device can model the sofa to obtain a corresponding three-dimensional model of the target object. Of course, the target object and the first image are not limited to this; no restrictions are imposed here.

[0063] In the process of modeling a target object based on the first image, near-eye display devices, even with only a limited number of detailed images captured from the target object's perspective, can obtain a 3D model of the target object using the size information determined from the first image. This improves the ease of modeling the target object. Correspondingly, as the number of first images required for modeling decreases, it helps save computing power and hardware resources needed for modeling the target object, thereby increasing the modeling speed.

[0064] When a near-eye display device models a target object based on a first image to obtain a 3D model of the target object, it improves the ease of modeling the target object. The 3D model of the target object allows the near-eye display device to perform a virtual trial of the target object, and further enables the near-eye display device to evaluate whether the target object is compatible with a preset environment, thus improving the ease of assessment of the near-eye display device's compatibility with the target object and the preset environment.

[0065] S103. Evaluate the matching degree between the second image and the 3D model of the target object to obtain the matching degree evaluation result between the 3D model of the target object and the target object.

[0066] For example, when a user needs to assess whether a target object is suitable for a preset environment—that is, when the user has a virtual trial requirement for the target object—the 3D model of the target object is compared with the target object to determine its compatibility. If the compatibility assessment result reaches or exceeds a preset threshold, the 3D model of the target object is considered a match. At this point, the user can virtually try out the 3D model of the target object using a near-eye display device. Based on the virtual trial experience, the user can assess whether the target object is suitable for the preset environment and thus decide whether to purchase it. For instance, if the 3D model of the target object matches the target object, and the user determines that the 3D model of the target object is suitable for the preset environment corresponding to the near-eye display device, the user can purchase the target object. Conversely, if the 3D model of the target object does not match the target object, there is no need to further determine whether the target object itself is suitable for the preset environment corresponding to the near-eye display device, nor is it necessary to determine whether to purchase the target object.

[0067] For example, a second image can be used to assess whether a 3D model of the target object matches the target object. The number of second images may include one or more, and there is no limitation herein.

[0068] In some implementations, the second image may include at least one of a picture associated with the target object and text information associated with the target object. The near-eye display device can obtain a matching degree evaluation result between the target object's 3D model and the target object by performing a matching degree evaluation on the second image and the target object's 3D model.

[0069] In some implementations, the near-eye display device can acquire first feature information of key points in a target object in a second image based on the second image. Correspondingly, the near-eye display device can acquire second feature information of the same key point on the three-dimensional model of the target object. The first and second feature information of the key point can include the positional features and appearance features of the key point. Positional features can include the position of the key point, the spatial relationship between the key point and other key points, etc., without limitation. Appearance features can include the morphological features and texture features of the key point. The morphological features of the key point include its shape, position, and number, while the texture features include the shape of the texture, the distribution of the texture, and the direction of the texture, without limitation. After acquiring the first and second feature information of the key point of the target object, the near-eye display device can compare whether they match. If the number of key points matching the first and second feature information is greater than or equal to a first quantity threshold, the near-eye display device can determine that the matching degree evaluation result is greater than or equal to a preset similarity threshold. If the number of key points matching the first feature information and the second feature information is less than a first quantity threshold, the near-eye display device can determine that the matching degree evaluation result is less than a preset similarity threshold. The first quantity threshold can be preset or set by the user. For example, the first quantity threshold can be determined based on the number of key points of the target object, and there is no restriction on this.

[0070] In other embodiments, the second image includes an image related to the target object. The near-eye display device can project the 3D model of the target object onto different reference planes to obtain model images of the 3D model of the target object from different viewpoints. The near-eye display device can acquire a model image from the corresponding viewpoint based on the viewpoint corresponding to the second image. Accordingly, the near-eye display device can evaluate the similarity between the second image and the model image from the same viewpoint. For example, the near-eye display device can determine the similarity between the second image and the model image from the same viewpoint using a preset image comparison algorithm. Accordingly, the near-eye display device can determine the matching degree evaluation result between the 3D model of the target object and the target object based on the similarity between the second image and the model image from multiple viewpoints. For example, if the number of second images with a similarity greater than or equal to a preset similarity threshold is greater than or equal to a second quantity threshold, the near-eye display device can determine that the matching degree evaluation result is greater than or equal to the preset similarity threshold. As another example, if the number of second images with a similarity greater than or equal to the preset similarity threshold is less than the second quantity threshold, the near-eye display device can determine that the matching degree evaluation result is less than the preset similarity threshold. The preset similarity threshold can be pre-set or user-defined, without restriction. The second quantity threshold can also be pre-set or user-defined; for example, it can be determined based on the number of second images, without restriction. However, it is not limited to this. In determining the similarity between each second image and its corresponding model image, the near-eye display device can also determine one of the mean, mode, or median of the similarity between all the second images to determine whether one of these values ​​is greater than or equal to the preset similarity threshold. If one of these values ​​is greater than or equal to the preset similarity threshold, the near-eye display device can, for example, determine that the matching evaluation result is greater than or equal to the preset similarity threshold.

[0071] After evaluating the matching degree between the second image and the 3D model of the target object, and obtaining the matching degree evaluation result between the 3D model of the target object and the target object, the matching degree evaluation result can be used by the near-eye display device to determine whether to execute the virtual trial function of the target object. For example, if the matching degree evaluation result is greater than or equal to a preset similarity threshold, the near-eye display device can determine that the 3D model of the target object matches the target object, and thus can execute the virtual trial function of the target object. If the matching degree evaluation result is less than the preset similarity threshold, the near-eye display device can determine that the 3D model of the target object does not match the target object, and thus can not execute or temporarily not execute the virtual trial function of the target object. During the virtual trial of the target object, if the user's virtual trial experience of the 3D model of the target object can match the user's physical trial experience of the target object, it will help improve the intuitiveness and accuracy of the near-eye display device's assessment of the adaptability of the target object to the preset environment.

[0072] S104. When the matching degree evaluation result is greater than or equal to the preset similarity threshold, the preset three-dimensional size data of the environment is obtained through the near-eye display device, and the three-dimensional model of the target object in the preset environment is displayed in the near-eye display device based on the preset three-dimensional size data of the environment.

[0073] If the 3D model of the target object is determined to be greater than or equal to a preset similarity threshold based on the matching evaluation results, the near-eye display device can perform a virtual trial function for the target object. For example, the near-eye display device can acquire the 3D dimension data of a preset environment and, based on this data, display the 3D model of the target object within that environment. This allows the user to intuitively experience and assess whether the target object is suitable for the preset environment. Because the 3D model matches the target object, the user's virtual trial of the target object using the near-eye display device is equivalent to trying out the physical object, thus providing a basis for purchasing decisions. This improves the accuracy of the user's assessment of the compatibility between the target object and the preset environment, reducing costs such as money and time incurred due to the incorrect purchase of unsuitable goods.

[0074] Specifically, when a user is in a preset environment, the near-eye display device projects a 3D model image of the target object into their eyes, achieving visual fusion between the 3D model and the preset environment, thus enabling a virtual trial of the target object. Simultaneously, the user can adjust the position and angle of the 3D model, allowing for a trial experience within the preset environment. This ensures that the virtual trial experience is similar to or identical to the actual trial experience, improving the accuracy of the virtual trial. Based on this, when the 3D model matches the target object, the user's assessment of whether the 3D model fits the preset environment is equivalent to assessing whether the actual object fits the preset environment. Therefore, the user can decide whether to purchase the target object based on the virtual trial experience of the 3D model.

[0075] In some implementations, the first positional correspondence of key points of the target object in multiple first images is determined; the structural features of the target object are determined based on the first positional correspondence; and the target object is modeled based on the structural features to obtain a three-dimensional model of the target object.

[0076] For example, key points of the target object may include the outline points, center points, etc. of the target object, without limitation.

[0077] For example, when a near-eye display device acquires multiple first images, these multiple first images may include the same key point of the target object. The near-eye display device can determine the first positional correspondence of the same key point of the target object in the multiple first images.

[0078] Taking a cuboid as the target object, and the first image including the left view, front view, and top view of the cuboid as an example. The cuboid may include multiple key points. The near-eye display device can determine that the key point located in the upper left of the front view of the cuboid and the key point located in the upper right of the left view of the cuboid are the same key point, and the key point located in the upper left of the front view of the cuboid and the key point located in the lower left of the top view of the cuboid are the same key point. Therefore, based on the positions of the same key point of the cuboid in the front view, left view, and top view of the cuboid, the first positional correspondence of the same key point of the cuboid in the front view, left view, and top view can be determined. By analogy, the near-eye display device can determine the first positional correspondence of multiple key points of the target object in multiple first images.

[0079] Once the first positional correspondence is determined, the near-eye display device can determine the structural features of the target object based on the first positional correspondence.

[0080] Taking a cuboid as the target object, and including keypoints A and B, as an example: If both keypoints A and B correspond to a keypoint located in the upper right corner of the left view of the cuboid, keypoint A corresponds to a keypoint located in the upper left corner of the front view of the cuboid, and keypoint B corresponds to a keypoint located in the upper right corner of the front view of the cuboid, then the near-eye display device can determine that keypoints A and B are different keypoints, and that keypoint A is located to the left of keypoint B in both the front and top views of the cuboid. Accordingly, the near-eye display device can determine the geometric relationship between keypoints A and B based on their positional relationship in different views. Similarly, the near-eye display device can determine the geometric relationship between multiple keypoints of the target object based on the first positional correspondence of each keypoint in multiple first images.

[0081] Accordingly, near-eye display devices can determine the structural features of a target object by integrating the geometric relationships between multiple key points. For example, near-eye display devices can utilize principles such as triangulation to determine the structural features of a target object based on the geometric relationships between multiple key points. The structural features of the target object can include its three-dimensional structural information.

[0082] Given the structural features of the target object, the near-eye display device can use these features to model the target object and obtain a three-dimensional model of it.

[0083] When the correspondence between the key points of a target object and their corresponding positions in multiple first images is determined, this correspondence can be used to determine the structural features of the target object, thus improving the ease of determining these features for near-eye display devices. The structural features of the target object can then be used by the near-eye display device to model the object and obtain a 3D model, further enhancing the ease of modeling the target object.

[0084] In some implementations, based on a preset deep learning algorithm, the first image is subjected to appearance feature extraction processing to obtain the appearance features of the target object; based on a preset natural language processing algorithm, the first image is subjected to text feature extraction processing to obtain the text features of the target object.

[0085] Based on the structural features, appearance features, and text features of the target object, a modeling process is performed on the target object to obtain a three-dimensional model of the target object.

[0086] For example, a target object can have appearance features. The appearance features of a target object can include the morphological features of key points, texture features, etc. Among them, the morphological features of key points include the shape, position, and number of key points, and the texture features of key points include the shape of the texture, the distribution of the texture, the direction of the texture, etc., without limitation.

[0087] In the process of modeling a target object, near-eye display devices can determine the appearance features of the target object to determine the appearance of the three-dimensional model of the target object.

[0088] Near-eye display devices can utilize preset deep learning algorithms to extract appearance features from a first image to obtain the appearance features of the target object. Deep learning algorithms can include Neural Radiance Fields (NeRF) models, improved models of NeRF models, Recurrent MVSNet for High-resolution Multi-view Stereo Depth Inference (R-MVSNet), etc., and are not limited here.

[0089] Taking a sofa as an example, the target object can include fabric sofas, leather sofas, etc., which may have corresponding textures and wrinkles. Near-eye display devices can use preset deep learning algorithms to extract appearance features such as textures and wrinkles to obtain the sofa's appearance characteristics. Of course, the target object is not limited to sofas; it can also include fabric curtains, window curtains, etc., without further limitation.

[0090] Given a defined appearance feature of the target object, a near-eye display device can combine the object's structural and appearance features to model it, resulting in a 3D model. For example, the texture of the 3D model can match the texture of the actual object, and the wrinkles in the 3D model can match the wrinkles in the actual object. Therefore, the 3D model can reflect the appearance of the target object entity, such as texture and wrinkles, allowing the appearance features of the 3D model to match the appearance of the actual object, thereby improving the accuracy of the near-eye display device's modeling of the target object.

[0091] For example, the target object may possess textual features. These textual features can be determined based on textual information related to the target object included in the first image. For instance, the textual information related to the target object may include its size information, material information, etc., without limitation. The target object's size information, color scheme, texture, etc., can be used by the near-eye display device to model the target object.

[0092] During the modeling process of a target object, near-eye display devices can determine the text features of the target object to enable the near-eye display device to determine the three-dimensional model of the target object.

[0093] Near-eye display devices can use preset natural language processing algorithms to extract text features from the first image to obtain the text features of the target object.

[0094] Taking a table as an example, near-eye display devices can use pre-defined natural language processing algorithms to extract textual features of the table, such as its size, color scheme, and texture. Of course, the target object is not limited to a table, and this is not a limitation here.

[0095] Given the defined textual features of a target object, a near-eye display device can combine the object's structural and textual features to model it, resulting in a 3D model. For example, the dimensions of the 3D model can match the dimensions of the actual object, and the material of the 3D model can match the color scheme and texture of the actual object. Therefore, the 3D model can reflect textual information related to the target object entity, such as dimensions, color scheme, and texture, which improves the matching degree between the 3D model and the actual object, thereby enhancing the accuracy of the near-eye display device's modeling of the target object.

[0096] Near-eye display devices can also integrate the structural features, appearance features, and text features of the target object to model the target object and obtain a three-dimensional model of the target object.

[0097] Based on this, when a near-eye display device determines at least one of the appearance features and text features of the target object, it can combine the structural features of the target object to model the target object and obtain a three-dimensional model of the target object, which is beneficial to improving the accuracy of modeling the target object.

[0098] In some implementations, the key points of the target object are determined to have a second positional correspondence in the second image and the three-dimensional model of the target object; based on a preset feature comparison algorithm, the second image and the three-dimensional model of the target object are compared according to the second positional correspondence to obtain the feature comparison result corresponding to the three-dimensional model of the target object; the three-dimensional model of the target object is reprojected according to the camera parameters of the near-eye display device to obtain a third image corresponding to the three-dimensional model of the target object; a reprojection error evaluation algorithm is used to determine the pixel position error of the target object in the second image and the third image; and the matching degree evaluation result between the three-dimensional model of the target object and the target object is determined according to the feature comparison result and the pixel position error.

[0099] For example, a near-eye display device can determine the position of the same key point of a target object in both a second image and a three-dimensional model of the target object. Accordingly, the near-eye display device can determine a second positional correspondence between the key points of the target object in the second image and the three-dimensional model of the target object based on the positions of the same key point of the target object in both the second image and the three-dimensional model of the target object.

[0100] The second image may include an image related to the target object. Upon acquiring the second image, the near-eye display device can obtain the viewpoint corresponding to the second image. For example, the near-eye display device can estimate the viewpoint of the second image to obtain the viewpoint corresponding to the second image. Accordingly, based on the viewpoint corresponding to the second image, the near-eye display device can determine the model image of the target object's 3D model under the viewpoint corresponding to the second image. Furthermore, the near-eye display device can perform feature extraction and feature matching on the second image and the model image of the target object's 3D model under the viewpoint corresponding to the second image to obtain the second positional correspondence between the key points of the target object in the second image and the target object's 3D model.

[0101] For example, if the second image includes a side view of the target object, the near-eye display device can acquire a model image of the target object's 3D model that includes the side view of the target object's 3D model. The near-eye display device can perform feature extraction and feature matching on the second image and the side view of the target object's 3D model to determine the second positional correspondence between the key points of the target object in the second image and the target object's 3D model.

[0102] Given that the key points of the target object are determined to correspond to the second position in the second image and the three-dimensional model of the target object, the near-eye display device can perform feature comparison on the second image and the three-dimensional model of the target object based on a preset feature comparison algorithm and according to the second position correspondence.

[0103] Taking key point C of the target object as an example. Based on the position of key point C in the second image (position 1) and its position in the 3D model of the target object (position 2), a second positional correspondence between key point C and the 3D model of the target object can be determined. The near-eye display device can obtain the first feature information of key point C in the second image based on position 1. Correspondingly, the near-eye display device can obtain the second feature information of key point C in the 3D model of the target object based on position 2. The near-eye display device can determine whether the first feature information and the second feature information of key point C match. For example, if the positional features of key point C in the 3D model of the target object match the positional features of key point C in the second image, and the appearance features of key point C in the 3D model of the target object match the appearance features of key point C in the second image, the near-eye display device can determine that the first feature information and the second feature information of key point C match. Similarly, near-eye display devices can determine whether the first feature information of each of the multiple key points of the target object in the second image matches its second feature information in the three-dimensional model of the target object, and thus determine the feature comparison result corresponding to the three-dimensional model of the target object.

[0104] For example, a near-eye display device can perform reprojection processing on a 3D model of a target object based on the camera parameters of the near-eye display device. For instance, the camera parameters of the near-eye display device can be used to indicate the geometric mapping relationship from the 3D space corresponding to the near-eye display device to the 2D image plane corresponding to the near-eye display device. The camera parameters of the near-eye display device can be understood as the camera parameters of the shooting module set on the near-eye display device, such as one or more of the camera intrinsic parameters, camera extrinsic parameters, and camera distortion of the shooting module. The camera parameters of the near-eye display device can be determined by calibration of the near-eye display device before it leaves the factory, and are not limited here. Based on the camera parameters, the near-eye display device can transform elements in the 3D model of the target object, such as points, lines, and surfaces, from the 3D space corresponding to the near-eye display device to the 2D image plane corresponding to the near-eye display device according to the camera imaging principle, obtaining a third image corresponding to the 3D model of the target object. The third image corresponding to the 3D model of the target object is essentially obtained by simulating the actual shooting process of the second image by the near-eye display device. Based on this, near-eye display devices can determine whether the second image matches the third image in order to determine whether the 3D model of the target object matches the target object.

[0105] For example, near-eye display devices can use reprojection error assessment to determine the pixel position error of a target object in the second and third images.

[0106] Taking keypoint C of the target object as an example. If keypoint C is located at position 1 in the second image and position 3 in the third image, the near-eye display device can calculate the pixel position error of keypoint C between the second and third images based on positions 1 and 3. For example, the near-eye display device can calculate the distance between positions 1 and 3 to obtain the pixel position error corresponding to keypoint C. Similarly, the near-eye display device can determine the pixel position errors of multiple keypoints of the target object in the second and third images, thereby determining the pixel position error of the target object in the second and third images. For example, the pixel position error of the target object in the second and third images can include the pixel position errors of multiple keypoints of the target object in the second and third images.

[0107] Given the feature comparison results and pixel position errors corresponding to the 3D model of the target object, the near-eye display device can determine whether the 3D model of the target object matches the target object based on the feature comparison results and pixel position errors, and obtain the matching degree evaluation result between the 3D model of the target object and the target object.

[0108] For example, if, based on feature comparison results, the number of key points in the target object whose first feature information matches the second feature information is greater than or equal to a first threshold, and the number of key points in the target object whose pixel position error is less than or equal to a preset position error threshold is also greater than or equal to the first threshold, the near-eye display device can determine that the matching degree evaluation result corresponding to the 3D model of the target object is greater than or equal to a preset similarity threshold, which is equivalent to determining that the 3D model of the target object matches the target object. Conversely, if, based on feature comparison results, the number of key points in the target object whose first feature information matches the second feature information is less than the first threshold, and / or the number of key points in the target object whose pixel position error is less than or equal to a preset position error threshold is less than the first threshold, the near-eye display device can determine that the matching degree evaluation result corresponding to the 3D model of the target object is less than a preset similarity threshold, which is equivalent to determining that the 3D model of the target object does not match the target object.

[0109] In determining the matching degree evaluation result corresponding to the 3D model of the target object, the near-eye display device can use the second image and the 3D model of the target object to determine the feature comparison result corresponding to the 3D model of the target object. The feature comparison result can be used by the near-eye display device to evaluate whether the 3D model of the target object matches the target object. Correspondingly, the near-eye display device can also perform reprojection processing on the 3D model of the target object to obtain a third image corresponding to the 3D model of the target object. The near-eye display device can use the second image and the third image to determine the pixel position error of the target object corresponding to the second image and the third image. The pixel position error can be used by the near-eye display device to evaluate whether the 3D model of the target object matches the target object. When the near-eye display device can comprehensively determine the matching degree evaluation result corresponding to the 3D model of the target object by combining the feature comparison result and the pixel position error, performing a secondary verification of whether the 3D model of the target object matches the target object helps to improve the accuracy of the near-eye display device in determining the matching degree evaluation result corresponding to the 3D model of the target object.

[0110] In some implementations, when the matching degree evaluation result is less than a preset similarity threshold, the deviation type corresponding to the target object's 3D model is determined based on the matching degree evaluation result; the target object's 3D model is adjusted according to the deviation type to obtain the adjusted target object's 3D model.

[0111] For example, if the matching degree assessment result is less than a preset similarity threshold, the near-eye display device can determine that the 3D model of the target object does not match the target object. Based on this, the near-eye display device can adjust the 3D model of the target object so that the adjusted 3D model matches the target object.

[0112] For example, near-eye display devices can determine the type of deviation corresponding to the 3D model of the target object based on the matching degree evaluation results, and then adjust the 3D model of the target object according to the type of deviation.

[0113] The deviation type corresponding to the 3D model of the target object can include a first deviation type and a second deviation type. The first deviation type indicates a local mismatch between the 3D model of the target object and the target object. The second deviation type indicates an overall mismatch between the 3D model of the target object and the target object.

[0114] The matching evaluation results include at least one of the following: feature comparison results corresponding to the 3D model of the target object and pixel position errors.

[0115] If the matching degree evaluation result is less than the preset similarity threshold, the near-eye display device can determine whether the matching degree evaluation result is less than the preset similarity threshold due to a local mismatch between the target object's 3D model and the target object, or due to an overall mismatch between the target object's 3D model and the target object, based on the feature comparison results included in the matching degree evaluation result.

[0116] The feature comparison result corresponding to the 3D model of the target object can be determined based on the first feature information of the key points of the target object in the second image and the second feature information of the key points in the 3D model of the target object. Taking the first and second feature information as including the positional features and appearance features of the key points as an example: For a key point in the target object whose first and second feature information do not match, if the positional features of the key point in the second image match the positional features of the key point in the 3D model of the target object, but the appearance features of the key point in the second image do not match the appearance features of the key point in the 3D model of the target object, then the near-eye display device can determine that the local part of the 3D model of the target object does not match the target object, and determine that the overall structure and positional relationship of the 3D model of the target object matches the target object. Therefore, the near-eye display device can determine that the deviation type corresponding to the 3D model of the target object is the first deviation type. Correspondingly, if the positional features of the key point in the second image do not match the positional features of the key point in the 3D model of the target object, the near-eye display device can determine that the overall structure and positional relationship of the 3D model of the target object does not match the target object. Therefore, the near-eye display device can determine that the deviation type corresponding to the 3D model of the target object is the second deviation type.

[0117] If the matching degree evaluation result is less than the preset similarity threshold, the near-eye display device can determine whether the matching degree evaluation result is less than the preset similarity threshold due to a local mismatch between the target object's 3D model and the target object, or due to an overall mismatch between the target object's 3D model and the target object, based on the pixel position error included in the matching degree evaluation result.

[0118] For example, if the number of keypoints in the target object whose pixel position error is greater than or equal to a preset position error threshold is greater than or equal to a first threshold but less than a second threshold, the near-eye display device can determine that the matching degree evaluation result is less than the preset similarity threshold because the pixel position error corresponding to a relatively small number of keypoints is large. Therefore, the near-eye display device can determine whether the local part of the target object's 3D model matches the target object, and whether the overall structure and positional relationship of the target object's 3D model matches the target object. Consequently, the near-eye display device can determine that the deviation type corresponding to the target object's 3D model is the first deviation type. The second threshold is greater than the first threshold. Correspondingly, if the number of keypoints in the target object whose pixel position error is greater than or equal to a preset position error threshold is greater than the second threshold, the near-eye display device can determine that the matching degree evaluation result is less than the preset similarity threshold because the pixel position error corresponding to a relatively large number of keypoints is large. Therefore, the near-eye display device can determine that the overall structure and positional relationship of the target object's 3D model does not match the target object, and consequently, the deviation type corresponding to the target object's 3D model is the second deviation type.

[0119] Of course, this is not limited to this. Near-eye display devices can comprehensively analyze the feature comparison results corresponding to the 3D model of the target object and the pixel position error to determine the deviation type of the 3D model of the target object. For example, if a key point in the target object has a mismatch between its first and second feature information, but its positional features on the second image match its positional features on the 3D model of the target object, but its appearance features on the second image do not match its appearance features on the 3D model of the target object, and the number of key points in the target object whose pixel position error is greater than or equal to a preset position error threshold is greater than or equal to a first threshold and less than a second threshold, the near-eye display device can determine that the deviation type of the 3D model of the target object is the first deviation type. As another example, if a key point in the target object has a mismatch between its first and second feature information, but its positional features on the second image do not match its positional features on the 3D model of the target object, and the number of key points in the target object whose pixel position error is greater than or equal to a preset position error threshold is greater than or equal to a second threshold, the near-eye display device can determine that the deviation type of the 3D model of the target object is the second deviation type.

[0120] Once the type of deviation corresponding to the 3D model of the target object is determined, the near-eye display device can subsequently adopt different model adjustment strategies based on different deviation types to adjust the 3D model of the target object and obtain the adjusted 3D model of the target object, which will help improve the ease of adjustment of the 3D model of the target object by the near-eye display device.

[0121] In some implementations, when the deviation type is the first deviation type, a preset editing model is used to perform local adjustment processing on the three-dimensional model of the target object to obtain the adjusted three-dimensional model of the target object.

[0122] When the deviation type is the first deviation type, the near-eye display device can determine that there is a local deviation between the 3D model of the target object and the actual target object. Then, the near-eye display device can automatically perform local adjustment processing on the 3D model of the target object using a preset editing model.

[0123] For example, editing models can be used to automatically move or modify elements such as points, lines, and surfaces of a target object's 3D model.

[0124] For example, when a near-eye display device determines that the deviation type corresponding to the 3D model of the target object is the first deviation type, the near-eye display device can refer to the second image and automatically use the editing model to move or modify the points, lines, surfaces, and other elements of the 3D model of the target object so that the adjusted 3D model of the target object can match the second image.

[0125] For example, when a user is browsing a lamp with a complex design, if the near-eye display device determines that the 3D model of the target object has a first-type modeling deviation at a certain decorative detail of the lamp, the local adjustment method can be used to make fine corrections to that area.

[0126] In some implementations, when the deviation type is the second deviation type, the target object is reconstructed based on the first image and the second image to obtain an adjusted three-dimensional model of the target object.

[0127] When the deviation type is the second deviation type, the near-eye display device can determine that the three-dimensional model of the target object has an overall deviation compared to the actual target object. Then, the near-eye display device can combine the second image with the first image to reconstruct the target object and obtain the adjusted three-dimensional model of the target object.

[0128] During the reconstruction of the target object, the near-eye display device can use a re-fusion reconstruction method to integrate the first image with the second image, and use a preset three-dimensional reconstruction algorithm or adjust the algorithm parameters to reconstruct the three-dimensional model of the target object, thus obtaining the adjusted three-dimensional model of the target object.

[0129] When the target object is reconstructed based on the first and second images, errors or omissions in the original 3D model of the target object can be corrected to obtain an adjusted 3D model of the target object, which helps to improve the accuracy of modeling the target object.

[0130] In some implementations, the near-eye display device can record information related to the adjustment of the target object's 3D model so that users can view the adjustment progress and effect of the target object's 3D model, ensuring that the final adjusted target object 3D model can meet the user's virtual usage needs, for example, allowing users to evaluate whether the target object is suitable for a preset environment.

[0131] When near-eye display devices can adjust the 3D model of a target object according to the type of deviation, they can adopt corresponding model adjustment strategies. This improves the convenience and flexibility of adjusting the 3D model of the target object, thereby enhancing the user's virtual experience of the target object.

[0132] In some implementations, a spatial measurement process is performed on a preset environment using a near-eye display device to obtain three-dimensional size data of the preset environment; based on a preset spatial positioning algorithm, a three-dimensional model of the target object is displayed on the display interface of the near-eye display device according to the three-dimensional size data of the preset environment.

[0133] The preset environment can include the actual space where the near-eye display device is located. The near-eye display device can have spatial measurement capabilities. For example, it can be equipped with a LiDAR (Light Detection and Ranging) radar, a depth camera, etc., without limitation. The near-eye display device can use the LiDAR, depth camera, etc., to perform three-dimensional measurements of the preset environment to obtain its three-dimensional dimensional data.

[0134] For example, a near-eye display device can use LiDAR (Light Detection and Ranging) to emit a laser beam towards an object in a preset environment and receive its reflected light. Based on the emission and return times of the laser beam, the distance between the near-eye display device and the object is calculated. The near-eye display device can then synthesize the distances between itself and the objects in the preset environment to generate 3D point cloud data of the preset environment. This 3D point cloud data can be used to determine the 3D dimensions of the preset environment.

[0135] For example, near-eye display devices can acquire depth information of objects in a preset environment using a depth camera, combined with projected structured light, binocular visual parallax, or by measuring the time-of-flight of modulated light. Accordingly, the near-eye display device can generate a depth image based on the object's depth information. The near-eye display device can then convert the depth image into 3D point cloud data, thereby determining the 3D dimensions of the preset environment corresponding to the near-eye display device.

[0136] Having acquired the 3D dimensions of a preset environment, a near-eye display device can utilize a preset spatial positioning algorithm, such as Simultaneous Localization and Mapping (SLAM), combined with the positional information of objects within the preset environment corresponding to the near-eye display device, to display the 3D model of the target object on the display interface of the near-eye display device. When the target object is displayed on the near-eye display device's interface, the 3D model of the target object can be integrated with the preset environment. For example, while viewing the 3D model of the target object displayed on the near-eye display device's interface, the user can also see the preset environment in which the near-eye display device is located, effectively allowing the user to visually perceive a spatial overlay of the 3D model of the target object and the preset environment. Based on this, the user can intuitively view the 3D model of the target object within the preset environment through the near-eye display device, enabling the user to assess the compatibility between the target object and the preset environment.

[0137] During the process of displaying a 3D model of a target object under a preset environment in a near-eye display device, the near-eye display device can update the display of the 3D model of the target object under the preset environment.

[0138] In some implementations, in response to user interaction with the near-eye display device, the three-dimensional model of the target object under a preset environment is updated and displayed in the near-eye display device.

[0139] For example, the interactive operation may include a preset rotation operation. The preset rotation operation may include rotation gestures, rotation voice commands, etc., and is not limited thereto. A rotation gesture may include a gesture used to instruct the rotation of a target 3D model. A rotation voice command may include a voice command used to instruct the rotation of a target 3D model. And so on.

[0140] For example, the interactive operation may include a pose adjustment command. When displaying a 3D model of a target object under a preset environment on a near-eye display device, the near-eye display device may, in response to a user's pose adjustment command, adjust the position or orientation of the 3D model of the target object to update the display of the 3D model of the target object under the preset environment on the near-eye display device.

[0141] When users view the 3D model of the target object and the updated 3D model of the target object, they can intuitively see the trial effect of the target object in the preset environment corresponding to the near-eye display device from different angles. This allows users to evaluate whether the target object is suitable for the preset environment, which helps to improve the intuitiveness and accuracy of the near-eye display device's assessment of the suitability of the target object and the preset environment.

[0142] Taking a television as an example, when a user is selecting a television, they can use a near-eye display device to display a 3D model of the television in a preset environment, allowing the user to virtually try out the television. Correspondingly, during this virtual trial, the preset environment can be determined based on the user's living room, allowing the user to view the visual effect of the television's 3D model alongside the living room walls and other furniture.

[0143] Correspondingly, during the process of displaying a 3D model of the target object (TV) in a preset environment on the near-eye display device, the user can also control the near-eye display device to rotate the 3D model of the target object through interactive operations, such as preset rotation operations. While rotating the 3D model of the target object, the user can check whether the TV's back design and interface locations meet their purchasing needs.

[0144] Of course, user interaction with near-eye display devices is not limited to preset rotation operations and pose adjustment commands. For example, near-eye display devices can receive user control commands to adjust at least one of the following: the display angle and display position of the target object's 3D model in a preset environment; these are not limited here.

[0145] In some embodiments, the pose change of the near-eye display device under a preset environment is obtained by using at least one of the gyroscope and accelerometer of the near-eye display device; based on the pose change, the three-dimensional model of the target object under the preset environment is updated and displayed in the near-eye display device.

[0146] For example, when a user wears a near-eye display device to view a 3D model of a target object displayed in a preset environment, they can walk or make corresponding head movements within that environment. Correspondingly, the near-eye display device's posture within the preset environment will change according to the user's movement or head movements. Based on this, the near-eye display device can obtain the amount of pose change of the near-eye display device within the preset environment using at least one of a gyroscope or an accelerometer, and update the displayed 3D model of the target object in the preset environment according to the pose change. For instance, the near-eye display device can adjust at least one of the display angle and display position of the 3D model of the target object in the preset environment based on the pose change, thereby achieving an updated display of the 3D model of the target object within the preset environment.

[0147] Furthermore, taking a target object including a television and a preset environment including a user's living room as an example, when a user moves around in their living room, the near-eye display device can use at least one of its gyroscope and accelerometer to obtain the change in its pose within the preset environment. Based on this change in pose, the near-eye display device can update the display of the television's 3D model of the target object. For example, if a user moves from one side of the living room to the other, the near-eye display device can adjust the size and pose of the television's 3D model of the target object within the preset environment based on the change in pose. This allows the user to easily observe the degree of matching between the 3D model of the target object and the preset environment from multiple different angles, effectively simulating the display effect of a real target object placed in the preset environment, thus helping the user make better purchasing decisions.

[0148] When a near-eye display device updates the 3D model of a target object in a preset environment in response to user interaction, the user can view both the 3D model before and after the update. This allows the user to comprehensively evaluate whether the 3D model of the target object is suitable for the preset environment, thereby improving the intuitiveness and accuracy of the near-eye display device's assessment of the target object's suitability for the preset environment.

[0149] In another embodiment of this application, when a three-dimensional model of a target object in a preset environment is displayed in a near-eye display device, the near-eye display device can also obtain the distance between the target object and objects in the preset environment; and output a purchase suggestion for the target object based on the distance between the target object and the objects.

[0150] For example, before displaying a 3D model of a target object in a preset environment on a near-eye display device, the near-eye display device needs to acquire the 3D dimension data of the preset environment. Taking the preset environment as an example, which includes the environment in which the near-eye display device is located, the near-eye display device can use a distance sensor installed on the device to measure the distance between the near-eye display device and objects in the environment, thereby determining the position information of the objects. The near-eye display device can then integrate the position information of all objects to determine the spatial data of the environment in which it is located, i.e., determine the 3D dimension data of the preset environment. During the process of displaying the 3D model of the target object in the preset environment, the near-eye display device can automatically detect whether the distance between the 3D model of the target object and surrounding objects is reasonable based on the 3D dimension data of the preset environment, in order to evaluate whether the distance between the target object entity and its surrounding objects is reasonable.

[0151] Taking a television as an example, a near-eye display device can use the distance between a 3D model of the television and the wall to assess whether the distance between the television and the wall meets the heat dissipation requirements. If it is determined that the distance between the television and the wall does not meet the heat dissipation requirements, the near-eye display device may output a purchase recommendation for the television, such as "not recommending purchase" because the distance between the television and the wall does not meet the heat dissipation requirements. Of course, the target object and purchase recommendation are not limited to this, and are not restricted here.

[0152] Based on this, the purchase suggestions for the target object output by the near-eye display device can help users decide whether to purchase the target object. This helps users avoid problems with the use of the target object after purchase due to improper space planning, and reduces the situation where users need to spend extra time and effort to return and refund the target object because it is not suitable for the preset environment.

[0153] By displaying a 3D model of a target object in a preset environment on a near-eye display device, and updating the display of the 3D model of the target object in the preset environment in response to the user's interactive operation on the near-eye display device, spatial fusion between the 3D model of the target object and the preset environment can be achieved, as well as real-time interactive function of the 3D model of the target object. This can provide users with an immersive virtual shopping experience, allowing them to intuitively feel the placement effect of the target object in the actual space, thereby making a more informed purchasing decision.

[0154] The display method for a near-eye display device provided in the above embodiments includes: acquiring a target image captured by a target object, wherein the target object is the display content or physical object of a terminal device other than the near-eye display device, and the target image includes a first image and a second image, wherein the first image and the second image are different images of the same target object; performing modeling processing on the target object based on the first image to obtain a three-dimensional model of the target object; performing a matching degree evaluation on the second image and the three-dimensional model of the target object to obtain a matching degree evaluation result between the three-dimensional model of the target object and the target object; when the matching degree evaluation result is greater than or equal to a preset similarity threshold, acquiring preset environmental three-dimensional size data through the near-eye display device, and displaying the three-dimensional model of the target object in the preset environment in the near-eye display device based on the preset environmental three-dimensional size data.

[0155] When a target image of a target object is acquired by a near-eye display device, since the target image includes a first image and a second image, which are different images of the same target object, the near-eye display device can determine the 3D model of the target object based on the first image. The near-eye display device can also determine the matching degree evaluation result between the 3D model of the target object and the target object based on the second image, to assess whether the 3D model of the target object matches the target object. When the matching degree evaluation result is greater than or equal to a preset similarity threshold, it can be determined that the 3D model of the target object matches the entity of the target object. If the 3D model of the target object matches the target object, the near-eye display device can display the 3D model of the target object in the preset environment based on the acquired 3D dimension data of the preset environment. The 3D model of the target object in the preset environment displayed by the near-eye display device is interactive for users to simulate trying out the target object in the preset environment, which helps improve the intuitiveness of the near-eye display device's assessment of the adaptability between the target object and the preset environment. Correspondingly, when the matching degree evaluation result is greater than or equal to the preset similarity threshold, it can be ensured that the virtual trial experience when the user virtually tries out the 3D model of the target object through the near-eye display device is similar to or the same as the user's actual trial experience when trying out the target object. This can reduce the situation where users return and refund the target object after purchasing it because the target object is not compatible with the preset environment, thereby improving the accuracy of the near-eye display device's assessment of the compatibility between the target object and the preset environment.

[0156] Please see Figure 2 , Figure 2 This is a schematic block diagram of a display device for a near-eye display device according to an embodiment of this application. The display device can be configured within a near-eye display device or a server to execute the aforementioned display method for the near-eye display device. The near-eye display device may include AR glasses, MR glasses, AR helmets, MR helmets, etc., and is not limited thereto. The server can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms.

[0157] like Figure 2 As shown, the display device of the near-eye display device includes an image acquisition module 110, a modeling module 120, a matching degree evaluation module 130, and a model display module 140.

[0158] Image acquisition module 110 is used to acquire a target image captured by a target object, wherein the target object is the display content or physical object of a terminal device other than the near-eye display device, and the target image includes a first image and a second image, wherein the first image and the second image are different images of the same target object;

[0159] The modeling module 120 is used to perform modeling processing on the target object based on the first image to obtain a three-dimensional model of the target object;

[0160] The matching degree evaluation module 130 is used to evaluate the matching degree between the second image and the three-dimensional model of the target object, and obtain the matching degree evaluation result between the three-dimensional model of the target object and the target object.

[0161] The model display module 140 is used to acquire preset environmental three-dimensional size data through the near-eye display device when the matching degree evaluation result is greater than or equal to a preset similarity threshold, and display the three-dimensional model of the target object in the preset environment in the near-eye display device based on the preset environmental three-dimensional size data.

[0162] For example, the modeling module 120 includes a first correspondence determination submodule, a structural feature determination submodule, and a modeling submodule.

[0163] The first correspondence determination submodule is used to determine the first position correspondence of the key points of the target object in multiple first images.

[0164] The structural feature determination submodule is used to determine the structural features of the target object based on the first position correspondence.

[0165] The modeling submodule is used to perform modeling processing on the target object based on the structural features of the target object, so as to obtain a three-dimensional model of the target object.

[0166] For example, the modeling module 120 includes an appearance feature determination submodule and a text feature determination submodule.

[0167] The appearance feature determination submodule is used to perform appearance feature extraction processing on the first image based on a preset deep learning algorithm to obtain the appearance features of the target object.

[0168] The text feature determination submodule is used to perform text feature extraction processing on the first image based on a preset natural language processing algorithm to obtain the text features of the target object.

[0169] The modeling submodule includes the first modeling submodule.

[0170] The first modeling submodule is used to perform modeling processing on the target object based on the structural features, appearance features, and text features of the target object to obtain a three-dimensional model of the target object.

[0171] For example, the matching degree evaluation module 130 includes a second correspondence determination submodule, a feature comparison submodule, a reprojection submodule, an error determination submodule, and an evaluation result determination submodule.

[0172] The second correspondence determination submodule is used to determine the second positional correspondence between the key points of the target object in the second image and the three-dimensional model of the target object.

[0173] The feature comparison submodule is used to perform feature comparison on the second image and the three-dimensional model of the target object based on a preset feature comparison algorithm and according to the second position correspondence, so as to obtain the feature comparison result corresponding to the three-dimensional model of the target object.

[0174] The reprojection submodule is used to reproject the three-dimensional model of the target object according to the camera parameters of the near-eye display device to obtain a third image corresponding to the three-dimensional model of the target object.

[0175] The error determination submodule is used to determine the pixel position error of the target object in the second image and the third image using a reprojection error evaluation algorithm.

[0176] The evaluation result determination submodule is used to determine the matching degree evaluation result between the 3D model of the target object and the target object based on the feature comparison result and the pixel position error.

[0177] For example, the display device also includes a deviation determination submodule and a deviation adjustment submodule.

[0178] The deviation determination submodule is used to determine the deviation type corresponding to the 3D model of the target object based on the matching degree evaluation result when the matching degree evaluation result is less than a preset similarity threshold.

[0179] The deviation adjustment submodule is used to adjust the three-dimensional model of the target object according to the deviation type to obtain the adjusted three-dimensional model of the target object.

[0180] For example, the deviation adjustment submodule includes a first adjustment submodule and a second adjustment submodule.

[0181] The first adjustment submodule is used to perform local adjustment processing on the three-dimensional model of the target object using a preset editing model when the deviation type is the first deviation type, so as to obtain the adjusted three-dimensional model of the target object.

[0182] The second adjustment submodule is used to reconstruct the target object based on the first image and the second image when the deviation type is the second deviation type, so as to obtain the adjusted three-dimensional model of the target object.

[0183] For example, the model display module 140 includes a size data acquisition submodule and a display submodule.

[0184] The size data acquisition submodule is used to perform spatial measurement processing on the preset environment through the near-eye display device to obtain the three-dimensional size data of the preset environment.

[0185] The display submodule is used to display the three-dimensional model of the target object on the display interface of the near-eye display device based on a preset spatial positioning algorithm and the preset three-dimensional size data of the environment.

[0186] For example, the model display module 140 includes a first model update submodule.

[0187] The first model update submodule is used to update and display the three-dimensional model of the target object in the preset environment in response to the user's interactive operation on the near-eye display device.

[0188] For example, the model display module 140 includes a change acquisition submodule and a second model update submodule.

[0189] The change acquisition submodule is used to acquire the pose change of the near-eye display device under the preset environment by using at least one of the gyroscope and accelerometer of the near-eye display device.

[0190] The second model update submodule is used to update and display the three-dimensional model of the target object in the preset environment in the near-eye display device according to the pose change amount.

[0191] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the above-described apparatus and its modules and units can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0192] The method of this application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0193] For example, the above-described method and apparatus can be implemented as a computer program that runs on a near-eye display device or a server to control the near-eye display device. For example, the near-eye display device may include AR glasses, MR glasses, AR helmets, MR helmets, etc., without limitation. The server can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms.

[0194] Please see Figure 3 , Figure 3 This is a schematic block diagram of the structure of a near-eye display device provided in an embodiment of this application.

[0195] like Figure 3 As shown, the near-eye display device includes a memory and a processor. The memory and processor can be connected via a system bus, and the memory may include a storage medium and internal memory.

[0196] The storage medium can store the operating system and computer programs. When the computer program is executed, it enables the processor to perform any display method of the near-eye display device.

[0197] The processor provides computing and control capabilities to support the operation of the entire near-eye display device.

[0198] Internal memory provides an environment for the execution of computer programs stored in storage media. When these computer programs are executed by the processor, the processor can execute any display method of a near-eye display device.

[0199] Those skilled in the art will understand that Figure 3The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the near-eye display device to which the present application is applied. A specific near-eye display device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0200] It should be understood that a processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other convertible logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0201] In one embodiment, the processor is configured to execute a computer program and, when executing the computer program, perform the following steps:

[0202] Acquire a target image captured by a target object, wherein the target object is the display content or physical object of a terminal device other than the near-eye display device, and the target image includes a first image and a second image, wherein the first image and the second image are different images of the same target object;

[0203] Based on the first image, the target object is modeled to obtain a three-dimensional model of the target object;

[0204] A matching degree evaluation is performed on the second image and the 3D model of the target object to obtain the matching degree evaluation result between the 3D model of the target object and the target object.

[0205] When the matching degree evaluation result is greater than or equal to a preset similarity threshold, the preset three-dimensional size data of the environment is obtained through the near-eye display device, and the three-dimensional model of the target object in the preset environment is displayed in the near-eye display device based on the preset three-dimensional size data of the environment.

[0206] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the near-eye display device described above can be referred to the corresponding process in the aforementioned near-eye display device display method embodiments, and will not be repeated here.

[0207] This application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the method implemented can be referred to in various embodiments of the display method of the near-eye display device of this application.

[0208] The computer-readable storage medium can be an internal storage unit of the near-eye display device described in the foregoing embodiments, such as the hard disk or memory of the near-eye display device. Alternatively, the computer-readable storage medium can be an external storage device of the near-eye display device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the near-eye display device.

[0209] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0210] It should also be understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. It should be noted that, herein, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0211] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. The above descriptions are merely specific implementations of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A display method for a near-eye display device, characterized in that, include: Acquire a target image captured by a target object, wherein the target object is the display content or physical object of a terminal device other than the near-eye display device, and the target image includes a first image and a second image, wherein the first image and the second image are different images of the same target object; Based on the first image, the target object is modeled to obtain a three-dimensional model of the target object; A matching degree evaluation is performed on the second image and the 3D model of the target object to obtain the matching degree evaluation result between the 3D model of the target object and the target object. When the matching degree evaluation result is greater than or equal to a preset similarity threshold, the preset three-dimensional size data of the environment is obtained through the near-eye display device, and the three-dimensional model of the target object in the preset environment is displayed in the near-eye display device based on the preset three-dimensional size data of the environment. When the matching degree evaluation result is less than a preset similarity threshold, the deviation type corresponding to the three-dimensional model of the target object is determined based on the matching degree evaluation result; Based on the deviation type, the 3D model of the target object is adjusted to obtain the adjusted 3D model of the target object.

2. The display method according to claim 1, characterized in that, The step of modeling the target object based on the first image to obtain a three-dimensional model of the target object includes: Determine the first positional correspondence of the key points of the target object in multiple first images; Based on the first position correspondence, the structural features of the target object are determined; Based on the structural characteristics of the target object, the target object is modeled to obtain a three-dimensional model of the target object.

3. The display method according to claim 2, characterized in that, The display method further includes: Based on a preset deep learning algorithm, the first image is processed to extract appearance features to obtain the appearance features of the target object; Based on a preset natural language processing algorithm, text feature extraction is performed on the first image to obtain the text features of the target object; The step of modeling the target object based on its structural features to obtain a three-dimensional model of the target object includes: Based on the structural features of the target object and at least one of the appearance features and text features, the target object is modeled to obtain a three-dimensional model of the target object.

4. The display method according to claim 1, characterized in that, The step of evaluating the matching degree between the second image and the 3D model of the target object to obtain the matching degree evaluation result between the 3D model of the target object and the target object includes: Determine the second positional correspondence between the key points of the target object in the second image and the 3D model of the target object; Based on a preset feature comparison algorithm, the second image and the three-dimensional model of the target object are compared according to the second position correspondence to obtain the feature comparison result corresponding to the three-dimensional model of the target object; Based on the camera parameters of the near-eye display device, the three-dimensional model of the target object is reprojected to obtain a third image corresponding to the three-dimensional model of the target object. The reprojection error evaluation algorithm is used to determine the pixel position error of the target object in the second image and the third image. Based on the feature comparison results and the pixel position errors, the matching degree evaluation result between the 3D model of the target object and the target object is determined.

5. The display method according to claim 1, characterized in that, The step of adjusting the 3D model of the target object according to the deviation type to obtain the adjusted 3D model of the target object includes: When the deviation type is the first deviation type, the target object's three-dimensional model is locally adjusted using a preset editing model to obtain the adjusted target object's three-dimensional model. When the deviation type is the second deviation type, the target object is reconstructed based on the first image and the second image to obtain an adjusted three-dimensional model of the target object.

6. The display method according to any one of claims 1 to 4, characterized in that, When the matching degree evaluation result is greater than or equal to a preset similarity threshold, the step of acquiring preset environmental three-dimensional size data through the near-eye display device, and displaying the target object three-dimensional model in the preset environment in the near-eye display device based on the preset environmental three-dimensional size data includes: The preset environment is spatially measured and processed by the near-eye display device to obtain the three-dimensional dimensional data of the preset environment; Based on a preset spatial positioning algorithm, and according to the preset three-dimensional size data of the environment, the three-dimensional model of the target object is displayed on the display interface of the near-eye display device.

7. The display method according to any one of claims 1 to 4, characterized in that, Displaying the three-dimensional model of the target object under a preset environment in the near-eye display device includes: In response to user interaction with the near-eye display device, the three-dimensional model of the target object under the preset environment is updated and displayed on the near-eye display device.

8. The display method according to any one of claims 1 to 4, characterized in that, Displaying the three-dimensional model of the target object under a preset environment in the near-eye display device includes: The pose change of the near-eye display device under the preset environment is obtained by using at least one of the gyroscope and accelerometer of the near-eye display device. Based on the change in pose, the three-dimensional model of the target object under the preset environment is updated and displayed in the near-eye display device.

9. A display device for a near-eye display system, characterized in that, The display device includes: The image acquisition module is used to acquire a target image captured by a target object, wherein the target object is the display content or physical object of a terminal device other than the near-eye display device, and the target image includes a first image and a second image, wherein the first image and the second image are different images of the same target object; The modeling module is used to perform modeling processing on the target object based on the first image to obtain a three-dimensional model of the target object; The matching degree evaluation module is used to evaluate the matching degree between the second image and the three-dimensional model of the target object, and to obtain the matching degree evaluation result between the three-dimensional model of the target object and the target object. The model display module is used to acquire preset environmental three-dimensional size data through the near-eye display device when the matching degree evaluation result is greater than or equal to a preset similarity threshold, and display the three-dimensional model of the target object in the preset environment in the near-eye display device based on the preset environmental three-dimensional size data; The display device is also used for: When the matching degree evaluation result is less than a preset similarity threshold, the deviation type corresponding to the three-dimensional model of the target object is determined based on the matching degree evaluation result; Based on the deviation type, the 3D model of the target object is adjusted to obtain the adjusted 3D model of the target object.

10. A near-eye display device, characterized in that, The near-eye display device includes a memory and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program and, when executing the computer program, implement the steps of the display method of the near-eye display device as described in any one of claims 1 to 8.

11. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the display method of the near-eye display device as described in any one of claims 1 to 8.

Images