Commodity display method and apparatus, and device and medium

Abstract

A commodity display method and apparatus, and a device and a medium. When a commodity display request for requesting that a target commodity is displayed to a target user in a 3D form is received from a terminal device, a 3D model of the target commodity is acquired, so as to acquire, on the basis of pose indication information of the target user and from the 3D model of the target commodity, a first image and a second image that match the pose indication information, and then send the first image and the second image to the terminal device, such that the terminal device can respectively display the first image and the second image as a left-eye image and a right-eye image, thereby achieving the aim of displaying the target commodity with a 3D effect, and improving the display effect of the target commodity.

Description

Commodity display method, device, equipment and medium TECHNICAL FIELD

[0001] The present application relates to the field of information technology, and in particular to a commodity display method, device, equipment and medium. BACKGROUND

[0002] With the rapid development of e-commerce, the traditional physical store shopping mode brings many inconveniences to the shopping process of consumers due to the need to spend a lot of time and effort to go to the store to buy goods. In view of the deficiencies of the traditional shopping mode, the development of virtual reality (VR) shopping becomes crucial. Through VR technology, consumers can remotely experience a realistic virtual shopping environment, so that the shopping process can be limited by time and place, and the convenience and fun of shopping are improved. SUMMARY

[0003] The present application provides a commodity display method, device, equipment and medium to solve the deficiencies in the related art.

[0004] According to a first aspect of an embodiment of the present application, a commodity display method is provided, applied to a first server, the method comprising:

[0005] In response to a commodity display request of a terminal device, a 3D model of a target commodity is obtained, wherein the commodity display request is used to request to display the target commodity in 3D form to a target user;

[0006] Based on posture indication information of the target user, a first image and a second image matched with the posture indication information are obtained from the 3D model of the target commodity;

[0007] The first image and the second image are sent to the terminal device, and the first image and the second image are respectively displayed as left eye image and right eye image by the terminal device.

[0008] According to a second aspect of an embodiment of the present application, a commodity display device is provided, applied to a first server, the device comprising:

[0009] A first obtaining module is configured to obtain a 3D model of a target commodity in response to a commodity display request of a terminal device, wherein the commodity display request is used to request to display the target commodity in 3D form to a target user;

[0010] A second obtaining module is configured to obtain a first image and a second image matched with posture indication information of the target user from the 3D model of the target commodity based on the posture indication information of the target user;

[0011] The sending module is configured to send the first image and the second image to the terminal device, and the first image and the second image are respectively displayed as a left-eye image and a right-eye image by the terminal device.

[0012] According to a third aspect of the embodiments of the present application, a first server is provided, the first server comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements operations performed by the commodity display method according to the first aspect.

[0013] According to a fourth aspect of the embodiments of the present application, a computer readable storage medium is provided, the computer readable storage medium storing a program, and the program is executable on a processor to implement operations performed by the commodity display method according to the first aspect.

[0014] According to the above embodiments, when the terminal device requests to display the target commodity to the target user in a 3D form, the 3D model of the target commodity is obtained, and then the first image and the second image are obtained from the 3D model of the target commodity based on the posture indication information of the target user, and the first image and the second image are sent to the terminal device, so that the terminal device can display the first image and the second image as a left-eye image and a right-eye image respectively, so as to display the target commodity in a 3D effect, and improve the display effect of the target commodity.

[0015] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present application and, together with the specification, serve to explain the principles of the present application.

[0017] FIG. 1 is a schematic diagram of an application scenario of a commodity display method according to an embodiment of the present application.

[0018] FIG. 2 is a flowchart of a commodity display method according to an embodiment of the present application.

[0019] FIG. 3 is a flowchart of a model training process according to an embodiment of the present application.

[0020] FIG. 4 is a flowchart of a commodity display method according to an embodiment of the present application.

[0021] FIG. 5 is a flowchart of a commodity purchase process according to an embodiment of the present application.

[0022] Fig. 6 is a block diagram of a commodity display device according to an embodiment of the present application.

[0023] Fig. 7 is a structural schematic diagram of a server according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] The exemplary embodiments will be described in detail herein with reference to the attached drawings. In the following description, the same numbers are used to indicate the same or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all the implementations consistent with the present application. Instead, they only represent examples of devices and methods consistent with some aspects of the present application, as detailed in the appended claims.

[0025] In the related art, the three-dimensional (3D) shopping method such as VR is not very mature. Many so-called VR shopping actually only builds a 3D shopping scene, and the goods in the scene are basically in the form of 2D pictures, which cannot meet the needs of consumers to realize real-time viewing of 3D information of goods at the local end of the VR device, and lacks an immersive shopping experience.

[0026] Therefore, the present application provides a commodity display method, which can provide real-time 3D model information for various shopping software, and protect consumers to watch 3D information of goods from any angle in real time, thereby bringing a more immersive shopping experience to consumers.

[0027] Optionally, the commodity display method provided in the present application can be executed by a first server (or 3D model server). The server can be a server, a plurality of servers, a server cluster, a cloud computing platform, etc., but is not limited thereto.

[0028] Referring to Fig. 1, Fig. 1 is a schematic diagram of an application scenario of a commodity display method according to an embodiment of the present application. As shown in Fig. 1, the terminal device 101 can be a desktop computer, a notebook computer, a tablet computer, a smart phone, etc., but is not limited thereto. The terminal device 101 can be connected to a VR glasses through a wired or wireless connection mode, so as to display the picture on the terminal device 101 through the VR glasses, and the terminal device 101 can be used as a processor of the VR glasses. In addition, the VR glasses can be connected to a handle through a wired or wireless connection mode, and the handle can include buttons, joysticks, touchpads, etc., so that the user can interact with the terminal device and the VR glasses through the handle to realize the interaction and control of the user in the virtual environment.

[0029] Optionally, the terminal device 101 can be loaded with and run shopping software, and the user can control the terminal device 101 through the handle to send a request to the second server 102 (i.e., the background server of the shopping software) through the shopping software running on the terminal device 101 to request to view 3D information of a certain product. The second server 102 can respond to the request of the terminal device 101 to realize the acquisition of the product 3D information through the first server 103, and return the acquired product 3D information to the terminal device 101, so that the user can view the product 3D information through the VR glasses connected to the terminal device 101.

[0030] It should be noted that FIG. 1 is only an exemplary description of the application scenario of the product display method provided by the present application, and does not constitute a limitation on the application scenario of the product display method provided by the present application. In more possible implementation manners, the product display method provided by the present application can also be applied to other scenarios involving product display processes.

[0031] After introducing the application scenario of the product display method provided by the present application, the implementation manner of the product display method provided by the present application will be introduced below.

[0032] Referring to FIG. 2, FIG. 2 is a flowchart of a product display method according to an embodiment of the present application. As shown in FIG. 2, the method can be applied to a first server (or a 3D model server), and the method comprises the following steps:

[0033] Step 201: In response to a product display request of a terminal device, acquiring a 3D model of a target product, wherein the product display request is used to request to display the target product in 3D form to a target user.

[0034] In a possible implementation manner, the terminal device can send a product display request to the first server to display the target product in 3D form, and the first server can receive the product display request of the terminal device, so as to acquire the 3D model of the target product in response to the product display request.

[0035] Step 202: Based on pose indication information of the target user, acquiring a first image and a second image matched with the pose indication information from the 3D model of the target product.

[0036] It should be noted that the terminal device can be connected with VR glasses and a handle, and the user can view the visual interface of the terminal device through the VR glasses, and can interact with the terminal device through the handle.

[0037] The pose indication information can be used to determine the current head pose of the target user.

[0038] Optionally, the posture indication information can be posture data of the target user. For example, the posture data of the target user can be collected through a VR glasses worn by the target user, and the collected posture data can be taken as the posture indication information of the target user. Alternatively, the posture indication information can be handle interaction data. For example, the handle interaction data can be collected through an interactive handle of the VR glasses worn by the target user, and the collected handle interaction data can be taken as the posture indication information of the target user.

[0039] The first image and the second image are both 2D images.

[0040] Step 203: sending the first image and the second image to the terminal device, and the first image and the second image are respectively taken as left-eye images and right-eye images by the terminal device.

[0041] It should be noted that the 3D experience in VR is actually a synthesis of two parallax images by the human brain, and therefore, by sending the first image as a left-eye image and the second image as a right-eye image to the terminal device, the terminal device can display the received first image and second image in front of the left and right eyes of the user through the VR glasses worn by the user, so as to achieve a three-dimensional display effect.

[0042] Through the above embodiment, in the case that a product display request is received from the terminal device to display a target product to a target user in a 3D form, a 3D model of the target product can be obtained, and then, based on the posture indication information of the target user, a first image and a second image corresponding to the posture indication information can be obtained from the 3D model of the target product, and then, the first image and the second image can be sent to the terminal device, so that the terminal device can display the first image and the second image as left-eye images and right-eye images respectively, so as to display the target product in a 3D effect, and improve the display effect of the target product.

[0043] After introducing the basic implementation process of the present application, various optional implementation manners of the product display method provided by the present application are introduced below.

[0044] In some embodiments, before step 201, the method can further include the following steps:

[0045] Step 200: receiving a product display request sent by a terminal device.

[0046] Optionally, the product display request can be triggered by the terminal device according to a product display operation of a user on the terminal device.

[0047] For example, the shopping software can provide a first function entry for the user to view 3D information of a product. When the user browses the product through the shopping software, the user can trigger a product that the user wants to further understand. The terminal device can display a product introduction interface of the product triggered by the user in response to the triggering operation of the user. The product introduction interface can be used to display some basic information of the triggered product and a product image (which is a 2D image) of the triggered product. In addition, the terminal device can provide the first function entry in the product introduction interface. If the user triggers the first function entry to trigger the product display operation, the first server can generate a product display request in response to the triggering operation of the user on the first function entry and send the product display request.

[0048] For another example, when the user browses the product through the shopping software, if the user triggers a product that the user wants to further understand, the terminal device can trigger a product display operation in response to the triggering operation of the user to generate a product display request based on the product display operation and send the product display request. The triggered product is the target product.

[0049] It should be noted that the above is only two exemplary ways of triggering a product display request. In more possible implementation manners, other ways can also be used to trigger a product display request, which is not limited by the present application.

[0050] In a possible implementation manner, the terminal device can send the product display request to the second server, and the second server can forward the product display request to the first server.

[0051] The second server can be a background server corresponding to the shopping software currently running on the terminal device.

[0052] In a possible implementation manner, the first server can receive the product display request sent by the second server to achieve the purpose of receiving the product display request sent by the terminal device.

[0053] In some embodiments, a dedicated interface (hereinafter referred to as a target interface) of a background server of the first server can be provided to connect multiple shopping software to allow multiple shopping software to realize 3D display of products through the product display method provided by the present application.

[0054] In a possible implementation manner, the second server can send the product display request to the target interface of the first server, and the first server can receive the product display request sent by the second server through the target interface.

[0055] Optionally, the commodity display request can carry commodity information of the target commodity, so that the first server can acquire the 3D model of the target commodity by receiving the commodity display request.

[0056] After receiving the commodity display request through the above-mentioned embodiments, the 3D model of the target commodity can be acquired in response to the commodity display request of the terminal device through step 201.

[0057] In some embodiments, for step 201, when the 3D model of the target commodity is acquired in response to the commodity display request of the terminal device, the 3D model of the target commodity can be acquired in response to the commodity display request of the terminal device based on the commodity information of the target commodity.

[0058] Optionally, when the 3D model of the target commodity is acquired based on the commodity information of the target commodity, it can be achieved in the following way:

[0059] The commodity information of the target commodity is input into a three-dimensional reconstruction large model to generate the 3D model of the target commodity through the three-dimensional reconstruction large model.

[0060] Optionally, the commodity information can be a commodity image. That is, the commodity image of the target commodity can be input into the three-dimensional reconstruction large model to generate the 3D model of the target commodity through the three-dimensional reconstruction large model.

[0061] The three-dimensional reconstruction large model can be pre-trained. Optionally, the three-dimensional reconstruction large model can be a large pre-training model based on a transformer architecture.

[0062] Optionally, the three-dimensional reconstruction large model can be trained in the following way:

[0063] Based on the first data set, the initial deep learning model is unsupervised trained to obtain an initial three-dimensional reconstruction model; based on the second data set, the initial three-dimensional reconstruction model is supervised trained to obtain the three-dimensional reconstruction large model.

[0064] The first data set can be a super large-scale commodity image data set. For example, the first data set can be constructed by using widely open source data on the Internet and private data sets of commodity procurement platforms.

[0065] Optionally, after the first data set is acquired, the data in the first data set can also be subjected to data cleaning processing, so that the first data set subjected to data cleaning processing is used as a training data set for training the initial three-dimensional reconstruction model.

[0066] For example, the data cleaning process on the first data set can be implemented by removing duplicate data in the first data set, filtering sensitive data in the first data set, etc., to provide a clean and reliable training data set.

[0067] By performing data cleaning on the first data set, the quality and consistency of the training data can be ensured, thereby ensuring the training effect of the model.

[0068] It should be noted that after the initial three-dimensional reconstruction model is trained based on the first data set, the initial three-dimensional reconstruction model can be supervised fine-tuned based on the second data set to further optimize the model performance and processing effect, so that the final trained three-dimensional reconstruction large model can adapt to specific tasks and data sets.

[0069] Among them, the second data set can be a labeled data set, and the data amount of the second data set can be much smaller than that of the first data set.

[0070] Referring to FIG. 3, FIG. 3 is a flowchart illustrating a model training process according to an embodiment of the present application. As shown in FIG. 3, after the first data set for model training is collected through data collection, data cleaning can be performed based on the first data set, so that unsupervised pre-training is performed based on the first data set after data cleaning to obtain an initial three-dimensional reconstruction model, and then supervised fine-tuning is performed on the initial three-dimensional reconstruction model based on the second data set to obtain a three-dimensional reconstruction large model.

[0071] It should be noted that the above model training process can be performed in advance, so that the trained three-dimensional reconstruction large model can be deployed to the first server, so that the first server can call the three-dimensional reconstruction large model to generate a commodity 3D model when needed.

[0072] Optionally, the three-dimensional reconstruction large model can take a commodity image as the model input, and output a 3D model of the commodity through a series of processing inside the model. Optionally, the commodity 3D model can be a point cloud model, a polygon mesh model, a voxel model, etc., but is not limited thereto.

[0073] In some embodiments, when the 3D model of the target commodity is obtained based on the commodity information of the target commodity, it can also be achieved in the following way:

[0074] Based on the commodity information of the target commodity, a 3D model of the target commodity is determined from a plurality of alternative 3D models.

[0075] Among them, the plurality of alternative 3D models correspond to an alternative commodity respectively, that is, each alternative commodity can correspond to an alternative 3D model, and in addition, the plurality of alternative 3D models can be generated and stored in the first server in advance.

[0076] It should be noted that for any alternative commodity, the alternative 3D model of the alternative commodity can be obtained in various ways.

[0077] Optionally, the depth information of the alternative commodity can be determined based on the commodity images of the alternative commodity taken from different perspectives, so as to generate the alternative 3D model of the alternative commodity based on the depth information.

[0078] For example, the depth information of the alternative commodity can be obtained by using multiple cameras to take the same alternative commodity from different angles by a stereo vision method, so as to calculate the difference between images, thereby achieving the acquisition of the alternative 3D model of the alternative commodity. It should be noted that for an alternative commodity with rich texture, the stereo vision method can be used to generate the alternative 3D model.

[0079] Optionally, the alternative 3D model of the alternative commodity can be generated based on the light pattern projected onto the alternative commodity and the light pattern projected onto the alternative commodity.

[0080] For example, the alternative 3D model of the alternative commodity can be obtained by projecting a special light pattern onto the surface of the alternative commodity by a structured light method, and inferring the three-dimensional shape of the alternative commodity by observing the deformation of the light pattern on the surface of the alternative commodity. It should be noted that for an alternative commodity with complex surface texture, the structured light method can be used to generate the alternative 3D model.

[0081] Optionally, the alternative 3D model of the alternative commodity can be generated based on the commodity images of the alternative commodity taken from different perspectives and the geometric relationship between different shooting perspectives.

[0082] For example, the alternative 3D model of the alternative commodity can be obtained by observing the same alternative commodity from different perspectives by multiple cameras by a multi-view geometry method, and inferring the three-dimensional structure of the alternative commodity by calculating the geometric relationship between the cameras. Through the above method, the observation of the alternative commodity can be realized from multiple perspectives, so as to obtain more complete and more accurate three-dimensional structure information of the alternative commodity.

[0083] Optionally, the alternative 3D model of the alternative commodity can be generated by a neural network model based on the commodity images of the alternative commodity.

[0084] For example, a small-scale neural network model can be trained by a deep learning method, so that the neural network model can learn the mapping relationship from the input image to the output three-dimensional structure, thereby enabling the generation of the alternative 3D model by the neural network model.

[0085] Optionally, the neural network model can be a Convolutional Neural Network (CNN), a Sparse Convolutional Network, a Variational Autoencoder (VAE), a Recurrent Reconstruction Neural Network (R2N2), etc., but is not limited thereto.

[0086] The above are only several exemplary ways of constructing the alternative 3D model of the alternative commodity, and in more possible implementations, the generation of the alternative 3D model of the alternative commodity can also be implemented in other ways, which are not limited by the present application.

[0087] After the alternative 3D model of each alternative commodity is generated, the generated alternative 3D model can be stored so that the required commodity 3D model can be obtained from the stored alternative 3D model after receiving a commodity display request.

[0088] Optionally, the alternative 3D model and the commodity information of the alternative commodity corresponding thereto can be stored in association so that the required commodity 3D model can be obtained based on the commodity information. For example, the alternative 3D model and the commodity identifier of the alternative commodity corresponding thereto can be stored in association so that the required commodity 3D model can be obtained based on the commodity identifier.

[0089] In some embodiments, the above two ways of obtaining the 3D model of the target commodity can also be used in combination. Optionally, if there is an alternative 3D model corresponding to the target commodity, the 3D model of the target commodity is determined from the plurality of alternative 3D models; if there is no alternative 3D model corresponding to the target commodity, the commodity information (such as the commodity image) of the target commodity is input to the three-dimensional reconstruction large model to generate the 3D model of the target commodity through the three-dimensional reconstruction large model.

[0090] For example, for some newly emerging commodities or commodities with less available data, the reconstruction accuracy of the three-dimensional reconstruction large model can not be accurate enough, at which time, the alternative 3D model thereof can be constructed and stored in advance, so that the commodity 3D model can be determined from the stored alternative 3D model when needed. For relatively common or commodities with more available data, the generation of the commodity 3D model can be directly implemented through the three-dimensional reconstruction large model, so that the accuracy of the 3D model determination result can be further improved.

[0091] After the 3D model of the target commodity is determined through the above embodiments, the first image and the second image matched with the posture indication information of the target user can be obtained from the 3D model of the target commodity based on the posture indication information of the target user through step 202.

[0092] The posture indication information of the target user can be carried by the commodity display request, or can be sent to the 3D model server by the terminal device alone after the commodity display sending request is sent, which is not limited in the present application.

[0093] In some embodiments, for step 202, when the first image and the second image matched with the posture indication information of the target user are obtained from the 3D model of the target commodity based on the posture indication information of the target user, the following steps can be implemented:

[0094] Step 2021, based on the posture indication information of the target user, determining the first view angle corresponding to the left eye of the target user and the second view angle corresponding to the right eye of the target user.

[0095] The first view angle can be used to indicate the line of sight direction of the left eye of the target user, and the second view angle can be used to indicate the line of sight direction of the right eye of the target user.

[0096] Step 2022, based on the first view angle, intercepting the first image from the 3D model of the target commodity, and based on the second view angle, intercepting the second image from the 3D model of the target commodity.

[0097] Optionally, the first image matched with the line of sight direction of the left eye of the user can be intercepted from the 3D model of the target commodity based on the first view angle, and the second image matched with the line of sight direction of the right eye of the user can be intercepted from the 3D model of the target commodity based on the second view angle.

[0098] After the first image and the second image are obtained, the first image and the second image can be sent to the terminal device through step 203, so that the terminal device can display the first image and the second image as left eye image and right eye image respectively through the VR glasses in front of the user, so that the user can realize the viewing of the 3D effect of the commodity.

[0099] Optionally, the terminal device can render the first image to the left eye display area of the VR glasses, and render the second image to the right eye display area of the VR glasses, so as to display the first image in front of the left eye of the target user and the second image in front of the right eye of the target user, and the target user can obtain the experience of 3D display according to the parallax map of the first image and the second image.

[0100] Referring to FIG. 4, FIG. 4 is a flowchart illustrating a method for displaying a product according to an embodiment of the present application. As shown in FIG. 4, a user can trigger a product display request at a local VR terminal (including a computing device, VR glasses, and a handle), and the local VR terminal will start uploading real-time posture indication information of the user. The 3D model server can generate a 3D model of the product based on the product display request, and real-time obtain a product binocular image from the 3D model of the product based on the posture indication information of the user. The 3D model server can feed back the product binocular image to the local VR terminal in real time, so that the local VR terminal can realize real-time 3D display of the product.

[0101] It should be noted that during the process in which the user uses the VR glasses to view the 3D effect of the product, the posture of the user can change. Therefore, after sending the product display request, the terminal device can send real-time posture indication information to the 3D model server. Correspondingly, the 3D model server can receive the real-time posture indication information sent by the terminal device, so that each time a posture indication information is received, the 3D model server re-obtains the first image and the second image that match the currently received posture indication information from the 3D model of the target product based on the currently received posture indication information, and then re-sends the currently obtained first image and second image to the terminal device. The re-obtaining of the first image and the second image that match the currently received posture indication information from the 3D model of the target product is the same as the above process, and will not be described here.

[0102] Optionally, the terminal device can collect real-time posture indication information of a target user, and each time a posture indication information is collected, it is sent to the 3D model server, so that the 3D model server can realize real-time obtaining and synchronous display of the first image and the second image according to the real-time collected posture indication information.

[0103] In the above manner, the real-time performance and accuracy of the 3D effect display of the product can be effectively ensured.

[0104] Optionally, each time a new posture indication information is collected, the terminal device can also compare the newly collected posture indication information with the posture indication information collected at the previous moment. If it is determined that the newly collected posture indication information has changed compared with the posture indication information collected at the previous moment, the newly collected posture indication information is sent to the 3D model server again. Correspondingly, the 3D model server can re-obtain the first image and the second image again upon receiving the new posture indication information. It should be noted that for the moment when the posture indication information does not change, the terminal device can use the first image and the second image sent by the 3D model server to display the 3D effect of the product, and update the displayed 3D effect of the product upon receiving the first image and the second image sent by the 3D model server.

[0105] In this way, the processing pressure of the 3D model server can be effectively reduced.

[0106] Referring to FIG. 5, which is a flowchart illustrating a commodity purchasing process according to an embodiment of the present application, as shown in FIG. 5, a user can open a shopping software through a local VR terminal to select and purchase commodities through the shopping software, so that a commodity display request can be triggered based on a commodity to be purchased, the 3D model server can be interacted with through the commodity display request, and a commodity binocular image can be obtained from the 3D model server, so that the 3D effect of the commodity can be viewed based on the commodity binocular image, and the commodity can be purchased when the commodity meets the demand, thereby providing the user with a more immersive shopping experience.

[0107] Corresponding to the foregoing method embodiments, the embodiments of the present application also provide a commodity display device. Referring to FIG. 6, which is a block diagram of a commodity display device according to an embodiment of the present application, as shown in FIG. 6, the device can be applied to a first server, and the device includes:

[0108] A first acquisition module 601 is configured to acquire a 3D model of a target commodity in response to a commodity display request of a terminal device, wherein the commodity display request is used to request to display the target commodity to a target user in a 3D form.

[0109] A second acquisition module 602 is configured to acquire a first image and a second image matched with pose indication information from the 3D model of the target commodity based on the pose indication information of the target user.

[0110] A sending module 603 is configured to send the first image and the second image to the terminal device, and the first image and the second image are respectively displayed as a left-eye image and a right-eye image by the terminal device.

[0111] In some embodiments, the first acquisition module 601, when acquiring the 3D model of the target commodity, is configured to:

[0112] input commodity information of the target commodity into a three-dimensional reconstruction large model to generate the 3D model of the target commodity through the three-dimensional reconstruction large model.

[0113] In some embodiments, the three-dimensional reconstruction large model is pre-trained, and the device further includes a training module, wherein the training module is configured to:

[0114] unsupervisedly train an initial deep learning model based on a first data set to obtain an initial three-dimensional reconstruction model;

[0115] supervise training of the initial three-dimensional reconstruction model based on a second data set to obtain the three-dimensional reconstruction large model.

[0116] In some embodiments, the apparatus further includes:

[0117] a data cleaning module, configured to perform data cleaning processing on the data in the first data set.

[0118] In some embodiments, the first obtaining module 601, when used to obtain the 3D model of the target commodity, is configured to:

[0119] determine the 3D model of the target commodity from the plurality of candidate 3D models based on the commodity information of the target commodity.

[0120] In some embodiments, the plurality of candidate 3D models respectively correspond to a candidate commodity; and for any candidate commodity, the candidate 3D model of the candidate commodity is obtained in any of the following ways:

[0121] determine the depth information of the candidate commodity based on the commodity images of the candidate commodity taken from different perspectives, to generate the candidate 3D model of the candidate commodity based on the depth information;

[0122] generate the candidate 3D model of the candidate commodity based on the light pattern projected onto the candidate commodity and the light pattern obtained by projecting onto the candidate commodity;

[0123] generate the candidate model of the candidate commodity based on the commodity images of the candidate commodity taken from different perspectives and the geometric relationship between different shooting perspectives;

[0124] generate the candidate 3D model of the candidate commodity through a neural network model based on the commodity images of the candidate commodity.

[0125] In some embodiments, the first obtaining module 601, when used to obtain the 3D model of the target commodity, is configured to any of the following:

[0126] if there is a candidate 3D model corresponding to the target commodity, determine the 3D model of the target commodity from the plurality of candidate 3D models;

[0127] if there is no candidate 3D model corresponding to the target commodity, input the commodity information of the target commodity into a three-dimensional reconstruction large model to generate the 3D model of the target commodity through the three-dimensional reconstruction large model.

[0128] In some embodiments, the second obtaining module 602, when used to obtain the first image and the second image matching the pose indication information from the 3D model of the target commodity based on the pose indication information of the target user, is configured to:

[0129] determine the first perspective corresponding to the left eye of the target user and the second perspective corresponding to the right eye of the target user based on the pose indication information of the target user;

[0130] The first image is intercepted from the 3D model of the target commodity based on a first perspective, and the second image is intercepted from the 3D model of the target commodity based on a second perspective.

[0131] In some embodiments, the apparatus further includes:

[0132] The first receiving module is configured to receive the posture indication information sent by the terminal device in real time.

[0133] The second obtaining module 602 is further configured to, each time a posture indication information is received, reobtain, based on the currently received posture indication information, the first image and the second image matched with the currently received posture indication information from the 3D model of the target commodity.

[0134] The sending module 603 is further configured to re-send the currently obtained first image and second image to the terminal device.

[0135] In some embodiments, the apparatus further includes:

[0136] The second receiving module is configured to receive, through a target interface, a commodity display request sent by a second server, the second server being a background server corresponding to the shopping software currently running on the terminal device, and the commodity display request being generated by the terminal device in response to a commodity display operation and sent to the second server.

[0137] In some embodiments, the posture indication information of the target user is collected by a VR glasses worn by the target user, and the terminal device is connected with the VR glasses worn by the target user.

[0138] The first image and the second image are respectively displayed as a left-eye image and a right-eye image by the terminal device, including:

[0139] The first image is rendered by the terminal device to a left-eye display area of the VR glasses, and the second image is rendered by the terminal device to a right-eye display area of the VR glasses.

[0140] For the apparatus embodiment, since it basically corresponds to the method embodiment, the related parts are described in the part of the method embodiment. The apparatus embodiments described above are only schematic, and the modules described as separate components can or can not be physically separated, and the components displayed as modules can or can not be physical modules, i.e., they can be located in one place, or distributed on multiple network modules. According to actual needs, some or all of the modules can be selected to achieve the purpose of the scheme of the present application. Those skilled in the art can understand and implement it without creative labor.

[0141] The application further provides a first server (i.e., the 3D model server described above), as shown in FIG. 7, which is a structural schematic diagram of a first server according to an embodiment of the application. As shown in FIG. 7, the first server comprises a processor 710, a memory 720, and a network interface 730. The memory 720 is configured to store computer instructions executable on the processor 710. The processor 710 is configured to implement the commodity display method provided by any of the embodiments of the application when executing the computer instructions. The network interface 730 is configured to implement input and output functions. In more possible implementation manners, the first server can further comprise other hardware, which is not limited by the application.

[0142] The application further provides a computer readable storage medium. The computer readable storage medium can be in various forms, such as, in different examples, a RAM (Random Access Memory), a volatile memory, a non-volatile memory, a flash memory, a storage drive (such as a hard disk drive), a solid state disk, any type of storage disk (such as an optical disk, a DVD, etc.), or similar storage medium, or a combination thereof. In particular, the computer readable medium can also be paper or other suitable medium capable of printing programs. The computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the commodity display method provided by any of the embodiments of the application.

[0143] The application further provides a computer program product comprising a computer program. The computer program is executed by a processor to implement the commodity display method provided by any of the embodiments of the application.

[0144] Those skilled in the art should understand that one or more embodiments of the present application can be provided as a method, device, computing device, computer readable storage medium, or computer program product. Therefore, one or more embodiments of the present application can be in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, one or more embodiments of the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including, but not limited to, magnetic disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0145] Each of the embodiments in the present specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, for the embodiments corresponding to the computing device, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiments.

[0146] The above described embodiments of the present specification have been described. Other embodiments are within the scope of the present application. In some cases, the actions or steps recited in the present application can be performed in a different order than the order in which they are recited and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous or necessary.

[0147] Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, in tangibly-embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non-transitory program carrier for execution by, or to control the operation of, a product defect detection device. Alternatively or additionally, the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal that is generated to encode information for transmission to suitable receiver device for execution by a product defect detection device. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.

[0148] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit), and the apparatus can be implemented as special purpose logic circuitry.

[0149] Computers suitable for the execution of a computer program include, by way of example, general and / or special purpose microprocessors, or any other kind of central processing unit. Generally, a central processing unit will receive instructions and data from a read-only memory and / or a random access memory. The essential elements of a computer are a central processing unit for performing or executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive), to name just a few.

[0150] Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0151] While the specification contains many specifics, these should not be construed as limiting the scope of any invention or of any claim, but as merely providing illustrations of some of the embodiments of the inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features can be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination and the claimed combination can be directed to a subcombination or variation of a subcombination.

[0152] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring such an order, nor that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing can be advantageous. Moreover, the separation of various system modules and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated in a single software product or packaged into multiple software products.

[0153] Thus, particular embodiments of the subject matter have been described. Other embodiments are within the scope of the application. In some cases, actions recited in the claims can be performed in a different order and still achieve desirable results. In some cases, processes depicted in the figures are not necessarily sequential in nature, and thus some actions can be performed in parallel or concurrently. In some cases, certain actions can be omitted.

[0154] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

[0155] The above description is only the optional embodiments of the present application, and is not used to limit the present application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0156] It should be noted that the forming process adopted by the flow involved in the present application may, for example, include film forming processes such as deposition, sputtering, etc. and patterning processes such as etching, etc.

[0157] In the present application, the terms "first", "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. The term "a plurality of" refers to two or more, unless otherwise explicitly limited.

[0158] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

[0159] It should be understood that the present application is not limited to the precise construction that has been described above and shown in the accompanying drawings and that various modifications and changes can be made by those skilled in the art without departing from the scope of the present application. The scope of the application is indicated only by the appended claims.

Claims

1. A method of displaying merchandise, comprising: The method applied to a first server comprises: in response to a product display request of a terminal device, obtaining a 3D model of a target product, wherein the product display request is used to request to display the target product in a 3D form to a target user; based on posture indication information of the target user, obtaining a first image and a second image matched with the posture indication information from the 3D model of the target product; sending the first image and the second image to the terminal device, wherein the first image and the second image are respectively displayed as a left-eye image and a right-eye image by the terminal device.

2. The method of claim 1, wherein, The method for obtaining the 3D model of the target product comprises: inputting product information of the target product into a three-dimensional reconstruction large model to generate the 3D model of the target product through the three-dimensional reconstruction large model.

3. The method of claim 2, wherein, The three-dimensional reconstruction large model is pre-trained, and the training process of the three-dimensional reconstruction large model comprises: based on a first data set, unsupervised training is performed on an initial deep learning model to obtain an initial three-dimensional reconstruction model; based on a second data set, supervised training is performed on the initial three-dimensional reconstruction model to obtain the three-dimensional reconstruction large model.

4. The method of claim 3, wherein, Before the unsupervised training of the initial deep learning model based on the first data set, the method further comprises: performing data cleaning processing on the data in the first data set.

5. The method of claim 1, wherein, The method for obtaining the 3D model of the target product comprises: based on the product information of the target product, determining the 3D model of the target product from a plurality of alternative 3D models.

6. The method of claim 5, wherein, The plurality of alternative 3D models correspond to an alternative product respectively; and for any alternative product, the alternative 3D model of the alternative product is obtained by any of the following ways: based on product images of the alternative product taken from different perspectives, determining depth information of the alternative product to generate an alternative 3D model of the alternative product based on the depth information; based on a light pattern projected onto the alternative product and a light pattern obtained by projecting onto the alternative product, generating an alternative 3D model of the alternative product; based on product images of the alternative product taken from different perspectives and geometric relationships between different shooting perspectives, generating an alternative model of the alternative product; based on the product images of the alternative product, generating an alternative 3D model of the alternative product through a neural network model.

7. The method of claim 1, wherein, The method for obtaining the 3D model of the target product comprises any of the following: if there is an alternative 3D model corresponding to the target product, determining the 3D model of the target product from a plurality of alternative 3D models; if there is no alternative 3D model corresponding to the target product, inputting product information of the target product into a three-dimensional reconstruction large model to generate the 3D model of the target product through the three-dimensional reconstruction large model.

8. The method of claim 1, wherein, The method for obtaining the 3D model of the target product comprises any of the following: based on the posture indication information of the target user, determining a first perspective corresponding to a left eye of the target user and a second perspective corresponding to a right eye of the target user; The first image is intercepted from the 3D model of the target commodity based on the first view angle, and the second image is intercepted from the 3D model of the target commodity based on the second view angle.

9. The method of claim 1, wherein, After the first image and the second image are sent to the terminal device, the method further comprises: receiving posture indication information sent by the terminal device in real time; Upon receiving each posture indication information, a first image and a second image matched with the currently received posture indication information are re-acquired from the 3D model of the target commodity based on the currently received posture indication information; The currently acquired first image and second image are re-sent to the terminal device.

10. The method of claim 1, wherein, Before the 3D model of the target commodity is acquired in response to the commodity display request of the terminal device, the method further comprises: receiving, through a target interface, the commodity display request sent by a second server, the second server being a background server corresponding to a shopping software currently running on the terminal device, and the commodity display request being generated by the terminal device in response to a commodity display operation and sent to the second server.

11. The method of claim 1, wherein, The posture indication information of the target user is collected by a VR glasses worn by the target user, and the terminal device is connected with the VR glasses worn by the target user; The first image and the second image are respectively displayed as left-eye images and right-eye images by the terminal device, comprising: The first image is rendered to a left-eye display area of the VR glasses by the terminal device, and the second image is rendered to a right-eye display area of the VR glasses by the terminal device.

12. A merchandise display device, comprising: The apparatus is applied to a first server, and comprises: a first acquisition module configured to acquire a 3D model of a target commodity in response to a commodity display request of a terminal device, wherein the commodity display request is used to request to display the target commodity in a 3D form to a target user; a second acquisition module configured to acquire, based on posture indication information of the target user, a first image and a second image matched with the posture indication information from the 3D model of the target commodity; a sending module configured to send the first image and the second image to the terminal device, wherein the first image and the second image are respectively displayed as left-eye images and right-eye images by the terminal device.

13. A first server, characterized by The first server comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements operations performed by the commodity display method according to any one of claims 1 to 11 when executing the computer program.

14. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a program, and the program is executed by the processor to implement operations performed by the commodity display method according to any one of claims 1 to 11.

Images