A virtual simulation model generation method based on GAN architecture variant

Abstract

The present invention discloses a virtual simulation model generation method based on a GAN architecture variant, and relates to the technical field of virtual simulation modeling. The method is to first obtain multiple 3D images of a prototype virtual simulation model at multiple different viewing angles, and then for each viewing angle, import the corresponding 3D image into a 3D image generation network and a 3D image true and false discrimination network in the GAN architecture variant, output a corresponding new 3D image, and then generate a new virtual simulation model based on the new 3D image of each viewing angle by using an image-based virtual simulation model construction method, and finally, when there is no virtual simulation model similar to the new virtual simulation model in the judgment library, the new virtual simulation model is added to the virtual simulation model library, so that a new virtual simulation model with differentiation from the existing virtual simulation model can be automatically obtained without collecting a sequence of photos of the new object, so that the virtual simulation model library can be enriched at a lower cost and in a shorter period.

Description

Technical Field

[0001] The present invention belongs to the technical field of virtual simulation modeling, and in particular relates to a virtual simulation model generation method based on a GAN architecture variant. Background Art

[0002] Virtual simulation is a method of simulating the real world through computer technology and mathematical models. It can help people conduct experiments, training and simulation in a virtual environment to better understand and predict various phenomena such as physics, chemistry, and biology. Virtual simulation models are one of the important materials for building virtual environments. Specifically, they are 3D (3D) models, including 3D models of various buildings, people, vegetation, and machinery.

[0003] At present, there are three main methods for constructing virtual simulation models: (1) manually constructing virtual simulation models based on 3D modeling software. This method requires operators to have rich professional knowledge and be proficient in using modeling software. It is also complicated to operate and takes a long time. At the same time, the virtual simulation model finally constructed is not very realistic; (2) constructing virtual simulation models based on 3D scanners. This method requires expensive hardware equipment such as 3D scanners. Moreover, 3D scanners can only obtain the position information of objects. For the texture features of the object surface, most of them still require a lot of manual work to complete, making the whole process costly and time-consuming; (3) constructing virtual simulation models based on images. This method only requires a set of sequential photos of the object at different angles, and the virtual simulation model of the object can be automatically generated with the help of a computer.

[0004] Although the above-mentioned solution of building a virtual simulation model based on images has the characteristics of simple operation, high degree of automation, low cost and strong sense of reality, it still requires the pre-collection of sequence photos of the target object, which limits the extent to which the modeling cost can be reduced and the modeling cycle can be shortened. Therefore, how to automatically generate a differentiated new virtual simulation model based on the existing virtual simulation model so that the virtual simulation model library can be enriched at a lower cost and in a shorter period without collecting sequence photos of new objects is a topic that technicians in this field urgently need to study. Summary of the invention

[0005] The purpose of the present invention is to provide a virtual simulation model generation method based on a GAN architecture variant, a computer device, a computer-readable storage medium and a computer program product, so as to solve the problem that the existing solutions for building virtual simulation models based on images are limited in reducing the modeling cost and shortening the modeling cycle.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, a virtual simulation model generation method based on a GAN architecture variant is provided, comprising:

[0008] Extracting a virtual simulation model from a virtual simulation model library as a prototype virtual simulation model;

[0009] Acquire a plurality of 3D images of the prototype virtual simulation model at a plurality of different viewing angles, wherein the plurality of 3D images correspond one-to-one to the plurality of viewing angles;

[0010] For each of the multiple perspectives, the corresponding 3D image is imported into the 3D image generation network in the GAN architecture variant, and the corresponding new 3D image is outputted. Then, the corresponding 3D image and the new 3D image are imported into the 3D image true-false discrimination network in the GAN architecture variant, and the probability of the corresponding new 3D image belonging to the real category or the false category is outputted. If the probability that the corresponding new 3D image belongs to the real category is not within a preset first probability range or the probability that the corresponding new 3D image belongs to the false category is not within a preset second probability range, the 3D image generation network and the 3D image true-false discrimination network are applied to continue to generate the corresponding new 3D image and make a true-false discrimination until the probability that the corresponding new 3D image belongs to the real category is within the first probability range or the probability that the corresponding new 3D image belongs to the false category is within the second probability range;

[0011] According to the new 3D images from each perspective, a new virtual simulation model is generated by using a method of constructing a virtual simulation model based on the image;

[0012] Determining whether there is a virtual simulation model similar to the new virtual simulation model in the virtual simulation model library;

[0013] If it is determined that there is no virtual simulation model similar to the new virtual simulation model in the virtual simulation model library, the new virtual simulation model is added to the virtual simulation model library.

[0014] Based on the above invention content, a new solution for automatically generating a differentiated new virtual simulation model based on an existing virtual simulation model is provided, that is, first obtaining multiple 3D images of the prototype virtual simulation model at different multiple viewing angles, and then for each viewing angle, importing the corresponding 3D image into the 3D image generation network and the 3D image true and false discrimination network in the GAN architecture variant, outputting the corresponding new 3D image, and then generating a new virtual simulation model based on the new 3D image of each viewing angle by adopting the method of constructing a virtual simulation model based on the image, and finally adding the new virtual simulation model to the virtual simulation model library when it is determined that there is no virtual simulation model similar to the new virtual simulation model in the library, so that a new virtual simulation model that is differentiated from the existing virtual simulation model can be automatically obtained without collecting a sequence of photos of the new object, thereby replenishing the virtual simulation model library at a lower cost and in a shorter period, which is convenient for practical application and promotion.

[0015] In a possible design, a plurality of virtual depth cameras corresponding one-to-one to the plurality of viewing angles are arranged circumferentially and equidistantly around the prototype virtual simulation model.

[0016] In a possible design, for each of the multiple perspectives, the corresponding 3D image is imported into the 3D image generation network in the GAN architecture variant, and the corresponding new 3D image is outputted. Then, the corresponding 3D image and the new 3D image are imported into the 3D image true and false discrimination network in the GAN architecture variant, and the probability of the corresponding new 3D image belonging to the real category or the false category is outputted. If the probability that the corresponding new 3D image belongs to the real category is not within a preset first probability range or the probability that the corresponding new 3D image belongs to the false category is not within a preset second probability range, the 3D image generation network and the 3D image true and false discrimination network are used to continue to generate the corresponding new 3D image and make a true and false discrimination until the probability that the corresponding new 3D image belongs to the real category is within the first probability range or the probability that the corresponding new 3D image belongs to the false category is within the second probability range, including the following steps S311 to S313:

[0017] S311. For the first a viewing angle, and the corresponding 3D image and the circumferential first image in the plurality of viewing angles The new 3D image from the perspective is imported into the 3D image generation network in the GAN architecture variant, and the output is obtained in the first The new 3D image of each perspective is annularly extended along the upper edge and corresponds to the new 3D image, and then step S312 is executed, wherein: Indicates less than A positive integer, A total number of perspectives representing the plurality of perspectives;

[0018] S312. The 3D image of the first perspective and the new 3D image are imported into the 3D image true and false discrimination network in the GAN architecture variant, and the output is the first The probability that the new 3D image from each perspective belongs to the real category or the false category is calculated, and then step S313 is executed;

[0019] S313. If the The probability that the new 3D image from the perspective belongs to the real category is not within the preset first probability range or the first probability range. If the probability that the new 3D image from the perspective belongs to the false category is not within the preset second probability range, then the process returns to step S311, otherwise the process ends at the step S312. Generate a new 3D image from each perspective.

[0020] In one possible design, the end is in the plurality of viewing angles and the circumferential After generating a new 3D image from each viewing angle, the method further includes the following steps S321 to S323:

[0021] S321. For the first circumferential viewing angle among the multiple viewing angles, the corresponding new 3D image and the first The new 3D image from the first perspective is imported into the 3D image generation network, and the output is obtained in the first The new 3D image of each perspective is circularly extended along the upper edge and corresponds to the latest 3D image, and then step S322 is executed;

[0022] S322. Import the 3D image of the first viewing angle and the latest 3D image into the 3D image true and false discrimination network, output the probability that the latest 3D image of the first viewing angle belongs to the true category or the false category, and then execute step S323;

[0023] S323. If the probability that the latest 3D image of the first perspective belongs to the real category is not within the first probability range or the probability that the latest 3D image of the first perspective belongs to the false category is not within the second probability range, return to execute step S321, otherwise end the generation of the latest 3D image for the first perspective.

[0024] In one possible design, determining whether there is a virtual simulation model similar to the new virtual simulation model in the virtual simulation model library includes:

[0025] Extracting a 2D image of the model surface of the new virtual simulation model, and also extracting a 2D image of the model surface of each virtual simulation model in the virtual simulation model library;

[0026] For each of the virtual simulation models, calculating the image similarity between the corresponding model surface 2D image and the model surface 2D image of the new virtual simulation model;

[0027] If all the image similarities are less than or equal to a preset first threshold, it is determined that there is no virtual simulation model similar to the new virtual simulation model in the virtual simulation model library; otherwise, it is determined that there is a virtual simulation model similar to the new virtual simulation model in the virtual simulation model library.

[0028] In one possible design, determining whether there is a virtual simulation model similar to the new virtual simulation model in the virtual simulation model library includes:

[0029] Performing feature extraction processing on each virtual simulation model in the virtual simulation model library to obtain a first feature vector of each virtual simulation model, and adding all the first feature vectors to the vector library;

[0030] Performing the feature extraction process on the new virtual simulation model to obtain a second feature vector of the new virtual simulation model;

[0031] Performing ANN vector retrieval on the vector library based on the second feature vector;

[0032] If any first feature vector having a similarity with the second feature vector higher than a preset second threshold is retrieved in the vector library, it is determined that there is a virtual simulation model similar to the new virtual simulation model in the virtual simulation model library; otherwise, it is determined that there is no virtual simulation model similar to the new virtual simulation model in the virtual simulation model library.

[0033] In one possible design, when it is determined that there is a virtual simulation model similar to the new virtual simulation model in the virtual simulation model library, the method further includes:

[0034] Using the new virtual simulation model as another prototype virtual simulation model;

[0035] For the other prototype virtual simulation model, generating another corresponding new virtual simulation model based on the GAN architecture variant after changing the viewing angle parameter, wherein the viewing angle parameter includes the total number of viewing angles and / or the viewing angle position;

[0036] Determining whether there is a virtual simulation model similar to the another new virtual simulation model in the virtual simulation model library;

[0037] If it is determined that there is no virtual simulation model similar to the another new virtual simulation model in the virtual simulation model library, the another new virtual simulation model is added to the virtual simulation model library.

[0038] In a second aspect, the present invention provides a computer device comprising a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the virtual simulation model generation method as described in the first aspect or any possible design of the first aspect.

[0039] In a third aspect, the present invention provides a computer-readable storage medium having instructions stored thereon, which, when executed on a computer, executes the virtual simulation model generation method as described in the first aspect or any possible design of the first aspect.

[0040] In a fourth aspect, the present invention provides a computer program product, comprising a computer program or instructions, which, when executed by a computer, implements the virtual simulation model generation method as described in the first aspect or any possible design of the first aspect.

[0041] Beneficial effects of the above scheme:

[0042] The present invention creatively provides a new solution for automatically generating a differentiated new virtual simulation model based on an existing virtual simulation model, that is, first obtaining multiple 3D images of the prototype virtual simulation model at different multiple viewing angles, and then for each viewing angle, importing the corresponding 3D image into the 3D image generation network and the 3D image authenticity discrimination network in the GAN architecture variant, outputting the corresponding new 3D image, and then generating a new virtual simulation model based on the new 3D image of each viewing angle by adopting the image-based virtual simulation model construction method, and finally adding the new virtual simulation model to the virtual simulation model library when it is determined that there is no virtual simulation model similar to the new virtual simulation model in the library, so that a new virtual simulation model that is differentiated from the existing virtual simulation model can be automatically obtained without collecting serial photos of the new object, thereby replenishing the virtual simulation model library at a lower cost and in a shorter period of time, which is convenient for practical application and promotion. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 A schematic diagram of a flow chart of a virtual simulation model generation method based on a GAN architecture variant provided in an embodiment of the present application.

[0045] Figure 2 A schematic diagram of the positional relationship between the virtual camera and the prototype virtual simulation model in the virtual space provided in an embodiment of the present application.

[0046] Figure 3 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in combination with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can be obtained based on these embodiments without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0048] It should be understood that although the terms first and second, etc. may be used herein to describe various objects, these objects should not be limited by these terms. These terms are only used to distinguish one object from another object. For example, a first object can be referred to as a second object, and similarly, a second object can be referred to as a first object without departing from the scope of the exemplary embodiments of the present invention.

[0049] It should be understood that the term "and / or" that may appear in this document is merely a description of the association relationship between associated objects, indicating that there may be three relationships. For example, A and / or B can indicate three situations: A exists alone, B exists alone, or A and B exist at the same time. For another example, A, B and / or C can indicate the existence of any one of A, B and C or any combination of them. The term " / and" that may appear in this document describes another type of association object relationship, indicating that there may be two relationships. For example, A / and B can indicate two situations: A exists alone or A and B exist at the same time. In addition, the character " / " that may appear in this document generally indicates that the previous and next associated objects are in an "or" relationship.

[0050] Example

[0051] like Figure 1 As shown, the virtual simulation model generation method provided in the first aspect of this embodiment and based on the GAN architecture variant can be, but is not limited to, executed by a computer device with certain computing resources, such as a cloud server. Figure 1 As shown, the virtual simulation model generating method may include but is not limited to the following steps S1 to S6.

[0052] S1. Extract a virtual simulation model from the virtual simulation model library as a prototype virtual simulation model.

[0053] In step S1, the virtual simulation model library is used to centrally store multiple virtual simulation models that are different and pre-built based on existing 3D modeling technologies (such as artificially building virtual simulation models based on 3D modeling software, building virtual simulation models based on 3D scanners, and building virtual simulation models based on images, etc.) or the technology described in this embodiment, for example, storing three-dimensional models of various buildings, people, vegetation, and machinery, etc. In addition, the specific extraction method of the virtual simulation model can be random extraction, or it can be specified extraction based on human-computer interaction technology in response to manual operation, such as extracting a table virtual simulation model as the prototype virtual simulation model.

[0054] S2. Acquire multiple 3D images of the prototype virtual simulation model at multiple different viewing angles, wherein the multiple 3D images correspond one-to-one to the multiple viewing angles.

[0055] In step S2, the 3D image can be conventionally acquired by a virtual depth camera with a corresponding viewing angle. In order to ensure that the multiple 3D images can form a full-body depth image of the prototype virtual simulation model, preferably, multiple virtual depth cameras corresponding to the multiple viewing angles are arranged around the prototype virtual simulation model at equal intervals. Figure 2 As shown, for example, when the multiple perspectives include a main perspective , Left View , rear view and right perspective When the plurality of virtual cameras include a primary view virtual depth camera arranged in a clockwise direction, , left-view virtual depth camera , Rear-view virtual depth camera and right-view virtual depth camera Etc., through these cameras, the front-view 3D image, left-view 3D image, rear-view 3D image and right-view 3D image of the prototype virtual simulation model can be respectively acquired, thereby forming a full-body depth image of the prototype virtual simulation model.

[0056] S3. For each of the multiple perspectives, the corresponding 3D image is imported into the 3D image generation network in the GAN architecture variant, and the corresponding new 3D image is outputted. Then, the corresponding 3D image and the new 3D image are imported into the 3D image true-false discrimination network in the GAN architecture variant, and the probability of the corresponding new 3D image belonging to the real category or the false category is outputted. If the probability that the corresponding new 3D image belongs to the real category is not within a preset first probability range or the probability that the corresponding new 3D image belongs to the false category is not within a preset second probability range, the 3D image generation network and the 3D image true-false discrimination network are applied to continue to generate the corresponding new 3D image and make a true-false discrimination until the probability that the corresponding new 3D image belongs to the real category is within the first probability range or the probability that the corresponding new 3D image belongs to the false category is within the second probability range.

[0057] In step S3, the aforementioned GAN (Generative Adversarial Nets) architecture is a new framework for estimating generative models, specifically including a generator network and a discriminator network (both of which are deep convolutional neural networks), wherein the generator network is an upsampling network, which is used to upsample a noise vector (i.e., a vector in a probability latent space), and then generate a 2D image similar in shape to the input image in length, width, and channel, and the discriminator network is a downsampling network, which is used to judge the authenticity of input data (i.e., the 2D image output by the generator network) through a series of 2D convolution operations and a fully connected layer, thereby achieving the purpose of generating a real 2D image. Based on the existing GAN architecture, a GAN architecture variant including the 3D image generator network (e.g., specifically including at least 5 volumetric full convolutional layers) and the 3D image authenticity discrimination network (e.g., specifically including at least 5 volumetric convolutional layers) and used to generate a real 3D image can be conventionally modified: 3D-GAN architecture. Specifically, for each of the multiple perspectives, the corresponding 3D image is imported into the 3D image generation network in the GAN architecture variant, and the corresponding new 3D image is outputted. Then, the corresponding 3D image and the new 3D image are imported into the 3D image true and false discrimination network in the GAN architecture variant, and the probability of the corresponding new 3D image belonging to the real category or the false category is outputted. If the probability that the corresponding new 3D image belongs to the real category is not within a preset first probability range or the probability that the corresponding new 3D image belongs to the false category is not within a preset second probability range, the 3D image generation network and the 3D image true and false discrimination network are applied to continue to generate the corresponding new 3D image and make a true and false discrimination until the probability that the corresponding new 3D image belongs to the real category is within the first probability range or the probability that the corresponding new 3D image belongs to the false category is within the second probability range, including but not limited to the following steps S301 to S303.

[0058] S301. For any perspective among the multiple perspectives, the corresponding 3D image is imported into the 3D image generation network in the GAN architecture variant, and the corresponding new 3D image is output, and then step S302 is executed.

[0059] In step S301, in order to make the new 3D image have a different directivity from the 3D image, the corresponding 3D image and the difference description keywords input by the user at the corresponding viewing angle may be specifically imported into the 3D image generation network for any viewing angle.

[0060] S302. For any of the viewing angles, the corresponding 3D image and the new 3D image are imported into the 3D image true and false discrimination network in the GAN architecture variant, and the probability that the corresponding new 3D image belongs to the real category or the false category is output, and then step S303 is executed.

[0061] S303. For any of the viewing angles, if the probability that the corresponding new 3D image belongs to the real category is not within the preset first probability range or the probability that the corresponding new 3D image belongs to the false category is not within the preset second probability range, return to step S301, otherwise end the generation of the new 3D image for the corresponding viewing angle.

[0062] In step S303, the first probability range or the second probability range can be preset based on actual conditions. For example, the first probability range is set to [0.90, 0.98], or the second probability range is set to [0.02, 0.10].

[0063] In the step S3, in order to make the new 3D images of each perspective have relevance so that a new virtual simulation model can be successfully generated based on these new 3D images later, preferably, when a plurality of virtual depth cameras corresponding to the plurality of perspectives are arranged around the prototype virtual simulation model at equal intervals in a circular direction, for each perspective in the plurality of perspectives, the corresponding 3D image is imported into the 3D image generation network in the GAN architecture variant, and the corresponding new 3D image is output, and then the corresponding 3D image and the new 3D image are imported into the 3D image true and false discrimination network in the GAN architecture variant, and the output is obtained. The probability that the corresponding new 3D image belongs to the real category or the false category. If the probability that the corresponding new 3D image belongs to the real category is not within the preset first probability range or the probability that the corresponding new 3D image belongs to the false category is not within the preset second probability range, the 3D image generation network and the 3D image true and false discrimination network are applied to continue to generate the corresponding new 3D image and make a true and false discrimination until the probability that the corresponding new 3D image belongs to the real category is within the first probability range or the probability that the corresponding new 3D image belongs to the false category is within the second probability range, including but not limited to the following steps S311 to S313.

[0064] S311. For the first a viewing angle, and the corresponding 3D image and the circumferential first image in the plurality of viewing angles The new 3D image from the perspective is imported into the 3D image generation network in the GAN architecture variant, and the output is obtained in the first The new 3D image of each perspective is annularly extended along the upper edge and corresponds to the new 3D image, and then step S312 is executed, wherein: Indicates less than A positive integer, Represents the total number of perspectives of the multiple perspectives.

[0065] In step S311, for example Figure 2 As shown, the The left perspective may be the , the rear viewing angle or the right perspective (At this time, the The perspectives are respectively corresponding to the main perspectives , the left perspective or the rear viewing angle ), that is, for the left perspective , the left-view 3D image and the main viewing angle The generated new main view 3D image is imported into the 3D image generation network to generate a new left view 3D image extending to the left side of the new main view 3D image (because it is clockwise); for the rear view , the rear-view 3D image and the left-view image can be The generated new left-view 3D image is imported into the 3D image generation network to generate a new rear-view 3D image extending to the left side of the new left-view 3D image (because it is clockwise); for the right view , the right-view 3D image and the rear-view image can be The generated new rear-view 3D image is introduced into the 3D image generation network to generate a new right-view 3D image extending to the left side of the new rear-view 3D image (because it is clockwise). When equal to 1, the The new 3D image of the first perspective can be generated based on, but not limited to, the aforementioned steps S301 to S303. The corresponding 3D image and the difference description keywords input by the user at the corresponding viewing angle are imported into the 3D image generation network.

[0066] S312. The 3D image of the first perspective and the new 3D image are imported into the 3D image true and false discrimination network in the GAN architecture variant, and the output is the first The probability that the new 3D image from each perspective belongs to the real category or the false category is calculated, and then step S313 is performed.

[0067] S313. If the The probability that the new 3D image from the perspective belongs to the real category is not within the preset first probability range or the first probability range. If the probability that the new 3D image from the perspective belongs to the false category is not within the preset second probability range, then the process returns to step S311, otherwise the process ends at the step S312. Generate a new 3D image from each perspective.

[0068] In step S3, in order to make the first The new 3D image of the first viewing angle in the plurality of viewing angles is associated with the new 3D image of the first viewing angle in the circumferential direction (refer to Figure 2 , that is, to make the new right-view 3D image correlated with the new main-view 3D image), so that a new virtual simulation model can be successfully generated based on these new 3D images later, and further preferably, at the end of the first one in the plurality of viewing angles and in the circumferential direction After generating a new 3D image from each viewing angle, the method further includes but is not limited to the following steps S321 to S323.

[0069] S321. For the first circumferential viewing angle among the multiple viewing angles, the corresponding new 3D image and the first The new 3D image from the first perspective is imported into the 3D image generation network, and the output is obtained in the first The new 3D image of each viewing angle is annularly extended along the upper edge and corresponds to the latest 3D image, and then step S322 is executed.

[0070] In step S321, referring to Figure 2 , that is, for the main perspective , the new main 3D image and the right viewing angle The generated new right-view 3D image is imported into the 3D image generation network to generate the latest main-view 3D image that is extended to the left side of the new right-view 3D image (because it is clockwise), to replace the new main-view 3D image generated based on the aforementioned steps S301 to S303. In addition, in order to make the latest 3D image and the 3D image have different directivity, the corresponding new 3D image and the difference description keywords input by the user and at the corresponding perspective can also be specifically imported into the 3D image generation network for the first perspective.

[0071] S322. Import the 3D image of the first perspective and the latest 3D image into the 3D image true and false discrimination network, output the probability that the latest 3D image of the first perspective belongs to the real category or the false category, and then execute step S323.

[0072] S323. If the probability that the latest 3D image of the first perspective belongs to the real category is not within the first probability range or the probability that the latest 3D image of the first perspective belongs to the false category is not within the second probability range, return to execute step S321, otherwise end the generation of the latest 3D image for the first perspective.

[0073] S4. According to the new 3D images from each perspective, a new virtual simulation model is generated by adopting a method of constructing a virtual simulation model based on the image.

[0074] S5. Determine whether there is a virtual simulation model similar to the new virtual simulation model in the virtual simulation model library.

[0075] In the step S5, specifically, it is determined whether there is a virtual simulation model similar to the new virtual simulation model in the virtual simulation model library, including but not limited to the following steps S511 to S513.

[0076] S511. Extract a 2D image of the model surface of the new virtual simulation model, and also extract a 2D image of the model surface of each virtual simulation model in the virtual simulation model library.

[0077] S512. For each virtual simulation model, calculate the image similarity between the corresponding model surface 2D image and the model surface 2D image of the new virtual simulation model.

[0078] In step S512, the specific calculation process of the image similarity is prior art and will not be described in detail herein.

[0079] S513. If the similarities of all the images are less than or equal to a preset first threshold, it is determined that there is no virtual simulation model similar to the new virtual simulation model in the virtual simulation model library; otherwise, it is determined that there is a virtual simulation model similar to the new virtual simulation model in the virtual simulation model library.

[0080] In the step S513, the first threshold may be specifically set according to actual conditions, for example, 0.618.

[0081] S6. If it is determined that there is no virtual simulation model similar to the new virtual simulation model in the virtual simulation model library, then the new virtual simulation model is added to the virtual simulation model library.

[0082] In step S6, if it is determined that there is a virtual simulation model similar to the new virtual simulation model in the virtual simulation model library, the new virtual simulation model may be discarded to avoid duplication of virtual simulation models in the library.

[0083] Therefore, based on the virtual simulation model generation method described in the aforementioned steps S1 to S6, a new solution for automatically generating a differentiated new virtual simulation model based on an existing virtual simulation model is provided, that is, firstly obtaining multiple 3D images of the prototype virtual simulation model at multiple different perspectives, and then for each perspective, importing the corresponding 3D image into the 3D image generation network and the 3D image authenticity discrimination network in the GAN architecture variant, outputting the corresponding new 3D image, and then generating a new virtual simulation model based on the new 3D image of each perspective by adopting the method of constructing a virtual simulation model based on the image, and finally, when it is determined that there is no virtual simulation model similar to the new virtual simulation model in the library, the new virtual simulation model is added to the virtual simulation model library, so that a new virtual simulation model that is differentiated from the existing virtual simulation model can be automatically obtained without collecting a sequence of photos of the new object, and then the virtual simulation model library can be enriched at a lower cost and in a shorter period, which is convenient for practical application and promotion.

[0084] Based on the technical solution of the first aspect, the present embodiment also provides a possible design for how to quickly determine whether there is a virtual simulation model similar to the new virtual simulation model in the library, that is, to determine whether there is a virtual simulation model similar to the new virtual simulation model in the virtual simulation model library, including but not limited to the following steps S521 to S524.

[0085] S521. Perform feature extraction processing on each virtual simulation model in the virtual simulation model library to obtain a first feature vector of each virtual simulation model, and add all the first feature vectors to the vector library.

[0086] In the step S521, the specific process of the feature extraction processing is prior art and will not be described in detail here.

[0087] S522. Perform the feature extraction process on the new virtual simulation model to obtain a second feature vector of the new virtual simulation model.

[0088] S523. Perform ANN vector search on the vector library based on the second feature vector.

[0089] In step S523, the ANN (Approximate Nearest Neighbor) vector retrieval is an existing technology for processing similarity search in large-scale data sets, which is mainly used to quickly find data items similar to the query object in large data sets; this technology has applications in many fields, including vector databases, retrieval enhancement generation, large-scale information retrieval, recommendation systems, drug discovery, and image search. The core of ANN vector retrieval is to use efficient indexing and search algorithms to approximately find the nearest neighbors in the data set (note that it is not accurate to find the nearest neighbors). This approximate method allows a certain error while ensuring search efficiency, thereby significantly improving the search speed when processing large-scale data sets. In general, ANN vector retrieval technology can significantly improve the efficiency of similarity search in large-scale data sets while ensuring that the retrieval results have a certain degree of accuracy by using efficient indexing and search algorithms, as well as specific graph methods such as HNSW (Hierarchical Navigable Small World Graph, which is a graph method for vector retrieval. It is based on the idea of ​​skip lists and achieves efficient approximate nearest neighbor search by constructing a navigable small world graph. It organizes data by using a navigable small world graph at each layer to form a hierarchical structure, so that the retrieval process can gradually approach the target vector space from top to bottom, thereby improving the search efficiency. In the process of graph composition, HNSW can ensure the connectivity of the graph by cutting the edges to ensure the smooth progress of the search process). Therefore, by performing ANN vector retrieval on the vector library based on the second feature vector, the retrieval results can be obtained accurately and efficiently.

[0090] S524. If any first feature vector having a similarity with the second feature vector higher than a preset second threshold value is retrieved in the vector library, it is determined that there is a virtual simulation model similar to the new virtual simulation model in the virtual simulation model library; otherwise, it is determined that there is no virtual simulation model similar to the new virtual simulation model in the virtual simulation model library.

[0091] In step S524, the similarity may be based on, but not limited to, cosine similarity (which is obtained by measuring the similarity of two vector Angle CosineThe similarity between them is measured by the cosine value: the cosine value of the 0 degree angle is 1, while the cosine value of any other angle is not greater than 1, and its minimum value is -1, so the cosine value of the angle between two vectors determines whether the two vectors point roughly in the same direction; when the two vectors have the same direction, the cosine similarity value is 1; when the angle between the two vectors is 90°, the cosine similarity value is 0; and when the two vectors point in completely opposite directions, the cosine similarity value is -1; the aforementioned cosine similarity result is independent of the length of the vector, but only related to the direction of the vector; cosine similarity is usually used in positive space, so the value given is between -1 and 1) to obtain the conventional calculation. Specifically, when the similarity is expressed as a percentage, the second threshold value can be, but is not limited to, 61.8%.

[0092] Therefore, based on the aforementioned possible design one, it is also possible to avoid calculating the image similarity between each virtual simulation model in the library and the new virtual simulation model by performing ANN vector retrieval, and then quickly and accurately obtain the judgment result of whether there is a similar model in the library, further shortening the modeling cycle.

[0093] Based on the technical solution of the first aspect or possible design one, this embodiment also provides a possible design two for how to evolve and update the new virtual simulation model when a similar model exists in the library, that is, when it is determined that there is a virtual simulation model similar to the new virtual simulation model in the virtual simulation model library, the method also includes but is not limited to the following steps S701 to S704.

[0094] S701. Use the new virtual simulation model as another prototype virtual simulation model.

[0095] S702. For the other prototype virtual simulation model, generate another corresponding new virtual simulation model based on the GAN architecture variant after changing the viewing angle parameters, wherein the viewing angle parameters include but are not limited to the total number of viewing angles and / or viewing angle positions, etc.

[0096] In the step S702, the specific generation process of the other new virtual simulation model can refer to the aforementioned steps S2 to S4, which will not be described in detail here.

[0097] S703. Determine whether there is a virtual simulation model similar to the other new virtual simulation model in the virtual simulation model library.

[0098] In the step S703, the specific judgment process may refer to the aforementioned steps S511 to S513 or steps S521 to S524, which will not be described in detail here.

[0099] S704. If it is determined that there is no virtual simulation model similar to the another new virtual simulation model in the virtual simulation model library, then the another new virtual simulation model is added to the virtual simulation model library.

[0100] In step S704, if it is determined that there is a virtual simulation model similar to the other new virtual simulation model in the virtual simulation model library, another round of steps S701 to S704 may be performed until the generated new virtual simulation model is added to the virtual simulation model library.

[0101] Therefore, based on the aforementioned possible design 2, the new virtual simulation model can also be evolved and updated when similar models exist in the library until the generated new virtual simulation model is added to the virtual simulation model library, thereby avoiding waste of modeling results and further reducing modeling costs.

[0102] like Figure 3 As shown, the second aspect of this embodiment provides a computer device for executing the virtual simulation model generation method as described in the first aspect or any possible design in the first aspect, including a memory, a processor and a transceiver that are sequentially connected in communication, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program, and execute the virtual simulation model generation method as described in the first aspect or any possible design in the first aspect. For example, the memory may include, but is not limited to, a random access memory (Random-Access Memory, RAM), a read-only memory (Read-Only Memory, ROM), a flash memory (Flash Memory), a first-in-first-out memory (First Input First Output, FIFO) and / or a first-in-last-out memory (First Input Last Output, FILO), etc.; the processor may include, but is not limited to, a microprocessor of the STM32F105 series. In addition, the computer device may also include, but is not limited to, a power module, a display screen and other necessary components.

[0103] The working process, working details and technical effects of the aforementioned computer device provided in the second aspect of this embodiment can be referred to the virtual simulation model generation method described in the first aspect or any possible design of the first aspect, and will not be repeated here.

[0104] The third aspect of this embodiment provides a computer-readable storage medium storing instructions including the method for generating a virtual simulation model as described in the first aspect or any possible design in the first aspect, that is, the computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the method for generating a virtual simulation model as described in the first aspect or any possible design in the first aspect is executed. Wherein, the computer-readable storage medium refers to a carrier for storing data, which may include but is not limited to computer-readable storage media such as floppy disks, optical disks, hard disks, flash memories, USB flash drives and / or memory sticks, and the computer may be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices.

[0105] The working process, working details and technical effects of the aforementioned computer-readable storage medium provided in the third aspect of this embodiment can be referred to the virtual simulation model generation method described in the first aspect or any possible design in the first aspect, and will not be repeated here.

[0106] A fourth aspect of the present embodiment provides a computer program product, including a computer program or instructions, wherein the computer program or instructions, when executed by a computer, implements the virtual simulation model generation method as described in the first aspect or any possible design in the first aspect. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0107] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A virtual simulation model generation method based on GAN architecture variant, characterized in that: include: Extracting a virtual simulation model from a virtual simulation model library as a prototype virtual simulation model; Acquire a plurality of 3D images of the prototype virtual simulation model at a plurality of different viewing angles, wherein the plurality of 3D images correspond one to one to the plurality of viewing angles; For each of the multiple perspectives, the corresponding 3D image is imported into the 3D image generation network in the GAN architecture variant, and the corresponding new 3D image is outputted. Then, the corresponding 3D image and the new 3D image are imported into the 3D image true-false discrimination network in the GAN architecture variant, and the probability of the corresponding new 3D image belonging to the real category or the false category is outputted. If the probability that the corresponding new 3D image belongs to the real category is not within a preset first probability range or the probability that the corresponding new 3D image belongs to the false category is not within a preset second probability range, the 3D image generation network and the 3D image true-false discrimination network are applied to continue to generate the corresponding new 3D image and make a true-false discrimination until the probability that the corresponding new 3D image belongs to the real category is within the first probability range or the probability that the corresponding new 3D image belongs to the false category is within the second probability range; According to the new 3D images from each perspective, a new virtual simulation model is generated by using a method of constructing a virtual simulation model based on the image; Determining whether there is a virtual simulation model similar to the new virtual simulation model in the virtual simulation model library; If it is determined that there is no virtual simulation model similar to the new virtual simulation model in the virtual simulation model library, the new virtual simulation model is added to the virtual simulation model library.

2. The virtual simulation model generation method according to claim 1, characterized in that: A plurality of virtual depth cameras corresponding to the plurality of viewing angles are circumferentially arranged at equal intervals around the prototype virtual simulation model.

3. The virtual simulation model generation method according to claim 2, characterized in that: For each of the multiple perspectives, the corresponding 3D image is imported into the 3D image generation network in the GAN architecture variant, and the corresponding new 3D image is outputted. Then, the corresponding 3D image and the new 3D image are imported into the 3D image true-false discrimination network in the GAN architecture variant, and the probability of the corresponding new 3D image belonging to the real category or the false category is outputted. If the probability that the corresponding new 3D image belongs to the real category is not within a preset first probability range or the probability that the corresponding new 3D image belongs to the false category is not within a preset second probability range, the 3D image generation network and the 3D image true-false discrimination network are applied to continue to generate the corresponding new 3D image and make a true-false discrimination until the probability that the corresponding new 3D image belongs to the real category is within the first probability range or the probability that the corresponding new 3D image belongs to the false category is within the second probability range, including the following steps S311 to S313: S311. For the first a viewing angle, and the corresponding 3D image and the circumferential first image in the plurality of viewing angles The new 3D image from the perspective is imported into the 3D image generation network in the GAN architecture variant, and the output is obtained in the first The new 3D image of each perspective is annularly extended and corresponds to the new 3D image, and then step S312 is executed, wherein: Indicates less than A positive integer, A total number of perspectives representing the plurality of perspectives; S312. The 3D image of the first perspective and the new 3D image are imported into the 3D image true and false discrimination network in the GAN architecture variant, and the output is the first The probability that the new 3D image from each perspective belongs to the real category or the false category is calculated, and then step S313 is executed; S313. If the The probability that the new 3D image from the perspective belongs to the real category is not within the preset first probability range or the first probability range. If the probability that the new 3D image from the perspective belongs to the false category is not within the preset second probability range, then the process returns to step S311, otherwise the process ends at the step S312. Generate a new 3D image from each perspective.

4. The virtual simulation model generation method according to claim 3, characterized in that: At the end of the first After generating a new 3D image from each viewing angle, the method further includes the following steps S321 to S323: S321. For the first circumferential viewing angle among the multiple viewing angles, the corresponding new 3D image and the first The new 3D image from the first perspective is imported into the 3D image generation network, and the output is obtained in the first The new 3D image of each perspective is circularly extended along the upper edge and corresponds to the latest 3D image, and then step S322 is executed; S322. Import the 3D image of the first viewing angle and the latest 3D image into the 3D image true and false discrimination network, output the probability that the latest 3D image of the first viewing angle belongs to the true category or the false category, and then execute step S323; S323. If the probability that the latest 3D image of the first perspective belongs to the real category is not within the first probability range or the probability that the latest 3D image of the first perspective belongs to the false category is not within the second probability range, return to execute step S321, otherwise end the generation of the latest 3D image for the first perspective.

5. The virtual simulation model generation method according to claim 1, characterized in that: Determining whether there is a virtual simulation model similar to the new virtual simulation model in the virtual simulation model library includes: Extracting a 2D image of the model surface of the new virtual simulation model, and also extracting a 2D image of the model surface of each virtual simulation model in the virtual simulation model library; For each of the virtual simulation models, calculating the image similarity between the corresponding model surface 2D image and the model surface 2D image of the new virtual simulation model; If all the image similarities are less than or equal to a preset first threshold, it is determined that there is no virtual simulation model similar to the new virtual simulation model in the virtual simulation model library; otherwise, it is determined that there is a virtual simulation model similar to the new virtual simulation model in the virtual simulation model library.

6. The virtual simulation model generation method according to claim 1, characterized in that: Determining whether there is a virtual simulation model similar to the new virtual simulation model in the virtual simulation model library includes: Performing feature extraction processing on each virtual simulation model in the virtual simulation model library to obtain a first feature vector of each virtual simulation model, and adding all the first feature vectors to the vector library; Performing the feature extraction process on the new virtual simulation model to obtain a second feature vector of the new virtual simulation model; Performing ANN vector retrieval on the vector library based on the second feature vector; If any first feature vector having a similarity with the second feature vector higher than a preset second threshold is retrieved in the vector library, it is determined that there is a virtual simulation model similar to the new virtual simulation model in the virtual simulation model library; otherwise, it is determined that there is no virtual simulation model similar to the new virtual simulation model in the virtual simulation model library.

7. The virtual simulation model generation method according to claim 1, characterized in that: When it is determined that there is a virtual simulation model similar to the new virtual simulation model in the virtual simulation model library, the method further includes: Using the new virtual simulation model as another prototype virtual simulation model; For the other prototype virtual simulation model, generating another corresponding new virtual simulation model based on the GAN architecture variant after changing the viewing angle parameter, wherein the viewing angle parameter includes the total number of viewing angles and / or the viewing angle position; Determining whether there is a virtual simulation model similar to the another new virtual simulation model in the virtual simulation model library; If it is determined that there is no virtual simulation model similar to the another new virtual simulation model in the virtual simulation model library, the another new virtual simulation model is added to the virtual simulation model library.

8. A computer device, characterized in that: It includes a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the virtual simulation model generation method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed on the computer, the virtual simulation model generation method as described in any one of claims 1 to 7 is executed.

10. A computer program product comprising a computer program or instructions, characterized in that When the computer program or the instruction is executed by a computer, the virtual simulation model generation method according to any one of claims 1 to 7 is implemented.

Images