A method and system for generating virtual images of artworks

Abstract

The present invention discloses a method and system for generating virtual images of artworks, which relates to the technical field of virtual image generation, including uniformly arranging cameras to synchronously shoot images of artworks, performing stereo matching on the images of artworks, calculating disparity to establish a disparity map, and converting the disparity map into a depth map; processing the depth map, identifying regions and objects in the depth map, determining the image texture type according to the attributes of the regions and objects, and synthesizing high-quality textures; constructing a generative adversarial network, generating virtual images of artworks based on the depth map and high-quality textures; and displaying and storing virtual views of artworks. The present invention improves the quality of generating virtual images of artworks by shooting multi-directional images of artworks for processing to obtain a depth map of the artworks, identifying regions and objects in the depth map, and synthesizing high-quality textures of the artworks according to the attributes of the regions and objects.

Description

Technical Field

[0001] The present invention relates to the technical field of virtual image generation, and in particular to a method and system for generating virtual images of artworks. Background Art

[0002] In recent years, with the rapid development of computer vision and image processing technology, virtual image generation has been widely used in art creation, film and television production, virtual reality and other fields. Traditional image generation methods mainly rely on manual design and image processing software to generate high-quality works of art by manually adjusting the color, texture and light and shadow effects of the image. However, this method is time-consuming and labor-intensive, requires high professional skills from the creator, and it is difficult to achieve highly realistic virtual images. With the introduction of deep learning and generative adversarial networks (GAN), image generation technology has made major breakthroughs. GAN can automatically generate high-quality images through the mutual game between the generator and the discriminator, greatly simplifying the process of virtual image generation. However, the accuracy and quality of existing technologies in depth map processing and texture generation still need to be improved. The generated images often have problems with texture distortion and missing details, resulting in the lack of three-dimensionality and authenticity of the generated virtual images. Summary of the invention

[0003] In view of the problems existing in the above-mentioned existing virtual image generation method and system for artworks, the present invention is proposed.

[0004] Therefore, the problem to be solved by the present invention is that the accuracy and quality of the prior art in depth map processing and texture generation still need to be improved, and the generated images often have problems of texture distortion and missing details, resulting in the generated virtual images lacking three-dimensionality and authenticity.

[0005] To solve the above technical problems, the present invention provides the following technical solutions: a method for generating a virtual image of an artwork, comprising: evenly arranging cameras to synchronously capture images of the artwork, performing stereo matching on the images of the artwork, calculating disparity to establish a disparity map, and converting the disparity map into a depth map; processing the depth map, identifying regions and objects in the depth map, determining the image texture type according to the attributes of the regions and objects, and synthesizing high-quality textures; constructing a generative adversarial network to generate a virtual image of the artwork based on the depth map and the high-quality texture; and displaying and storing the virtual view of the artwork.

[0006] As a preferred solution of the method for generating a virtual image of an artwork according to the present invention, the evenly arranged cameras for synchronously capturing the artwork image refers to placing the artwork in the center of a closed room, evenly arranging cameras around the artwork, connecting all cameras to a central console, the central console using a calibration tool to calibrate the parameters of each camera and unify the shooting time of all cameras, the cameras synchronously capturing all-round artwork images and transmitting the images to the central console.

[0007] As a preferred solution of the method for generating a virtual image of an artwork according to the present invention, the steps of performing stereo matching on the artwork image, calculating disparity to establish a disparity map, and converting the disparity map into a depth map include:

[0008] Select the artwork images as the left and right views and calculate the matching cost:

[0009] C(x, y) = || I L (x, y)-I R (x, y)||,

[0010] Where C(x, y) is the matching cost of pixel point (x, y), I L (x, y) is the gray value of the pixel (x, y) in the left view, I R (x, y) is the grayscale value of the pixel (x, y) in the right view;

[0011] Aggregate using the aggregator window based on the matching cost:

[0012]

[0013] Where C'' is the matching cost after aggregation, W is the aggregation window, and (u, v) is the offset of the pixel point (x, y) relative to the center of the aggregation window;

[0014] Initialize the aggregation window size and use a greedy algorithm to select the disparity by minimizing the post-aggregation matching cost:

[0015] d(x,y)=argminC'',

[0016] Where d(x, y) is the disparity of the pixel point (x, y), and the disparity of all pixels is calculated to construct a disparity map;

[0017] Calculate the depth of the pixel (x, y) based on the disparity:

[0018]

[0019] Where D(x, y) is the depth of the pixel point (x, y), B is the baseline distance of the camera, and f is the focal length of the camera. The depth is calculated by traversing all the pixels in the disparity map and stored in the corresponding position to form a depth map of the artwork.

[0020] As a preferred solution of the method for generating a virtual image of an artwork of the present invention, the processing of the depth map and identifying regions and objects in the depth map refers to smoothing the artwork depth map using a Gaussian filter, and calculating the gradient of the smoothed artwork depth map using a Sobel operator to obtain the gradient amplitude and gradient direction, performing edge detection on the depth map, dividing pixel points in the depth map into edge points and non-edge points, connecting edge points to form edge lines and marking them on the depth map, and combining pixel points enclosed by the edge lines to form pixel blocks;

[0021] A UNet model is constructed and trained, and the trained UNet model is used to perform semantic segmentation on the depth map, outputting the semantic label of each pixel in the depth map, and identifying the pixel block category according to the semantic label, wherein the category includes region and object.

[0022] As a preferred solution of the virtual image generation method for artworks of the present invention, the step of determining the image texture type according to the region and object attributes and synthesizing high-quality textures means that after identifying the region and object in the depth map of the artwork, the texture type is determined according to the actual attributes of the region and object, and the texture parameters in the depth map are calculated:

[0023] A k =mean(D(x,y)|(x,y)∈R i ),

[0024]

[0025] Z k =std(D(x,y)|(x,y)∈R i ),

[0026] Among them A k is the amplitude of the kth pixel, q is the frequency, R i is the set of pixels in the i-th pixel block, Z k is the detail intensity of the k-th pixel, std represents the standard deviation of the pixel depth values ​​in the i-pixel block, and s is the texture scale;

[0027] Apply the Perlin algorithm to generate pixel base texture values:

[0028]

[0029] Where P k(x, y) is the base texture value of the k-th pixel, G(o, j) is a pseudo-random gradient generated by a pseudo-random number generator, Δx and Δy are interpolation intervals, the depth map is divided into grids according to the interpolation intervals, and o and j are the coordinates of the grid points;

[0030] Apply a detail enhancement algorithm to enhance the base texture value:

[0031] Q k (x, y) = P k (x,y)+Z k *sin(q*x)cos(q*y),

[0032] Where Q k (x, y) is the detail texture value of the k-th pixel;

[0033] Calculate the texture value synthesis weight according to the detail intensity:

[0034]

[0035] where α k is the basic texture value synthesis weight, β k is the detail texture value synthesis weight, max Z is the maximum detail intensity;

[0036] Synthesize high-quality textures based on texture value synthesis weights:

[0037] T(x, y) = α k *P k (x,y)+β k *Q k (x, y),

[0038] Where T(x, y) is the high-quality texture value of the pixel point (x, y);

[0039] Create an empty image matrix of the same size as the depth map, and fill the high-quality texture value calculated for each pixel into the empty image matrix to form a high-quality texture of the artwork.

[0040] As a preferred solution of the method for generating a virtual image of an artwork described in the present invention, wherein: the construction of a generative adversarial network to generate a virtual image of an artwork based on a depth map of the artwork and a high-quality texture of the artwork refers to the construction of a generative adversarial network, including a generator and a discriminator, wherein the generator includes a generative input layer, a generative hidden layer, and a generative output layer, wherein the generative input layer receives the depth map and the high-quality texture, and the generative output layer outputs the virtual image, and the discriminator includes a discriminative input layer, a discriminative hidden layer, and a discriminative output layer, wherein the discriminative input layer receives the image, and the discriminative output layer outputs the discriminative result;

[0041] Collect real artwork images, real image depth maps, and real image textures to form a training set, use the training set to train the generator and discriminator, synchronously calculate the training losses of the generator and discriminator, and iteratively update the parameters of the generator and discriminator. If the training losses of the generator and discriminator no longer decrease significantly during continuous iterations, stop the iteration and use the updated generator and discriminator parameters to update the generator and discriminator.

[0042] The depth map and high-quality texture of the artwork are input into the generator to obtain a virtual image of the artwork. The discriminator is used to judge the difference between the virtual image and the artwork. If the virtual image corresponds to the artwork, the virtual image of the artwork is output. If not, the generator and discriminator are retrained.

[0043] As a preferred solution of the method for generating virtual images of artworks described in the present invention, wherein: the displaying and storing the virtual images of artworks refers to that after the central control console obtains the virtual images of artworks, the virtual images and the photographed images are synchronously stored in the same category of the database, and the virtual images and the photographed images of artworks are displayed on the same screen and the artwork information is annotated.

[0044] Another object of the present invention is to provide a virtual image generation system for an artwork, comprising:

[0045] An image acquisition module is used to deploy and control cameras to capture images of artworks from multiple angles and send them to a central control console;

[0046] A depth processing module, used to perform stereo matching on the artwork image to calculate the disparity to form a disparity map, and calculate the depth map of the artwork based on the disparity map;

[0047] Texture synthesis module, used to identify the depth map areas and objects of the artwork and synthesize high-quality textures of the artwork based on the areas and objects;

[0048] An image generation module is used to construct and train a generative adversarial network to generate a virtual image of an artwork using the artwork depth map and high-quality texture of the artwork;

[0049] The storage and display module is used to store and display the generated virtual images of the artworks.

[0050] A computer device comprises: a memory and a processor; the memory stores a computer program, and the processor implements the steps of the above-mentioned method for generating a virtual image of an artwork when executing the computer program.

[0051] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned method for generating a virtual image of an artwork.

[0052] The beneficial effects of the present invention are as follows: the present invention obtains a depth map of the artwork by photographing multi-directional images of the artwork and processing them, identifies regions and objects in the depth map, and synthesizes high-quality textures of the artwork according to the attributes of the regions and objects, thereby improving the quality of generating a virtual image of the artwork. At the same time, a training generative adversarial network is constructed to generate a virtual image of the artwork using the depth map and high-quality texture, thereby improving the efficiency and accuracy of generating a virtual image of the artwork. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0054] Figure 1 A flowchart of a method for generating a virtual image of an artwork is shown.

[0055] Figure 2 Schematic diagram of the structure of a virtual image generation system for artworks. DETAILED DESCRIPTION

[0056] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.

[0057] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0058] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or selective embodiment that is mutually exclusive with other embodiments.

[0059] Example 1

[0060] Reference Figure 1 , which is the first embodiment of the present invention, and which provides a method for generating a virtual image of an artwork, the method for generating a virtual image of an artwork comprising:

[0061] S1. Evenly arrange cameras to synchronously capture images of artworks, perform stereo matching on the images of artworks, calculate disparity to establish a disparity map, and convert the disparity map into a depth map;

[0062] Specifically, evenly arranging cameras to synchronously capture images of artworks means placing the artwork in the center of a closed room, evenly arranging cameras around the artwork, connecting all cameras to a central console, using a calibration tool on the central console to calibrate parameters of each camera and unify the shooting time of all cameras, and the cameras synchronously capture all-round images of the artwork and transmit the images to the central console.

[0063] By evenly arranging the cameras, the images of the artworks can be fully captured from multiple angles. This multi-angle image capture method can obtain more comprehensive and detailed information about the artworks, providing a rich data foundation for subsequent 3D reconstruction and virtual image generation. The synchronous shooting of the cameras ensures that all cameras capture images at the same time, avoiding image inconsistencies caused by time differences. This process is achieved through the unified management of the synchronization signal source and the central control console, ensuring data consistency and synchronization, greatly improving the accuracy and authenticity of the final generated virtual image. All cameras are connected to the central console and calibrated and managed through the central console, which can efficiently process and manage large amounts of image data. The camera parameters are adjusted through the calibration tool of the central console to ensure that the images taken by all cameras can be accurately stitched and aligned, providing a high-quality data foundation for subsequent image processing and analysis.

[0064] Furthermore, stereo matching is performed on the artwork image, disparity is calculated to establish a disparity map, and the disparity map is converted into a depth map, including:

[0065] Select the artwork images as the left and right views and calculate the matching cost:

[0066] C(x, y) = || I L (x, y)-I R (x, y)||,

[0067] Where C(x, y) is the matching cost of pixel point (x, y), I L (x, y) is the gray value of the pixel (x, y) in the left view, I R (x, y) is the grayscale value of the pixel (x, y) in the right view;

[0068] Aggregate using the aggregator window based on the matching cost:

[0069]

[0070] Where C'' is the matching cost after aggregation, W is the aggregation window, and (u, v) is the offset of the pixel point (x, y) relative to the center of the aggregation window;

[0071] Initialize the aggregation window size and use a greedy algorithm to select the disparity by minimizing the post-aggregation matching cost:

[0072] d(x,y)=argminC'',

[0073] Where d(x, y) is the disparity of the pixel point (x, y), and the disparity of all pixels is calculated to construct a disparity map;

[0074] Calculate the depth of the pixel (x, y) based on the disparity:

[0075]

[0076] Where D(x, y) is the depth of the pixel point (x, y), B is the baseline distance of the camera, and f is the focal length of the camera. The depth is calculated by traversing all the pixels in the disparity map and stored in the corresponding position to form a depth map of the artwork.

[0077] By accurately calculating the matching cost of each pixel, the accuracy of disparity calculation can be effectively improved, thereby obtaining a high-precision disparity map. This process helps to capture the detailed information of the artwork and improve the detail expression of the final generated image. By aggregating the matching cost within the aggregation window, the impact of noise on the matching result can be reduced, and the robustness and accuracy of the matching can be improved. This process can effectively improve the quality of the disparity map, making the subsequent depth map generation more accurate. By selecting the disparity that minimizes the matching cost after aggregation through a greedy algorithm, the disparity map can be generated quickly and accurately. This method improves computational efficiency and is suitable for the processing of large-scale high-resolution images. It helps to quickly generate high-quality disparity maps. Through accurate depth calculation, a depth map reflecting the three-dimensional structure of the artwork can be generated. This process not only improves the stereoscopic and realism of the virtual image, but also provides high-quality three-dimensional data support for subsequent virtual image generation and processing.

[0078] S2, processing the depth map, identifying the regions and objects in the depth map, determining the image texture type according to the region and object attributes, and synthesizing high-quality textures;

[0079] Specifically, processing the depth map and identifying regions and objects in the depth map refers to smoothing the artwork depth map using a Gaussian filter, and using a Sobel operator to calculate the gradient of the smoothed artwork depth map to obtain the gradient amplitude and gradient direction to perform edge detection on the depth map, dividing the pixel points in the depth map into edge points and non-edge points, connecting the edge points to form edge lines and marking them on the depth map, and combining the pixel points enclosed by the edge lines to form pixel blocks;

[0080] A UNet model is constructed and trained, and the trained UNet model is used to perform semantic segmentation on the depth map, outputting the semantic label of each pixel in the depth map, and identifying the pixel block category according to the semantic label, wherein the category includes region and object.

[0081] Through the smoothing process of Gaussian filter, the noise in the depth map can be effectively removed and the image quality can be improved. The gradient calculation of the Sobel operator can accurately detect the edges in the image, obtain the gradient amplitude and direction, and provide basic data for the subsequent edge detection and pixel block formation. Edge detection can accurately distinguish the edge and non-edge areas in the image. By connecting the edge points to form edge lines, and combining the pixel points enclosed by the edge lines to form pixel blocks, different areas in the image can be effectively distinguished, providing clear regional division for subsequent semantic segmentation. The UNet model has a strong image segmentation capability. Through the semantic segmentation of the depth map, different areas and objects in the image can be accurately identified. The output semantic label of each pixel provides reliable classification information for the subsequent texture synthesis and virtual image generation. Through the recognition of semantic labels, the generation process of virtual images can be further optimized and adjusted to improve the accuracy and quality of generated images.

[0082] Further, determining the image texture type according to the region and object attributes and synthesizing high-quality texture means that after identifying the region and object in the depth map of the artwork, the texture type is determined according to the actual attributes of the region and object, and the texture parameters in the depth map are calculated:

[0083] A k =mean(D(x,y)|(x,y)∈R i ),

[0084]

[0085] Z k =std(D(x,y)|(x,y)∈R i ),

[0086] Among them A k is the amplitude of the kth pixel, q is the frequency, R i is the set of pixels in the i-th pixel block, Z k is the detail intensity of the k-th pixel, std represents the standard deviation of the pixel depth values ​​in the i-pixel block, and s is the texture scale, which is determined by the user;

[0087] Apply the Perlin algorithm to generate pixel base texture values:

[0088]

[0089] Where Pk (x, y) is the base texture value of the k-th pixel, G(o, j) is a pseudo-random gradient generated by a pseudo-random number generator, Δx and Δy are interpolation intervals, the depth map is divided into grids according to the interpolation intervals, and o and j are the coordinates of the grid points;

[0090] Apply a detail enhancement algorithm to enhance the base texture value:

[0091] Q k (x, y) = P k (x,y)+Z k *sin(q*x)cos(q*y),

[0092] Where Q k (x, y) is the detail texture value of the k-th pixel;

[0093] Calculate the texture value synthesis weight according to the detail intensity:

[0094]

[0095] where α k is the basic texture value synthesis weight, β k is the detail texture value synthesis weight, max Z is the maximum detail intensity;

[0096] Synthesize high-quality textures based on texture value synthesis weights:

[0097] T(x, y) = α k *P k (x,y)+β k *Q k (x, y),

[0098] Where T(x, y) is the high-quality texture value of the pixel point (x, y);

[0099] Create an empty image matrix of the same size as the depth map, and fill the high-quality texture value calculated for each pixel into the empty image matrix to form a high-quality texture of the artwork.

[0100] By determining the texture type according to the region and object attributes and calculating the relevant parameters, a texture that is more suitable for the characteristics of each pixel can be generated, thereby improving the accuracy and adaptability of texture synthesis. This process ensures the natural transition and realism of the texture and enhances the detail expression of the virtual image. The base texture generated by the Perlin algorithm has natural volatility and continuity, which can effectively avoid the repetition and rigidity of conventional noise textures. The generated base texture is more natural and realistic, providing a good foundation for subsequent detail enhancement. By superimposing the detail texture on the base texture through the detail enhancement algorithm, the detail expression of the texture can be significantly improved, making the texture more vivid and rich. This process The process ensures the generation of high-quality textures and provides a delicate visual effect for the final virtual image. By calculating the texture value synthesis weight, the ratio of basic texture and detail texture can be flexibly adjusted to ensure that the generated high-quality texture achieves the best balance between the overall effect and detail performance. By synthesizing high-quality textures, the generated texture has both overall naturalness and rich detail expression. An empty image matrix of the same size as the depth map is created, and the high-quality texture value calculated for each pixel is filled into the empty image matrix to form a high-quality texture of the artwork. This process ensures the high quality and high realism of the virtual image, and provides advanced technical support for artistic creation and virtual reality applications.

[0101] S3, build a generative adversarial network to generate virtual images of artworks based on depth maps and high-quality textures;

[0102] Specifically, constructing a generative adversarial network to generate a virtual image of an artwork based on a depth map of the artwork and high-quality textures of the artwork refers to constructing a generative adversarial network, including a generator and a discriminator, wherein the generator includes a generative input layer, a generative hidden layer, and a generative output layer, the generative input layer receives the depth map and the high-quality texture, and the generative output layer outputs the virtual image, and the discriminator includes a discriminative input layer, a discriminative hidden layer, and a discriminative output layer, the discriminative input layer receives the image, and the discriminative output layer outputs a discriminative result, which is used to judge whether the generated virtual image corresponds to the real artwork;

[0103] Collect real artwork images, real image depth maps, and real image textures to form a training set, use the training set to train the generator and discriminator, synchronously calculate the training losses of the generator and discriminator, and iteratively update the parameters of the generator and discriminator. If the training losses of the generator and discriminator no longer decrease significantly during continuous iterations, stop the iteration and use the updated generator and discriminator parameters to update the generator and discriminator.

[0104] The depth map and high-quality texture of the artwork are input into the generator to obtain a virtual image of the artwork. The discriminator is used to judge the difference between the virtual image and the artwork. If the virtual image corresponds to the artwork, the virtual image of the artwork is output. If not, the generator and discriminator are retrained.

[0105] By constructing a generative adversarial network, efficient and high-quality image generation can be achieved. The generator generates virtual images with high realism and rich details by receiving depth maps and high-quality textures; the discriminator ensures the quality of the generated image by discriminating the input image. This process greatly improves the accuracy and efficiency of image generation and provides advanced technical support for artistic creation. By collecting and using real artwork images, depth maps and textures to form a rich training set, the generator and discriminator can be effectively trained, and the parameters of the generator and discriminator are synchronously calculated and iteratively updated to ensure the efficiency and accuracy of the training process. The adversarial training of the generator and the discriminator enables the generated images to reach a high level in quality and realism. By inputting the depth map and high-quality texture into the generator, the generated virtual image can fully reflect the details and three-dimensional structure of the artwork. Through the judgment of the discriminator, it is ensured that the generated virtual image is highly consistent with the real artwork in visual effect. If the generated virtual image does not meet expectations, the generator and discriminator can be retrained to continuously optimize the generation results. This process not only improves the quality of image generation, but also improves the intelligence and automation level of the generation process.

[0106] S4, displaying and storing the virtual view of the artwork;

[0107] Specifically, displaying and storing the virtual image of the artwork means that after the central control console obtains the virtual image of the artwork, it synchronously stores the virtual image and the photographed image in the same category of the database, displays the virtual image and the photographed image of the artwork on the same screen and annotates the artwork information.

[0108] The central control console is responsible for processing and managing all image data. Through synchronous storage technology, virtual images and real images are stored in the database according to the same classification standards to facilitate subsequent queries and calls. This process improves the systematicness and efficiency of data management and ensures the orderly storage and rapid access of image data. During the storage process, the central control console uses advanced encryption technology and multiple backup measures to ensure the security and integrity of image data. Even if failures occur during data transmission and storage, data can be restored in time to prevent data loss and damage. Through the same-screen display technology, users can see virtual images and real images on the same screen at the same time for intuitive comparison and analysis. This display method not only enhances the user's visual experience, but also facilitates the discovery and correction of details in the virtual image, improving the accuracy and quality of image generation.

[0109] Example 2

[0110] Reference Figure 2 , which is a second embodiment of the present invention, and which is different from the previous embodiment, provides a virtual image generation system for artworks, which includes:

[0111] An image acquisition module is used to deploy and control cameras to capture images of artworks from multiple angles and send them to a central control console;

[0112] A depth processing module, used to perform stereo matching on the artwork image to calculate the disparity to form a disparity map, and calculate the depth map of the artwork based on the disparity map;

[0113] Texture synthesis module, used to identify the depth map areas and objects of the artwork and synthesize high-quality textures of the artwork based on the areas and objects;

[0114] An image generation module is used to construct and train a generative adversarial network to generate a virtual image of an artwork using the artwork depth map and high-quality texture of the artwork;

[0115] The storage and display module is used to store and display the generated virtual images of the artworks.

[0116] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0117] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0118] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0119] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0120] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for generating a virtual image of an artwork, characterized in that: include, Evenly arrange cameras to synchronously capture images of artworks, perform stereo matching on the images of artworks, calculate disparity to establish a disparity map, and convert the disparity map into a depth map; Process the depth map, identify the regions and objects in the depth map, determine the image texture type based on the region and object attributes, and synthesize high-quality textures; Build a generative adversarial network to generate virtual images of artworks based on depth maps and high-quality textures; Display and store virtual views of artworks; Determining the image texture type according to the region and object attributes and synthesizing high-quality texture means that after identifying the region and object in the depth map of the artwork, the texture type is determined according to the actual attributes of the region and object, and the texture parameters in the depth map are calculated: A k =mean(D(x,y)|(x,y)∈R i ), Z k =std(D(x,y)|(x,y)∈R i ), Among them A k is the amplitude of the kth pixel, q is the frequency, R i is the set of pixels in the i-th pixel block, Z k is the detail intensity of the k-th pixel, std represents the standard deviation of the pixel depth values ​​in the i-pixel block, s is the texture scale, and D(x, y) is the depth of the pixel (x, y); Apply the Perlin algorithm to generate pixel base texture values: Where P k (x, y) is the base texture value of the k-th pixel, G(o, j) is a pseudo-random gradient generated by a pseudo-random number generator, Δx and Δy are interpolation intervals, the depth map is divided into grids according to the interpolation intervals, and o and j are the coordinates of the grid points; Apply a detail enhancement algorithm to enhance the base texture value: Q k (x, y)=P k (x, y)+Z k *sin(q*x)cos(q*y), Where Q k (x, y) is the detail texture value of the k-th pixel; Calculate the texture value synthesis weight according to the detail intensity: where α k is the basic texture value synthesis weight, β k is the detail texture value synthesis weight, max Z is the maximum detail intensity; Synthesize high-quality textures based on texture value synthesis weights: T(x,y)=α k *P k (x,y)+β k *Q k (x,y), Where T(x, y) is the high-quality texture value of the pixel point (x, y); Create an empty image matrix of the same size as the depth map, and fill the high-quality texture value calculated for each pixel into the empty image matrix to form a high-quality texture of the artwork.

2. The method for generating a virtual image of an artwork according to claim 1, wherein: The evenly arranged cameras that synchronously capture images of the artwork refer to placing the artwork in the center of a closed room, evenly arranging cameras around the artwork, connecting all cameras to a central console, the central console using a calibration tool to calibrate parameters of each camera and unify the capture time of all cameras, the cameras synchronously capturing all-round images of the artwork and transmitting the images to the central console.

3. The method for generating a virtual image of an artwork as claimed in claim 2, characterized in that: The stereo matching of the artwork image, calculating the disparity to establish the disparity map, and converting the disparity map into a depth map comprises: Select the artwork images as the left and right views and calculate the matching cost: C(x,y)=||I L (x,y)-I R (x,y)||, Where C(x, y) is the matching cost of pixel point (x, y), I L (x, y) is the gray value of the pixel (x, y) in the left view, I R (x, y) is the grayscale value of the pixel (x, y) in the right view; Aggregate using the aggregator window based on the matching cost: Where C'' is the matching cost after aggregation, W is the aggregation window, and (u, v) is the offset of the pixel point (x, y) relative to the center of the aggregation window; Initialize the aggregation window size and use a greedy algorithm to select the disparity by minimizing the post-aggregation matching cost: d(x,y)=argminC'', Where d(x, y) is the disparity of the pixel point (x, y), and the disparity of all pixels is calculated to construct a disparity map; Calculate the depth of the pixel (x, y) based on the disparity: Where D(x, y) is the depth of the pixel point (x, y), B is the baseline distance of the camera, and f is the focal length of the camera. The depth is calculated by traversing all the pixels in the disparity map and stored in the corresponding position to form a depth map of the artwork.

4. The method for generating a virtual image of an artwork as claimed in claim 3, wherein: Processing the depth map and identifying regions and objects in the depth map refers to smoothing the artwork depth map using a Gaussian filter, and using a Sobel operator to calculate the gradient of the smoothed artwork depth map to obtain the gradient amplitude and gradient direction, performing edge detection on the depth map to divide pixel points in the depth map into edge points and non-edge points, connecting edge points to form edge lines and marking them on the depth map, and combining pixel points enclosed by the edge lines to form pixel blocks; A UNet model is constructed and trained, and the trained UNet model is used to perform semantic segmentation on the depth map, outputting the semantic label of each pixel in the depth map, and identifying the pixel block category according to the semantic label, wherein the category includes region and object.

5. The method for generating a virtual image of an artwork as claimed in claim 4, characterized in that: The constructing of a generative adversarial network to generate a virtual image of the artwork based on the artwork depth map and the artwork high-quality texture refers to constructing a generative adversarial network, including a generator and a discriminator, wherein the generator includes a generating input layer, a generating hidden layer and a generating output layer, the generating input layer receives the depth map and the high-quality texture, and the generating output layer outputs the virtual image, and the discriminator includes a discriminating input layer, a discriminating hidden layer and a discriminating output layer, the discriminating input layer receives the image, and the discriminating output layer outputs the discriminant result; Collect real artwork images, real image depth maps, and real image textures to form a training set, use the training set to train the generator and discriminator, synchronously calculate the training losses of the generator and discriminator, and iteratively update the parameters of the generator and discriminator. If the training losses of the generator and discriminator no longer decrease significantly during continuous iterations, stop the iteration and use the updated generator and discriminator parameters to update the generator and discriminator. The depth map and high-quality texture of the artwork are input into the generator to obtain a virtual image of the artwork. The discriminator is used to judge the difference between the virtual image and the artwork. If the virtual image corresponds to the artwork, the virtual image of the artwork is output. If not, the generator and discriminator are retrained.

6. The method for generating a virtual image of an artwork as claimed in claim 5, characterized in that: Displaying and storing the virtual image of the artwork means that after the central control console obtains the virtual image of the artwork, it synchronously stores the virtual image and the photographed image in the same category of the database, displays the virtual image and the photographed image of the artwork on the same screen and annotates the artwork information.

7. A virtual image generation system for a method for generating a virtual image of an artwork according to any one of claims 1 to 6, characterized in that: include, An image acquisition module is used to deploy and control cameras to capture images of artworks from multiple angles and send them to a central control console; A depth processing module, used to perform stereo matching on the artwork image to calculate the disparity to form a disparity map, and obtain a depth map of the artwork based on the disparity map; Texture synthesis module, used to identify the depth map areas and objects of the artwork and synthesize high-quality textures of the artwork based on the areas and objects; An image generation module is used to construct and train a generative adversarial network to generate a virtual image of an artwork using the artwork depth map and high-quality texture of the artwork; The storage and display module is used to store and display the generated virtual images of the artworks.

8. A computer device comprising: Memory and processor; The memory stores a computer program, wherein the processor implements the steps of the method for generating a virtual image of an artwork according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for generating a virtual image of an artwork according to any one of claims 1 to 6 are implemented.

Images