Training of a game image reconstruction network and a reconstruction method, device and storage medium thereof

Abstract

The application discloses a kind of game image reconstruction network training and its reconstruction method, equipment, storage medium, the method comprises: obtaining the content sample image data of performance real scene;Filtering sample game belonging to simulation management on mobile platform;Obtain the picture with the subject style of sample game as style sample image data;According to content sample image data and style sample image data, generative adversarial network is trained as game image reconstruction network, and game image reconstruction network is used to reconstruct the image data with the subject style of sample game.Generative adversarial network is trained in the mode of unpaired data, so that game image reconstruction network can reconstruct the subject style of sample game to image data, improve the efficiency of training, and the subject style of reconstruction sample game belongs to post-processing, in the case where given subject style, can reduce the workload of post-processing.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and in particular to a training method, device, and storage medium for a game image reconstruction network. Background Technology

[0002] In scenarios such as short videos and advertisements, users create various types of video data. After recording the original video data, the video data is usually post-processed to improve the quality of the video data.

[0003] Due to certain business needs, some post-processing involves transforming the style of video data to match the style of a particular game. Currently, the most common post-processing involves adding filters to the video data to transform it into other styles, such as retro, film, sunset, etc.

[0004] However, filters typically adjust the color values ​​of pixels and add other decorative elements, resulting in a relatively simple effect. It is difficult to achieve the style of the game by using multiple filters in combination. If the video data is designed according to the style of the game, it will greatly increase the threshold for video data production, resulting in a significant increase in the time required for video data production and low efficiency. Summary of the Invention

[0005] This invention provides a training and reconstruction method, device, and storage medium for a game image reconstruction network to solve the problem of how to efficiently realize the style of the game in the image.

[0006] According to one aspect of the present invention, a method for training a game image reconstruction network is provided, comprising:

[0007] Acquire sample image data representing real-world scenes;

[0008] Select sample games belonging to the simulation management genre on mobile platforms;

[0009] Obtain images with the main style of the sample game as style sample image data;

[0010] Based on the content sample image data and the style sample image data, a generative adversarial network is trained to become a game image reconstruction network, which is used to reconstruct image data with the main style of the sample game.

[0011] According to another aspect of the present invention, an image reconstruction method is provided, comprising:

[0012] Load the game image reconstruction network trained by the method according to any embodiment of the present invention;

[0013] Obtain the original image data to be reconstructed;

[0014] The original image data is input into the game image reconstruction network and reconstructed into candidate image data containing the main style of the sample game.

[0015] Post-processing is performed on the candidate image data to obtain the target image data.

[0016] According to another aspect of the present invention, a method for style reconstruction of video data is provided, comprising:

[0017] Load the game image reconstruction network trained by the method according to any embodiment of the present invention;

[0018] The content to be acquired is the original video data introducing the target game, which contains multiple frames of original image data;

[0019] The original image data is input into the game image reconstruction network and reconstructed into candidate image data containing the main style of the sample game.

[0020] Post-processing is performed on the candidate image data to obtain the target image data;

[0021] The target image data is replaced in the original video data to obtain the target video data.

[0022] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0023] At least one processor; and

[0024] A memory communicatively connected to the at least one processor; wherein,

[0025] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the training method, image reconstruction method, or video reconstruction method of the game image reconstruction network according to any embodiment of the present invention.

[0026] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, the computer program being configured to cause a processor to execute and implement the training method, image reconstruction method, or video reconstruction method of the game image reconstruction network according to any embodiment of the present invention.

[0027] In this embodiment, sample image data representing real-world scenes is acquired; sample games belonging to the simulation management genre on mobile platforms are selected; images with the main style of the sample games are acquired as style sample image data; and a generative adversarial network (GAN) is trained into a game image reconstruction network based on the content sample image data and the style sample image data. The game image reconstruction network is used to reconstruct image data with the main style of the sample games. Training the GAN using unpaired data allows the game image reconstruction network to reconstruct image data to the main style of the sample games, improving training efficiency. Reconstructing the main style of the sample games is a post-processing step; with a given main style, the workload of post-processing can be reduced, maintaining the threshold for video data production and minimizing the time required for video data production, thus greatly improving the efficiency of producing game-style video data.

[0028] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a flowchart of a training method for a game image reconstruction network according to Embodiment 1 of the present invention;

[0031] Figure 2 This is a flowchart of an image reconstruction method provided according to Embodiment 2 of the present invention;

[0032] Figures 3A to 3E This is an example diagram of an image reconstruction provided according to Embodiment 1 of the present invention;

[0033] Figure 4 This is a flowchart of a video reconstruction method provided in Embodiment 3 of the present invention;

[0034] Figure 5 This is a schematic diagram of the structure of a training device for a game image reconstruction network according to Embodiment 4 of the present invention;

[0035] Figure 6 This is a schematic diagram of the structure of an image reconstruction device according to Embodiment 5 of the present invention;

[0036] Figure 7This is a schematic diagram of the structure of a video reconstruction device according to Embodiment Six of the present invention;

[0037] Figure 8 This is a schematic diagram of the structure of an electronic device provided in Embodiment 7 of the present invention. Detailed Implementation

[0038] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0039] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0040] Example 1

[0041] Figure 1 This is a flowchart of a training method for a game image reconstruction network provided in Embodiment 1 of the present invention. This embodiment is applicable to training a game image reconstruction network that implements game styles. The method can be executed by a training device for the game image reconstruction network, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0042] Step 101: Obtain content sample image data representing real-world scenes.

[0043] In this embodiment, multiple frames of image data can be collected by applying to copyright holders, using publicly available datasets, or recording independently. The content of these image data records various scenes in the real world and can be used as the source of content, as samples for training game image reconstruction networks, and can be recorded as content sample image data.

[0044] The real-world scenario can include real natural environments, real buildings, real people, animals, etc., and this embodiment does not impose any restrictions on it.

[0045] Step 102: Filter sample games belonging to the simulation management category on mobile platforms.

[0046] In this embodiment, a game can be selected as the source of style when training the game image reconstruction network according to business needs, and it can be referred to as the sample game.

[0047] Furthermore, the sample game can run on mobile platforms, such as Android, iOS, and so on.

[0048] Of course, the sample game can be ported to other platforms, such as Windows, through emulators or other means, and this embodiment does not impose any restrictions on this.

[0049] Furthermore, the sample game is a simulation management game, in which the user takes on the role of a manager, operating and managing a virtual world within the game. The objects managed in simulation management games can include farms, sports clubs (such as basketball and football), modern businesses, ancient businesses, and so on.

[0050] In one example, the sample game is a forest-themed game where users can run a hot spring hotel with cats, providing rest and food areas, and cooperate with the cats to collect a large amount of raw materials to attract small animals back.

[0051] Generally, the same sample game is developed by the same team, and has a relatively unified and typical style in terms of art design.

[0052] Step 103: Obtain images with the main style of the sample game as style sample image data.

[0053] Generally, the style composition of sample games is quite complex. Some styles occupy a dominant position (i.e., the main style), which can focus the user's attention on the main style in a short time. Some styles occupy details and a non-main position (i.e., non-main styles), but users will also focus their attention on these non-main styles at times. For example, in the sample game in the example above, the main style is a warm color (such as light yellow-green) oil painting style. The various elements in the sample game are designed quite exquisitely, which can reflect a warm and healing feeling. The non-main style can be reflected in the changes of light and shadow, the continuity of lines, etc.

[0054] The learning difficulty of the main style is relatively low, while the learning difficulty of non-main styles is relatively high. Therefore, this embodiment can extract images with the main style of the sample game through various channels. These images are the source of the style and serve as samples for training the game image reconstruction network, and are denoted as style sample image data.

[0055] In one embodiment of the present invention, step 103 may include the following steps:

[0056] Step 1031: Obtain candidate video data recorded when the user controls the sample game.

[0057] Sample games, such as simulation management games, have visuals that change as the management progresses. Technicians can manually operate the sample games or use scripts to operate them and take screenshots to obtain style sample image data. However, the progress of the management depends on certain logic, and manual operation will consume a lot of time and effort. Scripts are not suitable for complex logic. In addition, a large number of diverse samples are needed to train the game image reconstruction network. If the sample game is directly launched and the screen is captured in the sample game as a sample, the efficiency may be low.

[0058] Therefore, in this embodiment, video data recorded when users control the sample game can be collected by means of inviting users to operate the sample game, applying to the copyright holder, or downloading from an open video platform. This data can be recorded as candidate video data, and style sample image data can be filtered from the candidate video data.

[0059] While the image quality of candidate video data may be somewhat lower than that of the sample game, it is generally sufficient for training the game image reconstruction network. The diverse methods of acquiring candidate video data make it easy to obtain a large amount of data. Furthermore, different users have enough time to play the sample game, resulting in similar image quality in the candidate video data. This satisfies the requirements for training the game image reconstruction network and greatly improves the efficiency of sample acquisition.

[0060] Step 1032: Select candidate video data whose content has the main style of the sample game as sample video data.

[0061] To improve the performance of the game image reconstruction network, candidate video data with content that has the main style of the sample game can be selected from all candidate video data as much as possible. These are denoted as sample video data. Then, the images in the sample video data are the images with the main style of the sample game.

[0062] In practical applications, the screening and input of sample video data can be handled by artists and / or technicians. When artists and / or technicians are familiar with the sample games, the screening efficiency is relatively high.

[0063] However, considering that there are many screens in the sample games, artists and / or technicians may not be familiar with the screens of each sample game. In addition, the candidate video data is long, and the process of artists and / or technicians browsing, familiarizing themselves with the sample games and screening candidate video data will take a long time. Therefore, we can analyze the characteristics of the main style of the sample games, automatically collect sample video data, and further provide it to artists and / or technicians for verification. This can greatly improve the efficiency of collecting sample video data.

[0064] In one embodiment of the present invention, step 1032 may further include the following steps:

[0065] Step 10321: Extract multiple frames of image data from the candidate video data as the first reference image data.

[0066] In this embodiment, multiple frames of image data can be extracted from candidate video data through random sampling, uniform sampling, or other methods, and are referred to as the first reference image data.

[0067] In one sampling method, command-line tools, library files, etc., can be used to divide candidate video data into multiple segments, denoted as video segments, with independent scenes as the segmentation nodes. Each video segment contains one or more independent scenes.

[0068] Furthermore, the detection methods include the following two:

[0069] 1. Threshold mode

[0070] For candidate video data with obvious scene boundaries, a threshold mode is applied. Each frame of image data is compared with the set black level. Based on the detection results, it is determined whether it is the boundary of a scene such as fade-in, fade-out, or cut to black, thereby dividing each scene in the candidate video data.

[0071] 2. Content Format

[0072] For candidate video data that switches rapidly between scenes, a content mode is applied. Each frame of image data is compared, and image data with significant content changes are identified as the segmentation nodes, thereby dividing the candidate video data into various scenes.

[0073] Generally, candidate video data containing a single scene can be segmented into a video segment. However, considering that some candidate video data containing a single scene is relatively short, the scene can be merged with other adjacent scenes, thereby segmenting candidate video data containing two or more connected scenes into a video segment. This embodiment does not impose any restrictions on this.

[0074] One frame of image data is extracted from each video segment at preset intervals and used as the first reference image data.

[0075] Since the content changes little between images in the same scene, sampling the first reference image data in the same scene can improve the uniformity of sampling.

[0076] Step 10322: If the primary color of the first reference image data matches the main style of the sample game, then mark the first reference image data as the second reference image data.

[0077] The sample game is a simulation management game. Generally, the screens that users see most often are those where management activities take place. Therefore, the base color that reflects the main style of the sample game can be set in advance, especially the base color that the overall screens in the management activities tend to have.

[0078] Among them, primary colors are the most basic colors. There are three commonly used primary colors: red (R), green (G), and yellow (B). Other colors can be mixed from these three primary colors in different proportions.

[0079] Traverse each frame of the first reference image data. During the traversal, identify the overall bias of the primary color of the current first reference image data and compare it with the primary color reflected in the main style of the sample game.

[0080] If the overall primary color of the current first reference image data matches the primary color of the main style of the sample game, then the current first reference image data can be filtered out and marked as the second reference image data.

[0081] If the primary color of the current first reference image data does not match the primary color of the main style of the sample game, then the current first reference image data can be ignored.

[0082] In one example, the gameplay of a sample game is mainly concentrated in a forest, especially the grassland within the forest. The forest is dark green, and the grassland is light green, so the base color of the main style of the sample game can be set as green.

[0083] For each pixel of the first reference image data, which includes the red component R, the green component G, and the blue component B, a binarization operation is performed on the green component G. That is, if the green component G is greater than or equal to a certain threshold, the green component G is set to 255; if the green component G is less than a certain threshold, the green component G is set to 0.

[0084] If binarization is completed, the number of pixels with a non-zero green component (i.e., 255) is counted in the first reference image data after binarization.

[0085] The percentage of the statistical quantity is the ratio between the number of pixels with a non-zero green component (i.e., 255) in the first reference image data after binarization and the total number of pixels.

[0086] If the proportion exceeds the preset first threshold, it means that the proportion of green component is relatively large. In this case, it is determined that the primary color of the first reference image data matches the main style of the sample game, and the first reference image data is marked as the second reference image data.

[0087] In this embodiment, binarization is performed on the green component and its proportion is calculated to determine the primary color of its overall tendency. The calculation is relatively simple and can reduce the calculation time.

[0088] Step 10323: If the second reference image data contains the target object that appears in the operation of the sample game, then mark the second reference image data as the third reference image data.

[0089] The sample game belongs to the simulation management game category. In the management activities, multiple elements may appear. Therefore, one or more elements that appear more frequently can be selected in advance and recorded as target objects.

[0090] For the selected second reference image data, the target object can be used as the detection target. The detection can be performed to see if the target object appears in the second reference image data. If the target object appears in the second reference image data, the second reference image data can be further filtered out and marked as the third reference image data.

[0091] In one example, the gameplay of a sample game is mainly concentrated in a forest, especially the grassland in the forest. There are several circular plots of land (some are bare land, and some are land inlaid with stones) distributed on the grassland as workplaces. Anthropomorphic cats cook, sleep, and take baths on these plots. In this example, the land and the cat can be set as target objects.

[0092] Using cats and land as targets, the second reference image data is input into the target detection network, and the target detection operation is performed on the second reference image data to obtain the target objects that appear in the sample game.

[0093] Furthermore, object detection networks can be one-stage or two-stage.

[0094] Two-stage refers to segment-to-segment object detection, which means that the object detection operation is completed in two steps. The first step is to use various convolutional neural networks as the backbone of the object detection network to extract features from the second reference image data, and perform coarse classification (distinguishing between foreground and background) and coarse localization (anchor) based on the features to obtain candidate regions. The second step is to classify the candidate regions (i.e., whether they contain land or cats) in the classification network of the object detection network.

[0095] For example, two-stage object detection operations may include R-CNN (Region-CNN), Fast R-CNN, Faster R-CNN, R-FCN (Region-based fully convolutional network), and so on.

[0096] One-stage refers to end-to-end object detection, which is completed in one step without searching for candidate regions separately. Instead, the second reference image data is input into a holistic network, and the generated detection result contains both the location and category information of the target object (i.e., land, cat).

[0097] For example, one-stage object detection operations may include SSD (Single Shot Multibox Detector), YOLO (You Only Look Once), and so on.

[0098] Generally speaking, two-stage detection has higher accuracy but slightly lower detection speed, while one-stage detection has higher speed but slightly lower accuracy. Those skilled in the art can choose one-stage or two-stage based on factors such as the resources of electronic equipment and the real-time requirements of detection. This embodiment does not impose any restrictions on this.

[0099] Since the outlines of the land in the sample game are roughly circular with a relatively large curvature, algorithms such as Canny, Sobel, and Laplacian can be used to detect the edges of the land (usually represented by a detection box). Some pixels can be collected on the edges, and these pixels can be substituted into the curvature formula to calculate the curvature of the edges. The confidence of the land output by the target detection network is then verified by the curvature.

[0100] For a cat (usually represented by a detection box) and land, the degree of overlap between the cat and the land can be calculated. The degree of overlap can be expressed as the Intersection over Union (IOU), which is the ratio between a first area and a second area. The first area is the area of ​​the intersection between the detection box representing the cat and the detection box representing the land, and the second area is the area of ​​the union between the detection box representing the cat and the detection box representing the land.

[0101] If the curvature is greater than or equal to the preset second threshold and the overlap is greater than or equal to the preset third threshold, it indicates that the curvature of the land output by the target detection network is large, and therefore the confidence level of the land is high. Furthermore, the overlap between the cat output by the target detection network and the land is high, and therefore the confidence level of the cat being located on the land is also high. Combining the two, it can be confirmed that the current scene is the operation activity of the sample game, which can reflect the main style of the sample game. Then the second reference image data can be marked as the third reference image data.

[0102] Since the target object is a cartoon character, it contains some missing information compared to a real image. The target detection network may have a certain false detection rate, which is particularly noticeable when the target detection network is trained with a small number of samples. In this example, the target object is calibrated by the curvature of the land and the degree of overlap between the land and the cat, which can improve the accuracy of target object detection.

[0103] Step 10324: Filter sample video data from the candidate video data with reference to the third reference image data.

[0104] In the candidate video data, the third reference image data reflects the distribution information of the main style. Since the user's operation has a certain continuity, the images that reflect the main style also have a certain continuity. Based on the distribution information reflected by the third reference image data, some continuous video data can be selected from the candidate video data as sample video data, so as to reduce or avoid collecting images containing non-main style in the sample video data.

[0105] In a practical implementation, the time point of the third reference image data can be marked on the timeline of the candidate video data.

[0106] By iterating through each time point in the candidate video data, the time interval between two adjacent time points can be calculated.

[0107] If the interval between two adjacent time points is less than or equal to the preset fourth threshold, it indicates that the time interval is short, there is a certain continuity in time, and the probability of belonging to a continuous scene is high. In this case, the two adjacent time points are connected to obtain the connected area.

[0108] If the interval between two adjacent time points is greater than the preset fourth threshold, it means that the time interval is long and there is no continuity in time. The probability of it being a continuous scene is low, so the two adjacent time points can be ignored, that is, the two adjacent time points are not connected.

[0109] After traversing all time points in the candidate video data, video data located within the connected area is extracted from the candidate video data as sample video data. Through coherence detection, the detection workload of the third reference image data can be reduced, thus reducing the time consumption.

[0110] Step 1033: Extract multiple frames of image data from the sample video data as style sample image data.

[0111] In the sample video data, multiple frames of image data can be extracted through methods such as scene segmentation, random sampling, and uniform sampling, and these are denoted as style sample image data.

[0112] For the number of style sample images, data augmentation processing can be performed on the style sample image data, wherein the data augmentation processing includes at least one of the following:

[0113] The style sample image data is randomly cropped.

[0114] The style sample image data is randomly blurred.

[0115] Of course, the data augmentation processing described above is merely an example. When implementing this embodiment, other data augmentation processing can be set according to actual circumstances, and this embodiment does not impose any limitations on this. Furthermore, in addition to the data augmentation processing described above, those skilled in the art can employ other data augmentation processing as needed, and this embodiment does not impose any limitations on this either.

[0116] Data augmentation can increase the quantity and diversity of style sample image data, that is, increase the quantity and diversity of samples, and try to keep them equivalent to the content sample image data, thereby improving the quality of training game image reconstruction networks.

[0117] Step 104: Train the generative adversarial network into a game image reconstruction network based on the content sample image data and style sample image data.

[0118] In this embodiment, a Generative Adversarial Network (GAN) can be pre-constructed.

[0119] Generally, a Generative Adversarial Network (GAN) consists of a generator and a discriminator. The generator is responsible for generating content based on random vectors; in this embodiment, the content is image data, especially image data with the style of the sample game. The discriminator is responsible for determining whether the received content is real; the discriminator usually provides a probability representing the authenticity of the content.

[0120] The generator and discriminator can use different structures. For the function of processing image data, these structures are not limited to artificially designed neural networks, such as convolutional layers, fully connected layers, etc. They can also be neural networks optimized by model quantization methods, neural networks searched for game style characteristics by NAS (Neural Architecture Search) methods, etc. This embodiment does not impose any restrictions on them.

[0121] Generative adversarial networks can be classified into the following types based on the different structures of generators and discriminators:

[0122] DCGAN (Deep Convolutional Generative Adversarial Network), CGAN (Conditional Generative Adversarial Network), CycleGAN (Periodic Generative Adversarial Network), CoGAN (Coupled Generative Adversarial Network), ProGAN (Progressive Growth Generative Adversarial Network), WGAN (Wasserstein Generative Adversarial Network), SAGAN (Self-Attention Generative Adversarial Network), BigGAN (Large Generative Adversarial Network), StyleGAN (Style-Based Generative Adversarial Network).

[0123] The generator and discriminator are in an adversarial relationship. This adversarial relationship refers to the alternating training process of the generative adversarial network. Taking the generation of game-style image data as an example, the generator generates both fake and real image data, which are then fed to the discriminator for judgment. The discriminator learns to distinguish between the two, giving high scores to real image data (i.e., game-style image data) and low scores to fake image data (i.e., non-game-style image data). Once the discriminator can skillfully judge the existing image data, the generator is instructed to continuously generate better fake image data with the goal of obtaining high scores from the discriminator, until it can fool the discriminator. This process is repeated until the discriminator's prediction probability for any image data is close to 0.5, meaning it can no longer distinguish between real and fake image data, at which point training can stop.

[0124] In this embodiment, content sample image data for recording real-world scenes and style sample image face data with the main style of the sample game are used as samples for training the generative adversarial network. The content sample image data is the source of the content, and the style sample image data is the source of the game style (i.e., the main style of the sample game). The generative adversarial network is trained in this way, and the trained generative adversarial network is denoted as the game image reconstruction network, so that the game image reconstruction network can be used to reconstruct image data with the main style of the sample game.

[0125] Furthermore, the samples used to train the generative adversarial network can be paired data, which can improve the performance of the generative adversarial network. However, this requires collecting real-world image data corresponding to the style sample image data. In reality, most style sample image data do not have corresponding real-world image data. Therefore, the generative adversarial network in this embodiment supports training using unpaired data, such as CycleGAN, StyleGAN, etc.

[0126] Taking the "Learning to Cartoonize Using White-box Cartoon Representations" network as an example, this network contains three modules that can divide the original image and style image into three representations:

[0127] 1. Surface characterization

[0128] Surface representations are extracted to represent smooth surfaces in image data. Given image data, weighted low-frequency components can be extracted, where color components and surface texture are preserved while edges, textures, and details are ignored. This can be used to achieve flexible and learnable feature representations of smooth surfaces.

[0129] 2. Structure representation

[0130] Structural representation effectively captures global structural information and sparse color blocks in cel-shaded cartoon style. It extracts segmented regions from the input image data and applies an adaptive coloring algorithm to each segmented region to generate a structural representation. This structural representation mimics the cel-shaded cartoon style, characterized by clear boundaries and sparse color blocks.

[0131] 3. Texture representation

[0132] Texture representations contain the details and edges of the drawn image. The input image data is converted into a single-channel intensity map, where color and brightness are removed, and relative pixel intensity is preserved. Texture representations guide the network to independently learn high-frequency texture details, excluding color and brightness patterns.

[0133] The style of image data output is controlled by balancing the weights of surface representation, structural representation, and texture representation.

[0134] In this embodiment, content sample image data representing real-world scenes is acquired; sample games belonging to the simulation management genre on mobile platforms are selected; images with the main style of the sample games are extracted from the sample games as style sample image data; a generative adversarial network (GAN) is trained into a game image reconstruction network based on the content sample image data and the style sample image data. The game image reconstruction network is used to reconstruct image data with the main style of the sample games. Training the GAN using unpaired data allows the game image reconstruction network to reconstruct image data to the main style of the sample games, improving training efficiency. Reconstructing the main style of the sample games is a post-processing step; with a given main style, the workload of post-processing can be reduced, maintaining the threshold for video data production and the time required for video data production, thus greatly improving the efficiency of producing game-style video data.

[0135] Example 2

[0136] Figure 2 This is a flowchart of an image reconstruction method provided in Embodiment 2 of the present invention. This embodiment is applicable to situations where image data is reconstructed to a game style based on a game image reconstruction network. This method can be executed by an image reconstruction device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 2 As shown, the method includes:

[0137] Step 201: Load the game image reconstruction network.

[0138] In a specific implementation, the game image reconstruction network can be trained in advance according to the method described in Embodiment 1 of the present invention, wherein the game image reconstruction network can be used to reconstruct image data with the style of the sample game theme.

[0139] When applying the game image reconstruction network, the game image reconstruction network and its parameters are loaded into memory for execution.

[0140] Step 202: Obtain the original image data to be reconstructed.

[0141] Generally, game image reconstruction networks have a large structure and consume a lot of resources. They are usually deployed on the server side. The server side can encapsulate the game image reconstruction network into interfaces, plugins, etc., to provide game style reconstruction services to users on local area networks or public networks. Users can use clients or browsers to call the interfaces, plugins, etc. to transmit the image data of the game style to be reconstructed to the server side. For easy distinction, the image data of the game style to be reconstructed is recorded as the original image data.

[0142] Of course, if electronic devices such as personal computers and laptops have sufficient local resources to support the operation of the game image reconstruction network, the game image reconstruction network can be loaded and run locally on the electronic device. In this case, the original image data of the game style to be reconstructed can be input through command line or other means. This embodiment does not impose any restrictions on this.

[0143] Among them, game style can refer to the main style reflected in the sample game as a whole.

[0144] Step 203: Input the original image data into the game image reconstruction network to reconstruct candidate image data containing the main style of the sample game.

[0145] In this embodiment, the original image data is input into the game image reconstruction network. The game image reconstruction network processes the original image data according to its structure and reconstructs the original image data into new image data containing the main style of the sample game while maintaining the content of the original image data. This new image data is denoted as candidate image data.

[0146] Step 204: Post-process the candidate image data to obtain the target image data.

[0147] In this embodiment, post-processing can be performed on the candidate image data to add some important non-subject styles to the candidate image data, so that the target image data has a more complete style in the sample game.

[0148] In one example, for such Figure 3A The original image data shown is reconstructed as follows, while preserving the content of the original image data. Figure 3B The candidate image data shown contains the main style of the sample game. Compared with the original image data, the candidate image data is more warm in color, that is, the color is more light yellow, which is particularly obvious in the grass on the ground.

[0149] In this example, methods such as RVM (Related Vector Machine) can be used to extract human figures from the original image data, thereby extracting features such as... Figure 3C The image data shown contains human image information.

[0150] In addition, algorithms such as Canny, Sobel, and Laplacian can be used to calculate edges on the original image data, which are recorded as the original edge information.

[0151] To preserve the image of the person, no outline is added. Instead, the original edge information corresponding to the image data is removed from the original edge information, resulting in the following: Figure 3D The target edge information is shown.

[0152] The target edge information is converted into colors that conform to the non-subjective style of the sample game, such as brown (141, 116, 100), etc.

[0153] If the conversion is complete, the original edge information is superimposed on the candidate image data to achieve the effect of outlining.

[0154] If the overlay is complete, mean shift processing is performed on the candidate image data to obtain, as shown below. Figure 3E The target image data shown.

[0155] Mean shifting is a smoothing filter for candidate image data at the color level. It can neutralize colors with similar color distributions, smooth color details, and erode smaller color areas. It can determine the size of color blocks and the average pixel range of colors by setting the spatial radius and color space radius.

[0156] Considering that edge detection depends on significant changes in light and shadow, and that the superimposed edges after edge detection are not particularly rounded, this embodiment first outlines the edges (i.e., superimposes the target edge information) and then removes the light and shadow (i.e., means shift processing), which ensures the quality of edge detection and makes the superimposed edges more rounded.

[0157] In this embodiment, a game image reconstruction network is loaded; the original image data to be reconstructed is obtained; the original image data is input into the game image reconstruction network to reconstruct candidate image data containing the main style of the sample game; post-processing is performed on the candidate image data to obtain the target image data. When training the game image reconstruction network, a generative adversarial network is trained using unpaired data, enabling the game image reconstruction network to reconstruct the image data to the main style of the sample game, improving training efficiency. Reconstructing the main style of the sample game is a post-processing step. Under a given main style, post-processing on the candidate image data makes the style of the sample game more complete. Post-processing is relatively simple, reducing the workload and maintaining the threshold for creating video data, thus significantly improving the efficiency of creating video data with game style.

[0158] Example 3

[0159] Figure 4 This is a flowchart of a video reconstruction method provided in Embodiment 3 of the present invention. This embodiment is applicable to situations where video data is reconstructed to a game style based on a game image reconstruction network. This method can be executed by a video reconstruction device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 4 As shown, the method includes:

[0160] Step 401: Load the game image reconstruction network.

[0161] In a specific implementation, the game image reconstruction network can be trained in advance according to the method described in Embodiment 1 of the present invention, wherein the game image reconstruction network can be used to reconstruct image data containing game style.

[0162] When applying the game image reconstruction network, the game image reconstruction network and its parameters are loaded into memory for execution.

[0163] Step 402: Obtain the original video data that introduces the target game.

[0164] In this embodiment, artists can create video data for the game to be promoted. The content of the video data is used to introduce the game. For easy distinction, the game is referred to as the target game, and the video data is referred to as the original video data.

[0165] The target game type may include MOBA, RPG, SLG, etc., and this embodiment does not limit it.

[0166] In practical implementation, the content of the raw video data can be divided into two main forms: game content and real-life storyline. The storyline can be further divided into the following categories:

[0167] 1. Pseudo-food sharing

[0168] The raw video data includes some food-related materials, which can attract users' attention. Secondly, it incorporates the gameplay of earning money to eat food, while also providing users with a clear goal for playing the game.

[0169] 2. Themes closely related to users' daily lives

[0170] The raw video data closely reflects users' current lifestyles, seamlessly integrating the game's selling points into various aspects of life, such as purchasing in-game items, eating, and buying snacks to earn money through the game. This type of material is relatively simple to produce, with simple scenes and low shooting difficulty. The first half of the material mainly focuses on dialogue between two people, while the second half features game integration segments.

[0171] 3. Situational drama

[0172] The raw video data contains material from situational dramas, and in some cases, celebrities wear costumes from the game to endorse the products. Some of the storylines are quite exaggerated in order to attract users' attention.

[0173] Generally, game image reconstruction networks have a large structure and consume a lot of resources. They are usually deployed on the server side. The server side can encapsulate the game image reconstruction network into interfaces, plugins, etc., to provide game style reconstruction services to users on local area networks or public networks. Users can transmit the original video data of the game style to be reconstructed to the server side by calling the interfaces, plugins, etc. through the client or browser.

[0174] Of course, if electronic devices such as personal computers and laptops have sufficient local resources to support the operation of the game image reconstruction network, the game image reconstruction network can be loaded and run locally on the electronic device. In this case, the original video data of the game style to be reconstructed can be input through command line or other means. This embodiment does not impose any restrictions on this.

[0175] Among them, game style can refer to the main style reflected in the sample game as a whole.

[0176] Step 403: Input the original image data into the game image reconstruction network to reconstruct candidate image data containing the main style of the sample game.

[0177] In this embodiment, the original image data is input into the game image reconstruction network. The game image reconstruction network processes the original image data according to its structure and reconstructs the original image data into new image data containing the main style of the sample game while maintaining the content of the original image data. This new image data is denoted as candidate image data.

[0178] Step 404: Perform post-processing on the candidate image data to obtain the target image data.

[0179] In this embodiment, post-processing can be performed on the candidate image data to add some important non-subject styles to the candidate image data, so that the target image data has a more complete style in the sample game.

[0180] In this example, methods such as RVM (Related Vector Machine) can be used to extract human figures from the original image data, thereby extracting features such as... Figure 3C The image data shown contains human image information.

[0181] In addition, algorithms such as Canny, Sobel, and Laplacian can be used to calculate edges on the original image data, which are recorded as the original edge information.

[0182] To preserve the image of the person, no outline is added. Instead, the original edge information corresponding to the image data is removed from the original edge information, resulting in the following: Figure 3D The target edge information is shown.

[0183] The target edge information is converted into colors that conform to the non-subjective style of the sample game, such as brown (141, 116, 100), etc.

[0184] If the conversion is complete, the original edge information is superimposed on the candidate image data to achieve the effect of outlining.

[0185] If the overlay is complete, mean shift processing is performed on the candidate image data to obtain, as shown below. Figure 3E The target image data shown.

[0186] Mean shifting is a smoothing filter for candidate image data at the color level. It can neutralize colors with similar color distributions, smooth color details, and erode smaller color areas. It can determine the size of color blocks and the average pixel range of colors by setting the spatial radius and color space radius.

[0187] Considering that edge detection depends on significant changes in light and shadow, and that the edges are not particularly rounded after detection, this example first outlines the edges (i.e., overlays the target edge information) and then removes the light and shadow (i.e., means shift processing). This ensures the quality of edge detection and makes the overlaid edges more rounded.

[0188] Step 405: Replace the original image data with the target image data in the original video data to obtain the target video data.

[0189] In the original video data, the target image data can be replaced with the corresponding original image data to obtain the target video data.

[0190] After that, you can add advertising element data related to the target game to the target video data to obtain advertising video data. The advertising element data includes the logo (icon) of the platform used to distribute the target game, banner ads, EC (ending clip, which generally contains information about the target game (such as name, platform for distributing the target game, etc.)), and so on.

[0191] Advertising video data is published on designated channels (such as news, short videos, novel reading, sports and health, etc.) so that when the client accesses the channel, the advertising video data is pushed to the client for playback. When the user is interested in the target game, he downloads the target game from the platform that distributes the game.

[0192] In this embodiment, a game image reconstruction network is loaded; the original video data introducing the target game is obtained, which contains multiple frames of original image data; the original image data is input into the game image reconstruction network to reconstruct candidate image data containing the main style of the sample game; post-processing is performed on the candidate image data to obtain the target image data; the target image data is then used to replace the original image data in the original video data to obtain the target video data. When training the game image reconstruction network, a generative adversarial network is trained using unpaired data, enabling the network to reconstruct the image data to the main style of the sample game, improving training efficiency. Reconstructing the main style of the sample game is a post-processing step. Given a specific main style, post-processing the candidate image data makes the style of the sample game more complete. Post-processing is relatively simple, reducing the workload and maintaining the threshold for creating video data, thus significantly improving the efficiency of creating video data with a game style.

[0193] Example 4

[0194] Figure 5 This is a schematic diagram of the structure of a training device for a game image reconstruction network provided in Embodiment 4 of the present invention. Figure 5 As shown, the device includes:

[0195] The content sample image data acquisition module 501 is used to acquire content sample image data representing a real scene.

[0196] The sample game screening module 502 is used to screen sample games belonging to the simulation management category on mobile platforms;

[0197] The style sample image data extraction module 503 is used to acquire images with the main style of the sample game as style sample image data.

[0198] Generative Adversarial Network (GAN) training module 504 is used to train the GAN into a game image reconstruction network based on the content sample image data and the style sample image data. The game image reconstruction network is used to reconstruct image data with the main style of the sample game.

[0199] In one embodiment of the present invention, the style sample image data extraction module 503 is further configured to:

[0200] Acquire candidate video data recorded while the user is controlling the sample game;

[0201] Candidate video data with content that has the main style of the sample game are selected as sample video data;

[0202] Multiple frames of image data are extracted from the sample video data to serve as style sample image data.

[0203] In one embodiment of the present invention, the style sample image data extraction module 503 is further configured to:

[0204] Multiple frames of image data are extracted from the candidate video data and used as the first reference image data;

[0205] If the primary color of the first reference image data matches the main style of the sample game, then the first reference image data is marked as the second reference image data;

[0206] If the second reference image data contains a target object that appears in the operation of the sample game, then the second reference image data is marked as the third reference image data;

[0207] The third reference image data is used to filter sample video data from the candidate video data.

[0208] In one embodiment of the present invention, the style sample image data extraction module 503 is further configured to:

[0209] The candidate video data is divided into multiple video segments, each of which has an independent scene;

[0210] In each video segment, one frame of image data is extracted at preset intervals and used as the first reference image data.

[0211] In one embodiment of the present invention, the style sample image data extraction module 503 is further configured to:

[0212] For each pixel of the first reference image data, a binarization operation is performed on the green component;

[0213] If binarization is completed, the number of pixels with a non-zero green component is counted.

[0214] The percentage of the stated quantity is calculated.

[0215] If the percentage exceeds a preset first threshold, then the primary color of the first reference image data is determined to match the main style of the sample game, and the first reference image data is marked as the second reference image data.

[0216] In one embodiment of the present invention, the style sample image data extraction module 503 is further configured to:

[0217] Using cats and land as targets, a target detection operation is performed on the second reference image data to obtain the target objects that appear in the sample game.

[0218] Detect the edges of the land;

[0219] Calculate the curvature of the edge;

[0220] Calculate the degree of overlap between the cat and the land;

[0221] If the curvature is greater than or equal to a preset second threshold, and the degree of overlap is greater than or equal to a preset third threshold, then the second reference image data is marked as the third reference image data.

[0222] In one embodiment of the present invention, the style sample image data extraction module 503 is further configured to:

[0223] Mark the time point of the third reference image data on the time axis of the candidate video data;

[0224] If the interval between two adjacent time points is less than or equal to a preset fourth threshold, then the two adjacent time points are connected to obtain a connected region.

[0225] Video data located within the connected area is extracted from the candidate video data and used as sample video data.

[0226] In one embodiment of the present invention, it further includes:

[0227] A data augmentation processing module is used to perform data augmentation processing on the style sample image data, wherein the data augmentation processing includes at least one of the following:

[0228] The style sample image data is randomly cropped.

[0229] The style sample image data is randomly blurred.

[0230] The training apparatus for the game image reconstruction network provided in this embodiment of the invention can execute the training method for the game image reconstruction network provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the training method for the game image reconstruction network.

[0231] Example 5

[0232] Figure 6 This is a schematic diagram of an image reconstruction device provided in Embodiment 5 of the present invention. Figure 6 As shown, the device includes:

[0233] A network reconstruction loading module 601 is used to load a game image reconstruction network trained by the method according to any embodiment of the present invention.

[0234] The raw image data acquisition module 602 is used to acquire the raw image data to be reconstructed;

[0235] The candidate image data generation module 603 is used to input the original image data into the game image reconstruction network to reconstruct candidate image data containing the main style of the sample game.

[0236] The target image data generation module 604 is used to perform post-processing on the candidate image data to obtain the target image data.

[0237] In one embodiment of the present invention, the target image data generation module 604 is further configured to:

[0238] Extract human image data from the original image data;

[0239] Calculate the original edge information from the original image data;

[0240] Remove the original edge information from the portrait image data to obtain the target edge information;

[0241] The target edge information is converted into colors that conform to the non-main theme style of the sample game;

[0242] If the conversion is complete, the original edge information is superimposed on the candidate image data;

[0243] If the overlay is complete, mean shift processing is performed on the candidate image data to obtain the target image data.

[0244] The image reconstruction apparatus provided in this embodiment of the invention can execute the image reconstruction method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the image reconstruction method.

[0245] Example 6

[0246] Figure 7 This is a schematic diagram of a video reconstruction device provided in Embodiment Six of the present invention. Figure 7 As shown, the device includes:

[0247] A network reconstruction loading module 701 is used to load a game image reconstruction network trained by the method according to any embodiment of the present invention.

[0248] The video data acquisition module 702 is used to acquire raw video data that introduces the target game, wherein the raw video data contains multiple frames of raw image data;

[0249] The candidate image data generation module 703 is used to input the original image data into the game image reconstruction network to reconstruct candidate image data containing the main style of the sample game.

[0250] The target image data generation module 704 is used to perform post-processing on the candidate image data to obtain target image data;

[0251] The target video data generation module 705 is used to replace the original image data with the target image data in the original video data to obtain target video data.

[0252] In one embodiment of the present invention, it further includes:

[0253] An advertising video data generation module is used to add advertising element data related to the target game to the target video data as advertising video data.

[0254] The advertising video data publishing module is used to publish the advertising video data on a designated channel, so that when the client accesses the channel, the advertising video data is pushed to the client for playback.

[0255] The video reconstruction apparatus provided in this embodiment of the invention can execute the video reconstruction method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the video reconstruction method.

[0256] Example 7

[0257] Figure 8 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0258] like Figure 8As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0259] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0260] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as training methods for game image reconstruction networks, or image reconstruction methods, or video reconstruction methods.

[0261] In some embodiments, the training method for the game image reconstruction network, or the image reconstruction method, or the video reconstruction method, may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the training method for the game image reconstruction network, or the image reconstruction method, or the video reconstruction method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the training method for the game image reconstruction network, or the image reconstruction method, or the video reconstruction method by any other suitable means (e.g., by means of firmware).

[0262] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0263] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0264] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0265] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0266] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0267] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0268] Example 8

[0269] This invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements the training method, image reconstruction method, or video reconstruction method of the game image reconstruction network provided in any embodiment of this invention.

[0270] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0271] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0272] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A training method for a game image reconstruction network, characterized in that, include: Acquire sample image data representing real-world scenes; Select sample games belonging to the simulation management genre on mobile platforms; Obtain images with the main style of the sample game as style sample image data; Based on the content sample image data and the style sample image data, a generative adversarial network is trained into a game image reconstruction network, which is used to reconstruct image data with the main style of the sample game. The step of acquiring images with the main style of the sample game as style sample image data includes: Acquire candidate video data recorded while the user is controlling the sample game; Multiple frames of image data are extracted from the candidate video data and used as the first reference image data; For each pixel of the first reference image data, a binarization operation is performed on the green component; If binarization is completed, the number of pixels with a non-zero green component is counted. The percentage of the stated quantity is calculated. If the proportion exceeds a preset first threshold, then the primary color of the first reference image data is determined to match the main style of the sample game, and the first reference image data is marked as the second reference image data. If the second reference image data contains a target object that appears in the operation of the sample game, then the second reference image data is marked as the third reference image data; The third reference image data is used to filter sample video data from the candidate video data; Multiple frames of image data are extracted from the sample video data to serve as style sample image data.

2. The method according to claim 1, characterized in that, The step of extracting multiple frames of image data from the candidate video data as first reference image data includes: The candidate video data is divided into multiple video segments, each of which has an independent scene; In each video segment, one frame of image data is extracted at preset intervals and used as the first reference image data.

3. The method according to claim 1, characterized in that, If the second reference image data contains a target object appearing in the sample game, then marking the second reference image data as the third reference image data includes: Using cats and land as targets, a target detection operation is performed on the second reference image data to obtain the target objects that appear in the sample game. Detect the edges of the land; Calculate the curvature of the edge; Calculate the degree of overlap between the cat and the land; If the curvature is greater than or equal to a preset second threshold, and the degree of overlap is greater than or equal to a preset third threshold, then the second reference image data is marked as the third reference image data.

4. The method according to claim 1, characterized in that, The process of filtering sample video data from the candidate video data using the third reference image data includes: Mark the time point of the third reference image data on the time axis of the candidate video data; If the interval between two adjacent time points is less than or equal to a preset fourth threshold, then the two adjacent time points are connected to obtain a connected region. Video data located within the connected area is extracted from the candidate video data and used as sample video data.

5. The method according to any one of claims 1-4, characterized in that, After acquiring the image with the main style of the sample game as style sample image data, the process further includes: Perform data augmentation processing on the style sample image data, wherein the data augmentation processing includes at least one of the following: The style sample image data is randomly cropped. The style sample image data is randomly blurred.

6. An image reconstruction method, characterized in that, include: Load the game image reconstruction network trained according to any one of claims 1-5; Obtain the original image data to be reconstructed; The original image data is input into the game image reconstruction network and reconstructed into candidate image data containing the main style of the sample game. Post-processing is performed on the candidate image data to obtain the target image data.

7. The method according to claim 6, characterized in that, The post-processing of the candidate image data to obtain the target image data includes: Extract human image data from the original image data; Calculate the original edge information from the original image data; Remove the original edge information from the portrait image data to obtain the target edge information; The target edge information is converted into colors that conform to the non-main theme style of the sample game; If the conversion is complete, the original edge information is superimposed on the candidate image data; If the overlay is complete, mean shift processing is performed on the candidate image data to obtain the target image data.

8. A method for style reconstruction of video data, characterized in that, include: Load the game image reconstruction network trained according to any one of claims 1-5; The content to be acquired is the original video data introducing the target game, which contains multiple frames of original image data; The original image data is input into the game image reconstruction network and reconstructed into candidate image data containing the main style of the sample game. Post-processing is performed on the candidate image data to obtain the target image data; The target image data is replaced in the original video data to obtain the target video data.

9. The method according to claim 8, characterized in that, Also includes: Add advertising element data related to the target game to the target video data to create advertising video data; The advertising video data is published on a designated channel so that when a client accesses the channel, the advertising video data is pushed to the client for playback.

10. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, which enables the at least one processor to perform the training method of the game image reconstruction network according to any one of claims 1-5, the image reconstruction method according to any one of claims 6-7, or the video reconstruction method according to any one of claims 8-9.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the training method of the game image reconstruction network according to any one of claims 1-5, the image reconstruction method according to any one of claims 6-7, or the video reconstruction method according to any one of claims 8-9.

Images