A game style search network training, game search method and related equipment

By training portrait and the whole-region style search network, and combining the two output game style search network, the problem of low accuracy when searching games is solved, efficient game style search is achieved, and user experience is improved.

CN115475388BActive Publication Date: 2024-11-05SHANGHAI INGOT NETWORK TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202211146329.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-09-20
Publication Date
2024-11-05
Estimated Expiration
2042-09-20

AI Technical Summary

Technical Problem

Due to the general demand and lack of purpose when searching for games, users have low search accuracy and low efficiency, making it difficult to quickly find the game they like.

Method used

By loading the image search network, we can obtain the game's style representative image data, detect portrait data, and train the portrait style search network and the whole-region style search network. Combining the two, we can output the game style search network, and extract the style features to improve search Accuracy.

Benefits of technology

It realizes efficient search for game styles, and users can find their favorite games more quickly in search results, improving the efficiency and accuracy of search operations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115475388B_ABST
    Figure CN115475388B_ABST
Patent Text Reader

Abstract

The present invention discloses a training method for a game style search network, a game search method and related equipment, the method comprising: loading an image search network; obtaining image data representing the style of a game for a plurality of games; detecting portrait data in the image data; training the image search network into a portrait style search network using the portrait data as a sample, the portrait style search network being used to extract the characteristics of the portrait data in terms of style; training the image search network into a global style search network using the image data as a sample, the global style search network being used to extract the characteristics of the image data in terms of style; and jointly outputting the portrait style search network and the global style search network into a game style search network. This embodiment provides a basis for game style search, realizes the search for games with similar styles, gives game search more scalability, and improves the accuracy of searching for games.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and in particular to a training of a game style search network, a game search method and related equipment. Background Art

[0002] Games are one of the most commonly used applications for users' leisure and entertainment. Different game manufacturers release games for different platforms at different times, so the accumulated number of games is huge.

[0003] Users have different preferences for different games. If users want to search for their favorite games, one way is to use the index divided by the game website, which can be semantics such as "classic", "hot", "recently playable", or game features such as "3D", "cartoon", etc. Another way is to use keywords to search for games in general search engines.

[0004] However, the user's search for games itself is of a general nature, often not very purposeful, and has a certain degree of scalability, resulting in low accuracy when searching using the above method.

[0005] At this time, users often browse through the search results one by one to find their favorite games, which makes the search operation less efficient. Summary of the invention

[0006] The present invention provides a game style search network training, a game search method and related equipment to solve the problem of how to improve the accuracy of game search.

[0007] According to one aspect of the present invention, a method for training a game style search network is provided, comprising:

[0008] Load the image search network;

[0009] For a plurality of games, respectively obtain image data representing the painting styles in the games;

[0010] Detecting portrait data in the image data;

[0011] Using the portrait data as a sample, the image search network is trained into a portrait style search network, wherein the portrait style search network is used to extract the style features of the portrait data;

[0012] Using the image data as a sample, the image search network is trained into a global painting style search network, wherein the global painting style search network is used to extract the characteristics of the image data in terms of painting style;

[0013] The portrait style search network and the global style search network are jointly output as a game style search network.

[0014] According to another aspect of the present invention, there is provided a game search method, comprising:

[0015] Loading a game style search network, wherein the game style search network includes a portrait style search network and a global style search network;

[0016] Receiving raw image data from an original game;

[0017] Detecting original portrait data in the original image data;

[0018] If it is detected that there is original portrait data, the original portrait data is input into the portrait style search network to extract the original portrait style features representing the style;

[0019] Searching for a target game similar in style to the original game according to the style features of the original portrait;

[0020] If no original portrait data is detected, the original image data is input into the global painting style search network to extract the original global painting style features representing the painting style;

[0021] A target game having a similar painting style to the original game is searched for according to the original global painting style feature.

[0022] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0023] at least one processor; and

[0024] a memory communicatively connected to the at least one processor; wherein,

[0025] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the training method of the game style search network or the game search method described in any embodiment of the present invention.

[0026] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, wherein the computer program is used to enable a processor to implement the method for training a game style search network or the method for game search as described in any embodiment of the present invention when executed.

[0027] In this embodiment, an image search network is loaded; image data representing the painting style in the game is obtained for a plurality of games; portrait data is detected in the image data; the image search network is trained as a portrait painting style search network using the portrait data as a sample, and the portrait painting style search network is used to extract the characteristics of the portrait data in the painting style; the image search network is trained as a global painting style search network using the image data as a sample, and the global painting style search network is used to extract the characteristics of the image data in the painting style; the portrait painting style search network and the global painting style search network are jointly output as a game painting style search network. This embodiment reuses the image search network of the same structure to train the portrait painting style search network and the global painting style search network, so that the characteristics of the painting style of the portrait data and the whole frame image data are unified, and the characteristics of two levels are divided, which provides a basis for the two-level game painting style search, and then realizes the search for games with similar painting styles, giving the game search more scalability, and improving the accuracy of searching for games. Then, in most cases, users can intuitively browse games and find their favorite games in the search results, thereby improving the efficiency of the search operation.

[0028] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0030] Figure 1 is a flowchart of a method for training a game style search network provided according to Embodiment 1 of the present invention;

[0031] Figure 2 It is a schematic diagram of a process of training a portrait painting style search network according to the first embodiment of the present invention;

[0032] Figure 3 It is a schematic diagram of a process of training a global painting style search network according to the first embodiment of the present invention;

[0033] Figure 4 is a flowchart of a game search method provided according to Embodiment 2 of the present invention;

[0034] Figure 5 This is an example diagram of a search game provided according to the second embodiment of the present invention;

[0035] Figure 6 2 is a schematic diagram of a structure of a training device for a game style search network provided according to a third embodiment of the present invention;

[0036] Figure 7 is a structural schematic diagram of a game search device provided according to a fourth embodiment of the present invention;

[0037] Figure 8 It is a schematic diagram of the structure of an electronic device provided by Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0038] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0039] It should be noted that the terms "first", "second", "original", "candidate", "target", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0040] Embodiment 1

[0041] Figure 1 This is a flowchart of a method for training a game style search network provided in the first embodiment of the present invention. This embodiment is applicable to the case of a game style search network for game training search style. The method can be executed by a training device for a game style search network. The training device for a game style search network can be implemented in the form of hardware and / or software. The training device for a game style search network can be configured in an electronic device. Figure 1 As shown, the method includes:

[0042] Step 101: Load the image search network.

[0043] In this embodiment, an image search network can be constructed in advance for the game, and the image search network is used to extract features of image data. The image search network with the same structure can be reused and trained to extract features of the painting style of portrait data in the game, and to extract features of the painting style of the entire frame image data in the game.

[0044] Furthermore, the structure of the image search network is not limited to artificially designed neural networks, but can also be a neural network optimized by a model quantization method, a neural network searched for painting style characteristics by a NAS (Neural Architecture Search) method, and so on. This embodiment does not impose any restrictions on this.

[0045] Step 102: For a plurality of games, respectively obtain image data representing the painting styles in the games.

[0046] In this embodiment, a plurality of games released for different platforms (such as Android, iOS, Windows, professional game consoles, etc.) can be continuously screened out, waiting to be entered into the database and provided for search.

[0047] Among them, the types of these games may include MOBA (Multiplayer Online Battle Arena), RPG (Role-playing game), SLG (Simulation Game), etc., which are not limited in this embodiment.

[0048] Generally speaking, games are produced by a fixed team, and the game graphics are basically drawn or modeled by the production staff in the team (especially the artists). Therefore, the style of the game to a large extent refers to the painting style of the game creator, which makes the game have a relatively unified style on the screen, that is, the style of constructing the game screen refers to the brushwork and composition of painting. For the same scene, different teams may use different styles to express it, and highlight its understanding on the screen with artistic means, leaving people with different feelings, so the game has a more unique style on the screen.

[0049] For each game, we can use manual screenshots, learn summaries of video data containing game content, and other methods to obtain multi-frame image data representing its style.

[0050] In one method, some application distribution platforms (which can be called game libraries for games) require game developers to upload multiple frames of image data introducing the game when releasing the game. In order to introduce the game in a limited page, game developers often upload image data that reflects the game content and style.

[0051] Then, in this method, multiple games can be searched in the specified game library. If a game is found, the image data published in the game library to introduce the game is looked up. As the representative of the game style, the game maker is familiar with the game, and the image data he chooses to introduce the game is often representative of the game style. By reusing the image data introducing the game in the game library as the representative of the game style, not only the accuracy of collecting the style representative is guaranteed, but also the workload of collecting the image data representing the game style is reduced.

[0052] Step 103: Detect portrait data in the image data.

[0053] In the specific implementation, the details that reflect the painting style include the painting style in terms of character design (including appearance and body shape (such as the character's nose, eyes, ears, muscles, and hair line processing), clothing, hairstyle), background layout, etc.

[0054] Generally speaking, the portrait data in the game includes the head, clothing and other parts, which can reflect the character design. Therefore, in this embodiment, it is possible to detect whether there is portrait data in each frame of image data through face positioning, target detection algorithm and other methods. If there is portrait data, the portrait data is captured from the frame of image data.

[0055] In a specific implementation, face detection may be performed on image data, thereby identifying face data in the image data.

[0056] Among them, face detection is also called face key point detection, positioning or face alignment. It means locating the key areas of the face, including eyebrows, eyes, nose, mouth, facial contours, etc., given face data.

[0057] Furthermore, face detection includes at least one of the following methods:

[0058] 1. Use artificially extracted features, such as Haar features, use the features to train classifiers, and use the classifiers for face detection.

[0059] 2. Inherit face detection from general object detection algorithms, for example, using Faster R-CNN to detect faces.

[0060] 3. Use cascaded convolutional neural networks, for example, Cascade CNN (Cascaded Convolutional Neural Networks), MTCNN (Multi-task Cascaded Convolutional Networks).

[0061] Considering that simply marking facial data is sufficient to extract portrait data, the requirements for face detection algorithms are relatively low, noise is allowed, and general convolutional neural networks such as MTCNN can be used for face detection.

[0062] If face data is recognized, the face data is expanded in the surrounding areas in the image data to obtain portrait data.

[0063] If no face data is recognized, it is determined that no portrait data is detected in the image data.

[0064] In an extended method, the width and height of the face data can be queried.

[0065] Multiply the width by the first coefficient to get the first step length.

[0066] Multiply the height by the second coefficient to get the second step size.

[0067] The height is multiplied by a third coefficient to obtain a third step length, wherein the third coefficient is greater than the second coefficient.

[0068] Generally, the first coefficient, the second coefficient and the third coefficient may be default empirical values. Considering that different types of games have different situations in designing portrait data, which may tend to be realistic or Q-version, the first coefficient, the second coefficient and the third coefficient may be formulated according to the type of game, and this embodiment does not impose any restrictions on this.

[0069] Then, in the image data, the face data is expanded by the first step length along the horizontal direction (including the horizontal left direction and the horizontal right direction), the face data is expanded by the second step length along the vertical upward direction, and the face data is expanded by the third step length along the vertical downward direction to obtain portrait data.

[0070] In this method, portrait data is roughly located based on face data, and the operation of intercepting portrait data is simplified while meeting the requirements of training an image search network, thereby improving the efficiency of intercepting portrait data.

[0071] Step 104: Use the portrait data as samples to train the image search network into a portrait style search network.

[0072] In this embodiment, the image search network is trained using portrait data as training samples, and is recorded as a portrait style search network when the training is completed, so that the portrait style search network is used to extract the style features of the portrait data.

[0073] In one embodiment of the present invention, step 104 may include the following steps:

[0074] Step 1041: Input the portrait data into the image search network to extract the style features and obtain the portrait style features.

[0075] In this embodiment, the portrait data is input into the image search network, and the image search network processes the portrait data according to its own structural logic, extracts style features from the portrait data, and records them as portrait style features.

[0076] In a specific implementation, the image search network may be any network that can extract features of image data, in particular, a convolutional neural network (CNN). Therefore, the feature extraction process of different image search networks is different.

[0077] In one structure, the image search network includes a first residual network, a second residual network, a third residual network and a fourth residual network, wherein the first residual network, the second residual network, the third residual network and the fourth residual network all belong to residual networks ResNet of different depths (i.e., different layers), and the residual networks ResNet of different depths are formed by stacking basic residual blocks.

[0078] Among them, the basic residual block of some depths is recorded as Basicblock, which contains 2 convolutions; the basic residual block of some depths is recorded as Bottleneck, which contains 3 convolutions.

[0079] Compared with general convolutional neural networks, the residual network has an additional shortcut branch. The original structure is called the main branch. The shortcut branch allows the residual network to transmit the gradient directly to the previous network through this shortcut during backpropagation, thereby alleviating the problem of network degradation.

[0080] Taking Bottleneck as an example, the structure of the first residual network is It means that the first residual network distributes 3 identical structures in sequence, and 3 convolution layers are distributed in each structure in sequence. The convolution kernel of the first convolution layer is 1×1, and the number of output channels is 64. The convolution kernel of the second convolution layer is 3×3, and the feature dimension reduction operation is performed, and the number of output channels is 64. The convolution kernel of the third convolution layer is 1×1, and the feature dimension increase operation is performed, and the number of output channels is 256. In this way, the number of input and output channels of the 3×3 convolution layer in the middle of the structure can be made smaller, thereby improving efficiency. In addition, the feature matrix output on the main branch and the feature matrix output on the shortcut branch are of the same shape by first reducing and then increasing, so as to perform addition operations. After that, the features input to the structure are added to the features output by the structure (that is, the numbers of the features at the same position are added) as new features.

[0081] The structure of the second residual network is The structure of the third residual network is The structure of the fourth residual network is

[0082] In some cases, preprocessing is performed when portrait data is input into the image search network, including sequentially performing convolution operations, batch normalization operations (Batch Normalization, BN), maximum pooling operations, and so on.

[0083] If the processing of each residual network is called a stage, then Figure 2 As shown in the figure, the image search network extracts portraits and is divided into four stages:

[0084] 1. Stage 1

[0085] In the first stage, the portrait data is input into the first residual network to extract features and the features extracted by the first residual network are downsampled to obtain the first portrait candidate features.

[0086] 2. Stage 2

[0087] In the second stage, the first portrait candidate features are input into the second residual network for extraction and the features extracted by the second residual network are downsampled to obtain the second portrait candidate features.

[0088] In the first and second stages, downsampling the features can enhance the transformation invariance of the features and reduce the feature parameters to prevent overfitting, which is manifested in the gradual reduction of the feature size and the gradual increase of the number of channels.

[0089] 3. Stage 3

[0090] In the third stage, the second portrait candidate features are input into the third residual network to extract features to obtain the third portrait candidate features.

[0091] 4. Stage 4

[0092] In the fourth stage, the third portrait candidate features are input into the fourth residual network to extract features and obtain portrait style features.

[0093] In the third and fourth stages, ignoring downsampling can provide richer features.

[0094] This embodiment extracts features from image data through the first stage, the second stage, the third stage, and the fourth stage, and can extract high-level features from the image data.

[0095] Of course, the above-mentioned structure of the image search network is only used as an example. When implementing this embodiment, other structures of the image search network can be set according to actual conditions, and this embodiment does not limit this. In addition, in addition to the above-mentioned structure of the image search network, those skilled in the art can also adopt other judgment and processing methods according to actual needs, and this embodiment does not limit this.

[0096] Step 1042: Calculate the first portrait loss value when the portrait style feature is classified according to the style on multiple channels.

[0097] In this embodiment, a two-branch training image search network is provided, wherein one branch is the backbone of the training image search network, and the other branch assists the training image search network.

[0098] In the branch serving as the trunk, the loss of the image search network when extracting portrait style features and classifying portrait data according to style on multiple color channels (i.e., dividing portrait data with the same or similar style into the same category) is comprehensively calculated and recorded as the first portrait loss value.

[0099] In one embodiment of the present invention, step 1042 may further include the following steps:

[0100] 10421. Perform global description processing with pooling on sub-blocks of the portrait style feature on multiple channels to obtain a first portrait channel feature.

[0101] In this embodiment, if Figure 2 As shown, the portrait style features can be split into multiple (n) sub-blocks according to the channel. For each sub-block, the global description processing (Globaldescriptor, GD) with pooling (i.e., pooling operation) is performed on the channel in turn to obtain the first portrait channel feature, that is, the first portrait channel feature is a global feature on a certain channel.

[0102] The pooling operation includes at least one of the following:

[0103] And pooling operation SPoC Pooling, maximum pooling operation Max Pooling, and generalized average pooling operation GeMPooling.

[0104] In the specific implementation, the global description operation is expressed as follows:

[0105]

[0106] f={f 1 …f c …f C} T

[0107] Among them, let the portrait style feature be C×H×W, where C is the number of channels, H is the height, W is the width, and f c is the first portrait channel feature output by channel c(c∈C), X c is the sub-block (H×W) of each channel, P c Represents a pooling operation.

[0108] 10422. Perform a full connection operation on each first portrait channel feature to reduce the dimension into a second portrait channel feature.

[0109] 10423. Perform a second norm normalization process on each second portrait channel feature to obtain a third portrait channel feature.

[0110] like Figure 2 As shown in the figure, for each channel output feature (i.e., the first portrait channel feature), the fully connected layer (FC) fully connected operation can be performed in sequence to reduce the dimension, and the L2 (second norm) normalization processing can be performed on the reduced dimension feature (i.e., the second portrait channel feature). This process can be expressed as:

[0111]

[0112] Among them, a i represents the i-th pooling operation, is the feature of the third portrait channel, f is the feature of the first portrait channel, and W is the parameter of the fully connected layer.

[0113] 10424. Perform a series operation on the third portrait channel feature to obtain the first portrait global feature.

[0114] 10425. Perform a second norm normalization process on the first portrait global feature to obtain a second portrait global feature.

[0115] In this embodiment, if Figure 2 As shown, the third portrait channel features of each channel can be concatenated to obtain the first portrait global feature, and the first portrait global feature is normalized by L2 to obtain the second portrait global feature, that is, the second portrait global feature is the global feature of all channels, which is expressed as follows:

[0116]

[0117] Among them, a i represents the i-th pooling operation, is the third portrait channel feature, It is the global feature of the second portrait.

[0118] 10426. Use the second portrait global feature to calculate the loss of the additive angle interval for classification, and obtain the first portrait loss value.

[0119] In this embodiment, if Figure 2 As shown, the second portrait global feature is substituted into the additive angular margin loss function (ArcFace Loss), and the loss of the additive angular margin for classification is calculated, which is recorded as the first portrait loss value.

[0120] In ArcFace, the vectors and weights in the second portrait global feature are normalized, and the angle interval m is added to the angle θ between the vector and the weight. The angle between the deep feature and its corresponding weight is penalized in an additive manner. That is, ArcFace directly maximizes the classification boundary in the angle space θ, thereby enhancing the intra-class compactness and inter-class differences of the portrait data. The so-called penalty means that adding m during training will reduce θ. The angle interval has a more direct impact on the angle than the cosine interval, so there is a constant linear angle margin in geometry.

[0121] Step 1043: Calculate the second portrait loss value when the portrait style feature is classified according to the style on a single channel.

[0122] In the auxiliary branch, some results (i.e., features output on a single channel) in the main branch are reused to calculate the loss of the image search network when extracting portrait style features and classifying portrait data according to style on a single color channel (i.e., portrait data with the same or similar styles are divided into the same category), which is recorded as the second portrait loss value.

[0123] In the specific implementation, Figure 2 As shown, the first first portrait channel feature output on the first color channel in the branch serving as the trunk can be extracted, and a batch normalization operation (BN) is performed on the first first portrait channel feature to obtain the fourth portrait channel feature.

[0124] The fourth portrait channel feature is input into the fully connected layer (FC), and a fully connected operation is performed on the fourth portrait channel feature to map it into the fifth portrait channel feature.

[0125] Substitute the fifth portrait channel feature into the Softmax and other activation functions to activate the fifth portrait channel feature and obtain the sixth portrait channel feature.

[0126] Substitute the sixth portrait channel feature into the additive angle interval loss function (ArcFace Loss), use the sixth portrait channel feature to calculate the loss of the additive angle interval for classification, and obtain the second portrait loss value.

[0127] In this embodiment, the two steps of training branches and training the image search network are combined into one. The branches allow the image search network to better describe the distance between categories of portrait data, making the training faster and more stable.

[0128] In addition, in order to prevent overfitting and learn better features of image data, measures such as label smoothing and temperature adjustment may be added to the auxiliary branch, which is not limited in this embodiment.

[0129] Step 1044: Update the parameters of the image search network by combining the first portrait loss value and the second portrait loss value.

[0130] In this embodiment, the image search network may be back-propagated with reference to the first portrait loss value and the second portrait loss value to update the parameters of the image search network.

[0131] In this specific implementation, the first portrait loss value and the second portrait loss value can be merged into a total portrait loss value in a linear or nonlinear manner.

[0132] Among them, for the linear method, considering the difference in the role of the main trunk and the auxiliary, the weight of the first portrait loss value may be higher than the weight of the second portrait loss value.

[0133] Backpropagation is performed on the image search network and the branches serving as the trunk and the auxiliary branches. The total portrait loss value is substituted into optimization algorithms such as SGD (stochastic gradient descent) and Adam (Adaptive momentum). The gradients of the image search network and the branches serving as the trunk and the auxiliary branches are calculated respectively, so as to update the parameters of the image search network, the parameters of the branches serving as the trunk, and the parameters of the auxiliary branches according to the gradient.

[0134] In this embodiment, the operations involved in the branches serving as the trunk and the auxiliary branches are relatively simple, and will not add too large a structure to the image search network for training, nor will it modify the original structure of the image search network. In addition, each branch is controlled and adjusted separately, with fewer logical modifications, and can be well integrated into the image search network, thereby improving the efficiency of training the image search network.

[0135] Step 1045 , determine whether the preset first training condition is met; if so, execute step 1046 , if not, return to execute step 1041 .

[0136] Step 1046: Output the parameters of the image search network to the first configuration file.

[0137] In this embodiment, a first training condition can be set in advance as a condition for stopping using portrait data to train the image search network. For example, the number of iterative training reaches a certain threshold, the amplitude of the change in the total portrait loss value for multiple consecutive times is less than a certain threshold, the total portrait loss value is less than a certain threshold, and so on. In each round of iterative training, it is determined whether the data recorded during the current iterative training meets the first training condition.

[0138] If the first training condition is met, it can be considered that the image search network training using the portrait data is completed. At this time, the image search network is recorded as a portrait style search network, and the parameters in the portrait style search network are output and persisted in the first configuration file config.

[0139] If the first training condition is not met, the next round of iterative training can be entered, and steps 1041 to 1045 are executed again, and the iterative training is repeated in this way until the image search network is transformed into a portrait style search network using portrait data.

[0140] Step 105: Use the image data as samples to train the image search network into a global style search network.

[0141] In addition to the details of painting, the style of the game also includes the style of the game's overall picture, such as the overall color style, tone style, light style, overall picture style (the picture's sense of space and layering), etc.

[0142] In this embodiment, the image search network is trained using the entire frame of image data as a training sample, and when the training is completed, it is recorded as a global style search network, so that the global style search network is used to extract the style features of the image data.

[0143] Furthermore, the entire frame of image data may or may not contain portrait data, which is not limited in this embodiment.

[0144] In one embodiment of the present invention, step 105 may include the following steps:

[0145] Step 1051: Input the image data into the image search network to extract the style features and obtain the global style features.

[0146] In this embodiment, the whole frame of image data is input into the image search network, and the image search network processes the whole frame of image data according to its own structural logic, extracts style features from the whole frame of image data, and records them as global style features.

[0147] In one structure, the image search network includes a first residual network, a second residual network, a third residual network and a fourth residual network.

[0148] If the processing of each residual network is called a stage, then Figure 3 As shown in the figure, the image search network extracts portraits and is divided into four stages:

[0149] 1. Stage 1

[0150] In the first stage, the image data is input into the first residual network to extract features and the features extracted by the first residual network are downsampled to obtain the first global candidate features.

[0151] 2. Stage 2

[0152] In the second stage, the first global candidate features are input into the second residual network for extraction and the features extracted by the second residual network are downsampled to obtain second global candidate features.

[0153] 3. Stage 3

[0154] In the third stage, the second global candidate features are input into the third residual network to extract features to obtain the third global candidate features.

[0155] 4. Stage 4

[0156] In the fourth stage, the third global candidate feature is input into the fourth residual network to extract features and obtain the global style feature.

[0157] Step 1052: cluster the portrait data to obtain categories of the image data.

[0158] In this embodiment, the portrait data is clustered by using methods such as K-Means, hierarchical clustering, and density clustering, and the category of the portrait data is assigned as the category of the image data, thereby performing semi-supervised training.

[0159] Step 1053: Using the category as a label, calculate the first global loss value when the global style feature is classified according to the style on multiple channels.

[0160] In this embodiment, a two-branch training image search network is provided, wherein one branch is the backbone of the training image search network, and the other branch assists the training image search network.

[0161] In the branch serving as the trunk, the loss of the image search network when extracting global style features and classifying the entire frame of image data according to the style on multiple color channels (that is, classifying image data with the same or similar style into the same category) is comprehensively calculated and recorded as the first global loss value.

[0162] In the specific implementation, Figure 3As shown, a global description process (GD) with pooling is performed on the sub-blocks of the global style feature on multiple channels to obtain a first global channel feature; a fully connected operation (FC) is performed on each first global channel feature to reduce the dimension to a second global channel feature; a normalization process of the second norm (L2) is performed on each second global channel feature to obtain a third global channel feature; all third global channel features are concatenated to obtain a first global global feature; and a normalization process of the second norm (L2) is performed on the first global global feature to obtain a second global global feature.

[0163] The category is used as a label (Tag / Label), and the second global feature is used to calculate the cross entropy for classification to obtain the first global loss value.

[0164] Furthermore, the category and the second global feature are substituted into the cross entropy loss function (CE Loss) to calculate the cross entropy for classification and obtain the first global loss value.

[0165] Among them, cross entropy is an indicator of output uncertainty caused by the mismatch between the possibility of actual output and the predicted possibility.

[0166] Exemplarily, the cross entropy loss function is as follows:

[0167] Loss = ∑-y·ln(x)

[0168] Among them, y is the label. When the image data actually belongs to the current category, y is 1. When the image data does not belong to the current category, y is 0. x is the probability of predicting each category to which the image data belongs.

[0169] Step 1054: Using the category as a label, calculate the second global loss value when classifying based on the style on a single channel of the global style feature.

[0170] In the auxiliary branch, some results (i.e., features output on a single channel) in the main branch are reused to calculate the loss of the image search network when extracting global style features and classifying the entire frame of image data according to the style on a single color channel (i.e., classifying image data with the same or similar styles into the same category), which is recorded as the second global loss value.

[0171] In the specific implementation, Figure 3As shown, a batch normalization operation (BN) is performed on the first global channel feature to obtain the fourth global channel feature; a fully connected operation (FC) is performed on the fourth global channel feature to map it to the fifth global channel feature; the fifth global channel feature is activated using functions such as Softmax to obtain the sixth global channel feature; the category is used as the label, and the sixth global channel feature is used to calculate the cross entropy (CE Loss) for classification to obtain the second global loss value.

[0172] Step 1055: Update the parameters of the image search network by combining the first global loss value and the second global loss value.

[0173] In this embodiment, the first global loss value and the second global loss value may be comprehensively referenced to perform back propagation on the image search network to update the parameters of the image search network.

[0174] In this specific implementation, the first global loss value and the second global loss value may be fused into a global total loss value in a linear or nonlinear manner.

[0175] Among them, for the linear method, considering the difference in the role of the backbone and the auxiliary, the weight of the first global loss value may be higher than the weight of the second global loss value.

[0176] Backpropagation is performed on the image search network and the branches serving as the trunk and the auxiliary branches. The total portrait loss value is substituted into optimization algorithms such as SGD and Adam. The gradients of the image search network and the branches serving as the trunk and the auxiliary branches are calculated respectively, so as to update the parameters of the image search network and the parameters of the branches serving as the trunk and the auxiliary branches according to the gradient.

[0177] Step 1056 , determine whether the preset second training condition is met; if so, execute step 1057 , if not, return to execute step 1051 .

[0178] Step 1057: Output the parameters of the image search network to the second configuration file.

[0179] In this embodiment, a second training condition can be set in advance as a condition for stopping using portrait data to train the image search network. For example, the number of iterative training reaches a certain threshold, the amplitude of the change in the global total loss value for multiple consecutive times is less than a certain threshold, the global total loss value is less than a certain threshold, and so on. In each round of iterative training, it is determined whether the data recorded during the current iterative training meets the second training condition.

[0180] If the second training condition is met, it can be considered that the image search network training using the image data is completed. At this time, the image search network is recorded as a global style search network, and the parameters in the global style search network are output and persisted in the second configuration file config.

[0181] If the second training condition is not met, the next round of iterative training can be entered, and steps 1051 to 1056 are executed again, and the iterative training is repeated in this way until the image search network is transformed into a global style search network using image data.

[0182] In this embodiment, since step 105 is basically similar to step 104 in the application of the image search network, the description is relatively simple, and the relevant parts can be referred to the partial description of step 104, and this embodiment will not be described in detail here.

[0183] Step 106: jointly output the portrait style search network and the global style search network into a game style search network.

[0184] In a specific implementation, the portrait style search network and the global style search network can be trained asynchronously. If both the portrait style search network and the global style search network have been trained, the portrait style search network and the global style search network can be jointly output and recorded as a game style search network.

[0185] Furthermore, the portrait style search network and the global style search network reuse the image search network with the same structure. On this basis, the portrait style search network sets a first configuration file, which records the parameters of the image search network for extracting the style features of the portrait data. The global style search sets a second configuration file, which records the parameters of the image search network for extracting the style features of the entire frame of image data.

[0186] Generally speaking, when users want to search for games with similar styles, they usually search on general search engines using the game name, "style", "similar (or identical, close, similar, etc.)" and other keywords. The search results are basically some records left by other users on public websites such as blogs and forums. In most cases, games with similar styles cannot be found.

[0187] If a user uses a search engine's image search service and provides image data of a game, the search results will basically be image data of the same game. In most cases, games with a similar style cannot be found.

[0188] In this embodiment, an image search network is loaded; image data representing the painting style in the game is obtained for a plurality of games; portrait data is detected in the image data; the image search network is trained as a portrait painting style search network using the portrait data as a sample, and the portrait painting style search network is used to extract the characteristics of the portrait data in the painting style; the image search network is trained as a global painting style search network using the image data as a sample, and the global painting style search network is used to extract the characteristics of the image data in the painting style; the portrait painting style search network and the global painting style search network are jointly output as a game painting style search network. This embodiment reuses the image search network of the same structure to train the portrait painting style search network and the global painting style search network, so that the characteristics of the painting style of the portrait data and the whole frame image data are unified, and the characteristics of two levels are divided, which provides a basis for the two-level game painting style search, and then realizes the search for games with similar painting styles, giving the game search more scalability, and improving the accuracy of searching for games. Then, in most cases, users can intuitively browse games and find their favorite games in the search results, thereby improving the efficiency of the search operation.

[0189] Embodiment 2

[0190] Figure 4 This is a flowchart of a game search method provided in the first embodiment of the present invention. This embodiment can be applied to the case of using a game style search network to search for games with similar styles. The method can be executed by a game search device, which can be implemented in the form of hardware and / or software, and can be configured in an electronic device. Figure 4 As shown, the method includes:

[0191] Step 401, load the game style search network.

[0192] In a specific implementation, the game style search network can be pre-trained according to the method of embodiment 1 of the present invention, wherein the game style search network includes a portrait style search network and a global style search network, the portrait style search network is used to extract the style features of the portrait data, and the global style search network is used to extract the style features of the image data.

[0193] When the game style search network is applied for searching, the portrait style search network and its parameters, and the global style search network and its parameters are loaded into the memory for running.

[0194] In a specific implementation, if the portrait style search network and the global style search network reuse the same structure of image search network training, then the image search network can be loaded and encapsulated as a main class. The image search network usually runs in the form of an instance (i.e., an instantiated object). When resources are sufficient, two instances can be generated for the image search network.

[0195] A first configuration file is loaded into the image search network (one of the instances) to obtain a portrait style search network, wherein the first configuration file records parameters obtained by training the image search network with the goal of extracting style features of portrait data.

[0196] A second configuration file is loaded into the image search network (another instance) to obtain a global style search network, wherein the second configuration file records parameters obtained by training the image search network with the goal of extracting style features of the entire frame of image data.

[0197] Furthermore, the portrait style search network and the global style search network reuse the image search network training with the same structure, and load the first configuration file and the second configuration file in the form of instances, etc., which can reduce the usage of storage, memory and other resources.

[0198] Step 402: Receive original image data from an original game.

[0199] In this embodiment, if Figure 5 As shown, the user can collect original image data 510 in an original game by taking screenshots, taking photos, searching, etc. The original image data 510 can be a part of the original game screen or the entire original game screen.

[0200] Generally speaking, the structure and parameters of the image search network are relatively large, occupying more resources, and are usually deployed on a server. The server can encapsulate the generative adversarial network into an API (Application Programming Interface) to provide game search services to users on the intranet or the public network.

[0201] Since the business scenarios in which the server is used are different, the manner of providing the game search service is also different, and this embodiment does not impose any limitation on this.

[0202] If the server is a business server of a game website on the public network, it can provide a game search function to users of the public network in the form of a vertical search engine.

[0203] If the server is a business server for searching websites on the public network, the game search function can be provided to users on the public network in the form of one of the functions of a general search engine (such as searching for games by image, etc.).

[0204] If the server is a business server in an intranet, the game search function can be provided to users in the intranet in the form of product retrieval.

[0205] Then, the user can log in to the client, and the client calls the API interface to transmit the original image data to the server, and sends a search request to the server to search for other games with similar styles to the original game.

[0206] Step 403: Detect original portrait data in the original image data.

[0207] In actual applications, when users play games, they focus more on the characters in the game and pay attention to the character designs of different characters. Therefore, the original portrait data is the focus of the painting style, such as Figure 5 As shown, the existence of original portrait data 511 can be detected in the original image data 510 by means of face positioning, target detection algorithm, etc. If the original portrait data 511 exists, the original portrait data 511 is intercepted from the original image data 510.

[0208] In one embodiment of the present invention, step 403 may include the following steps:

[0209] Recognize face data in raw image data;

[0210] If face data is recognized, the original face data is expanded in the original image data to obtain original portrait data;

[0211] If no face data is recognized, it is determined that no original portrait data is detected in the original image data.

[0212] Furthermore, when expanding the face data, the width and height of the face data can be queried;

[0213] Multiply the width by the first coefficient to get the first step length;

[0214] Multiply the height by the second coefficient to get the second step length;

[0215] Multiplying the height by a third coefficient to obtain a third step length, wherein the third coefficient is greater than the second coefficient;

[0216] In the original image data, the face data is expanded by a first step length in the horizontal direction, a second step length in the vertical upward direction, and a third step length in the vertical downward direction to obtain the original portrait data.

[0217] Step 404: If it is detected that there is original portrait data, the original portrait data is input into a portrait style search network to extract original portrait style features representing the style.

[0218] When the original image data contains original portrait data, the original portrait data is input into the portrait style search network. The portrait style search network processes the original portrait data according to its own structural logic, extracts style features from the original portrait data, and records them as original portrait style features.

[0219] In one embodiment of the present invention, the portrait style search network includes a first residual network, a second residual network, a third residual network and a fourth residual network;

[0220] In this embodiment, step 404 may include the following steps:

[0221] Inputting the original portrait data into the first residual network to extract features and perform downsampling to obtain first portrait candidate features;

[0222] Input the first portrait candidate feature into the second residual network to extract and downsample it to obtain the second portrait candidate feature;

[0223] Inputting the second portrait candidate feature into the third residual network to extract features, thereby obtaining the third portrait candidate feature;

[0224] The third portrait candidate features are input into the fourth residual network to extract features and obtain the original portrait style features.

[0225] Step 405: Search for a target game similar in style to the original game based on the original portrait style features.

[0226] In this embodiment, if Figure 5 As shown, different games can be pre-collected into the game library 520, then other games similar in style (especially in the style of the portrait data) to the original game can be searched based on the original portrait style features and recorded as target games.

[0227] In the specific implementation, Figure 5 As shown, some and all preset games can be recalled from the game library 520, which are recorded as candidate games 521, wherein the candidate game 521 has a candidate portrait style feature, which is a feature that characterizes the style and is extracted from the candidate portrait data in the candidate game using a portrait style search network.

[0228] The first similarity between the original portrait style features and the candidate portrait style features is calculated using methods such as cosine angle, so that multiple candidate games are screened out based on the first similarity as the target game 522 that is similar in style to the original game, and the information of the target game 522 (such as name, introduction, user evaluation, download address, etc.) is made available for users to browse and screen.

[0229] Generally, the candidate games can be sorted in order according to the first similarity, that is, the candidate games are sorted in order from large to small according to the first similarity, and the first n (n is a positive integer) candidate games are extracted as target games with similar style to the original game.

[0230] Of course, in addition to sorting, a first threshold value (ie, the similarity is greater than or equal to the first threshold value) and other conditions for screening target games may also be set for the candidate games, and this embodiment does not impose any limitation on this.

[0231] Step 406: If no original portrait data is detected, the original image data is input into the global style search network to extract the original global style features representing the style.

[0232] When the original image data does not contain original portrait data, the entire frame of original image data is input into the global style search network. The global style search network processes the entire frame of original image data according to its own structural logic, extracts style features from the entire frame of original image data, and records them as original global style features.

[0233] In one embodiment of the present invention, the global style search network includes a first residual network, a second residual network, a third residual network and a fourth residual network;

[0234] In this embodiment, step 405 may include the following steps:

[0235] Inputting the original image data into the first residual network to extract features and perform downsampling to obtain first global candidate features;

[0236] Input the first global candidate feature into the second residual network to extract and downsample to obtain a second global candidate feature;

[0237] Inputting the second global candidate feature into the third residual network to extract features, thereby obtaining a third global candidate feature;

[0238] The third global candidate feature is input into the fourth residual network to extract features and obtain the original global painting style feature.

[0239] Step 407: Search for a target game similar in style to the original game based on the original global style features.

[0240] In this embodiment, different games can be pre-collected into the game library, then other games similar in style (especially in the style of the entire frame image data) to the original game can be searched based on the original global style features and recorded as target games.

[0241] In a specific implementation, some and then all preset games can be recalled from the game library and recorded as candidate games, wherein the candidate games have candidate global style features, which are features that characterize the style and are extracted from the entire frame of candidate image data in the candidate games using a global style search network.

[0242] The second similarity between the original global style feature and the candidate global style feature is calculated using methods such as cosine angle, so as to screen out multiple candidate games based on the second similarity as target games that are similar in style to the original game.

[0243] Generally, the candidate games can be sorted in order according to the second similarity, that is, the candidate games are sorted in order from large to small according to the second similarity, and the first m (m is a positive integer) candidate games are extracted as target games with similar style to the original game.

[0244] Of course, in addition to sorting, a second threshold value (ie, the similarity is greater than or equal to the second threshold value) and other conditions for screening target games may also be set for the candidate games, and this embodiment does not impose any limitation on this.

[0245] In this embodiment, since the portrait data, portrait style search network, and global style search network are basically similar to the applications of the first embodiment, the description is relatively simple. For the relevant parts, please refer to the partial description of the first embodiment. This embodiment will not be described in detail here.

[0246] In this embodiment, a game style search network is loaded, and the game style search network includes a portrait style search network and a global style search network; original image data from the original game is received; original portrait data is detected in the original image data; if the original portrait data is detected, the original portrait data is input into the portrait style search network to extract the original portrait style features representing the style; according to the original portrait style features, a target game similar in style to the original game is searched; if the original portrait data is not detected, the original image data is input into the global style search network to extract the original global style features representing the style; according to the original global style features, a target game similar in style to the original game is searched. This embodiment divides the features of the style of the portrait data and the whole frame image data at the two levels, and then searches for games with similar styles at the two levels of the portrait data and the whole frame image data, giving the game search more scalability and improving the accuracy of searching for games. Then, in most cases, users can intuitively browse games and find their favorite games in the search results, thereby improving the efficiency of the search operation.

[0247] Embodiment 3

[0248] Figure 6This is a schematic diagram of a training device for a game style search network provided by the fourth embodiment of the present invention. Figure 6 As shown, the device comprises:

[0249] An image search network loading module 601 is used to load an image search network;

[0250] An image data acquisition module 602 is used to acquire image data representing the painting styles in each of the multiple games;

[0251] A portrait data detection module 603, used to detect portrait data in the image data;

[0252] A portrait style search network training module 604 is used to train the image search network into a portrait style search network using the portrait data as a sample, wherein the portrait style search network is used to extract the style features of the portrait data;

[0253] A global painting style search network training module 605 is used to train the image search network into a global painting style search network using the image data as a sample, wherein the global painting style search network is used to extract the characteristics of the image data in terms of painting style;

[0254] The game style search network output module 606 is used to jointly output the portrait style search network and the global style search network into a game style search network.

[0255] In one embodiment of the present invention, the image data acquisition module 602 is further used for:

[0256] Search for multiple games in a specified game library;

[0257] If the game is found, image data published in the game library and used to introduce the game is searched as a representative of the painting style in the game.

[0258] In one embodiment of the present invention, the portrait data detection module 603 is further used to:

[0259] identifying facial data in the image data;

[0260] If the face data is recognized, the face data is expanded in the image data to obtain portrait data;

[0261] If the face data is not recognized, it is determined that no portrait data is detected in the image data.

[0262] In one embodiment of the present invention, the portrait data detection module 603 is further used to:

[0263] Query the width and height of the face data;

[0264] Multiplying the width by a first coefficient to obtain a first step length;

[0265] multiplying the height by a second coefficient to obtain a second step length;

[0266] multiplying the height by a third coefficient to obtain a third step length, wherein the third coefficient is greater than the second coefficient;

[0267] In the image data, the face data is expanded by the first step length in the horizontal direction, the second step length in the vertical upward direction, and the third step length in the vertical downward direction to obtain portrait data.

[0268] In one embodiment of the present invention, the portrait style search network training module 604 is further used to:

[0269] Inputting the portrait data into the image search network to extract the characteristics of the painting style, thereby obtaining the portrait style characteristics;

[0270] Calculating a first portrait loss value when the portrait style feature is classified according to the style on multiple channels;

[0271] Calculating a second portrait loss value when the portrait style feature is classified according to the style on a single channel;

[0272] updating the parameters of the image search network in combination with the first portrait loss value and the second portrait loss value;

[0273] Determine whether a preset first training condition is met; if so, output the parameters of the image search network to the first configuration file; if not, return to execute the step of inputting the portrait data into the image search network to extract the style features and obtain the portrait style features.

[0274] In one embodiment of the present invention, the image search network includes a first residual network, a second residual network, a third residual network and a fourth residual network; the portrait style search network training module 604 is also used for:

[0275] The step of inputting the portrait data into the image search network to extract the style features and obtain the portrait style features comprises:

[0276] Inputting the portrait data into the first residual network to extract features and perform downsampling to obtain first portrait candidate features;

[0277] Inputting the first portrait candidate feature into the second residual network to extract and downsample to obtain a second portrait candidate feature;

[0278] Inputting the second portrait candidate feature into the third residual network to extract features, thereby obtaining a third portrait candidate feature;

[0279] The third portrait candidate feature is input into the fourth residual network to extract features and obtain portrait style features.

[0280] In one embodiment of the present invention, the portrait style search network training module 604 is further used to:

[0281] Performing a global description process with pooling on the sub-blocks of the portrait style feature on multiple channels respectively to obtain a first portrait channel feature;

[0282] Performing a full connection operation on each of the first portrait channel features to reduce the dimension into a second portrait channel feature;

[0283] Performing a normalization process of the second norm on each of the second portrait channel features respectively to obtain a third portrait channel feature;

[0284] Performing a series operation on all the third portrait channel features to obtain a first portrait global feature;

[0285] Performing a normalization process of a second norm on the first portrait global feature to obtain a second portrait global feature;

[0286] The second portrait global feature is used to calculate the loss of the additive angle interval for classification to obtain the first portrait loss value.

[0287] In one embodiment of the present invention, the portrait style search network training module 604 is further used to:

[0288] Performing a batch normalization operation on the first portrait channel feature to obtain a fourth portrait channel feature;

[0289] Performing a fully connected operation on the fourth portrait channel feature to map it into a fifth portrait channel feature;

[0290] activating the fifth portrait channel feature to obtain a sixth portrait channel feature;

[0291] The sixth portrait channel feature is used to calculate the loss of the additive angle interval for classification, and a second portrait loss value is obtained.

[0292] In one embodiment of the present invention, the global painting style search network training module 605 is further used to:

[0293] Inputting the image data into the image search network to extract the features of the painting style and obtain the global painting style features;

[0294] Clustering the portrait data to obtain categories of the image data;

[0295] Using the category as a label, calculating a first global loss value when the global style feature is classified according to the style on multiple channels;

[0296] Using the category as a label, calculating a second global loss value when classifying a single channel of the global style feature according to the style;

[0297] updating the parameters of the image search network in combination with the first global loss value and the second global loss value;

[0298] Determine whether the preset second training condition is met; if so, output the parameters of the image search network to the second configuration file, if not, return to execute the step of inputting the global data into the image search network to extract the style features and obtain the global style features.

[0299] In one embodiment of the present invention, the image search network includes a first residual network, a second residual network, a third residual network and a fourth residual network; the global style search network training module 605 is also used for:

[0300] Inputting the image data into the first residual network to extract features and perform downsampling to obtain first global candidate features;

[0301] Inputting the first global candidate feature into the second residual network to extract and downsample to obtain a second global candidate feature;

[0302] Inputting the second global candidate feature into the third residual network to extract features, thereby obtaining a third global candidate feature;

[0303] The third global candidate feature is input into the fourth residual network to extract features, thereby obtaining a global painting style feature.

[0304] In one embodiment of the present invention, the global painting style search network training module 605 is further used to:

[0305] Performing a global description process with pooling on the sub-blocks of the global painting style feature on multiple channels respectively to obtain a first global channel feature;

[0306] Performing a full connection operation on each of the first global channel features to reduce the dimension into a second global channel feature;

[0307] Performing a normalization process of the second norm on each of the second global channel features respectively to obtain a third global channel feature;

[0308] Perform a series operation on all the third global channel features to obtain a first global feature;

[0309] Performing a normalization process of a second norm on the first global feature to obtain a second global feature;

[0310] The category is used as a label and the second global feature is used to calculate the cross entropy for classification to obtain a first global loss value.

[0311] In one embodiment of the present invention, the global painting style search network training module 605 is further used to:

[0312] Performing a batch normalization operation on the first global channel feature to obtain a fourth global channel feature;

[0313] Performing a fully connected operation on the fourth global channel feature to map it into a fifth global channel feature;

[0314] activating the fifth global channel feature to obtain a sixth global channel feature;

[0315] The category is used as a label and the sixth global channel feature is used to calculate a cross entropy for classification to obtain a second global loss value.

[0316] The training device for the game style search network provided in the embodiment of the present invention can execute the training method for the game style search network provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the training method for the game style search network.

[0317] Embodiment 4

[0318] Figure 7 This is a schematic diagram of the structure of a game search device provided by Embodiment 4 of the present invention. Figure 7 As shown, the device comprises:

[0319] A game style search network loading module 701 is used to load a game style search network, wherein the game style search network includes a portrait style search network and a global style search network;

[0320] The original image data receiving module 702 is used to receive the original image data from the original game;

[0321] The original portrait data detection module 703 is used to detect the original portrait data in the original image data;

[0322] The original portrait style feature extraction module 704 is used to input the original portrait data into the portrait style search network to extract the original portrait style features representing the style if the original portrait data is detected;

[0323] A first target game search module 705, configured to search for a target game similar in style to the original game according to the style feature of the original portrait;

[0324] The original global painting style feature extraction module 706 is used to input the original image data into the global painting style search network to extract the original global painting style feature representing the painting style if no original portrait data is detected;

[0325] The second target game search module 707 is used to search for a target game similar in style to the original game according to the original global style feature.

[0326] In one embodiment of the present invention, the game style search network loading module 701 is further used for:

[0327] Load the image search network;

[0328] Loading a first configuration file in the image search network to obtain a portrait style search network, wherein the first configuration file records parameters obtained by training the image search network with the goal of extracting style features of portrait data;

[0329] A second configuration file is loaded into the image search network to obtain a global style search network, wherein the second configuration file records parameters obtained by training the image search network with the goal of extracting style features of the entire frame of image data.

[0330] In one embodiment of the present invention, the original portrait data detection module 703 is further used to:

[0331] Recognizing facial data in the original image data;

[0332] If the face data is recognized, the original face data is expanded in the original image data to obtain original portrait data;

[0333] If the face data is not recognized, it is determined that the original portrait data is not detected in the original image data.

[0334] In one embodiment of the present invention, the original portrait data detection module 703 is further used to:

[0335] Query the width and height of the face data;

[0336] Multiplying the width by a first coefficient to obtain a first step length;

[0337] multiplying the height by a second coefficient to obtain a second step length;

[0338] multiplying the height by a third coefficient to obtain a third step length, wherein the third coefficient is greater than the second coefficient;

[0339] In the original image data, the face data is expanded by the first step length in the horizontal direction, the second step length in the vertical upward direction, and the third step length in the vertical downward direction to obtain the original portrait data.

[0340] In one embodiment of the present invention, the portrait style search network includes a first residual network, a second residual network, a third residual network and a fourth residual network; the original portrait style feature extraction module 704 is also used for:

[0341] Inputting the original portrait data into the first residual network to extract features and perform downsampling to obtain first portrait candidate features;

[0342] Inputting the first portrait candidate feature into the second residual network to extract and downsample to obtain a second portrait candidate feature;

[0343] Inputting the second portrait candidate feature into the third residual network to extract features, thereby obtaining a third portrait candidate feature;

[0344] The third portrait candidate feature is input into the fourth residual network to extract features, so as to obtain the original portrait style feature.

[0345] In one embodiment of the present invention, the first target game search module 705 is further used to:

[0346] Recalling preset candidate games, the candidate games having candidate portrait style features, the candidate portrait style features being features representing style extracted from candidate portrait data in the candidate games using the portrait style search network;

[0347] Calculating a first similarity between the original portrait style feature and the candidate portrait style feature;

[0348] A plurality of the candidate games are screened out according to the first similarity as target games having a similar style to the original game.

[0349] In one embodiment of the present invention, the global style search network includes a first residual network, a second residual network, a third residual network and a fourth residual network; the original global style feature extraction module 706 is also used for:

[0350] Inputting the original image data into the first residual network to extract features and perform downsampling to obtain first global candidate features;

[0351] Inputting the first global candidate feature into the second residual network to extract and downsample to obtain a second global candidate feature;

[0352] Inputting the second global candidate feature into the third residual network to extract features, thereby obtaining a third global candidate feature;

[0353] The third global candidate feature is input into the fourth residual network to extract features, so as to obtain the original global painting style feature.

[0354] In one embodiment of the present invention, the second target game search module 707 is further used to:

[0355] Recalling preset candidate games, the candidate games having candidate global style features, the candidate global style features being features representing the style extracted from the candidate image data of a whole frame in the candidate games using the global style search network;

[0356] Calculating a second similarity between the original global painting style feature and the candidate global painting style feature;

[0357] A plurality of the candidate games are screened out according to the second similarity as target games having a similar style to the original game.

[0358] The game search device provided in the embodiment of the present invention can execute the game search method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the game search method.

[0359] Embodiment 5

[0360] Figure 8 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0361] like Figure 8As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0362] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0363] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as the training method of the game style search network or the game search method.

[0364] In some embodiments, the training method of the game style search network or the game search method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the training method of the game style search network or the game search method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the training method of the game style search network or the game search method by any other appropriate means (e.g., by means of firmware).

[0365] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0366] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0367] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0368] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0369] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0370] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0371] Embodiment 6

[0372] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements a training method for a game style search network or a game search method as provided in any embodiment of the present invention.

[0373] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0374] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0375] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for training a game style search network, characterized in that: include: Load the image search network; For a plurality of games, respectively obtain image data representing the painting styles in the games; Detecting portrait data in the image data; Using the portrait data as a sample, the image search network is trained into a portrait style search network, wherein the portrait style search network is used to extract the style features of the portrait data; Using the image data as a sample, the image search network is trained into a global painting style search network, wherein the global painting style search network is used to extract the characteristics of the image data in terms of painting style; The portrait style search network and the global style search network are jointly output as a game style search network; The step of training the image search network into a portrait style search network using the portrait data as a sample includes: Inputting the portrait data into the image search network to extract the characteristics of the painting style, thereby obtaining the portrait style characteristics; Calculating a first portrait loss value when the portrait style feature is classified according to the style on multiple channels; Calculating a second portrait loss value when the portrait style feature is classified according to the style on a single channel; updating the parameters of the image search network in combination with the first portrait loss value and the second portrait loss value; Determine whether a preset first training condition is met; if so, output the parameters of the image search network to the first configuration file; if not, return to execute the step of inputting the portrait data into the image search network to extract the style features and obtain the portrait style features; The step of training the image search network into a global painting style search network using the image data as a sample includes: Inputting the image data into the image search network to extract the features of the painting style and obtain the global painting style features; Clustering the portrait data to obtain categories of the image data; Using the category as a label, calculating a first global loss value when the global style feature is classified according to the style on multiple channels; Using the category as a label, calculating a second global loss value when classifying a single channel of the global style feature according to the style; updating the parameters of the image search network in combination with the first global loss value and the second global loss value; Determine whether the preset second training condition is met; if so, output the parameters of the image search network to the second configuration file, if not, return to execute the step of inputting the image data into the image search network to extract the style features and obtain the global style features.

2. The method according to claim 1, characterized in that The step of obtaining, for each of the plurality of games, image data representing the painting styles in the games includes: Search for multiple games in a specified game library; If the game is found, image data published in the game library and used to introduce the game is searched as a representative of the painting style in the game.

3. The method according to claim 1, characterized in that The detecting of portrait data in the image data comprises: identifying facial data in the image data; If the face data is recognized, the face data is expanded in the image data to obtain portrait data; If the face data is not recognized, it is determined that no portrait data is detected in the image data.

4. The method according to claim 3, characterized in that The step of expanding the face data in the image data to obtain the portrait data includes: Query the width and height of the face data; Multiplying the width by a first coefficient to obtain a first step length; multiplying the height by a second coefficient to obtain a second step length; multiplying the height by a third coefficient to obtain a third step length, wherein the third coefficient is greater than the second coefficient; In the image data, the face data is expanded by the first step length in the horizontal direction, the second step length in the vertical upward direction, and the third step length in the vertical downward direction to obtain portrait data.

5. The method according to claim 1, characterized in that The image search network includes a first residual network, a second residual network, a third residual network and a fourth residual network; The step of inputting the portrait data into the image search network to extract the style features and obtain the portrait style features comprises: Inputting the portrait data into the first residual network to extract features and perform downsampling to obtain first portrait candidate features; Inputting the first portrait candidate feature into the second residual network to extract and downsample to obtain a second portrait candidate feature; Inputting the second portrait candidate feature into the third residual network to extract features, thereby obtaining a third portrait candidate feature; The third portrait candidate feature is input into the fourth residual network to extract features and obtain portrait style features.

6. The method according to claim 1, characterized in that The calculating of the first portrait loss value when the portrait style feature is classified according to the style on multiple channels includes: Performing a global description process with pooling on the sub-blocks of the portrait style feature on multiple channels respectively to obtain a first portrait channel feature; Performing a full connection operation on each of the first portrait channel features to reduce the dimension into a second portrait channel feature; Performing a normalization process of the second norm on each of the second portrait channel features respectively to obtain a third portrait channel feature; Performing a series operation on all the third portrait channel features to obtain a first portrait global feature; Performing a normalization process of a second norm on the first portrait global feature to obtain a second portrait global feature; The second portrait global feature is used to calculate the loss of the additive angle interval for classification to obtain the first portrait loss value.

7. The method according to claim 6, characterized in that The calculating of the second portrait loss value when the portrait style feature is classified according to the style on a single channel includes: Performing a batch normalization operation on the first portrait channel feature to obtain a fourth portrait channel feature; Performing a fully connected operation on the fourth portrait channel feature to map it into a fifth portrait channel feature; activating the fifth portrait channel feature to obtain a sixth portrait channel feature; The sixth portrait channel feature is used to calculate the loss of the additive angle interval for classification, and a second portrait loss value is obtained.

8. The method according to claim 1, characterized in that The image search network includes a first residual network, a second residual network, a third residual network and a fourth residual network; The step of inputting the image data into the image search network to extract the characteristics of the painting style and obtain the global painting style characteristics comprises: Inputting the image data into the first residual network to extract features and perform downsampling to obtain first global candidate features; Inputting the first global candidate feature into the second residual network to extract and downsample to obtain a second global candidate feature; Inputting the second global candidate feature into the third residual network to extract features, thereby obtaining a third global candidate feature; The third global candidate feature is input into the fourth residual network to extract features, thereby obtaining a global painting style feature.

9. The method according to claim 1, characterized in that: The step of using the category as a label and calculating a first global loss value when the global style feature is classified according to the style on multiple channels includes: Performing a global description process with pooling on the sub-blocks of the global painting style feature on multiple channels respectively to obtain a first global channel feature; Performing a full connection operation on each of the first global channel features to reduce the dimension into a second global channel feature; Performing a normalization process of the second norm on each of the second global channel features respectively to obtain a third global channel feature; Perform a series operation on all the third global channel features to obtain a first global feature; Performing a normalization process of a second norm on the first global feature to obtain a second global feature; The category is used as a label and the second global feature is used to calculate the cross entropy for classification to obtain a first global loss value.

10. The method according to claim 9, characterized in that The step of using the category as a label to calculate a second global loss value when classifying a single channel of the global style feature according to the style includes: Performing a batch normalization operation on the first global channel feature to obtain a fourth global channel feature; Performing a fully connected operation on the fourth global channel feature to map it into a fifth global channel feature; activating the fifth global channel feature to obtain a sixth global channel feature; The category is used as a label and the sixth global channel feature is used to calculate a cross entropy for classification to obtain a second global loss value.

11. A game search method, characterized in that: include: Loading a game style search network trained by the method according to any one of claims 1 to 10, wherein the game style search network includes a portrait style search network and a global style search network; Receiving raw image data from an original game; Detecting original portrait data in the original image data; If it is detected that there is original portrait data, the original portrait data is input into the portrait style search network to extract the original portrait style features representing the style; Searching for a target game similar in style to the original game according to the original portrait style feature; If no original portrait data is detected, the original image data is input into the global painting style search network to extract the original global painting style features representing the painting style; A target game having a similar painting style to the original game is searched for according to the original global painting style feature.

12. The method according to claim 11, characterized in that The loading of the game style search network includes: Load the image search network; Loading a first configuration file in the image search network to obtain a portrait style search network, wherein the first configuration file records parameters obtained by training the image search network with the goal of extracting style features of portrait data; A second configuration file is loaded into the image search network to obtain a global style search network, wherein the second configuration file records parameters obtained by training the image search network with the goal of extracting style features of the entire frame of image data.

13. The method according to claim 11 or 12, characterized in that: The step of searching for a target game similar in style to the original game according to the original portrait style feature includes: Recalling preset candidate games, the candidate games having candidate portrait style features, the candidate portrait style features being features representing style extracted from candidate portrait data in the candidate games using the portrait style search network; Calculating a first similarity between the original portrait style feature and the candidate portrait style feature; A plurality of the candidate games are screened out according to the first similarity as target games having a similar style to the original game.

14. The method according to claim 11 or 12, characterized in that: The step of searching for a target game similar in style to the original game according to the original global style feature includes: Recalling preset candidate games, the candidate games having candidate global style features, the candidate global style features being features representing the style extracted from the candidate image data of a whole frame in the candidate games using the global style search network; Calculating a second similarity between the original global painting style feature and the candidate global painting style feature; A plurality of the candidate games are screened out according to the second similarity as target games having a similar style to the original game.

15. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the training method of the game style search network described in any one of claims 1-10 or the game search method described in any one of claims 11-14.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is used to enable a processor to implement the training method of a game style search network described in any one of claims 1-10 or the game search method described in any one of claims 11-12 when executed.