Simulation data set generation method and device, equipment and medium

Abstract

The embodiment of the invention discloses a simulation data set generation method and device, equipment and a medium. The method comprises the following steps: determining node attribute information according to a scene graph of a real data set, and determining a first number of candidate node attribute lists according to the node attribute information; rendering according to node attributes in the candidate node attribute list to obtain a candidate simulation data set corresponding to the candidate node attribute list; image feature extraction is carried out on the candidate simulation data sets and the real data sets, and distribution difference parameters between the candidate simulation data sets and the real data sets are sequentially determined according to the extracted image features; and determining a target node attribute list according to the distribution difference parameters and the candidate node attribute list, and determining a target simulation data set according to the target node attribute list. According to the scheme, the simulation data set similar to the real data set in distribution is automatically generated, the distribution difference between the real data set and the simulation data set is reduced, and the simulation data set is better used for model training.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, device, equipment and medium for generating a simulation data set. Background Art

[0002] As the complexity of business scenarios continues to increase, image detection and recognition model training requires a richer set of training images. Existing training images are usually obtained by camera equipment. Due to the constraints of the on-site installation environment, the number and richness of the obtained training images are difficult to meet the requirements. Subsequent image annotation often uses manual annotation, which is inefficient, slow, and difficult to ensure the quality of annotation. In this case, simulation data sets came into being and were gradually used for model training.

[0003] In a related scheme, when generating a simulation data set, a simulation scene is constructed based on the target scene to generate a rendered image, the annotation information of the rendered image is determined to obtain an initial annotated image, and then the image style transfer technology is used to transfer the image style of the initial annotated image based on the real image of the target scene to obtain multiple target annotated images with real image style.

[0004] However, the above scheme requires the artificial construction of a large number of simulated scenes for real target scenes. The construction process requires meticulous modeling and parameter adjustment, which consumes a lot of manpower. In addition, style transfer can only reduce the style difference between real images and simulated images, but cannot solve the distribution difference between real images and simulated images. Therefore, the generated simulated images cannot accurately reflect the real scenes. Summary of the invention

[0005] The present invention provides a method, device, equipment and medium for generating a simulation data set, which determines the attribute value that can render the smallest difference from the real data distribution based on the attribute optimization method, automatically generates a simulation data set with a distribution similar to the real data set, effectively reduces the distribution difference between the real data set and the simulation data set, and enables the simulation data set to be better used for model training.

[0006] According to one aspect of the present invention, a method for generating a simulation data set is provided, the method comprising:

[0007] Determine node attribute information according to a scene graph of a real data set, and determine a first number of candidate node attribute lists according to the node attribute information;

[0008] Rendering is performed according to the node attributes in the candidate node attribute list to obtain a candidate simulation data set corresponding to the candidate node attribute list;

[0009] Extracting image features from the candidate simulated data sets and the real data sets, and determining distribution difference parameters between each candidate simulated data set and the real data set in turn according to the extracted image features;

[0010] A target node attribute list is determined according to the distribution difference parameter and the candidate node attribute list, and a target simulation data set is determined according to the target node attribute list.

[0011] According to another aspect of the present invention, there is provided a device for generating a simulation data set, comprising:

[0012] A candidate node attribute list determination module, used to determine node attribute information according to a scene graph of a real data set, and determine a first number of candidate node attribute lists according to the node attribute information;

[0013] A candidate simulation data set determination module, used for rendering according to the node attributes in the candidate node attribute list to obtain a candidate simulation data set corresponding to the candidate node attribute list;

[0014] a distribution difference parameter determination module, used to extract image features from the candidate simulation data set and the real data set, and determine the distribution difference parameters between each candidate simulation data set and the real data set in turn according to the extracted image features;

[0015] The target simulation data set determination module is used to determine a target node attribute list according to the distribution difference parameter and the candidate node attribute list, and determine a target simulation data set according to the target node attribute list.

[0016] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0017] at least one processor; and

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for generating a simulation data set described in any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for generating a simulation data set described in any embodiment of the present invention when executed.

[0021] The technical solution of the embodiment of the present invention determines node attribute information according to the scene graph of the real data set, and determines a first number of candidate node attribute lists according to the node attribute information; renders according to the node attributes in the candidate node attribute list to obtain a candidate simulation data set corresponding to the candidate node attribute list; extracts image features from the candidate simulation data set and the real data set, and determines the distribution difference parameters between each candidate simulation data set and the real data set in turn according to the extracted image features; determines a target node attribute list according to the distribution difference parameters and the candidate node attribute list, and determines a target simulation data set according to the target node attribute list. This technical solution determines the attribute value that can render the smallest distribution difference with the real data based on the attribute optimization method, automatically generates a simulation data set with a distribution similar to the real data set, effectively reduces the distribution difference between the real data set and the simulation data set, and enables the simulation data set to be better used for model training.

[0022] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 is a flow chart of a method for generating a simulation data set provided according to Embodiment 1 of the present invention;

[0025] Figure 2 is a schematic diagram of a probabilistic scene grammar scene graph provided according to Embodiment 1 of the present invention;

[0026] Figure 3 is a schematic diagram of a method for generating a simulation data set provided according to Embodiment 1 of the present invention;

[0027] Figure 4 is a flow chart of a method for generating a simulation data set provided in accordance with Embodiment 2 of the present invention;

[0028] Figure 5 is a schematic diagram of the structure of a device for generating a simulation data set provided according to Embodiment 3 of the present invention;

[0029] Figure 6 It is a structural schematic diagram of an electronic device for implementing a method for generating a simulation data set according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", "target", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] Embodiment 1

[0033] Figure 1 This is a flow chart of a method for generating a simulation data set provided in the first embodiment of the present invention. This embodiment is applicable to the case of automatically generating a simulation data set with a distribution similar to that of a real data set. The method can be executed by a device for generating a simulation data set. The device for generating a simulation data set can be implemented in the form of hardware and / or software. The device for generating a simulation data set can be configured in an electronic device with data processing capabilities. Figure 1 As shown, the method includes:

[0034] S110, determining node attribute information according to a scene graph of a real data set, and determining a first number of candidate node attribute lists according to the node attribute information.

[0035] Among them, the scene graph is a common method for representing 3D scenes. The elements of the scene can be represented by a simple hierarchical structure, and each node has a set of attributes (for example, type, position, angle, etc.). Among them, the hierarchy defines the parent-child dependency of the scene. In order to generate diverse and effective 3D scenes, the typical method is to specify the generation process of the graph through probabilistic scene grammar. Among them, probabilistic scene grammar is a method for generating 3D scenes, which defines the structure and properties of the scene and generates the scene through parameter distribution sampling. It is usually used in the field of games and graphics, and can generate diverse and effective virtual environments. It should be noted that the structure of the scene is defined by the grammar, and the attributes are obtained from a fixed parameter distribution sampling. It is usually necessary to manually formulate a reasonable parameter distribution according to the actual application scenario.

[0036] In this embodiment, a scene graph of a real data set is first generated by a probabilistic scene grammar method. Figure 2 This is a schematic diagram of a probabilistic scenario grammar scenario graph provided by the first embodiment of the present invention, wherein the scenario is a high-speed traffic scenario. Figure 2 As shown in the figure, first add a camera node, then add a main road node based on the camera node, then add lane nodes on the main road node, and then add car nodes on each lane node. The attributes of the camera node include position and angle, and the attributes of the car node include position, horizontal angle, and car model.

[0037] In this embodiment, it can be assumed that a probabilistic scene grammar has been obtained through prior knowledge, from which the initial scene graph can be sampled. The structure of each scene graph obtained by sampling is correct. The purpose of this solution is to modify the node attributes of each scene graph so that the converted scene is similar to the distribution of the real scene when rendered. Therefore, in essence, this process retains the graph structure generated by the probabilistic scene grammar, but transforms the distribution of attributes.

[0038] After generating the scene graph of the real data set, the node attribute information can be determined according to the scene graph of the real data set. The node attribute information includes the attributes of each node in the scene graph and the search space corresponding to each attribute. Among them, the search space corresponding to the attribute can be understood as the actual value range of the attribute. For example, the horizontal angle value range of a car is [0°-359°]. Then, a first number of candidate node attribute lists are determined according to the node attribute information. Specifically, for the attributes of all nodes in all scene graphs, the attribute value is determined by randomly selecting values ​​from the search space corresponding to each attribute, thereby obtaining a candidate node attribute list. Repeating the above random selection process can obtain a first number of candidate node attribute lists. Among them, the first number refers to the number of pre-set candidate node attribute lists, which can be set according to actual needs.

[0039] For example, assume that Q scene graphs are generated, each scene graph has an average of M nodes, each node has an average of L attributes, and the total number of attributes is N = Q*M*L. Assume that the i-th attribute is a i , the search space of the i-th attribute is S i , then the attributes of all nodes in all scene graphs can be recorded as A = {a 1 ,a 2 ,…,a N}, and the search space of all attributes is recorded as S = {S 1 ,S 2 ,…,S N For A = {a 1 ,a 2 ,…,a N}, respectively from S = {S 1 ,S 2 ,…,S N}, randomly select the corresponding attribute value to obtain a candidate node attribute list. Assuming the first number is 10, repeat the above process 10 times to obtain 10 candidate node attribute lists.

[0040] It should be noted that in order to reduce the number of parameters that need to be adjusted in the simulation scenario, thereby reducing the time and complexity of subsequent attribute optimization, some attributes can be selected for adjustment according to the scenario task. Figure 2 For example, assuming that the scenario task is vehicle detection, the goal of this solution is to obtain a set of simulated data sets with content distribution similar to the real vehicle detection data set. At this time, only the following attributes can be adjusted: 1. The position and angle of the camera; 2. The position, horizontal angle and model of the car. For other attributes, such as weather, camera noise, lighting, number of lanes, number of cars, etc., the initial distribution values ​​of the probabilistic scenario grammar can be directly used.

[0041] S120, rendering according to the node attributes in the candidate node attribute list to obtain a candidate simulation data set corresponding to the candidate node attribute list.

[0042] In this embodiment, after determining the first number of candidate node attribute lists, the node attributes in the candidate node attribute lists can be rendered to obtain candidate simulation data sets corresponding to the candidate node attribute lists. The number of candidate simulation data sets is consistent with the number of candidate node attribute lists, that is, there are as many candidate node attribute lists as there are candidate simulation data sets. For example, assuming there are K candidate node attribute lists, it can be expressed as i=1,2,…K. For K candidate node attribute lists, rendering is performed according to the attribute values ​​in each candidate node attribute list, and K candidate simulation data sets can be obtained accordingly. Since each attribute changes independently, the K candidate simulation data sets can be specifically expressed as i=1,2,…K, where R represents the renderer.

[0043] S130, extracting image features from the candidate simulation data sets and the real data sets, and determining distribution difference parameters between each candidate simulation data set and the real data set in turn according to the extracted image features.

[0044] In this embodiment, after obtaining the candidate simulation data set corresponding to the candidate node attribute list, image feature extraction can be performed on the candidate simulation data set and the real data set to reflect the distribution difference between them by extracting the feature similarity between the candidate simulation data set and the real data set. Among them, the image feature can be an image structure feature. Exemplarily, a feature extraction algorithm (such as SIFT, HOG) can be used to extract image structure features, wherein SIFT represents scale-invariant feature transformation and HOG represents directional gradient histogram. A neural network model can also be used to extract image structure features. For example, a feature extractor can be trained through self-supervised learning including a GAN network, and image structure features can be extracted through the feature extractor. In this embodiment, the image feature extraction method is not specifically limited and can be flexibly set according to actual needs.

[0045] In this embodiment, optionally, image feature extraction is performed on the candidate simulation data set and the real data set, including: image feature extraction is performed on the candidate simulation data set and the real data set based on a pre-trained feature extraction model; wherein the training process of the feature extraction model includes: obtaining a real image in the real data set, and dividing the real image into regions to obtain HOG features of multiple regions; selecting a preset number of target regions from multiple regions according to the HOG features, and occluding the target regions to obtain an occluded image; restoring the occluded image based on the image generation model to obtain a generated image; distinguishing the generated image from the real image based on the image discrimination model to obtain a distinction result; training the image generation model and the image discrimination model according to the real image in the real data set until the model converges; and determining the feature extraction model according to the feature extraction module in the image discrimination model.

[0046] In this embodiment, a feature extraction model (such as a GAN network model) can be pre-trained, and image structure feature extraction can be performed on the candidate simulation data set and the real data set according to the feature extraction model. When training the feature extraction model, first obtain the real image in the real data set, and divide the real image into regions to obtain the HOG features of multiple regions. This embodiment does not specifically limit the number and method of regional division, which can be set according to actual needs. For example, the real image can be evenly divided into 16×16 cell areas. When calculating the HOG features of each region, the gradient size of each pixel in the region can be calculated first, and then the HOG features of the region can be determined based on the gradient size of each pixel in the region.

[0047] After obtaining the HOG features of multiple regions, a preset number of target regions can be selected from the multiple regions according to the HOG features, and the target regions can be occluded to obtain an occluded image. The preset number may refer to a preset number of target regions, which can be set according to actual needs. For example, a preset number of regions with the highest pixel gradient intensity can be selected from multiple regions according to the HOG features as target regions. In this way, the region with the richest structural features can be found, so that the subsequent network can focus on learning the structural features of the entire image.

[0048] After obtaining the occluded image, the occluded image can be restored based on the image generation model to obtain the generated image. Among them, the image generation model can be a generator, whose task is to receive an occluded image and generate an output as close to the real image as possible according to the unoccluded part in the occluded image (i.e., obtain the generated image), thereby realizing the restoration of the occluded image. Exemplarily, the network structure of the image generation model may include an input layer, an encoder, a decoder, and an output layer. Among them, the encoder includes a plurality of convolutional layers, and the decoder includes a plurality of transposed convolutional layers. Specifically, the input layer can be used to receive the input image, the encoder can be used to extract features and downsample, the decoder can be used to generate features and upsample, and the output layer can be used to generate RGB images. It should be noted that in the network structure of the image generation model, the size, convolution size, step size, and activation function of each layer can be set according to actual needs, and this is not specifically limited in this embodiment. For example, the activation function can be LeakyReLU or ReLU.

[0049] After obtaining the generated image, the generated image and the real image can be distinguished based on the image discrimination model to obtain a distinction result. Among them, the distinction result can be represented by a score. The image discrimination model can be a discriminator, whose task is to receive an image (generated image or real image) and judge as accurately as possible whether the image is a real image or a generated image determined by the image generation model, thereby distinguishing between the generated image and the real image. Exemplarily, the network structure of the image discrimination model may include an input layer, multiple convolutional layers and an output layer. Among them, the input layer can be used to receive the generated image and the real image, the convolutional layer can be used to extract features and downsample, and the output layer can be used to output the authenticity score of each area. It should be noted that in the network structure of the image discrimination model, the size, convolution size, step size and activation function of each layer can be set according to actual needs, and this is not specifically limited in this embodiment. For example, the activation function can be LeakyReLU or Sigmoid.

[0050] According to the real images in the real data set, the image generation model and the image discrimination model are alternately trained until the model converges, indicating that the model training is completed. Among them, training the image generation model can enable it to restore the occluded image, and training the image discrimination model can enable it to distinguish between the real image and the generated image. After the model training is completed, the feature extraction model can be determined according to the feature extraction module in the image discrimination model. Exemplarily, the feature extraction module is composed of an input layer and a convolution layer, that is, the part of the image discrimination model other than the output layer is the feature extraction module.

[0051] The self-supervised learning task designed in this scheme can automatically generate input images, and the label is the original real image. There is no need to calibrate new data, which saves labor costs. Compared with directly using the existing pre-trained model, the feature extraction model trained by this task can block the part of the area with the strongest HOG feature intensity, so that the feature extraction model is forced to rely on other visible parts to infer and restore the area with the richest HOG features. The feature extraction model is encouraged to make more use of image context information, including global structure and features of adjacent areas, to better understand the overall structure of the image. Therefore, the feature extraction model is forced to pay more attention to the structural features of the image in the self-supervised learning task, thereby improving the model's ability to extract image structural features, so as to extract features that can better highlight the image structure.

[0052] After image structural features are extracted from the candidate simulation data sets and the real data sets, distribution difference parameters between each candidate simulation data set and the real data set can be determined in turn according to the extracted image structural features. The distribution difference parameters can be used to characterize the degree of distribution difference between the candidate simulation data set and the real data set. For example, the distribution difference parameters can be Inception Score (IS), Fréchet Inception Distance (FID), Perceptual Path Length (PPL), etc.

[0053] For example, taking FID as an example, the output result of the last convolution layer in the feature extraction module can be used as the image structure feature. Assuming that the size of the feature map of the last convolution layer is 16x16x512, in order to calculate FID, it is necessary to flatten this feature map into a vector and choose to average it on the spatial dimension 16x16 to obtain a 512-dimensional vector. Therefore, each image input into the image discrimination model can obtain a 512-dimensional feature vector. And the real dataset D r To extract image structure features, take the simulation data set as an example, the feature vector extracted from a single scene graph is recorded as Q scene graphs can be used to extract Q 512-dimensional feature vectors, and the average vector is recorded as Considering Q 512-dimensional vectors as a matrix F, the covariance matrix can be obtained as Similarly, we can get the feature distribution mean vector μ of the real data set r and the covariance matrix ∑ r . Thus, the expression of the distribution difference parameter can be obtained as Where tr is the trace of the matrix. The above formula can be used to calculate the FID between each candidate simulation data set and the real data set. The smaller the FID, the smaller the distribution difference between the candidate simulation data set and the real data set, that is, the more similar the candidate simulation data set and the real data set are.

[0054] S140, determining a target node attribute list according to the distribution difference parameter and the candidate node attribute list, and determining a target simulation data set according to the target node attribute list.

[0055] In this embodiment, after determining the distribution difference parameters between each candidate simulation data set and the real data set, the target node attribute list can be determined according to the distribution difference parameters and the candidate node attribute list. Optionally, determining the target node attribute list according to the distribution difference parameters and the candidate node attribute list includes: selecting a second number of first intermediate node attribute lists from the candidate node attribute list according to the distribution difference parameters; randomly selecting two groups of first exchange node attribute lists from the first intermediate node attribute list in turn, and exchanging the node attributes in the first exchange node attribute list to generate a new first exchange node attribute list until a first number of new first exchange node attribute lists are generated as the updated first number of candidate node attribute lists; if the number of updates of the candidate node attribute list reaches a first preset number of iterations, determining the first number of candidate node attribute lists after the current update as the reference node attribute list; determining the distribution difference parameters between the reference simulation data set and the real data set corresponding to the reference node attribute list, and selecting the reference node attribute list with the smallest distribution difference parameter as the target node attribute list.

[0056] In this embodiment, when determining the target node attribute list, firstly, a second number of first intermediate node attribute lists are selected from the candidate node attribute list according to the distribution difference parameter. The second number may refer to a preset number of first intermediate node attribute lists, which may be set according to actual needs. For example, assuming that the number of candidate node attribute lists (i.e., the first number) is K, the second number L may be set to Optionally, selecting a second number of first intermediate node attribute lists from the candidate node attribute list according to the distribution difference parameter includes: determining a normalized difference score of each candidate node attribute list according to the distribution difference parameter; determining a candidate selection probability of each candidate node attribute list according to the normalized difference score; and selecting a second number of first intermediate node attribute lists from the candidate node attribute list according to the candidate selection probability; wherein the expression of the candidate selection probability is: Among them, P i represents the candidate selection probability of the i-th candidate node attribute list, Fit i Represents the normalized difference score of the attribute list of the i-th candidate node.

[0057] For example, assume that the distribution difference parameter is FID and the number of candidate node attribute lists is K. First, according to the formula Determine the difference score of each candidate node attribute list and use the formula Normalize the difference score to get the normalized difference score. Then use the formula Determine the candidate selection probability of each candidate node attribute list, and then select from the candidate node attribute list according to the candidate selection probability The node attribute list is used as the first intermediate node attribute list.

[0058] Through such a setting, this scheme can filter out a part of the candidate node attribute list that has a large distribution difference with the real data set based on the candidate selection probability.

[0059] Exemplarily, select from a list of K candidate node attributes After the first intermediate node attribute list, In the first intermediate node attribute lists, two sets of node attribute lists are randomly selected as the first exchange node attribute lists, and M values ​​in the two sets of first exchange node attribute lists are randomly exchanged, for example, and Random M values ​​in and Until K new first switching node attribute lists are generated as updated K candidate node attribute lists. The value of M can be set according to actual needs, and this embodiment does not make any specific limitation on this.

[0060] According to the above process, the candidate node attribute list is iteratively updated. During the iterative update process, it is necessary to determine whether the number of updates reaches the first preset number of iterations, wherein the first preset number of iterations may refer to a preset reference value of the number of updates of the candidate node attribute list, which may be set according to actual needs. If the number of updates of the candidate node attribute list reaches the first preset number of iterations, the iterative update process ends, and at this time, the first number of candidate node attribute lists after the current update (i.e., the first number of candidate node attribute lists obtained by the last update) may be determined as the reference node attribute list. Taking the distribution difference parameter as FID as an example, after determining the reference node attribute list, the FID between the reference simulation data set and the real data set corresponding to the reference node attribute list is determined, and the reference node attribute list with the smallest FID is selected as the target node attribute list, and the target simulation data set is determined according to the target node attribute list. At this point, the content distribution between the simulation data set rendered using the target node attribute list and the real data set is most similar.

[0061] Figure 3 Schematic diagram of a method for generating a simulation data set provided in Embodiment 1 of the present invention. Figure 3 As shown in Figure 1, we can first generate scene graphs of several real data sets based on the probabilistic scene grammar, and then parameterize the attributes of each node in the scene graph to obtain a node attribute list. Then, we render the simulated data set based on the attribute values ​​in the node attribute list, and perform image feature extraction on the trained feature extraction model (i.e., Figure 3 Extraction features in) and distribution difference parameter comparison (i.e. Figure 3Then, the attribute optimization method is used to find the optimal attribute list to minimize the distribution difference parameter value. At this time, the content distribution of the rendered simulation data set is most similar to that of the real data set.

[0062] The technical solution of the embodiment of the present invention determines node attribute information according to the scene graph of the real data set, and determines a first number of candidate node attribute lists according to the node attribute information; renders according to the node attributes in the candidate node attribute list to obtain a candidate simulation data set corresponding to the candidate node attribute list; extracts image features from the candidate simulation data set and the real data set, and determines the distribution difference parameters between each candidate simulation data set and the real data set in turn according to the extracted image features; determines a target node attribute list according to the distribution difference parameters and the candidate node attribute list, and determines a target simulation data set according to the target node attribute list. This technical solution determines the attribute value that can render the smallest distribution difference with the real data based on the attribute optimization method, automatically generates a simulation data set with a distribution similar to the real data set, effectively reduces the distribution difference between the real data set and the simulation data set, and enables the simulation data set to be better used for model training.

[0063] In this embodiment, optionally, after exchanging the node attributes in the first exchange node attribute list, the method further includes: determining whether there is a mutation node attribute in the first exchange node attribute list according to a preset attribute mutation probability; if so, determining the mutation value of the mutation node attribute according to the value range of the mutation node attribute.

[0064] Among them, the preset attribute mutation probability may refer to a preset attribute mutation probability, which may be set according to actual needs. For example, the preset attribute mutation probability may be set to 0.001. In this embodiment, after the node attributes in the first exchange node attribute list are exchanged, it may also be determined whether there is a mutation node attribute in the first exchange node attribute list according to the preset attribute mutation probability. If so, it indicates that an attribute value has mutated, and at this time, an attribute value other than the current value may be randomly selected within the value range of the attribute value as the mutation value of the mutation node attribute.

[0065] Through such a setting, this scheme can better simulate the randomness of node attributes in the node attribute list, which helps to better determine the target node attribute list.

[0066] Embodiment 2

[0067] Figure 4 This is a flow chart of a method for generating a simulation data set provided in the second embodiment of the present invention. This embodiment is optimized based on the above embodiment.

[0068] like Figure 4As shown, the method of this embodiment specifically includes the following steps:

[0069] S210, determining node attribute information according to a scene graph of a real data set, and determining a first number of candidate node attribute lists according to the node attribute information.

[0070] S220, rendering according to the node attributes in the candidate node attribute list to obtain a candidate simulation data set corresponding to the candidate node attribute list.

[0071] S230 , extracting image features from the candidate simulation data sets and the real data sets, and determining distribution difference parameters between each candidate simulation data set and the real data set in sequence according to the extracted image features.

[0072] S240: Select a second number of first intermediate node attribute lists from the candidate node attribute lists according to the distribution difference parameter.

[0073] S250, randomly selecting two groups of first switching node attribute lists from the first intermediate node attribute list in turn, and exchanging the node attributes in the first switching node attribute list to generate new first switching node attribute lists, until a first number of new first switching node attribute lists are generated as the updated first number of candidate node attribute lists.

[0074] S260: If the number of updates of the candidate node attribute list reaches a first preset number of iterations, determine a first number of candidate node attribute lists after current updates as reference node attribute lists.

[0075] Among them, the specific implementation method of S210-S260 can refer to the relevant description in Example 1, and will not be repeated here.

[0076] S270, determining a task performance indicator according to the execution task of the target simulation data set.

[0077] The execution task may refer to a deep learning model trained with a target simulation data set, which may be determined based on the actual application scenario. For example, the execution task may be vehicle detection. Exemplarily, the task performance indicators may include accuracy, precision, recall, F1 value, mean square error, etc., which may be selected based on the actual application scenario.

[0078] It should be noted that due to the similar content distribution between the simulated dataset and the real dataset, some problems such as category imbalance, data offset, noise, and data non-smoothing in the real dataset itself will also affect the quality of the simulated dataset. To address this problem, the attribute list can be fine-tuned based on the task performance indicators after obtaining the reference node attribute list.

[0079] S280, performing a task according to a reference simulation data set corresponding to the reference node attribute list, and obtaining a reference task performance indicator value corresponding to the reference node attribute list.

[0080] For example, n node attribute lists with the highest normalized difference scores Fit can be selected from K reference node attribute lists. For n reference node attribute lists, n reference simulation data sets can be rendered according to their attribute values. The obtained reference simulation data sets are used to train the task model, and the m task performance indicators of each task model are counted to obtain the reference task performance indicator values ​​corresponding to the reference node attribute lists, which can be specifically expressed as fit i =∑k i *t i Among them, t i represents the specific task performance index, which can be selected according to the content of the actual task. i Indicates the weight of the task performance indicator.

[0081] S290, determining a target node attribute list according to the reference task performance indicator value and the reference node attribute list, and determining a target simulation data set according to the target node attribute list.

[0082] In this embodiment, after obtaining the reference task performance index value corresponding to the reference node attribute list, the target node attribute list can be determined according to the reference task performance index value and the reference node attribute list. Optionally, determining the target node attribute list according to the reference task performance index value and the reference node attribute list includes: selecting a third number of second intermediate node attribute lists from the reference node attribute list according to the reference task performance index value; randomly selecting two groups of second exchange node attribute lists from the second intermediate node attribute list in turn, and exchanging the node attributes in the second exchange node attribute list to generate a new second exchange node attribute list, until a third number of new second exchange node attribute lists are generated as the updated third number of reference node attribute lists; if the number of updates of the reference node attribute list reaches the second preset number of iterations, then determining the third number of reference node attribute lists after the current update as the adjustment node attribute list; determining the task performance index value of the adjustment simulation data set corresponding to the adjustment node attribute list, and selecting the adjustment node attribute list with the highest task performance index value as the target node attribute list.

[0083] The third number may refer to the number of preset second intermediate node attribute lists, which may be set according to actual needs. The second preset number of iterations may refer to the reference value of the number of updates of the reference node attribute list, which may be set according to actual needs.

[0084] For example, after determining the reference task performance index value, the formula To determine the probability of each reference node attribute list being selected. Where i = 1, 2, ..., n. Based on the probability of each reference node attribute list being selected, select from n reference node attribute lists. From the selected l second intermediate node attribute lists, two groups of node attribute lists are randomly selected in turn as the second exchange node attribute lists, and the M values ​​in the two groups of second exchange node attribute lists are randomly exchanged, for example, and Random M values ​​in and Until n new second switching node attribute lists are generated as the updated n reference node attribute lists.

[0085] The reference node attribute list is iteratively updated according to the above process. During the iterative update process, it is necessary to determine whether the number of updates reaches the second preset number of iterations. If the number of updates of the reference node attribute list reaches the first preset number of iterations, the iterative update process ends. At this time, the third number of reference node attribute lists after the current update (i.e., the third number of reference node attribute lists obtained by the last update) can be determined as the adjustment node attribute list. Then determine the task performance index value of the adjustment simulation data set corresponding to the adjustment node attribute list, and select the adjustment node attribute list with the highest task performance index value as the target node attribute list, and then determine the target simulation data set according to the target node attribute list.

[0086] It should be noted that in addition to automatically adjusting the attribute values ​​through performance indicators, if the distribution defects of the real data set itself have been analyzed, the distribution of the simulated data set can also be adjusted by manually adjusting the attribute values ​​to compensate for the defects of the real data set.

[0087] The technical solution of the embodiment of the present invention can determine the attribute value that can render the smallest difference from the real data distribution based on the attribute optimization method, and automatically generate a simulation data set with a distribution similar to the real data set, effectively reducing the distribution difference between the real data set and the simulation data set, so that the simulation data set can be better used for model training. At the same time, when the simulation data set basically meets the distribution of the real data set, the distribution of samples can be adjusted by internally exchanging attribute values ​​to obtain a simulation data set with better task performance.

[0088] Embodiment 3

[0089] Figure 5This is a schematic diagram of the structure of a device for generating a simulation data set provided in the third embodiment of the present invention. The device can execute the method for generating a simulation data set provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method. Figure 5 As shown, the device comprises:

[0090] A candidate node attribute list determination module 310, configured to determine node attribute information according to a scene graph of a real data set, and determine a first number of candidate node attribute lists according to the node attribute information;

[0091] A candidate simulation data set determination module 320 is used to render according to the node attributes in the candidate node attribute list to obtain a candidate simulation data set corresponding to the candidate node attribute list;

[0092] A distribution difference parameter determination module 330 is used to extract image features from the candidate simulation data set and the real data set, and determine the distribution difference parameters between each candidate simulation data set and the real data set in turn according to the extracted image features;

[0093] The target simulation data set determination module 340 is used to determine a target node attribute list according to the distribution difference parameter and the candidate node attribute list, and determine a target simulation data set according to the target node attribute list.

[0094] Optionally, the target simulation data set determination module 340 includes:

[0095] A first intermediate node attribute list determining unit, configured to select a second number of first intermediate node attribute lists from the candidate node attribute list according to the distribution difference parameter;

[0096] a candidate node attribute list updating unit, configured to randomly select two groups of first switching node attribute lists from the first intermediate node attribute list in sequence, and exchange node attributes in the first switching node attribute list to generate new first switching node attribute lists, until a first number of new first switching node attribute lists are generated as the updated first number of candidate node attribute lists;

[0097] A reference node attribute list determining unit, configured to determine a first number of candidate node attribute lists after current update as reference node attribute lists if the number of updates of the candidate node attribute list reaches a first preset number of iterations;

[0098] The target node attribute list determination unit is used to determine the distribution difference parameter between the reference simulation data set and the real data set corresponding to the reference node attribute list, and select the reference node attribute list with the smallest distribution difference parameter as the target node attribute list.

[0099] Optionally, the first intermediate node attribute list determining unit is specifically configured to:

[0100] Determining the normalized difference scores of the attribute lists of each candidate node respectively according to the distribution difference parameters;

[0101] Determining a candidate selection probability of each candidate node attribute list according to the normalized difference score;

[0102] Selecting a second number of first intermediate node attribute lists from the candidate node attribute list according to the candidate selection probability;

[0103] Among them, the expression of the candidate selection probability is:

[0104]

[0105] Among them, P i represents the candidate selection probability of the i-th candidate node attribute list, Fit i Represents the normalized difference score of the attribute list of the i-th candidate node.

[0106] Optionally, the target simulation data set determination module 340 further includes:

[0107] An attribute judgment unit, configured to determine whether there is a mutated node attribute in the first exchange node attribute list according to a preset attribute mutation probability after exchanging the node attributes in the first exchange node attribute list;

[0108] A mutation value determination unit is used to determine the mutation value of the mutation node attribute according to the value range of the mutation node attribute, if it exists.

[0109] Optionally, the distribution difference parameter determination module 330 is used to:

[0110] Performing image feature extraction on the candidate simulation data set and the real data set based on a pre-trained feature extraction model;

[0111] The training process of the feature extraction model includes:

[0112] Acquire a real image in the real data set, and divide the real image into regions to obtain HOG features of multiple regions;

[0113] Selecting a preset number of target areas from multiple areas according to the HOG features, and blocking the target areas to obtain a blocked image;

[0114] Restoring the blocked image based on an image generation model to obtain a generated image;

[0115] Distinguishing the generated image from the real image based on an image discrimination model to obtain a differentiation result;

[0116] Performing model training on the image generation model and the image discrimination model according to real images in the real data set until the models converge;

[0117] The feature extraction model is determined according to the feature extraction module in the image discrimination model.

[0118] Optionally, the target simulation data set determination module 340 further includes:

[0119] A task performance indicator determination unit, configured to determine a task performance indicator according to an execution task of the target simulation data set after determining that the first number of candidate node attribute lists currently updated are reference node attribute lists;

[0120] A reference task performance indicator value determination unit, used to perform a task according to a reference simulation data set corresponding to the reference node attribute list, and obtain a reference task performance indicator value corresponding to the reference node attribute list;

[0121] The second target node attribute list determining unit is used to determine the target node attribute list according to the reference task performance indicator value and the reference node attribute list.

[0122] Optionally, the second target node attribute list determining unit is specifically configured to:

[0123] Selecting a third number of second intermediate node attribute lists from the reference node attribute list according to the reference task performance indicator value;

[0124] randomly selecting two groups of second switching node attribute lists from the second intermediate node attribute lists in sequence, and exchanging node attributes in the second switching node attribute lists to generate new second switching node attribute lists, until a third number of new second switching node attribute lists are generated as the updated third number of reference node attribute lists;

[0125] If the number of updates of the reference node attribute list reaches a second preset number of iterations, determining the third number of reference node attribute lists after the current update as adjustment node attribute lists;

[0126] Determine the task performance index value of the adjusted simulation data set corresponding to the adjusted node attribute list, and select the adjusted node attribute list with the highest task performance index value as the target node attribute list.

[0127] A device for generating a simulation data set provided in an embodiment of the present invention can execute a method for generating a simulation data set provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0128] Embodiment 4

[0129] Figure 6 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0130] like Figure 6 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0131] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0132] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a method for generating a simulation data set.

[0133] In some embodiments, the method for generating a simulation data set may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for generating a simulation data set described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the method for generating a simulation data set by any other appropriate means (e.g., by means of firmware).

[0134] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0135] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0136] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0137] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0138] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0139] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0140] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0141] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for generating a simulation data set, characterized in that: The method comprises: Determine node attribute information according to a scene graph of a real data set, and determine a first number of candidate node attribute lists according to the node attribute information; Rendering is performed according to the node attributes in the candidate node attribute list to obtain a candidate simulation data set corresponding to the candidate node attribute list; Extracting image features from the candidate simulated data sets and the real data sets, and determining distribution difference parameters between each candidate simulated data set and the real data set in turn according to the extracted image features; A target node attribute list is determined according to the distribution difference parameter and the candidate node attribute list, and a target simulation data set is determined according to the target node attribute list.

2. The method according to claim 1, characterized in that: Determining a target node attribute list according to the distribution difference parameter and the candidate node attribute list includes: Selecting a second number of first intermediate node attribute lists from the candidate node attribute list according to the distribution difference parameter; randomly selecting two groups of first switching node attribute lists from the first intermediate node attribute lists in sequence, and exchanging node attributes in the first switching node attribute lists to generate new first switching node attribute lists, until a first number of new first switching node attribute lists are generated as the updated first number of candidate node attribute lists; If the number of updates of the candidate node attribute list reaches a first preset number of iterations, determining the first number of candidate node attribute lists currently updated as reference node attribute lists; Determine the distribution difference parameter between the reference simulation data set and the real data set corresponding to the reference node attribute list, and select the reference node attribute list with the smallest distribution difference parameter as the target node attribute list.

3. The method according to claim 2, characterized in that Selecting a second number of first intermediate node attribute lists from the candidate node attribute list according to the distribution difference parameter includes: Determining the normalized difference scores of the attribute lists of each candidate node respectively according to the distribution difference parameters; Determining a candidate selection probability of each candidate node attribute list according to the normalized difference score; Selecting a second number of first intermediate node attribute lists from the candidate node attribute list according to the candidate selection probability; Among them, the expression of the candidate selection probability is: Among them, P i represents the candidate selection probability of the i-th candidate node attribute list, Fit i Represents the normalized difference score of the attribute list of the i-th candidate node.

4. The method according to claim 2, characterized in that: After exchanging the node attributes in the first exchange node attribute list, the method further includes: Determining whether there is a mutation node attribute in the first exchange node attribute list according to a preset attribute mutation probability; If so, the mutation value of the mutation node attribute is determined according to the value range of the mutation node attribute.

5. The method according to claim 1, characterized in that Performing image feature extraction on the candidate simulation data set and the real data set includes: Performing image feature extraction on the candidate simulation data set and the real data set based on a pre-trained feature extraction model; The training process of the feature extraction model includes: Acquire a real image in the real data set, and divide the real image into regions to obtain HOG features of multiple regions; Selecting a preset number of target areas from multiple areas according to the HOG features, and blocking the target areas to obtain a blocked image; Restoring the blocked image based on an image generation model to obtain a generated image; Distinguishing the generated image from the real image based on an image discrimination model to obtain a differentiation result; Performing model training on the image generation model and the image discrimination model according to real images in the real data set until the models converge; The feature extraction model is determined according to the feature extraction module in the image discrimination model.

6. The method according to any one of claims 2 to 5, characterized in that: After determining that the first number of candidate node attribute lists currently updated are reference node attribute lists, the method further includes: Determining a task performance indicator according to the execution task of the target simulation data set; Performing a task according to a reference simulation data set corresponding to the reference node attribute list to obtain a reference task performance indicator value corresponding to the reference node attribute list; A target node attribute list is determined according to the reference task performance indicator value and the reference node attribute list.

7. The method according to claim 6, characterized in that Determining a target node attribute list according to the reference task performance indicator value and the reference node attribute list includes: Selecting a third number of second intermediate node attribute lists from the reference node attribute list according to the reference task performance indicator value; randomly selecting two groups of second switching node attribute lists from the second intermediate node attribute lists in sequence, and exchanging node attributes in the second switching node attribute lists to generate new second switching node attribute lists, until a third number of new second switching node attribute lists are generated as the updated third number of reference node attribute lists; If the number of updates of the reference node attribute list reaches a second preset number of iterations, determining the third number of reference node attribute lists after the current update as adjustment node attribute lists; Determine the task performance index value of the adjusted simulation data set corresponding to the adjusted node attribute list, and select the adjusted node attribute list with the highest task performance index value as the target node attribute list.

8. A device for generating a simulation data set, characterized in that: The device comprises: A candidate node attribute list determination module, used to determine node attribute information according to a scene graph of a real data set, and determine a first number of candidate node attribute lists according to the node attribute information; A candidate simulation data set determination module, used for rendering according to the node attributes in the candidate node attribute list to obtain a candidate simulation data set corresponding to the candidate node attribute list; a distribution difference parameter determination module, used to extract image features from the candidate simulation data set and the real data set, and determine the distribution difference parameters between each candidate simulation data set and the real data set in turn according to the extracted image features; The target simulation data set determination module is used to determine a target node attribute list according to the distribution difference parameter and the candidate node attribute list, and determine a target simulation data set according to the target node attribute list.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for generating a simulation data set according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for generating a simulation data set according to any one of claims 1 to 7 when executed.

Images