Synthetic image data authenticity evaluation method, electronic equipment and storage medium

Abstract

The invention relates to the technical field of image data processing, and particularly provides a composite image data authenticity evaluation method, electronic equipment and a storage medium, and the method comprises the steps: obtaining to-be-evaluated composite image data and real image data; based on the to-be-evaluated synthetic image data and the real image data, multi-dimensional image data comparison results between the to-be-evaluated synthetic image data and the real image data are obtained, and the multi-dimensional image data comparison results comprise feature comparison results of different level features and / or utility performance index results; obtaining a authenticity evaluation result of the to-be-evaluated synthetic image data based on a comparison result of the multi-dimensional image data; obtaining a to-be-evaluated composite image data authenticity evaluation result based on a feature comparison result of the different level features; and feature comparison results and utility performance index results of a plurality of levels are covered, comprehensive evaluation is carried out from details to local parts and then to global parts, and the accuracy and consistency of evaluation are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and specifically provides a method for evaluating the authenticity of synthetic image data, an electronic device and a storage medium. Background Art

[0002] The authenticity evaluation methods for synthetic image data currently mainly include: (1) Data distribution-based method: By analyzing the mean and variance of the synthetic image data on a specific color channel (such as the RGB channel), and comparing them with the distribution of the real image data, the authenticity of the synthetic image is evaluated.

[0003] (2) Model-based feature distribution judgment methods: including sKVD (semantically aligned KernelVGG Distance), a method that compares features after aligning semantics; or using the KID indicator (Kernel Inception Distance), which measures the difference between the two distributions by calculating the difference in the maximum eigenvalues ​​of the two Gaussian kernel matrices in the feature space of the generated sample and the real sample; the lower the KID, the closer the generated model is to the real distribution.

[0004] Although the above methods have achieved certain results in the authenticity evaluation of synthetic image data, there are still some shortcomings: The evaluation method based only on data distribution cannot judge the authenticity of image data from a geometric perspective, nor can it effectively measure whether there are unreal parts in the image data, such as over-processed noise images; Among the methods that evaluate only based on the feature distribution of the model, sKVD and KID are very sensitive to the quality of image data; however, real image data often has problems such as blur and exposure, and the sKVD and KID models essentially compare synthetic image data with real image data at the semantic level. Due to the diversity and complexity of real data, this comparison may have the problem of Domain Shift, resulting in low scores for ordinary real data.

[0005] Therefore, the existing synthetic image evaluation methods are too single, either global evaluation or too fine granularity, resulting in poor accuracy and consistency of the evaluation results.

[0006] Accordingly, the art needs a new synthetic image data authenticity evaluation scheme to solve the above problems. Summary of the invention

[0007] In order to overcome the above defects, the present invention is proposed to solve or at least partially solve the technical problem that the existing evaluation means are too single, either global evaluation or too fine granularity, resulting in poor accuracy and consistency of evaluation results.

[0008] In a first aspect, a method for evaluating the authenticity of synthetic image data is provided, the method comprising: Acquire synthetic image data to be evaluated and real image data; wherein the synthetic image data to be evaluated and the real image data contain at least some of the same target categories; Based on the synthetic image data to be evaluated and the real image data, obtaining a comparison result of multiple dimensional image data between the synthetic image data to be evaluated and the real image data, wherein the comparison results of multiple dimensional image data include feature comparison results of features at different levels and / or utility performance index results; The authenticity evaluation result of the synthetic image data to be evaluated is obtained based on the comparison results of the multiple dimensional image data.

[0009] In a technical solution of the above-mentioned synthetic image data authenticity evaluation method, obtaining the authenticity evaluation result of the synthetic image data to be evaluated based on the comparison results of the multiple dimensional image data includes: Based on the preset weights of the comparison results of the image data in multiple dimensions, the feature comparison results and / or utility performance index results of the features at different levels are weighted to obtain the authenticity evaluation result of the synthetic image data to be evaluated.

[0010] In a technical solution of the above-mentioned synthetic image data authenticity evaluation method, the method for obtaining the feature comparison results of the features at different levels includes: The synthetic image data to be evaluated and the real image data are input into a trained image data comparison model to obtain feature comparison results of different levels of features between the synthetic image data to be evaluated and the real image data.

[0011] In a technical solution of the above-mentioned synthetic image data authenticity evaluation method, the feature comparison results of the features at different levels include detail feature comparison results, local feature comparison results and global feature comparison results, and the trained image data comparison model includes a feature extraction model and a feature comparison model; obtaining the feature comparison results of the features at different levels between the synthetic image data to be evaluated and the real image data includes: Based on the feature extraction model, the bottom-level features, middle-level features and high-level features corresponding to the synthetic image data to be evaluated and the real image data are respectively obtained; Based on the feature comparison model, the underlying features, middle-level features and high-level features corresponding to the synthetic image data to be evaluated and the real image data, the detail feature comparison result, the local feature comparison result and the global feature comparison result are obtained respectively.

[0012] In a technical solution of the above-mentioned synthetic image data authenticity evaluation method, the feature extraction model includes a bottom-level feature extraction module, a middle-level feature extraction module and a high-level feature extraction module; based on the feature extraction model, the bottom-level features, middle-level features and high-level features corresponding to the synthetic image data to be evaluated and the real image data are respectively obtained, including: Based on the underlying feature extraction module and the LAB color channel, the underlying features corresponding to the synthetic image data to be evaluated and the real image data are obtained; Based on the middle-level feature extraction module, obtaining the middle-level features corresponding to the synthetic image data to be evaluated and the real image data respectively; Based on the high-level feature extraction module, high-level features corresponding to the synthetic image data to be evaluated and the real image data are obtained.

[0013] In a technical solution of the above-mentioned synthetic image data authenticity evaluation method, the utility performance index includes an average intersection-over-union index, and the trained image data comparison model also includes a utility performance index model; the method also includes obtaining the utility performance index result based on the following steps: Obtaining a labeling frame of the synthetic image data to be evaluated and its true target category; Training a preset segmentation model based on the synthetic image data to be evaluated to obtain a trained segmentation model; Based on the trained segmentation model, obtaining a prediction frame and a predicted target category of the synthetic image data to be evaluated; Based on the predicted frame of the synthetic image data to be evaluated and its predicted target category, the labeled frame of the synthetic image data to be evaluated and its true target category, and the utility performance indicator model, the average intersection-over-union ratio indicator result is obtained; wherein the utility performance indicator model is constructed based on the following formula:

[0014] in, for:

[0015] in, is the average intersection-and-union ratio indicator; is the intersection-over-union ratio between the c-th target category annotation box and the predicted box; is the total number of target categories; For the annotation box, is the prediction box.

[0016] In a technical solution of the above-mentioned synthetic image data authenticity evaluation method, the feature comparison result of the features at different levels includes the feature similarity of the features at different levels; and the method for obtaining the feature similarity of the features at different levels includes: Based on the synthetic image data to be evaluated and the real image data, obtaining one-dimensional feature vectors corresponding to the features at different levels of the synthetic image data to be evaluated and the real image data respectively; Based on a calculation method of cosine similarity calculation or Euclidean distance calculation, similarity calculation is performed on the one-dimensional feature vectors of the synthetic image data to be evaluated and the real image data at different levels of features to obtain feature similarities of the features at different levels.

[0017] In a technical solution of the above-mentioned synthetic image data authenticity evaluation method, obtaining a comparison result of image data in multiple dimensions between the synthetic image data to be evaluated and the real image data based on the synthetic image data to be evaluated and the real image data further includes: Performing data alignment on the synthetic image data to be evaluated and the real image data; wherein the data alignment at least includes semantic alignment; Based on the aligned synthetic image data to be evaluated and the real image data, image data comparison results of multiple dimensions between the synthetic image data to be evaluated and the real image data are obtained.

[0018] In a second aspect, an electronic device is provided, comprising at least one processor; and a memory communicatively connected to the at least one processor; wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the method described in any one of the technical solutions of the above-mentioned method for evaluating the authenticity of synthetic image data is implemented.

[0019] In a third aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, wherein the program codes are suitable for being loaded and run by a processor to execute the method described in any one of the technical solutions of the above-mentioned method for evaluating the authenticity of synthetic image data.

[0020] The above one or more technical solutions of the present invention have at least one or more of the following beneficial effects: In implementing the technical solution for evaluating the authenticity of synthetic image data provided by the present invention, synthetic image data to be evaluated and real image data are obtained; based on the synthetic image data to be evaluated and the real image data, multiple dimensional image data comparison results between the synthetic image data to be evaluated and the real image data are obtained, and the multiple dimensional image data comparison results include feature comparison results and / or utility performance index results of features at different levels; based on the multiple dimensional image data comparison results, authenticity evaluation results of the synthetic image data to be evaluated are obtained; based on the multiple dimensional image data comparison results, authenticity evaluation results of the synthetic image data to be evaluated are obtained; covering multiple levels of feature comparison results and utility performance index results, a comprehensive evaluation is conducted from details to local to global, thereby significantly improving the accuracy and consistency of the evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The disclosure of the present invention will become more easily understood with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present invention. Among them: Figure 1 is a flow chart of main steps of a method for evaluating the authenticity of synthetic image data according to an embodiment of the present invention; Figure 2 is a flowchart of main steps of a method for obtaining utility performance indicator results according to an embodiment of the present invention; Figure 3 It is a schematic diagram of the main structure of an electronic device according to an embodiment of the present invention.

[0022] Reference numerals: 31: memory; 32: processor. DETAILED DESCRIPTION

[0023] Some embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0024] In the description of the present invention, "module" and "processor" may include hardware, software or a combination of the two. A module may include hardware circuits, various suitable sensors, communication ports, and memories, and may also include software parts, such as program codes, or a combination of software and hardware. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, hardware, or a combination of the two. Computer-readable storage media include any suitable medium that can store program codes, such as a magnetic disk, a hard disk, an optical disk, a flash memory, a read-only memory, a random access memory, and the like. The term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B. The term "at least one A or B" or "at least one of A and B" has a similar meaning to "A and / or B", and may include only A, only B, or A and B. The singular terms "one" and "the" may also include plural forms.

[0025] The relevant user personal information that may be involved in the various embodiments of the present invention is strictly in accordance with the requirements of laws and regulations, following the principles of legality, legitimacy and necessity, based on the reasonable purposes of business scenarios, to process the personal information that users actively provide during the use of products / services or generated due to the use of products / services, as well as the personal information obtained with the user's authorization.

[0026] The user personal information processed by the present invention will vary depending on the specific product / service scenario, and shall be based on the specific scenario in which the user uses the product / service, and may involve the user's account information, device information, driving information, vehicle information or other related information. The present invention will treat the user's personal information and its processing with a high degree of diligence.

[0027] The present invention attaches great importance to the security of user personal information and has adopted reasonable and feasible security protection measures that meet industry standards to protect user information and prevent personal information from being accessed, disclosed, used, modified, damaged or lost without authorization.

[0028] Here we first explain some terms involved in the present invention.

[0029] Bottom-level features: obtained based on the bottom-level feature extraction module in the feature extraction model, which can reflect the original detail features of the image data, including the contour, edge, color, texture, noise, etc. of the image data.

[0030] Mid-level features: obtained based on the mid-level feature extraction module in the feature extraction model, which can reflect the local features of the image data, including the more complex local shapes, texture information, partial object contours or other local features of the image data; based on the underlying features, the mid-level features such as the geometric shape and partial structure of the image data are further extracted, such as the abstract features of the granularity of a tire or a human body.

[0031] High-level features: obtained based on the high-level feature extraction module in the feature extraction model, which can reflect the global characteristics of the image data and contain information such as the structure and global semantics of the image data, such as the approximate structure and main objects of the entire image data.

[0032] Utility performance indicators: Based on the specific problems solved by the model trained with synthetic image data, the indicators used to evaluate the performance of the model trained with synthetic image data are utility performance indicators; for example, when the trained model is a segmentation model, the mIOU indicator (Mean Intersection over Union) is used to evaluate the model; when the trained model is a detection model or a classification model, the mAP indicator (mean Average Precision) is used to evaluate the model.

[0033] See attached Figure 1 , Figure 1 FIG. 1 is a flow chart showing the main steps of a method for evaluating the authenticity of synthetic image data according to an embodiment of the present invention. Figure 1 As shown, the authenticity evaluation method of synthetic image data in the embodiment of the present invention mainly includes the following steps S101 to S103.

[0034] Step S101: obtaining synthetic image data and real image data to be evaluated; In this embodiment, the synthetic image data to be evaluated and the real image data contain at least some of the same target categories; for example, from a semantic level, the synthetic image data to be evaluated and the real image data both include the same target categories such as moving vehicles, vehicles parked on the roadside, trees, and people.

[0035] In this embodiment, after obtaining the synthetic image data to be evaluated and the real image data, the method further includes: Preprocessing the synthetic image data and the real image data to be evaluated by one or more preprocessing operations in image enhancement or normalization processing to obtain the preprocessed synthetic image data and the real image data to be evaluated; Perform data alignment on the preprocessed synthetic image data to be evaluated and the real image data.

[0036] In one embodiment, data alignment includes at least semantic alignment, using image recognition technology to extract key semantic information from the synthetic image data to be evaluated and the real image data, such as the location and category of the object in the image data, etc., matching the extracted key semantic information with the known semantic truth value, establishing a semantic mapping relationship, and adjusting the semantic mapping relationship through an iterative optimization algorithm, so that the synthetic image data to be evaluated and the real image data are more consistent at the semantic level. It should be noted that the data alignment method here is only an exemplary description, and different choices can be made in actual applications according to actual needs, as long as the synthetic image data to be evaluated and the real image data can be consistent at the semantic level.

[0037] Step S102: obtaining a comparison result of multiple dimensional image data between the synthetic image data to be evaluated and the real image data based on the synthetic image data to be evaluated and the real image data; the comparison result of multiple dimensional image data includes feature comparison results of features at different levels and / or utility performance index results; In this embodiment, the method for obtaining feature comparison results of features at different levels includes: The synthetic image data to be evaluated and the real image data are input into the trained image data comparison model to obtain feature comparison results of different levels of features between the synthetic image data to be evaluated and the real image data.

[0038] In this embodiment, the feature comparison results of features at different levels include detail feature comparison results, local feature comparison results and global feature comparison results, and the trained image data comparison model includes a feature extraction model and a feature comparison model; The feature comparison results of different levels of features between the synthetic image data to be evaluated and the real image data include: Based on the feature extraction model, the underlying features, middle-level features and high-level features corresponding to the synthetic image data to be evaluated and the real image data are obtained respectively; Based on the feature comparison model, the underlying features, middle-level features and high-level features corresponding to the synthetic image data to be evaluated and the real image data, the detail feature comparison results, local feature comparison results and global feature comparison results are obtained respectively.

[0039] In one embodiment, the feature extraction model includes a bottom-level feature extraction module, a middle-level feature extraction module, and a high-level feature extraction module; based on the feature extraction model, the bottom-level features, middle-level features, and high-level features corresponding to the synthetic image data to be evaluated and the real image data are obtained respectively, including: Based on the underlying feature extraction module and the LAB color channel, the underlying features corresponding to the synthetic image data to be evaluated and the real image data are obtained; Based on the middle-level feature extraction module, the middle-level features corresponding to the synthetic image data to be evaluated and the real image data are obtained; Based on the high-level feature extraction module, high-level features corresponding to the synthetic image data to be evaluated and the real image data are obtained.

[0040] Optionally, the feature extraction model can be set to a CNN network model (Convolutional Neural Networks) including six convolutional layers, where the first two convolutional layers are preset as bottom-level feature extraction modules, the middle two convolutional layers are middle-level feature extraction modules, and the last two convolutional layers are high-level feature extraction modules. It should be noted that the CNN network model in this embodiment includes six convolutional layers and the selection of the corresponding number of layers of the bottom-level, middle-level, and high-level feature extraction modules is only an exemplary description. In actual applications, the number of layers of the CNN network model and the selection of the corresponding number of layers of the bottom-level, middle-level, and high-level feature extraction modules can be selected differently according to actual needs.

[0041] The synthetic image data to be evaluated and the real image data are taken as input pairs and input into the underlying feature extraction module to perform a preliminary convolution operation to obtain the underlying features including color, edge, and texture corresponding to the synthetic image data to be evaluated and the real image data.

[0042] The synthetic image data to be evaluated and the real image data are taken as an input pair and input into the middle-level feature extraction module, and one or more operation steps of convolution, activation and / or pooling are performed to obtain the middle-level features including more complex local shapes and structures corresponding to the synthetic image data to be evaluated and the real image data.

[0043] The synthetic image data to be evaluated and the real image data are taken as an input pair and input into a high-level feature extraction module, and one or more operation steps of convolution, activation, pooling and / or full connection are performed to obtain high-level features corresponding to the synthetic image data to be evaluated and the corresponding real image data, including the structure and global semantics of the image data.

[0044] In one embodiment, when the synthetic image data to be evaluated and the real image data are input into the bottom feature extraction module, the middle feature extraction module and the high-level feature extraction module respectively, the image sizes of the synthetic image data to be evaluated and the real image data input are different for different feature extraction modules; wherein the image size of the synthetic image data to be evaluated and the real image data input into the bottom feature extraction module is the smallest, the image size of the synthetic image data to be evaluated and the real image data input into the middle feature extraction module is larger than the image size of the synthetic image data to be evaluated and the real image data input into the bottom feature extraction module, and smaller than the image size of the synthetic image data to be evaluated and the real image data input into the high-level feature extraction module. For example, the image size of the synthetic image data to be evaluated and the real image data input into the bottom feature extraction module can be selected as 32*32 pixels, the image size of the synthetic image data to be evaluated and the real image data input into the middle feature extraction module can be selected as 128*128 pixels; the image size of the synthetic image data to be evaluated and the real image data input into the high-level feature extraction module can be selected as 515*512 pixels.

[0045] It should be noted that the selection of the feature extraction model and the division of the levels of the bottom-level feature extraction module, the middle-level feature extraction module and the high-level feature extraction module and the image size of the input synthetic image data to be evaluated and the real image data are only exemplary descriptions. In actual applications, different choices can be made as needed, as long as the feature extraction of the bottom-level features, middle-level features and high-level features corresponding to the synthetic image data to be evaluated and the real image data can be achieved.

[0046] In this embodiment, the feature comparison results of features at different levels include feature similarities of features at different levels; based on the feature comparison model, the underlying features, the middle features, and the high-level features corresponding to the synthetic image data to be evaluated, and the real image data, respectively, the detailed feature comparison results, the local feature comparison results, and the global feature comparison results are obtained, including: Based on the feature comparison model, the underlying features, middle-level features, and high-level features corresponding to the synthetic image data to be evaluated and the real image data, the feature similarity of features at different levels is obtained; In one embodiment, a method for obtaining feature similarity of features at different levels includes: Flatten the underlying features, middle-level features, and high-level features corresponding to the synthetic image data to be evaluated and the real image data, respectively, to obtain a one-dimensional underlying feature vector, a one-dimensional middle-level feature vector, and a one-dimensional high-level feature vector corresponding to the synthetic image data to be evaluated and the real image data, respectively; Based on a calculation method in cosine similarity calculation or Euclidean distance calculation, similarity calculation is performed on the one-dimensional bottom feature vector, one-dimensional middle feature vector and one-dimensional high-level feature vector corresponding to the synthetic image data to be evaluated and the real image data, respectively, to obtain the feature similarity of features at different levels.

[0047] In this embodiment, the utility performance index includes an average intersection-over-union index, and the trained image data comparison model also includes a utility performance index model; see the attached Figure 2 , Figure 2 FIG. 1 is a flow chart showing the main steps of a method for obtaining a utility performance indicator result according to an embodiment of the present invention. Figure 2 As shown, the method also includes obtaining a utility performance indicator result based on the following steps: Step S201: obtaining the annotation box of the synthetic image data to be evaluated and its real target category; the synthetic image data to be evaluated contains multiple target categories, and the target categories and their annotation boxes are known; Step S202: training a preset segmentation model based on the synthetic image data to be evaluated to obtain a trained segmentation model; Step S203: obtaining a prediction frame and a predicted target category of the synthetic image data to be evaluated based on the trained segmentation model; Step S204: Based on the predicted frame of the synthetic image data to be evaluated and its predicted target category, the labeled frame of the synthetic image data to be evaluated and its true target category, and the utility performance indicator model, an average intersection-over-union ratio indicator result is obtained; wherein the utility performance indicator model is constructed based on the following formula:

[0048] in, for:

[0049] in, is the average intersection-and-union ratio indicator; is the intersection-over-union ratio between the c-th target category annotation box and the predicted box; is the total number of target categories; For the annotation box, is the prediction box.

[0050] It should be noted that the selection of the preset segmentation model and the utility performance index when performing utility performance evaluation on the synthetic image data to be evaluated is only for illustrative purposes. In actual application, the appropriate trained model and its corresponding utility performance index can be selected as needed, as long as the utility performance index evaluation can be achieved.

[0051] Step S103: obtaining a authenticity evaluation result of the synthetic image data to be evaluated based on the comparison results of the image data in multiple dimensions; In this embodiment, obtaining the authenticity evaluation result of the synthetic image data to be evaluated based on the comparison results of the image data in multiple dimensions includes: Based on the preset weights of the comparison results of image data in multiple dimensions, the feature comparison results and / or utility performance index results of features at different levels are weighted to obtain the authenticity evaluation results of the synthetic image data to be evaluated.

[0052] In one embodiment, the comparison results of multiple dimensional image data include feature comparison results of features at different levels, and the method for obtaining the authenticity evaluation results of the synthetic image data to be evaluated includes: obtaining the authenticity evaluation results of the synthetic image data to be evaluated based on preset weights of the feature comparison results of features at different levels.

[0053] In one embodiment, the comparison results of multiple dimensional image data include utility performance index results, and the method for obtaining the authenticity evaluation results of the synthetic image data to be evaluated includes: obtaining the authenticity evaluation results of the synthetic image data to be evaluated based on the utility performance index results.

[0054] In one embodiment, the comparison results of multiple dimensional image data include feature comparison results of features at different levels and utility performance index results, and the method for obtaining the authenticity evaluation results of the synthetic image data to be evaluated includes: obtaining the authenticity evaluation results of the synthetic image data to be evaluated based on the preset weights of the feature comparison results of features at different levels and the preset weights of the utility performance index results.

[0055] It should be pointed out that, although the various steps in the above embodiments are described in a specific order, those skilled in the art will understand that, in order to achieve the effects of the present invention, different steps do not have to be executed in such an order, and they can be executed simultaneously (in parallel) or in other orders. These adjusted solutions are equivalent to the technical solutions described in the present invention, and therefore will also fall within the scope of protection of the present invention.

[0056] It is understood by those skilled in the art that the present invention implements all or part of the processes in the method of the above embodiment, and can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electric carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code.

[0057] Another aspect of the present invention provides a computer-readable storage medium.

[0058] In an embodiment of a computer-readable storage medium according to the present invention, the computer-readable storage medium may be configured to store a program for executing the method for evaluating the authenticity of synthetic image data of the above method embodiment, and the program may be loaded and run by a processor to implement the above method for evaluating the authenticity of synthetic image data. For ease of explanation, only the parts related to the embodiment of the present invention are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present invention. The computer-readable storage medium may be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present invention is a non-transitory computer-readable storage medium.

[0059] Another aspect of the present invention provides an electronic device.

[0060] In an embodiment of an electronic device according to the present invention, the electronic device may include at least one processor; and a memory connected to the at least one processor in communication; wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the method described in any of the above embodiments is implemented. Figure 3 , Figure 3 FIG. 3 exemplarily shows that the memory 31 and the processor 32 are communicatively connected via a bus.

[0061] The electronic device described in the present invention may be, but is not limited to, a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, a vehicle-mounted device, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), an augmented reality (AR) or virtual reality (VR) device, etc., and the embodiments of the present invention are not limited to this.

[0062] So far, the technical solution of the present invention has been described in conjunction with an embodiment shown in the accompanying drawings, but it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A method for evaluating the authenticity of synthetic image data, characterized in that: include: Acquire synthetic image data to be evaluated and real image data; wherein the synthetic image data to be evaluated and the real image data contain at least some of the same target categories; Based on the synthetic image data to be evaluated and the real image data, obtaining a comparison result of multiple dimensional image data between the synthetic image data to be evaluated and the real image data, wherein the comparison results of multiple dimensional image data include feature comparison results of features at different levels and / or utility performance index results; The authenticity evaluation result of the synthetic image data to be evaluated is obtained based on the comparison results of the multiple dimensional image data.

2. The method for evaluating the authenticity of synthetic image data according to claim 1, characterized in that: Obtaining the authenticity evaluation result of the synthetic image data to be evaluated based on the comparison results of the multiple dimensional image data includes: Based on the preset weights of the comparison results of the image data in multiple dimensions, the feature comparison results and / or utility performance index results of the features at different levels are weighted to obtain the authenticity evaluation result of the synthetic image data to be evaluated.

3. The method for evaluating the authenticity of synthetic image data according to claim 1, characterized in that: The method for obtaining the feature comparison results of the features at different levels includes: The synthetic image data to be evaluated and the real image data are input into a trained image data comparison model to obtain feature comparison results of different levels of features between the synthetic image data to be evaluated and the real image data.

4. The method for evaluating the authenticity of synthetic image data according to claim 3, characterized in that: The feature comparison results of the features at different levels include detail feature comparison results, local feature comparison results and global feature comparison results, and the trained image data comparison model includes a feature extraction model and a feature comparison model; obtaining the feature comparison results of the features at different levels between the synthetic image data to be evaluated and the real image data includes: Based on the feature extraction model, the bottom-level features, middle-level features and high-level features corresponding to the synthetic image data to be evaluated and the real image data are respectively obtained; Based on the feature comparison model, the underlying features, middle-level features and high-level features corresponding to the synthetic image data to be evaluated and the real image data, the detail feature comparison result, the local feature comparison result and the global feature comparison result are obtained respectively.

5. The method for evaluating the authenticity of synthetic image data according to claim 4, characterized in that: The feature extraction model includes a bottom-level feature extraction module, a middle-level feature extraction module and a high-level feature extraction module; Obtaining the underlying features, middle-level features and high-level features corresponding to the synthetic image data to be evaluated and the real image data based on the feature extraction model includes: Based on the underlying feature extraction module and the LAB color channel, the underlying features corresponding to the synthetic image data to be evaluated and the real image data are obtained; Based on the middle-level feature extraction module, obtaining middle-level features corresponding to the synthetic image data to be evaluated and the real image data respectively; Based on the high-level feature extraction module, high-level features corresponding to the synthetic image data to be evaluated and the real image data are obtained.

6. The method for evaluating the authenticity of synthetic image data according to claim 3, characterized in that: The utility performance index includes an average intersection-over-union index, and the trained image data comparison model also includes a utility performance index model; the method also includes obtaining the utility performance index result based on the following steps: Obtaining a labeling frame of the synthetic image data to be evaluated and its true target category; Training a preset segmentation model based on the synthetic image data to be evaluated to obtain a trained segmentation model; Based on the trained segmentation model, obtaining a prediction frame and a predicted target category of the synthetic image data to be evaluated; Based on the predicted frame of the synthetic image data to be evaluated and its predicted target category, the labeled frame of the synthetic image data to be evaluated and its true target category, and the utility performance indicator model, the average intersection-over-union ratio indicator result is obtained; wherein the utility performance indicator model is constructed based on the following formula: in, for: in, is the average intersection-and-union ratio indicator; is the intersection-over-union ratio between the c-th target category annotation box and the predicted box; is the total number of target categories; For the annotation box, is the prediction box.

7. The method for evaluating the authenticity of synthetic image data according to claim 1, characterized in that: The feature comparison result of the features at different levels includes the feature similarity of the features at different levels; The method for obtaining the feature similarity of features at different levels includes: Based on the synthetic image data to be evaluated and the real image data, obtaining one-dimensional feature vectors corresponding to the features at different levels of the synthetic image data to be evaluated and the real image data respectively; Based on a calculation method of cosine similarity calculation or Euclidean distance calculation, similarity calculation is performed on the one-dimensional feature vectors of the synthetic image data to be evaluated and the real image data at different levels of features to obtain feature similarities of the features at different levels.

8. The method for evaluating the authenticity of synthetic image data according to any one of claims 1 to 7, characterized in that: Based on the synthetic image data to be evaluated and the real image data, obtaining a comparison result of image data in multiple dimensions between the synthetic image data to be evaluated and the real image data further includes: Performing data alignment on the synthetic image data to be evaluated and the real image data; wherein the data alignment at least includes semantic alignment; Based on the aligned synthetic image data to be evaluated and the real image data, image data comparison results of multiple dimensions between the synthetic image data to be evaluated and the real image data are obtained.

9. An electronic device, characterized in that: include: at least one processor; and, a memory connected to the at least one processor; Wherein, a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the method for evaluating the authenticity of synthetic image data according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by a processor to execute the synthetic image data authenticity assessment method according to any one of claims 1 to 8.

Images