A method, device and storage medium for extracting image data characteristics

Abstract

The embodiment of the present application discloses a method, device and storage medium for extracting image data characteristics, which are used to extract high-quality image data characteristics. The embodiment of the present application provides a method for extracting image data characteristics, including: extracting feature data of multiple images from image data to be evaluated respectively to obtain image features of the multiple images; performing multi-scale image entropy evaluation on the multiple images according to the image features of the multiple images to obtain image entropy of the multiple images; searching for class representative images from the multiple images according to the image features of the multiple images; generating image data characteristics of the image data to be evaluated according to the image entropy of the multiple images and the class representative image, wherein the image data characteristics include: image entropy of the multiple images and the class representative image.

Description

Technical Field

[0001] The present application relates to the field of image data processing, and in particular to a method, device and storage medium for extracting image data characteristics. Background Art

[0002] Computer vision technology can be used to complete image processing tasks. For example, a general image dataset consists of pictures of natural landscapes, common objects, etc. The general image dataset is input into a computer vision model to complete various image-related processing tasks.

[0003] The above-mentioned computer vision models are built on image datasets of natural landscapes, common objects, etc. However, in data trading scenarios, it is impossible to use the above-mentioned computer vision technology to obtain targeted processing results that promote data trading. Data can be traded as a commodity, but there is currently no relevant existing technology solution for how to efficiently extract data features. Summary of the invention

[0004] The embodiments of the present application provide a method, device and storage medium for extracting image data characteristics, which are used to extract high-quality image data characteristics.

[0005] To solve the above technical problems, the present application provides the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides a method for extracting characteristics of image data, comprising:

[0007] Extracting feature data of a plurality of images from the image data to be evaluated respectively to obtain image features of the plurality of images;

[0008] Performing multi-scale image entropy evaluation on the multiple images respectively according to the image features of the multiple images to obtain image entropies of the multiple images;

[0009] searching for a class representative image from the multiple images according to image features of the multiple images;

[0010] The image data characteristics of the image data to be evaluated are generated according to the image entropies of the multiple images and the class representative image, and the image data characteristics include: the image entropies of the multiple images and the class representative image.

[0011] In combination with the first aspect, in a possible implementation manner of the first aspect, the method further includes: performing image clarity detection on the multiple images respectively according to image features of the multiple images to obtain image clarity results of the multiple images;

[0012] in,

[0013] The step of generating the image data characteristics of the image data to be evaluated according to the image entropy of the plurality of images and the class representative image comprises:

[0014] Image data characteristics of the image data to be evaluated are generated according to the image entropies of the multiple images, the class representative images and the image clarity results of the multiple images, and the image data characteristics also include: the image clarity results of the multiple images.

[0015] In combination with the first aspect, in a possible implementation manner of the first aspect, generating the image data characteristics of the image data to be evaluated according to the image entropy of the multiple images, the class representative image, and the image clarity results of the multiple images includes:

[0016] identifying clarity outlier images from the plurality of images according to the image clarity results of the plurality of images;

[0017] Image data characteristics of the image data to be evaluated are generated according to the image entropies of the multiple images, the class representative images and the clarity outlier images, and the image data characteristics specifically include: the clarity outlier images.

[0018] In combination with the first aspect, in a possible implementation manner of the first aspect, the method further includes:

[0019] Obtaining image feature mean values ​​of the multiple images according to the image features of the multiple images;

[0020] Determine an outlier image among the multiple images according to a difference between an image feature of each image in the multiple images and a mean value of the image features;

[0021] in,

[0022] The step of generating the image data characteristics of the image data to be evaluated according to the image entropy of the plurality of images and the class representative image comprises:

[0023] Image data characteristics of the image data to be evaluated are generated according to the image entropy of the multiple images, the class representative images and the outlier images in the multiple images. The image data characteristics also include: outlier images in the multiple images.

[0024] In combination with the first aspect, in a possible implementation of the first aspect, searching for a class representative image from the multiple images according to image features of the multiple images includes:

[0025] Performing image clustering processing according to the image features of the multiple images to obtain multiple image categories;

[0026] According to each of the plurality of image categories, class representative images corresponding to the plurality of image categories are generated.

[0027] In combination with the first aspect, in a possible implementation manner of the first aspect, the method further includes:

[0028] For each image category in the plurality of image categories, determining a class center of each image category;

[0029] Determine the category outlier images in each image category according to the difference between the image feature of each image in each image category and the category center;

[0030] in,

[0031] The step of generating the image data characteristics of the image data to be evaluated according to the image entropy of the plurality of images and the class representative image comprises:

[0032] Image data characteristics of the image data to be evaluated are generated according to the image entropy of the multiple images, the class representative images and the category outlier images in each image category, and the image data characteristics also include: category outlier images in each image category.

[0033] In combination with the first aspect, in a possible implementation manner of the first aspect, performing multi-scale image entropy evaluation on the multiple images respectively according to image features of the multiple images includes:

[0034] For each of the plurality of images, determining image blocks of each image at a plurality of image block scales;

[0035] According to the grayscale distribution probability of the image grayscale feature of each image block in the image grayscale feature distribution interval at various image block scales, the image entropy of each image at various image block scales is obtained.

[0036] In a second aspect, an embodiment of the present application further provides a device for extracting characteristics of image data, including:

[0037] A feature extraction module, used to extract feature data of multiple images from the image data to be evaluated, so as to obtain image features of the multiple images;

[0038] An image entropy evaluation module, used to perform multi-scale image entropy evaluation on the multiple images respectively according to the image features of the multiple images to obtain image entropies of the multiple images;

[0039] A class representative image search module, used to search for a class representative image from the multiple images according to image features of the multiple images;

[0040] A data characteristic generating module is used to generate image data characteristics of the image data to be evaluated according to the image entropies of the multiple images and the class representative image, wherein the image data characteristics include: the image entropies of the multiple images and the class representative image.

[0041] In the second aspect of the present application, the component modules of the device for extracting image data characteristics can also execute the steps described in the aforementioned first aspect and various possible implementations. For details, please refer to the aforementioned description of the first aspect and various possible implementations.

[0042] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer executes the method described in the first aspect above.

[0043] In a fourth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method described in the first aspect above.

[0044] In a fifth aspect, an embodiment of the present application provides a device for extracting image data characteristics. The communication device may include entities such as a terminal device or a chip, and the device for extracting image data characteristics includes: a processor and a memory; the memory is used to store instructions; the processor is used to execute the instructions in the memory, so that the communication device executes a method as described in any one of the first aspects above.

[0045] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0046] In an embodiment of the present application, first, feature data of multiple images are extracted from the image data to be evaluated to obtain image features of the multiple images; then, multi-scale image entropy evaluation is performed on the multiple images according to the image features of the multiple images to obtain image entropy of the multiple images; and class representative images are found from the multiple images according to the image features of the multiple images; finally, image data characteristics of the image data to be evaluated are generated according to the image entropy of the multiple images and the class representative images, and the image data characteristics include: image entropy of the multiple images and class representative images. In the embodiment of the present application, image data characteristics of the image data to be evaluated can be generated by image entropy of the multiple images and class representative images, so the image data characteristics output in the embodiment of the present application include: image entropy of the multiple images and class representative images, the class representative images can make a concise summary of the image data to be evaluated, and represent the data conditions of the multiple images, and the image entropy can represent the amount of information of the image data to be evaluated, so the embodiment of the present application can achieve the extraction of high-quality image data characteristics. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1A flowchart of a method for extracting image data characteristics provided in an embodiment of the present application;

[0048] Figure 2 A schematic diagram of the composition structure of an image data characteristic provided in an embodiment of the present application;

[0049] Figure 3 A schematic diagram of the composition structure of an outlier image provided in an embodiment of the present application;

[0050] Figure 4 A schematic diagram of a scene for implementing a framework for extracting image data characteristics provided in an embodiment of the present application;

[0051] Figure 5 A schematic diagram of the composition structure of an image data characteristic extraction device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The embodiments of the present application provide a method, device and storage medium for extracting image data characteristics, which are used to extract high-quality image data characteristics.

[0053] The embodiments of the present application are described below in conjunction with the accompanying drawings.

[0054] The embodiments of the present application are applicable to scenarios for the evaluation of data characteristics. For example, in the case of data being traded as a commodity, both parties to the commodity transaction need to evaluate the characteristics and quality of the commodity in order to reach an agreement on the value of the commodity and facilitate the transaction. In the embodiments of the present application, image data can be used as the data to be evaluated. It is also necessary to evaluate the characteristics, quality, value, etc. of its data elements before the transaction, so that the buyer can judge whether the data product meets his needs.

[0055] In addition, an important feature of data is that it is very easy to copy and spread. It is necessary to prevent buyers or third parties from illegally obtaining data. Therefore, the data evaluation method needs to consider avoiding data leakage in both the transmission and use stages. The risk of data leakage can be solved through offline evaluation, which requires that the amount of data evaluation method calculation should not be too large so that it can be implemented on portable devices such as laptops. Potential buyers of data need to obtain data evaluation results before the transaction occurs, which requires that the data evaluation results cannot contain all the information contained in the data, but should also retain key information such as data characteristics, quality, and value. For data in different modes such as text, images, and numbers, data evaluation methods should be designed separately to serve the buyer's needs for data evaluation, while not exposing all data to protect the interests of the seller.

[0056] The embodiments of the present application are applicable to feature extraction of various image data, and can perform high-quality data feature evaluation on image data for data transactions.

[0057] Next, some of the terms involved in the embodiments of the present application are explained:

[0058] Deep learning is a mathematical model building paradigm that requires large-scale computing and a large amount of manually annotated data.

[0059] Model training refers to the process of building a deep learning model.

[0060] Labeling refers to manually creating a description for the data, such as labeling a picture as a "cat picture" or "dog picture", or summarizing a paragraph of an article into a sentence.

[0061] The label is the result of the above annotation.

[0062] Clustering is a method of classifying multiple data according to their internal connections. For example, there are 60 temperatures in City A for 30 days in winter and summer of a certain year. Without knowing which season a certain temperature belongs to, clustering can be used to classify the 60 temperatures into two categories according to the temperature.

[0063] Grayscale can represent the brightness of pixels in black and white images. In the embodiment of the present application, the grayscale value range can be 0-255, where 0 represents pure black pixels and 255 represents pure white pixels. Color images can be converted into grayscale images.

[0064] Data elements refer to a collection of data that can be used as resources in production and business activities and can create economic value. They are new production factors following land, labor, capital, and technology. The premise for data elements to play their value is circulation and trading.

[0065] Data quality is one of the factors that measure data value. The higher the data quality, the higher its value, the higher the price in the transaction process, and the greater the economic value it can create.

[0066] The processing of image data in the embodiments of the present application can be achieved through computer vision. Computer vision mainly studies the detection, recognition, and processing of images using computers. The main purpose is to use computers to replace manual work to complete image-related tasks. The main research issues of computer vision are as follows:

[0067] Image classification: Classify images into their categories. For example, classify a series of animal photos into categories such as cats, dogs, etc.

[0068] Object detection: Mark the key objects in the image with boxes. For example, identify all the faces in a group photo and mark them with several boxes.

[0069] Image segmentation: Further segmentation is performed based on target detection, such as marking all pixels belonging to faces and pixels belonging to the background in a group photo separately.

[0070] Clarity detection: used to objectively evaluate the clarity of an image, such as the gradient-based clarity detection method. The larger the gradient, the sharper the edges of objects in the image and the higher the clarity.

[0071] It should be noted that in the embodiments of the present application, images can also be referred to as pictures, which will not be distinguished in subsequent examples.

[0072] In the prior art, most of the computer vision models with high accuracy are deep learning models with large number of parameters and large amount of computation required in the model building stage. Repeatedly building models on the image dataset to be evaluated requires high computing power and data annotation costs and takes a long time. In addition, computer vision models are targeted at specific tasks in a certain field of computer vision, such as classification, segmentation, clarity assessment and other results. Existing computer vision models give a result for each image in the image dataset. If the amount of data is large, the result given is also relatively lengthy, which is not conducive to the buyer to concisely understand the characteristics of the dataset. The above-mentioned computer vision model cannot be used to collect the characteristics of image datasets in data trading scenarios.

[0073] In view of this, an embodiment of the present application provides a method for extracting image data characteristics, which can be applicable to a terminal device, for example, the terminal device can be a mobile phone terminal, a notebook terminal, or a tablet terminal. The method for extracting image data characteristics mainly includes:

[0074] 101. Extract feature data of multiple images from image data to be evaluated respectively to obtain image features of the multiple images.

[0075] In the embodiment of the present application, firstly, the image data to be evaluated is obtained, and the image data to be evaluated includes: data of multiple images. There is no limitation on the type of image, and there are many ways to obtain image data. For example, in a data transaction scenario, the image to be evaluated can be obtained according to the request message of the client.

[0076] Among them, for the acquired image data to be evaluated, feature data of each image can be extracted to obtain image features of multiple images, and the image features can specifically be image vectors. For example, the image features can specifically include image grayscale features,

[0077] An example is given below: a deep image classification model with a small number of parameters is selected, the pre-trained weights of the model are loaded, all images in the image data to be evaluated are input into the deep image classification model respectively, and the feature vector of the last layer of the model corresponding to each image is obtained as the image feature.

[0078] After acquiring the image features of multiple images, subsequent steps 102 and 103 may be performed, wherein there is no distinction in the order of execution between 102 and 103. Figure 1 The following takes the example of executing 102 first and then executing 103 for explanation.

[0079] 102. Perform multi-scale image entropy evaluation on the multiple images respectively according to the image features of the multiple images to obtain image entropies of the multiple images.

[0080] Among them, the image features of multiple images can be used to calculate the image entropy of each image. Image entropy can represent the amount of information of the image data to be evaluated, and image entropy can also be called information entropy. Image entropy reflects the amount of information contained in the image. The larger the image entropy, the greater the amount of information contained in the image. In the embodiment of the present application, multi-scale image entropy evaluation can be performed for each image. Multi-scale image entropy evaluation refers to the use of a variety of different image block sizes for entropy evaluation, so that the image entropy of multiple images can be obtained.

[0081] In some embodiments of the present application, step 102 performs multi-scale image entropy evaluation on the multiple images according to the image features of the multiple images, including:

[0082] For each of the multiple images, determine the image block of each image at multiple image block scales;

[0083] According to the grayscale distribution probability of the image grayscale feature of each image block in the image grayscale feature distribution interval at various image block scales, the image entropy of each image at various image block scales is obtained.

[0084] Among them, in the embodiment of the present application, there is no limitation on the value of the image block scale. For example, the image block scale can be represented by Ns, and the value of Ns can be an interval range. There is no limitation on the value of each image block scale within the interval range.

[0085] When each image segmentation scale is determined, each image in the multiple images can be segmented to obtain the image segmentation of each image. Based on the image features of each image in the aforementioned step 101, the image features of the image segmentation of each image can be obtained. For example, the image feature can be an image grayscale feature, and the image grayscale feature distribution interval refers to the image grayscale value range, for example, the image grayscale feature distribution interval can be 0-255. According to the grayscale distribution probability of the image grayscale feature of the image segmentation of each image in the image grayscale feature distribution interval at a plurality of image segmentation scales, the image entropy of each image at a plurality of image segmentation scales can be obtained. In the embodiment of the present application, the image entropy can be calculated for a plurality of image segmentation scales, so as to realize multi-scale image entropy evaluation of a plurality of images.

[0086] 103. Find a class representative image from multiple images based on the image features of multiple images.

[0087] Among them, the image features of multiple images can be used to find a class representative image from multiple images. The class representative image refers to an image that can represent the content of all images in the category to which the image belongs. The class representative image can succinctly summarize the image data to be evaluated and represent the data of multiple images.

[0088] In some embodiments of the present application, step 103 of searching for a class representative image from the multiple images according to the image features of the multiple images includes:

[0089] Perform image clustering processing according to the image features of multiple images to obtain multiple image categories;

[0090] According to each of the multiple image categories, a plurality of class representative images corresponding to the image categories are generated.

[0091] Among them, clustering related technology can be used to perform cluster analysis with image features as input to obtain the image category to which each image feature belongs. Based on different image categories, class representative images are searched, so that class representative images that can represent the image category to which they belong can be found.

[0092] 104. Generate image data characteristics of the image data to be evaluated according to the image entropies and class representative images of the multiple images, where the image data characteristics include: image entropies and class representative images of the multiple images.

[0093] In the embodiment of the present application, after executing the aforementioned steps 102 and 103 to obtain the image entropy and class representative images of multiple images, the image data characteristics of the image data to be evaluated are generated according to the image entropy and class representative images of the multiple images. The image data characteristics refer to data characteristics that can be used for data transaction evaluation, and the image data characteristics include: image entropy and class representative images of multiple images. It is not limited that in addition to the image entropy and class representative images of multiple images, the image data characteristics in the embodiment of the present application may also include other data characteristics, which are not limited at this time. For example, the structure, purpose, usage, advantages and disadvantages of image data can all be used as image data characteristics.

[0094] Furthermore, in some embodiments of the present application, in addition to executing the aforementioned steps 101 to 104, the method provided in the embodiments of the present application further includes:

[0095] The image data features are stored in an image data transaction database.

[0096] Among them, after generating the image data characteristics of the image data to be evaluated, the embodiment of the present application can also store the image data characteristics of the image data to be evaluated in an image data transaction database, so that when it is necessary to evaluate the image data to be evaluated, the corresponding image data characteristics can be obtained from the image data transaction database, thereby providing high-quality image data characteristics for data transactions.

[0097] Furthermore, in some embodiments of the present application, in addition to executing the aforementioned steps 101 to 103, the method provided in the embodiments of the present application further includes:

[0098] An outlier image analysis is performed on the multiple images according to the image features of the multiple images to obtain the outlier images.

[0099] Among them, an outlier image refers to an image whose data characteristics are too different from those of multiple images in the image data to be evaluated, and the outlier image cannot represent the data characteristics of multiple images.

[0100] The aforementioned step 104 generates image data characteristics of the image data to be evaluated according to the image entropy of the plurality of images and the class representative image, including:

[0101] According to the image entropy, class representative images and outlier images of the multiple images, image data characteristics of the image data to be evaluated are generated, and the image data characteristics include: the image entropy, class representative images and outlier images of the multiple images.

[0102] like Figure 2 As shown, an image data transaction database provided by an embodiment of the present application may include image data characteristics, and the image data characteristics may include at least one of the following: image entropy of multiple images, class representative images, and outlier images.

[0103] Furthermore, in some embodiments of the present application, in addition to executing the aforementioned steps 101 to 103, the method provided in the embodiments of the present application further includes:

[0104] For each image category in a plurality of image categories, determining a class center for each image category;

[0105] The class outlier images in each image class are determined based on the difference between the image features and the class center of each image in each image class.

[0106] Among them, for each image category in multiple image categories, the class center of each image category is determined according to the mean value of image features in the same image category. After determining the class center, the difference between the image features of each image in each image category and the class center is calculated, and the image with a difference exceeding a threshold is determined as a class outlier image, which refers to an image in the same image category with a large difference from the class center.

[0107] The aforementioned step 104 generates image data characteristics of the image data to be evaluated according to the image entropy of the plurality of images and the class representative image, including:

[0108] Image data characteristics of the image data to be evaluated are generated according to the image entropy of multiple images, class representative images and class outlier images in each image class. The image data characteristics also include: class outlier images in each image class.

[0109] The image data characteristics generated in the embodiment of the present application also include category outlier images in each image category, and a more comprehensive description of the image data to be evaluated can be achieved through the category outlier images in each image category.

[0110] Furthermore, in some embodiments of the present application, in addition to executing the aforementioned steps 101 to 103, the method provided in the embodiments of the present application further includes:

[0111] Image clarity detection is performed on the multiple images respectively according to the image features of the multiple images to obtain image clarity results of the multiple images.

[0112] Among them, a clarity detection technology can be used to take the whole image feature as input to obtain the image clarity result of each image, for example, the image clarity result can include an image clarity score. According to the above clarity detection technology, the operation can be repeated for each of the multiple images to obtain the image clarity result of each image.

[0113] in,

[0114] The aforementioned step 104 generates image data characteristics of the image data to be evaluated according to the image entropy of the plurality of images and the class representative image, including:

[0115] Image data characteristics of the image data to be evaluated are generated according to the image entropy of the multiple images, the class representative images and the image clarity results of the multiple images. The image data characteristics also include: the image clarity results of the multiple images.

[0116] like Figure 2 As shown, an image data transaction database provided by an embodiment of the present application may include image data characteristics, and the image data characteristics may include at least one of the following: image entropy of multiple images, class representative images, and image clarity results of multiple images.

[0117] The image data characteristics generated in the embodiment of the present application also include image clarity results of multiple images, and the image clarity results of multiple images can be used to accurately describe the clarity of the image data to be evaluated.

[0118] Furthermore, in some embodiments of the present application, the image data characteristics of the image data to be evaluated are generated according to the image entropy of the multiple images, the class representative images and the image clarity results of the multiple images, including:

[0119] identifying clarity outlier images from the multiple images according to image clarity results of the multiple images;

[0120] Image data characteristics of the image data to be evaluated are generated according to the image entropy, class representative images and clarity outlier images of the plurality of images. The image data characteristics specifically include: clarity outlier images.

[0121] After obtaining the image clarity results of the multiple images, the clarity mean is determined according to the image clarity results of the multiple images. After determining the clarity mean, the difference between the image clarity result of each image and the clarity mean is calculated, and the image whose difference exceeds a threshold is determined as a clarity outlier image, which refers to an image whose difference between the image clarity result and the clarity mean is too large.

[0122] The image data characteristics generated in the embodiment of the present application also include clarity outlier images, through which a more comprehensive description of the image data to be evaluated can be achieved.

[0123] Furthermore, in some embodiments of the present application, in addition to executing the aforementioned steps 101 to 103, the method provided in the embodiments of the present application further includes:

[0124] Obtaining image feature means of the multiple images according to the image features of the multiple images;

[0125] The outlier images in the multiple images are determined according to the difference between the image feature of each image in the multiple images and the mean value of the image features.

[0126] After the image features of the multiple images are acquired, the feature mean is determined according to the image features of the multiple images. After the feature mean is determined, the difference between the image features of each image and the feature mean is calculated, and the image whose difference exceeds the threshold is determined as an outlier image among the multiple images. The outlier image among the multiple images can also be called a feature outlier image, which refers to an image whose image feature has a large difference with the feature mean.

[0127] Wherein, step 104 generates image data characteristics of the image data to be evaluated according to the image entropy of the plurality of images and the class representative image, including:

[0128] Image data characteristics of the image data to be evaluated are generated according to the image entropy of the multiple images, the class representative images and the outlier images in the multiple images. The image data characteristics also include: outlier images in the multiple images.

[0129] The image data characteristics generated in the embodiment of the present application also include outlier images among the multiple images, and a more comprehensive description of the image data to be evaluated can be achieved through the outlier images among the multiple images.

[0130] like Figure 3 As shown, in the embodiment of the present application, the outlier image may include at least one of the following: an outlier image in multiple images, a category outlier image, and a definition outlier image. Please refer to the description of various outlier images in the above content, which will not be repeated here.

[0131] Through the above examples, it can be seen that in the embodiment of the present application, the feature data of multiple images are first extracted from the image data to be evaluated to obtain the image features of the multiple images; then, multi-scale image entropy evaluation is performed on the multiple images according to the image features of the multiple images to obtain the image entropy of the multiple images; and class representative images are found from the multiple images according to the image features of the multiple images; finally, image data characteristics of the image data to be evaluated are generated according to the image entropy of the multiple images and the class representative images, and the image data characteristics include: image entropy of the multiple images and class representative images. In the embodiment of the present application, the image data characteristics of the image data to be evaluated can be generated by the image entropy of the multiple images and the class representative images. Therefore, the image data characteristics output in the embodiment of the present application include: image entropy of the multiple images and class representative images. The class representative images can make a concise summary of the image data to be evaluated and represent the data conditions of the multiple images. The image entropy can represent the amount of information of the image data to be evaluated. Therefore, the embodiment of the present application can achieve the extraction of high-quality image data characteristics.

[0132] In order to make the technical solution of the present application clearer and easier to understand, the method for extracting image data characteristics of the present application is described in detail below in conjunction with a specific image data characteristics extraction scenario.

[0133] like Figure 2 As shown, it is a schematic diagram of a scene for implementing a framework for extracting image data characteristics provided by an embodiment of the present application, which mainly includes the following steps 1-7, which are described in detail below:

[0134] Step 1: Obtain image data to be evaluated.

[0135] The image data to be evaluated may include an image data set consisting of multiple images. The amount of image data to be evaluated is large, and it is difficult to evaluate the data characteristics by manually processing the images. In the embodiment of the present application, the multiple images in the image data to be evaluated may be related or independent of each other, which is not limited here.

[0136] Step 2: Image feature extraction.

[0137] Select a deep image classification model with a small number of parameters to avoid excessive calculations. Load the model pre-training weights, input all images in the image data set into the deep image classification model, and obtain the image features corresponding to each image, which can also be called the image vectorization result.

[0138] For example, the vector dimension of the last layer in the model is the smallest. To simplify subsequent calculations, only the vector of the last layer is selected as the image feature to obtain the feature vector of the last layer of the model corresponding to each image. For example, the feature vector of the last layer of the model corresponding to each image can be expressed as x i , as an image feature, the image feature can also be called an image vectorization result. In the embodiment of the present application, each image corresponds to an image feature, and the image feature can be in vector form.

[0139] Step 3: Image clustering.

[0140] Clustering-related technical methods are used to perform clustering with the vectorized results of the image as input to obtain the category to which each image vector belongs.

[0141] Step 4: Image clarity detection.

[0142] The clarity detection related technical methods are adopted, with the whole image as input to obtain the clarity score of the image, and the operation is repeated for each image in the picture collection to obtain the clarity score of each image.

[0143] For example, using the Laplace template to calculate image clarity, each image i can be calculated to obtain a numerical score, denoted as s i . Assume that there are N images in the data set, and the average clarity is The variance is

[0144] Step 5: Multi-scale image entropy evaluation.

[0145] Among them, the grayscale value range of the image is 0 to 255. Suppose the image has N g Gray levels, gray levels divide the gray value range equally. g One of the gray levels is n g , then n g Satisfies the following relationship: 0≤n g ≤N g , calculate the grayscale distribution of image pixels at grayscale level n g The probability of p(n g ), the image entropy is

[0146] The image entropy reflects the amount of information contained in an image. The larger the image entropy, the greater the amount of information contained in the image. In the embodiments of the present application, the spatial distribution of grayscale also needs to be considered when calculating the image entropy. A picture with practical significance has the following characteristics: at a microscopic scale, the grayscale changes little; but at a macroscopic scale, the grayscale will have certain changes. In the embodiments of the present application, the method of calculating the image entropy at multiple scales is adopted. According to the multi-scale image entropy, multi-scale image entropy anomaly pictures can be generated. For example, the multi-scale image entropy anomaly pictures may include meaningless noise pictures or blank pictures.

[0147] For example, let the length and width of a picture be H and W pixels respectively. The picture is evenly divided into blocks with a length and width of H / N s and W / N s respectively. N s represents the size of the image block. For example, N s can be an integer ranging from 1 to 10. For these 10 block division methods, calculate the information entropy of each block under each block division method, and calculate the average information entropy of the blocks for each block division method. Denote this information entropy as Ent(N s ).

[0148] For a picture with practical significance, when N s is small, the block size is large, the grayscale change is large, and Ent(N s ) should be large. When N s is large, the block size is small, the grayscale change is small, and Ent(N s ) should be small. For a picture, if there exists a certain N s , for example, 1 ≤ N s ≤ 9, such that Ent(N s ) < Ent(N s +1), then the picture may be a meaningless noise picture or a blank picture.

[0149] In the embodiments of the present application, it is stipulated that if there exists an N s such that Ent does not satisfy the trend of continuously decreasing, it can be determined that the image may be a meaningless image. It can be understood that most of the pictures selected based on the above rules are meaningless, but there are also a small number of meaningful ones. Therefore, there are various implementation manners for the description of the image data characteristics in the embodiments of the present application, and it is not limited to meaningless images.

[0150] For example, Ent(N1) > Ent(N2) > Ent(N3) < Ent(N4) < Ent(N5), there exists an N s = 3 such that Ent(N3) < Ent(N4), and it is considered that the image may be a meaningless image.

[0151] It should be noted that in step 5, N s The value can be 1 to 10, 1 to 100, 1 to 1000, etc.

[0152] Step 6: Find the representative image of the class.

[0153] In step 3, the category to which each image belongs has been obtained. For images belonging to the same category c, the vectorized representation x of the image features can be calculated. i The mean value E(x i ), take the mean as the class center, and record the distance between the image feature and the class center as d i (c), record x i Each component of For example, Euclidean distance Find the 5 images closest to the center of the class as class representative images. These images can generally represent the content of all images in the category.

[0154] It should be noted that there are many clustering techniques that can be used in the embodiments of the present application. Taking k-means clustering as an example, multiple rounds of iterative calculations are required, and each round of iteration requires the calculation of several class centers and Euclidean distances. Here, 1, 5, 10, or 20 images can be selected as class representative images according to the actual application scenario.

[0155] It should be noted that in the above step 6, the distance measurement method is not limited to Euclidean distance, and may also be Manhattan distance, Chebyshev distance, etc.

[0156] Step 7: Find outlier images.

[0157] Assume that there are N images in the image data set in step 1, and the vector representation of the image features is x i And the vector mean of all image features E(x i ) i The variance of For a certain image, when the following conditions are met (d i -E(d i )) 2 When >3D, the image i is considered to be an outlier image, and D is the variance of the distance, which is calculated as above.

[0158] Assume that there are N image categories c in step 3 c images, the distance d between the image features and the class center in this category i The variance of (c) is D c , when (d i (c)-E(d i (c))) 2 >3D c , the image is considered to be an outlier image.

[0159] Assume that the clarity scores of all images obtained in step 4 are Sh(i), and the variance D sh for Calculate the degree of deviation d between the clarity score of each image and the mean clarity score Sh (i) = (Sh(i) - E(Sh(i))) 2 , if d Sh (i)>3D Sh , the image is considered to be a clarity outlier image. Among them, the outlier image is quite different from most images in the image data set, and may be an image of poor quality or irrelevant to the category.

[0160] It should be noted that in the above step 7, the outlier images are determined based on 3 times the variance, and may also be based on 5 times the variance, 3 times the standard deviation, etc.

[0161] The image data characteristics are generated through steps 1-7 provided in the aforementioned embodiment of the present application. The image data characteristics may include the following contents:

[0162] (1) Five representative images of each image category are obtained from step 6. The image data features include representative images of each category, which is equivalent to a concise summary of the image data information, so that the data buyer can roughly understand the data situation with representative images without knowing all the information.

[0163] (2) Outlier images, including outlier images in the overall image set, outlier images in the category, and clarity outlier images, are derived from step 7; the image data characteristics include outlier images, which cannot be represented, so the buyer can have a more comprehensive understanding of the data set.

[0164] (3) The mean and variance of the image clarity scores in the image set are derived from step 4. The image data characteristics include the mean and variance of the image clarity scores. The higher the mean, the clearer the image, and the smaller the variance, the more stable the image clarity changes. This content can be used to evaluate the quality of image data clarity.

[0165] (4) Multi-scale image entropy anomaly pictures, which come from step 5. The image data features include multi-scale image entropy anomaly pictures. These images may be meaningless images. If there are too many such images, the value of the image dataset may be low.

[0166] It can be seen from the examples of the aforementioned embodiments that the embodiments of the present application use a pre-trained image model for feature extraction and do not involve model construction. Therefore, it does not require large-scale computing power, data labeling, etc., and the cost is low.

[0167] It is possible to achieve a high-quality description of the image data to be traded. For example, in a data trading scenario, without transferring the data, it is possible for the data buyer to understand the data quality and content. The image data characteristics given in the embodiments of the present application are all summarized and concise, which is convenient for the data buyer to quickly grasp the data situation, and can enable the buyer who needs to purchase the data to quickly determine the image data characteristics, thereby determining the purchase intention.

[0168] See also Figure 5 As shown, an image data feature extraction device provided in an embodiment of the present application mainly includes the following components:

[0169] A feature extraction module 501 is used to extract feature data of multiple images from the image data to be evaluated, so as to obtain image features of the multiple images;

[0170] An image entropy evaluation module 502 is used to perform multi-scale image entropy evaluation on the multiple images according to the image features of the multiple images to obtain image entropies of the multiple images;

[0171] A class representative image search module 503, used to search for a class representative image from the multiple images according to image features of the multiple images;

[0172] The data characteristic generating module 504 is used to generate the image data characteristics of the image data to be evaluated according to the image entropy of the multiple images and the class representative image, wherein the image data characteristics include: the image entropy of the multiple images and the class representative image.

[0173] By way of example, the aforementioned embodiments are used to illustrate that, first, feature data of multiple images are extracted from the image data to be evaluated to obtain image features of the multiple images; then, multi-scale image entropy evaluation is performed on the multiple images according to the image features of the multiple images to obtain image entropy of the multiple images; and class representative images are found from the multiple images according to the image features of the multiple images; finally, image data characteristics of the image data to be evaluated are generated according to the image entropy of the multiple images and the class representative images, and the image data characteristics include: image entropy of the multiple images and class representative images. In the embodiment of the present application, image data characteristics of the image data to be evaluated can be generated by image entropy of the multiple images and class representative images. Therefore, the image data characteristics output in the embodiment of the present application include: image entropy of the multiple images and class representative images. The class representative images can make a concise summary of the image data to be evaluated and represent the data conditions of the multiple images. Image entropy can represent the amount of information of the image data to be evaluated. Therefore, the embodiment of the present application can achieve the extraction of high-quality image data characteristics.

[0174] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0175] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0176] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0177] Those skilled in the art can understand that the flowchart shown in the figure is only an example in which the embodiments of the present application can be implemented, and the scope of application of the embodiments of the present application is not limited by any aspect of the flowchart.

[0178] In the several embodiments provided in the present application, it should be understood that the disclosed methods, devices and equipment can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0179] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0180] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0181] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for extracting characteristics of image data, characterized in that: include: Extracting feature data of a plurality of images from the image data to be evaluated respectively to obtain image features of the plurality of images; Performing multi-scale image entropy evaluation on the multiple images respectively according to the image features of the multiple images to obtain image entropies of the multiple images; searching for a class representative image from the multiple images according to image features of the multiple images; Obtaining image feature mean values ​​of the multiple images according to the image features of the multiple images; Determine an outlier image among the multiple images according to a difference between an image feature of each image in the multiple images and a mean value of the image features; generating image data characteristics of the image data to be evaluated according to the image entropies of the multiple images and the class representative image, the image data characteristics comprising: image entropies of the multiple images and the class representative image; in, The step of generating the image data characteristics of the image data to be evaluated according to the image entropy of the plurality of images and the class representative image comprises: Image data characteristics of the image data to be evaluated are generated according to the image entropy of the multiple images and the class representative image. The image data characteristics also include: outlier images in the multiple images.

2. The method according to claim 1, characterized in that The method further includes: performing image clarity detection on the multiple images respectively according to the image features of the multiple images to obtain image clarity results of the multiple images; in, The step of generating the image data characteristics of the image data to be evaluated according to the image entropy of the plurality of images and the class representative image comprises: Image data characteristics of the image data to be evaluated are generated according to the image entropies of the multiple images, the class representative images and the image clarity results of the multiple images, and the image data characteristics also include: the image clarity results of the multiple images.

3. The method according to claim 2, characterized in that The step of generating the image data characteristics of the image data to be evaluated according to the image entropy of the multiple images, the class representative image, and the image clarity results of the multiple images includes: identifying clarity outlier images from the plurality of images according to the image clarity results of the plurality of images; Image data characteristics of the image data to be evaluated are generated according to the image entropies of the multiple images, the class representative images and the clarity outlier images, and the image data characteristics specifically include: the clarity outlier images.

4. The method according to claim 1, characterized in that: The step of searching for a class representative image from the multiple images according to the image features of the multiple images includes: Performing image clustering processing according to the image features of the multiple images to obtain multiple image categories; According to each of the plurality of image categories, class representative images corresponding to the plurality of image categories are generated.

5. The method according to claim 4, characterized in that The method further comprises: For each image category in the plurality of image categories, determining a class center of each image category; Determine the category outlier images in each image category according to the difference between the image feature of each image in each image category and the category center; in, The step of generating the image data characteristics of the image data to be evaluated according to the image entropy of the plurality of images and the class representative image comprises: Image data characteristics of the image data to be evaluated are generated according to the image entropy of the multiple images, the class representative images and the category outlier images in each image category, and the image data characteristics also include: category outlier images in each image category.

6. The method according to claim 1, characterized in that The performing multi-scale image entropy evaluation on the multiple images respectively according to the image features of the multiple images comprises: For each of the plurality of images, determining image blocks of each image at a plurality of image block scales; According to the grayscale distribution probability of the image grayscale feature of each image block in the image grayscale feature distribution interval at various image block scales, the image entropy of each image at various image block scales is obtained.

7. The method according to claim 1, characterized in that The method further comprises: The image data characteristics are stored in an image data transaction database.

8. A device for extracting characteristics of image data, characterized in that: The image data characteristic extraction device comprises: Memory for storing computer programs or computer instructions; A processor is used to execute the computer program or computer instructions stored in the memory, so that the image data characteristic extraction device performs the method according to any one of claims 1 to 7.

9. A computer storage medium for storing a computer program, wherein when the computer program is executed, it is used to implement the method according to any one of claims 1 to 7.

Images