Image sample selection method and apparatus, electronic device, and storage medium

Abstract

Embodiments of the present application provide a kind of image sample selection method, device, electronic equipment and storage medium, belong to active learning technical field.The method includes: obtaining image sample data set and the category data set corresponding to image sample data set, image sample data set includes multiple initial image samples to be selected, category data set includes multiple known category information;From different feature dimensions, the feature extraction operation of initial image sample is carried out to obtain corresponding image feature;The posterior probability value that image feature under different feature dimensions and each known category information match is calculated, the category of initial image sample is predicted according to posterior probability value, obtains multiple prediction category information;Image evidence information is obtained by the prediction category information indicating different belonging category is counted, if the value of image evidence information exceeds the first threshold value preset, it is determined that the corresponding initial image sample is target image sample.The quality of the image sample selected by the present application can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of active learning, and particularly relates to an image sample selection method and device, electronic equipment and a storage medium. BACKGROUND

[0002] An image classification model is used to identify target content in an input image and assign the target content to a predefined category. In order to improve the training efficiency and quality of the image classification model, valuable image samples need to be selected from a large number of samples.

[0003] In related technologies, samples with high information entropy characteristics or low information entropy characteristics are selected as selected image samples from a large number of samples, where information entropy refers to the amount of information in data. However, image samples with high information entropy characteristics are most likely to be noise samples, which cannot improve or even reduce the training effect of the image classification model; image samples with low information entropy characteristics contain too little valuable training information, which also makes the training effect of the image classification model poor. That is, the image classification model has the problem of poor quality of selected image samples in the training process. SUMMARY

[0004] The main purpose of the embodiments of the present application is to provide an image sample selection method, device, electronic equipment and storage medium, which aims to improve the quality of selected image samples in the training process of the image classification model.

[0005] To achieve the above purpose, a first aspect of the embodiments of the present application provides an image sample selection method, which comprises:

[0006] obtaining an image sample data set and a category data set corresponding to the image sample data set, wherein the image sample data set comprises a plurality of initial image samples to be selected, and the category data set comprises a plurality of known category information;

[0007] performing a feature extraction operation on the initial image samples from different feature dimensions to obtain corresponding image features;

[0008] calculating posterior probability values of the image features under different feature dimensions and each known category information, and predicting the category to which the initial image samples belong according to the posterior probability values to obtain a plurality of predicted category information;

[0009] statistically obtaining image evidence information from the predicted category information indicating different categories, and determining that the corresponding initial image sample is a target image sample when the value of the image evidence information exceeds a preset first threshold.

[0010] In some embodiments, the posterior probability value of matching the image feature in the different feature dimension to each known category information is calculated, including:

[0011] Based on all image features corresponding to each initial image sample, a feature mean value representing the average expression of the initial image sample is calculated, and a feature variance value representing the difference between each image feature and the feature mean value is calculated;

[0012] Based on a preset first constant value and the feature variance value, a first multiplication value is calculated, and based on a preset second constant value, the feature mean value and the feature variance value, a second multiplication value is calculated;

[0013] Based on the first multiplication value, the second multiplication value and a preset third constant value, the posterior probability value representing the matching of the image feature to each known category information is calculated.

[0014] In some embodiments, the belonging category of the initial image sample is predicted according to the posterior probability value, and a plurality of predicted category information is obtained, including:

[0015] Based on the plurality of posterior probability values of the image feature, the known category information corresponding to the largest numerical value of the posterior probability value is determined as the predicted category information of the initial image sample in the current feature dimension;

[0016] Based on the predicted category information corresponding to each image feature, the plurality of predicted category information is obtained.

[0017] In some embodiments, after calculating the posterior probability value of matching the image feature in the different feature dimension to each known category information, it further includes:

[0018] A preset prior probability value is obtained, and the prior probability value is used to represent the probability of the initial image sample belonging to any known category information;

[0019] The product of the prior probability value and the posterior probability value is calculated to obtain a first probability value;

[0020] The belonging category of the initial image sample is predicted according to the first probability value, and a plurality of predicted category information is obtained;

[0021] The predicted category information indicating different belonging categories is counted to obtain image evidence information, and when the numerical value of the image evidence information exceeds a preset second threshold value, the corresponding initial image sample is determined as a target image sample.

[0022] In some embodiments, after obtaining the preset prior probability value, it further includes:

[0023] A preset total probability value is obtained, and the total probability value is used to represent the sum of the probability of the initial image sample including each image feature;

[0024] calculating a ratio between the first probability value and the total probability value to obtain a second probability value;

[0025] predicting a category to which the initial image sample belongs according to the second probability value to obtain a plurality of predicted category information;

[0026] statistically obtaining image evidence information from the predicted category information indicating different categories, and determining that the corresponding initial image sample is a target image sample when a value of the image evidence information exceeds a preset third threshold.

[0027] In some embodiments, after the image evidence information is obtained by statistically obtaining the predicted category information indicating different categories, the method further includes:

[0028] selecting the corresponding initial image sample in order from large to small according to a value of each image evidence information;

[0029] until the number of selected initial image samples reaches a preset number threshold, obtaining the target image sample.

[0030] In some embodiments, after determining that the corresponding initial image sample is the target image sample, the method further includes:

[0031] obtaining a target model to be trained, and adding a training label to the target image sample;

[0032] training the target model based on the target image sample with the training label to obtain a trained target model.

[0033] To achieve the above object, a second aspect of the embodiment of the present application provides a device for selecting an image sample, the device comprising:

[0034] an obtaining module, configured to obtain an image sample dataset and a category dataset corresponding to the image sample dataset, wherein the image sample dataset comprises a plurality of initial image samples to be selected, and the category dataset comprises a plurality of known category information;

[0035] a feature extraction module, configured to perform a feature extraction operation on the initial image sample from different feature dimensions to obtain corresponding image features;

[0036] a prediction module, configured to calculate a posterior probability value of matching the image features under different feature dimensions with each known category information, and predict a category to which the initial image sample belongs according to the posterior probability value to obtain a plurality of predicted category information;

[0037] a target image sample determination module, configured to statistically obtain image evidence information from the predicted category information indicating different categories, and determine that the corresponding initial image sample is a target image sample when a value of the image evidence information exceeds a preset first threshold.

[0038] To achieve the above object, a third aspect of embodiments of the present application provides an electronic device, comprising a memory and a processor, the memory stores a computer program, and the processor implements the method of the first aspect when executing the computer program.

[0039] To achieve the above object, a fourth aspect of embodiments of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method of the first aspect.

[0040] The present application provides an image sample selection method and device, electronic equipment and storage medium, which obtains an image sample data set and a category data set corresponding to the image sample data set, wherein the image sample data set includes a plurality of initial image samples to be selected, and the category data set includes a plurality of known category information; performs feature extraction on the initial image samples from different feature dimensions to obtain corresponding image features; calculates the posterior probability value of the matching between the image features under different feature dimensions and each known category information, and predicts the category to which the initial image sample belongs according to the posterior probability value to obtain a plurality of predicted category information; in this way, the possible classification results of the image features of the initial image samples under different feature dimensions can be more comprehensively considered, compared with the method of relying only on a single feature or a simple classifier, the embodiments of the present application can more accurately determine the category to which the initial image sample belongs based on a plurality of feature dimensions, so as to select more valuable samples from them; then, the predicted category information indicating different categories is counted to obtain image evidence information, and when the value of the image evidence information exceeds a preset first threshold, the corresponding initial image sample is determined as a target image sample; when the value of the image evidence information exceeds the preset first threshold, it indicates that a certain number of predicted category information conflicts, which indicates that the initial image sample has a high amount of information and has a high uncertainty for the image classification model, so the initial image sample can be selected as a valuable target image sample, thereby improving the quality of the image samples selected in the training process of the image classification model. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 is an optional application scenario diagram of the image sample selection device provided by the embodiments of the present application;

[0042] Figure 2 is an optional flowchart of the image sample selection method provided by the embodiments of the present application;

[0043] Figure 3 is an implementation flowchart of step 103 in Figure 2 is an implementation flowchart of step 103 in

[0044] Figure 4 is Figure 2 another implementation flowchart of step 103 in

[0045] Figure 5 is an optional initial image sample schematic diagram of the image sample selection method provided by the embodiments of the present application;

[0046] Figure 6 is Figure 2 an implementation flowchart after step 103 in

[0047] Figure 7 is Figure 6 an implementation flowchart after step 401 in

[0048] Figure 8 is Figure 2 an implementation flowchart of step 104 in

[0049] Figure 9 is an implementation flowchart after step 104 in Figure 2

[0050] Figure 10 is an optional structure schematic diagram of the image sample selection device provided by the embodiments of the present application;

[0051] Figure 11 is a hardware structure schematic diagram of the electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0052] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0053] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a manner different from the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification and claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0055] First, the terms involved in the present application are analyzed:​

[0056] Convolutional neural network is an important model in the field of artificial intelligence (AI), which contains a feedforward neural network for convolutional computation and has a deep structure. It realizes automatic learning and feature extraction of data by simulating the structure and function of human brain neural network. The core idea is to extract features through convolution and pooling operations, map the input data to a high-dimensional feature space, and then classify or regress the features through a fully connected layer.

[0057] Artificial intelligence: is a new technical science of theory, method, technology and application system for simulating, extending and expanding human intelligence; artificial intelligence is a branch of computer science, artificial intelligence tries to understand the essence of intelligence, and produce a new intelligent machine that can react in a similar way to human intelligence. The research in this field includes robots, language recognition, image recognition, natural language processing and expert systems. Artificial intelligence can simulate the information process of human consciousness and thinking. Artificial intelligence is also the theory, method, technology and application system of using digital computer or digital computer controlled machine to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain the best results.

[0058] The image classification model is used to identify the target content in the input image and assign it to a predefined category. In order to improve the training efficiency and quality of the image classification model, valuable image samples need to be selected from a large number of samples.

[0059] In the related art, samples with high information entropy characteristics or low information entropy characteristics are selected as selected image samples from a large number of samples, wherein information entropy refers to the amount of information in the data. However, image samples with high information entropy characteristics are more likely to be noise samples, which cannot improve or even reduce the training effect of the image classification model; image samples with low information entropy characteristics contain too little valuable training information, which also makes the training effect of the image classification model poor. That is, the selected image samples have poor quality in the training process of the image classification model.

[0060] Therefore, the embodiments of the present application provide a method and device for selecting image samples, an electronic device and a storage medium, which aims to improve the quality of the selected image samples.

[0061] The method and device for selecting image samples provided by the embodiments of the present application are specifically described by the following embodiments. First, the method for selecting image samples in the embodiments of the present application is described.

[0062] It should be noted that, in the embodiments of the present application, when it is necessary to obtain user's consent or agreement according to user's basic information or information related to user's characteristics such as user's identity, the collection, use and processing of the data will comply with relevant laws, regulations and standards. In addition, when the embodiments of the present application need to obtain sensitive personal information of the user, the separate consent or separate agreement of the user will be obtained first, and then the necessary data for the normal operation of the embodiments of the present application will be obtained, for example, when the embodiments of the present application obtain the initial image sample taken by the user, the consent of the relevant user will be obtained first, or if the initial image sample obtained involves privacy including but not limited to the user's portrait privacy, personal information and the like, the consent of the relevant user will be obtained first, otherwise the initial image sample used in the embodiments of the present application cannot be obtained.

[0063] The image sample selection method, device, electronic equipment and storage medium provided by the embodiments of the present application are described in detail as follows. First, the application scenario of the image sample selection device (for the convenience of description, hereinafter also referred to as "selection device") in the embodiments of the present application is described, as shown in Figure 1 Figure 1 is a schematic diagram of an optional application scenario of the image sample selection device provided by the embodiments of the present application. First, the obtained initial image sample and known category information are input into the selection device provided by the embodiments of the present application. The selection device can perform feature extraction operation on the initial image sample from different feature dimensions, and predict the category to which the corresponding image feature belongs under different feature dimensions, so as to select the initial image sample with multiple conflicting predicted category information (image evidence information) as the target image sample. It can be understood that such target image sample represents that there are multiple possibilities when the image classification model predicts the image content, which indicates that such sample has certain training value, thereby improving the quality of the selected image sample.

[0064] After understanding the application scenario of the training device proposed in the present application, the image sample selection method proposed in the embodiments of the present application is described.

[0065] In the embodiments of the present application, the selection device will be described from the dimension. The selection device can be integrated in a computer device, such as a server. As shown in Figure 2 Figure 2 is an optional flowchart of the image sample selection method provided by the embodiments of the present application, Figure 2 ​​The method in the method can include but is not limited to the following steps 101 to step 104, when the selection device executes the selection method of the image sample (for the convenience of description, it can also be referred to as "selection method" simply), the specific process is as follows, it needs to be pointed out that the embodiment of the present application is not limited to the selection method of the image sample Figure 2 The order of steps 101 to 104 in the method is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0066] Step 101, obtaining an image sample data set and a category data set corresponding to the image sample data set, wherein the image sample data set includes a plurality of initial image samples to be selected, and the category data set includes a plurality of known category information.

[0067] The step 101 is described in detail below.

[0068] Wherein, the initial image sample refers to a sample including different image contents, according to different image classification tasks, the value of the same initial image sample to the corresponding image classification model is also different. If an initial image sample can help improve the performance of the image classification model obtained by training, the initial image sample is called high value.

[0069] Further, the initial image sample can be obtained by a user in real time; or the initial image sample can be obtained by an open image data set; or the initial image sample can be obtained by intelligent synthesis of image editing software; of course, the present application embodiment does not limit the way of obtaining the initial image sample, which can be set according to actual situation.

[0070] Further, the open image data set can be a large visual data set (ImageNet data set) for visual object recognition software research, Canadian Institute for Advanced Research (Canadian Institute for Advanced Research, CI FAR data set) and other data sets that can contain images of target objects to be recognized and classified; wherein, the ImageNet data set further includes a tiny ImageNet data set, and the CI FAR data set further includes CI FAR10, CI FAR100 and other data sets. That is, the image sample data set used by the selection device when executing the selection method can be selected according to actual situation, and the present application embodiment does not make specific limitation.

[0071] The plurality of known category information included in the category data set is used to indicate the belonging category of the target object included in the initial image sample. If the image classification model to be trained has certain image classification capability, the category of the target object that it can recognize is called capability category information. It should be noted that the capability category information is different from the known category information. For example, an image classification model A has certain image classification capability, but it can only recognize and classify the target object as a dog from the input image, and the dog is called capability category information. In order to improve the image classification capability of the image classification model A, the image sample data set and the category data set are obtained in the embodiment of the application. The plurality of known category information included in the category data set is used to indicate the belonging category of the target object corresponding to the plurality of initial image samples in the image sample data set. The known category information includes dogs, cats, and tigers.

[0072] Further, the number of initial image samples and the range of corresponding known category information can be adaptively adjusted according to actual conditions, and the embodiment of the application does not limit this.

[0073] Step 102: performing a feature extraction operation on the initial image sample from different feature dimensions to obtain corresponding image features.

[0074] Step 102 is described in detail below.

[0075] The feature dimension refers to the image attribute constructed or recognized in the feature extraction process. Different feature dimensions include but are not limited to color, texture, shape, spatial relationship, etc. For different feature dimensions, the feature extraction operation performed by the selection device is also different accordingly. For example, when the feature dimension is the color feature dimension, the feature extraction operation can be color histogram statistics, color moment calculation, color space conversion, etc. When the feature dimension is the texture feature dimension, the feature extraction operation can be a gray level co-occurrence matrix, local binary calculation, wavelet transform, etc.

[0076] Taking the color dimension as an example, the feature dimension is different color channels (Color Channel), which are red channel (Red Channel), green channel (Green Channel) and blue channel (Blue Channel). Specifically, the feature extraction operation is performed on the initial image sample from different color channels, and different image features are correspondingly obtained:

[0077] (1) Red image feature: based on the red channel, the information related to the red color is captured from the initial image sample to obtain the red image feature;

[0078] (2) Green image feature: based on the green channel, the information related to the green color is captured from the initial image sample to obtain the green image feature;

[0079] (3) Blue image features: based on the blue channel, capture information related to blue from the initial image samples, to obtain blue image features.

[0080] Alternatively, a convolutional neural network is used to perform feature extraction operations on the initial image samples, and as the initial image samples pass through different convolutional layers, activation functions and pooling layers of the convolutional neural network, the three primary color channels will gradually be converted into more "feature channels". Here, the "feature channels" do not directly refer to colors visible to the naked eye, but rather abstract feature representations of the initial image samples at different processing layers of the convolutional neural network. For example, the last processing layer of a residual convolutional neural network model (ResNet18) contains 256 channels.

[0081] It should be noted that the specific image attributes represented by the feature dimensions and the corresponding specific dimension numbers can be set according to actual conditions, and the embodiments of the present application do not limit this.

[0082] Step 103, calculate the posterior probability value of the image features under different feature dimensions and each known category information, and predict the category to which the initial image sample belongs according to the posterior probability value, to obtain multiple predicted category information.

[0083] The step 103 is described in detail below.

[0084] The posterior probability value is used to represent the probability of the image features given the known category information. After obtaining the posterior probability value, the category to which the target object in the initial image sample belongs is predicted to obtain the predicted category information, so as to more comprehensively consider the possible classification results of the image features of the initial image sample under different feature dimensions. Compared with methods that rely only on a single feature or a simple classifier, the embodiments of the present application can more accurately determine the category to which the initial image sample belongs based on multiple feature dimensions, so as to screen out more valuable samples.

[0085] In some embodiments, Figure 3 is Figure 2 An implementation flowchart of step 103 in the step 103, calculating the posterior probability value of the image features under different feature dimensions and each known category information, includes the following steps 201 to 203:

[0086] Step 201, based on all image features corresponding to each initial image sample, calculate the feature mean value representing the average expression of the features of the initial image sample, and calculate the feature variance value representing the difference between each image feature and the feature mean value.

[0087] Step 202, based on the preset first constant and the characteristic variance value, a first multiplication value is calculated, and based on the preset second constant, the characteristic mean value and the characteristic variance value, a second multiplication value is calculated.

[0088] Step 203, based on the first multiplication value, the second multiplication value and the preset third constant, a posterior probability value representing the matching of the image feature and the known category information is calculated.

[0089] The steps 201 to 203 are described in detail as follows.

[0090] In some embodiments, the posterior probability value P(x j |Y) corresponding to each image feature is calculated according to the following formula <1>:

[0091]

[0092] Wherein μ is the characteristic mean value, which is obtained by adding all image features and dividing by the total number of image features; σ is the characteristic variance value, which is calculated by calculating the difference between each image feature and the characteristic mean value, then calculating the square result of the difference, and finally determining the average value of all square results as the characteristic variance value; x j represents the jth image feature; e is a natural constant.

[0093] Wherein, is a preset first constant, is a first multiplication value; is a second constant, is a second multiplication value; e is a preset third constant.

[0094] In some embodiments, Figure 4 is Figure 2 Another implementation flowchart of step 103 in the above method is provided, which predicts the category to which the initial image sample belongs according to the posterior probability value, to obtain a plurality of predicted category information, including the following steps 301 to 302:

[0095] Step 301, based on the plurality of posterior probability values of the image features, the known category information corresponding to the largest posterior probability value is determined as the predicted category information of the initial image sample under the current characteristic dimension.

[0096] Step 302, based on the predicted category information corresponding to each image feature, a plurality of predicted category information is obtained.

[0097] The steps 301 to 302 are described in detail as follows.

[0098] In some embodiments, after the posterior probability value of the matching degree of each image feature to each known category information is calculated, the category to which the initial image sample belongs is predicted based on the posterior probability value, and the known category information corresponding to the posterior probability value with the maximum value is taken as the predicted category information of the initial image sample in the current feature dimension. More specifically, the category to which the target object in the initial image sample belongs is predicted, and the target object can refer to the object with the largest area ratio in the initial image sample, or can refer to each object appearing in the initial image sample, which can be set according to actual conditions.

[0099] The predicted category information refers to a label used to indicate which category the corresponding initial image sample belongs to, such as Figure 5 as shown in Figure 5 is an optional initial image sample schematic diagram of the selection method of the image sample provided by the embodiments of the present application. In example A, the true category to which a certain initial image sample belongs should be "tiger", however, the predicted category information obtained by the embodiments of the present application in different feature dimensions includes tiger, cat and dog, that is, each image feature corresponds to a predicted category information, and therefore one initial image sample includes multiple predicted category information.

[0100] Step 104: The predicted category information indicating different categories is counted to obtain image evidence information, and when the value of the image evidence information exceeds a preset first threshold, the corresponding initial image sample is determined as a target image sample.

[0101] The step 104 is described in detail below.

[0102] The image evidence information is used to represent the number of predicted category information with different categories, for example, in the above example A, the image evidence information is 3. The greater the value of the image evidence information, the greater the number of mutually conflicting predicted category information, that is, the probability that the initial image sample belongs to different known categories in different feature dimensions is high.

[0103] The preset first threshold is used to represent the expected value of the predicted category information with the conflict characteristic, and when the value of the image evidence information exceeds the preset first threshold, it is indicated that a certain number of predicted category information have conflicts, and it is further indicated that the initial image sample has a high information amount, and has high uncertainty for the image classification model, and therefore the initial image sample can be selected as a valuable target image sample.

[0104] It should be noted that the specific value of the first threshold can be set according to the number of feature dimensions, and the embodiments of the present application do not limit this.

[0105] It can be understood that the finally selected multiple target image samples should be able to meet the balance between purity and information quantity; wherein, the purity refers to the type of target sample expected to be selected by the image classification model, for example, a certain image classification model expects to select the target sample type as "cat", then if the obtained initial image sample belongs to the category of "snake", the initial image sample will reduce the purity of the image sample data set; the information quantity refers to the number of conflicting prediction category information, which is used to represent the inconsistency of the same initial image sample when predicting in different feature dimensions. The multiple target image samples meeting the balance between purity and information quantity can help the trained image classification model to more accurately define the category boundary in the training process, and labeling these target image samples can reduce the waste of labeling resources while ensuring the training value.

[0106] In some embodiments, Figure 6 is Figure 2 An implementation flowchart after step 103 in the method 100, after calculating the posterior probability value of the image feature in different feature dimensions matching each known category information, further comprising steps 401 to 404:

[0107] Step 401, obtaining a preset prior probability value, the prior probability value is used to represent the probability size of the initial image sample belonging to any known category information in advance.

[0108] Step 402, calculating the product of the prior probability value and the posterior probability value to obtain a first probability value.

[0109] Step 403, predicting the category to which the initial image sample belongs according to the first probability value, to obtain multiple prediction category information.

[0110] Step 404, counting the prediction category information indicating different categories to obtain image evidence information, when the value of the image evidence information exceeds a preset second threshold, determining that the corresponding initial image sample is a target image sample.

[0111] The steps 401 to 404 are described in detail as follows.

[0112] The prior probability value refers to the probability size of the initial image sample belonging to the known category before extracting each image feature from the initial image sample; the prior probability value does not depend on any specific data or observation results occurring at present, but is obtained based on long-term experience accumulation, historical data or expert judgment and the like. It should be noted that in actual application, multiple methods may be combined to obtain more accurate and reliable prior probability values, which can be set according to actual conditions, and the embodiments of the present application do not limit this.

[0113] In some embodiments, by the following formula <2> A first probability value P is calculated so that the category of the initial image sample can be predicted based on the first probability value P:

[0114] P=P(Y)∏ j P(x j |Y) <2>

[0115] Where P(Y) is the prior probability value; P(x) j |Y) represents the posterior probability value; ∏ represents the multiplication sign, used to characterize the continuous product of a series of numbers. Thus, the first probability value takes into account both the prior probability of determining the category of the initial image sample before the observation data is available, and the posterior probability of determining the category of the initial image sample after the image features are observed, making it suitable for low feature dimensions.

[0116] Furthermore, after calculating the first probability value of the degree of matching between each image feature and each known category information, the category to which the initial image sample belongs is predicted based on the first probability value, and the known category information corresponding to the highest first probability value is taken as the category to which the corresponding image feature belongs.

[0117] The preset second threshold is used to characterize the expected predicted category information value with conflict characteristics. When the value of the image evidence information exceeds the preset second threshold, it indicates that a certain number of predicted category information have conflicted, which in turn indicates that the initial image sample has a high amount of information and a high degree of uncertainty for the image classification model. Therefore, the initial image sample can be selected as a valuable target image sample.

[0118] It should be noted that the specific value of the second threshold can be set according to the number of feature dimensions, and this application embodiment does not impose any restrictions on this.

[0119] In some embodiments, Figure 7 yes Figure 6 The implementation flowchart following step 401, after obtaining the pre-set prior probability value, also includes the following steps 501 to 504:

[0120] Step 501: Obtain the pre-set full probability value. The full probability value is used to characterize the sum of the probabilities of each image feature included in the pre-agreed initial image sample.

[0121] Step 502: Calculate the ratio between the first probability value and the total probability value to obtain the second probability value.

[0122] Step 503: Predict the category of the initial image sample based on the second probability value to obtain multiple predicted category information.

[0123] Step 504: Statistically determine the predicted category information of different categories to obtain image evidence information. When the value of the image evidence information exceeds the preset third threshold, determine the corresponding initial image sample as the target image sample.

[0124] Steps 501 to 504 are described in detail below.

[0125] In another alternative embodiment, the following formula can also be obtained based on Naive Bayes. <3> The second probability value P(Y|X) is calculated so that the category of the initial image sample can be predicted based on the second probability value P(Y|X):

[0126]

[0127] Wherein, P(x 1 ,x 2 ,…,x n The total probability (n) represents the sum of the probabilities of each image feature occurring, also known as the full probability value, where n represents the number of image features corresponding to each initial image sample. The full probability value can be obtained based on historical data. In the absence of sufficient data, an approximate full probability value can be obtained by relying on expert opinions. Of course, the method for obtaining the full probability value can be adaptively adjusted according to the actual situation, and this application does not limit this.

[0128] Furthermore, after calculating the second probability value of the degree of matching between each image feature and each known category information, the category to which the initial image sample belongs is predicted based on the second probability value, and the known category information corresponding to the highest second probability value is taken as the category to which the corresponding image feature belongs.

[0129] The preset third threshold is used to characterize the expected predicted category information value with conflict characteristics. When the value of the image evidence information exceeds the preset third threshold, it indicates that a certain number of predicted category information have conflicted, which indicates that the initial image sample has a high amount of information and a high degree of uncertainty for the image classification model. Therefore, the initial image sample can be selected as a valuable target image sample.

[0130] It should be noted that the specific value of the third threshold can be set according to the number of feature dimensions, and this application embodiment does not impose any restrictions on this.

[0131] In some embodiments, Figure 8 yes Figure 2 A flowchart of step 104 in the figure, after obtaining image evidence information by statistically indicating the predicted category information of different categories, also includes the following steps 601 to 602:

[0132] In step 601, the initial image samples are selected in descending order of the numerical values of the image evidence information.

[0133] In step 602, the target image samples are obtained when the number of the selected initial image samples reaches the preset number threshold.

[0134] The steps 601 to 602 are described in detail as follows.

[0135] In some embodiments, in addition to determining the target image samples according to the first threshold, the second threshold or the third threshold, the target image samples can also be selected by the preset number threshold. Specifically, the initial image features are sorted according to the numerical values of the corresponding image evidence information, and the initial image samples are selected in descending order of the numerical values according to the preset number threshold, and the target image samples are obtained when the number of the selected initial image samples reaches the preset number threshold.

[0136] It should be noted that the specific value of the number threshold can be set according to actual conditions, and the embodiments of the present application do not limit this.

[0137] For example, the image sample dataset includes an initial image sample a, an initial image sample b and an initial image sample c, wherein the image evidence information value of the initial image sample a is 3, the image evidence information value of the initial image sample b is 2, the image evidence information value of the initial image sample c is 1, and the number threshold is 2. Therefore, the target image samples can be determined as the initial image sample a and the initial image sample b.

[0138] In order to better understand the significant benefits of the image sample selection method proposed in the embodiments of the present application, the following comparative data are listed in Table 1. Table 1 shows the accuracy difference between the image sample selection method proposed in the embodiments of the present application and other methods when the same dataset is selected for image sample selection:

[0139] Table 1

[0140]

[0141] Among them, the comparison method includes: learning from open-set annotation (Learning from Open-Set Annotation, LfOSA), image matching method (Minimum Quadrilaterals for the Estimation of Corresponding Point sets, MQ-Net), randomized selection method (Randomized Algorithm, Random), uncertain information entropy selection method (Entropy), and certain information entropy selection method (Certainty).

[0142] It can be understood that when the image sample active selection operation is based on the same data set, the accuracy of the image sample selection method proposed in the embodiments of the present application is higher than that of other related methods, so that the quality of the image samples selected by the image classification model in the training process can be improved.

[0143] It should be noted that the data in Table 1 is obtained by performing sample active selection experiments on the CI FAR10-3 data set by each method. Of course, the data set used in the experiment can also be the CI FAR10-4 data set, the CI FAR10-5 data set, etc. It can be selected according to the actual situation, and the embodiments of the present application do not limit this.

[0144] In some embodiments, Figure 9 is Figure 2 An implementation flowchart after step 104 in the method 100 is shown in FIG. 7. After determining that the corresponding initial image sample is the target image sample, the following steps 701-702 are further included:

[0145] Step 701: obtaining a target model to be trained, and adding a training label to the target image sample.

[0146] Step 702: training the target model based on the target image sample with the training label to obtain a trained target model.

[0147] The steps 701-702 are described in detail as follows.

[0148] In some embodiments, after the selection of the target image sample is completed, a training label can be added to the selected target image sample, so that the target image sample with the training label can be used to train the target model to be trained later.

[0149] Further, the target image sample can be added with a training label in a manual manner, or added with a training label by using an automatic or semi-automatic model. The training label can be a category label or a numerical label with a mapping relationship, and the specific form of the label is not limited in the embodiments of the present application. It can be understood that, since the target image samples selected in the embodiments of the present application all have certain training value, thus, the waste of labeling resources caused by blindly labeling valueless samples is avoided, and the cost of data labeling is significantly reduced.

[0150] As shown in Figure 10 , Figure 10 is an optional structural schematic diagram of an image sample selection device provided by the embodiments of the present application, and the device comprises:

[0151] The acquisition module 801 is configured to acquire an image sample dataset and a category dataset corresponding to the image sample dataset, wherein the image sample dataset comprises a plurality of initial image samples to be selected, and the category dataset comprises a plurality of known category information.

[0152] The feature extraction module 802 is configured to perform a feature extraction operation on the initial image samples from different feature dimensions to obtain corresponding image features.

[0153] The prediction module 803 is configured to calculate a posterior probability value of matching the image features under different feature dimensions with each known category information, and predict the category to which the initial image sample belongs according to the posterior probability value to obtain a plurality of predicted category information.

[0154] The target image sample determination module 804 is configured to count the predicted category information indicating different categories to obtain image evidence information, and determine that the corresponding initial image sample is a target image sample when the value of the image evidence information exceeds a preset first threshold.

[0155] The application provides an image sample selection method and device, an electronic device and a storage medium. The method comprises the following steps: obtaining an image sample data set and a category data set corresponding to the image sample data set, wherein the image sample data set comprises a plurality of initial image samples to be selected, and the category data set comprises a plurality of known category information; performing feature extraction on the initial image samples from different feature dimensions to obtain corresponding image features; calculating a posterior probability value of matching the image features under different feature dimensions with the known category information, and predicting the category to which the initial image sample belongs according to the posterior probability value to obtain a plurality of predicted category information. In this way, the possible classification results of the image features of the initial image sample under different feature dimensions can be comprehensively considered. Compared with a method of only relying on a single feature or a simple classifier, the embodiment of the application can more accurately determine the category to which the initial image sample belongs based on a plurality of feature dimensions, so as to select more valuable samples from the initial image samples. Then, image evidence information is obtained by counting the predicted category information indicating different categories. When the value of the image evidence information exceeds a preset first threshold, the corresponding initial image sample is determined as a target image sample. When the value of the image evidence information exceeds the preset first threshold, it indicates that a certain number of predicted category information conflicts, which indicates that the initial image sample has a high information amount and has a high uncertainty for the image classification model. Therefore, the initial image sample can be selected as a valuable target image sample, thereby improving the quality of the image samples selected in the training process of the image classification model.

[0156] The specific implementation of the image sample selection device is basically the same as the specific embodiment of the image sample selection method described above, and will not be repeated here.

[0157] The embodiment of the application further provides an electronic device. The electronic device comprises a memory and a processor. The memory stores a computer program. The processor implements the image sample selection method described above when executing the computer program. The electronic device can be any intelligent terminal such as a tablet computer or a vehicle-mounted computer.

[0158] As shown in Figure 11 , Figure 11 is a hardware structure schematic diagram of the electronic device provided by the embodiment of the application. The electronic device comprises:

[0159] The processor 901 can be implemented in a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute related programs to implement the technical solutions provided by the embodiment of the application.

[0160] The memory 902 can be implemented in the form of a Read Only Memory (ROM), a static storage device, a dynamic storage device, or a Random Access Memory (RAM), etc. The memory 902 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present specification are implemented by software or firmware, the related program codes are stored in the memory 902 and are called and executed by the processor 901 to perform the image sample selection method of the embodiments of the present application.

[0161] The input / output interface 903 is configured to realize information input and output.

[0162] The communication interface 904 is configured to realize the communication interaction between the device and other devices. The communication can be realized by wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WI FI, Bluetooth, etc.).

[0163] The bus 905 transmits information between various components (such as the processor 901, the memory 902, the input / output interface 903, and the communication interface 904) of the device.

[0164] The processor 901, the memory 902, the input / output interface 903, and the communication interface 904 are connected to each other through the bus 905 to realize the communication connection between them in the device.

[0165] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the above-mentioned image sample selection method.

[0166] The memory is a non-transitory computer readable storage medium, which can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory can include a high-speed random access memory and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0167] The embodiments described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0168] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation to the embodiments of the present application, and can include more or fewer steps than the figures, or combine certain steps, or different steps.

[0169] The device embodiments described above are merely illustrative, and units described as separate components can or can not be physically separate, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments.

[0170] Those skilled in the art can understand that all or some steps in the above disclosed method, functions of modules / units in the system and device can be implemented as software, firmware, hardware and appropriate combinations thereof.

[0171] The terms "first", "second", "third", "fourth" and the like in the specification of the present application and the above-described drawings, if any, are used to distinguish similar objects, and do not necessarily have to describe a particular order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0172] It should be understood that in the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the association between the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases: only A, only B, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0173] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented by other manners. For example, the apparatus embodiments described above are merely illustrative, for example, the division of the above units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, apparatuses or units, and can be electrical, mechanical or other forms.

[0174] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they can be located in one place or distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0175] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0176] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part of the prior art that makes a contribution or the whole or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0177] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, but this does not limit the scope of the rights of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the rights of the embodiments of the present application.

Claims

1. A method of selecting image samples, characterized by, The method comprises: obtaining an image sample data set and a category data set corresponding to the image sample data set, wherein the image sample data set comprises a plurality of initial image samples to be selected, and the category data set comprises a plurality of known category information; performing feature extraction on the initial image samples from different feature dimensions to obtain corresponding image features; calculating posterior probability values of the image features under different feature dimensions and each known category information, and predicting the category to which the initial image sample belongs according to the posterior probability values to obtain a plurality of predicted category information; statistically obtaining image evidence information from the predicted category information indicating different categories, and determining that the corresponding initial image sample is a target image sample when the value of the image evidence information exceeds a preset first threshold.

2. The method of claim 1, wherein, The calculation of the posterior probability values of the image features under different feature dimensions and each known category information comprises: calculating a feature mean value representing the average expression of the initial image sample based on all the image features corresponding to each initial image sample, and calculating a feature variance value representing the difference between each image feature and the feature mean value; calculating a first multiplication value based on a preset first constant and the feature variance value, and calculating a second multiplication value based on a preset second constant, the feature mean value and the feature variance value; calculating the posterior probability value representing the matching of the image features and each known category information based on the first multiplication value, the second multiplication value and a preset third constant.

3. The method of claim 1, wherein, The prediction of the category to which the initial image sample belongs according to the posterior probability values to obtain a plurality of predicted category information comprises: determining the known category information corresponding to the posterior probability value with the maximum value as the predicted category information of the initial image sample under the current feature dimension based on the plurality of posterior probability values of the image features; obtaining a plurality of predicted category information based on the predicted category information corresponding to each image feature.

4. The method of claim 1, wherein, After the calculation of the posterior probability values of the image features under different feature dimensions and each known category information, the method further comprises: obtaining a preset prior probability value, which is used to represent the probability of the initial image sample belonging to any known category information in advance; calculating the product of the prior probability value and the posterior probability value to obtain a first probability value; predicting the category to which the initial image sample belongs according to the first probability value to obtain a plurality of predicted category information; statistically obtaining image evidence information from the predicted category information indicating different categories, and determining that the corresponding initial image sample is a target image sample when the value of the image evidence information exceeds a preset second threshold.

5. The method of claim 4, wherein, After obtaining the preset prior probability value, the method further comprises: obtaining a preset total probability value, which is used to represent the sum of the probabilities of the initial image sample including each image feature in advance; calculating the ratio between the first probability value and the total probability value to obtain a second probability value; predicting a class to which the initial image sample belongs according to the second probability value, to obtain a plurality of predicted class information; statistically counting the predicted class information indicating different classes to obtain image evidence information, and determining that the corresponding initial image sample is a target image sample when a value of the image evidence information exceeds a preset third threshold.

6. The method of claim 1, wherein, After the image evidence information is obtained by statistically counting the predicted class information indicating different classes, the method further includes: selecting the corresponding initial image sample in order from large to small according to a value of each image evidence information; until the number of selected initial image samples reaches a preset number threshold, the target image sample is obtained.

7. The method of claim 1, wherein, After it is determined that the corresponding initial image sample is a target image sample, the method further includes: obtaining a target model to be trained, and adding a training label to the target image sample; training the target model based on the target image sample with the training label, to obtain a trained target model.

8. An apparatus for selecting an image sample, characterized by The apparatus includes: an acquisition module configured to acquire an image sample dataset and a class dataset corresponding to the image sample dataset, wherein the image sample dataset includes a plurality of initial image samples to be selected, and the class dataset includes a plurality of known class information; a feature extraction module configured to perform a feature extraction operation on the initial image samples from different feature dimensions, to obtain corresponding image features; a prediction module configured to calculate a posterior probability value of matching the image features under different feature dimensions with each known class information, and predict a class to which the initial image sample belongs according to the posterior probability value, to obtain a plurality of predicted class information; a target image sample determination module configured to statistically count the predicted class information indicating different classes to obtain image evidence information, and determine that the corresponding initial image sample is a target image sample when a value of the image evidence information exceeds a preset first threshold.

9. An electronic device, comprising: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the image sample selection method of any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the image sample selection method of any one of claims 1 to 7.

Images