Data set generation method and device, image evaluation method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a data set generation method and device, an image evaluation method and device, electronic equipment and a storage medium. The data set generation method comprises the steps that an original data set is acquired; carrying out characteristic value calculation of each preset dimension on each piece of data in the original data set; and taking the characteristic value of each preset dimension obeying the corresponding preset distribution as a target, sampling the original data set, and obtaining a target data set. A data set can be obtained based on multi-dimensional feature sampling, and the model construction requirement can be met.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of computer, and particularly, to a data set generation and image evaluation method and device, electronic equipment and storage medium. BACKGROUND

[0002] In the prior art, uneven distribution of sample data will result in poor perception of the model for small data volume categories. Currently, the data of small data volume categories can be sampled with a large proportion to achieve the purpose of balanced distribution. However, this sampling method is relatively rough and cannot meet the growing demand for model construction. SUMMARY

[0003] Embodiments of the present disclosure provide a data set generation and image evaluation method, device, electronic equipment and storage medium, which can obtain a data set based on multi-dimensional feature sampling and meet the demand for model construction.

[0004] In a first aspect, embodiments of the present disclosure provide a data set generation method, comprising:

[0005] obtaining an original data set;

[0006] calculating feature values of each preset dimension for each data in the original data set;

[0007] sampling the original data set to obtain a target data set, with the feature values of each preset dimension subject to a corresponding preset distribution.

[0008] In a second aspect, embodiments of the present disclosure provide an image evaluation method, comprising:

[0009] obtaining an image to be evaluated;

[0010] evaluating the quality of the image to be evaluated by a preset evaluation model, wherein the preset evaluation model is constructed based on a target image set, and the target image set is generated based on any of the data set generation methods of the embodiments of the present disclosure.

[0011] In a third aspect, embodiments of the present disclosure further provide a data set generation device, comprising:

[0012] a first obtaining module configured to obtain an original data set;

[0013] a feature calculation module configured to calculate feature values of each preset dimension for each data in the original data set;

[0014] a sampling module configured to sample the original data set to obtain a target data set, with the feature values of each preset dimension subject to a corresponding preset distribution.

[0015] In a fourth aspect, the embodiments of the present disclosure further provide an image evaluation device, comprising:

[0016] a second obtaining module, configured to obtain a to-be-evaluated image;

[0017] an evaluation module, configured to perform quality evaluation on the to-be-evaluated image by using a preset evaluation model, wherein the preset evaluation model is constructed based on a target image set, and the target image set is generated based on any of the data set generation methods according to the embodiments of the present disclosure.

[0018] In a fifth aspect, the embodiments of the present disclosure further provide an electronic device, comprising:

[0019] one or more processors;

[0020] a storage device, configured to store one or more programs,

[0021] when the one or more programs are executed by the one or more processors, the one or more processors implement the data set generation or image evaluation method according to any of the embodiments of the present disclosure.

[0022] In a sixth aspect, the embodiments of the present disclosure further provide a storage medium containing computer executable instructions, which, when executed by a computer processor, are used to perform the data set generation or image evaluation method according to any of the embodiments of the present disclosure.

[0023] In the technical solution of the embodiments of the present disclosure, the data set generation method comprises: obtaining an original data set; performing feature value calculation of each preset dimension on each data in the original data set; and sampling the original data set to obtain a target data set, with the feature values of each preset dimension being subject to a corresponding preset distribution. By calculating the feature values of each data in the original data set in each preset dimension, and sampling the original data set with the feature values of each preset dimension being subject to a corresponding distribution, the target data set obtained by sampling can meet the model construction requirements in each preset dimension. BRIEF DESCRIPTION OF DRAWINGS

[0024] The above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the original and elements are not necessarily drawn according to the scale.

[0025] Figure 1 A flowchart of a data set generation method according to an embodiment of the present disclosure;

[0026] Figure 2 A flowchart of generating a target data set in a data set generation method according to an embodiment of the present disclosure;

[0027] Figure 3 A flowchart of an image evaluation method provided by an embodiment of the present disclosure is shown in FIG. 1.

[0028] Figure 4 A schematic block diagram of generating a target image set in an image evaluation method provided by an embodiment of the present disclosure is shown in FIG. 2.

[0029] Figure 5 A schematic block diagram of constructing a preset evaluation model in an image evaluation method provided by an embodiment of the present disclosure is shown in FIG. 3.

[0030] Figure 6 A structural schematic diagram of a data set generation apparatus provided by an embodiment of the present disclosure is shown in FIG. 4.

[0031] Figure 7 A structural schematic diagram of an image evaluation apparatus provided by an embodiment of the present disclosure is shown in FIG. 5.

[0032] Figure 8 A structural schematic diagram of an electronic device provided by an embodiment of the present disclosure is shown in FIG. 6. DETAILED DESCRIPTION

[0033] Embodiments of the present disclosure will be described in more detail by referring to the drawings. Although certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be implemented in various forms, and should not be construed as being limited to the embodiments set forth herein, but rather, these embodiments are provided so as to more completely and thoroughly understand the present disclosure. It is understood that the drawings and embodiments of the present disclosure are for exemplary purposes only, and are not intended to limit the scope of protection of the present disclosure.

[0034] It is understood that each step described in the method embodiments of the present disclosure can be executed in different orders, and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the execution of the steps shown. The scope of the present disclosure is not limited in this respect.

[0035] The term “comprising” and variations thereof as used herein are open-ended, that is, “including but not limited to”. The term “based on” is “based, at least in part, on”. The term “one embodiment” means “at least one embodiment”; the term “another embodiment” means “at least one additional embodiment”; the term “some embodiments” means “at least some embodiments”. Related definitions are given throughout the description.

[0036] It is noted that the concepts of “first”, “second”, etc. mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not intended to limit the order or interdependence of the functions performed by these devices, modules or units.

[0037] It should be noted that the modification of "one" and "multiple" mentioned in the present disclosure is illustrative rather than restrictive, and those skilled in the art should understand that "one or more" should be understood unless the context clearly indicates otherwise.

[0038] The names of the messages or information exchanged between the plurality of devices in the embodiments of the present disclosure are only for illustrative purposes, and are not used to limit the scope of the messages or information.

[0039] It can be understood that the data involved in the technical solution (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of the corresponding laws, regulations and relevant provisions.

[0040] Figure 1 A flowchart of a data set generation method provided by an embodiment of the present disclosure is shown. The embodiment of the present disclosure is applicable to the case of generating a data set, for example, the case of generating a data set for constructing a model. The method can be executed by a data set generation device, which can be implemented in the form of software and / or hardware, and can be configured in an electronic device, such as a computer.

[0041] As shown in Figure 1 The data set generation method provided by the embodiment can include:

[0042] S110, obtaining an original data set.

[0043] In the embodiment of the present disclosure, the original data set can be considered as a data set with sufficient redundancy, which can be understood as a data set after sufficient data augmentation. Based on this, the target data set obtained after sampling the original data set can still have the information of the original data set, avoiding the loss of effective information. Thus, the construction effect of the neural network model can be better guaranteed when the neural network model is constructed based on the target data set.

[0044] S120, calculating feature values of each data in the original data set in each preset dimension.

[0045] In the embodiment of the present disclosure, the preset dimension can include at least two dimensions, and the preset dimension can be set in advance based on at least one factor such as the modality (such as audio, image or text, etc.) of the data in the data set, the construction requirement of the model, etc. For example, for image data, the preset dimension can include dimensions such as brightness, sharpness and clarity. Wherein, the existing feature value calculation method can be used to calculate the feature values of each data in the original data set in each preset dimension.

[0046] S130, sampling the original data set to obtain a target data set, with the feature values of each preset dimension subject to the corresponding preset distribution.

[0047] In the embodiments of the present disclosure, for each preset dimension feature value, a desired distribution (i.e., a preset distribution) can be set in advance. The preset distributions corresponding to the feature values of different preset dimensions can be the same or different. The preset distribution may, for example, include a uniform distribution, a Gaussian distribution, or a distribution fitting the corresponding feature distribution in the original data set, and the like, without exhaustive listing.

[0048] The original data set can be sampled in cycles, and a certain number of data can be sampled in each cycle. After each cycle of sampling is completed, the feature value distribution of each data sampled in the current cycle in each preset dimension can be determined. The target data set can be determined in the following manner, for example: in response to the feature value distribution of each data sampled in the current cycle in each preset dimension satisfying the corresponding preset distribution, the cycle sampling is stopped, and the target data set is constructed based on the data sampled in the current cycle. For another example, the cycle can be stopped after a preset number of cycles of sampling, and the target cycle in which the feature value distribution of each preset dimension is closest to the corresponding preset distribution is determined from each cycle, and the target data set is constructed based on the data sampled in the target cycle.

[0049] By constructing the target data set based on multi-dimensional feature sampling, it can be ensured that the target data set meets the construction requirements of the neural network model in multiple preset dimensions, and more refined and multi-dimensional data set generation is achieved.

[0050] The technical solution of the embodiments of the present disclosure can include: obtaining an original data set; calculating the feature values of each data in the original data set in each preset dimension; and sampling the original data set to obtain a target data set, with the feature values of each preset dimension being subject to the corresponding preset distribution. By calculating the feature values of each data in the original data set in each preset dimension, and sampling the original data set with the feature values of each preset dimension being subject to the corresponding distribution, the target data set obtained by sampling can meet the model construction requirements in each preset dimension.

[0051] The embodiments of the present disclosure can be combined with the data set generation method provided in the above embodiments. The data set generation method provided in the embodiments of the present disclosure is described in detail. By minimizing the distribution distance between the feature value distribution of the sampled data in each preset dimension and the corresponding preset distribution, the sampling strategy can be determined to achieve the construction of the target data set.

[0052] Figure 2 A flowchart of generating a target data set in a data set generation method provided by the embodiments of the present disclosure is shown in FIG. 1. As shown in FIG. 1, the data set generation method provided in the embodiments of the present disclosure can include the following steps. Figure 2 As shown in FIG. 1, the data set generation method provided in the embodiments of the present disclosure can include the following steps.

[0053] S210, obtaining a first matrix for representing the original data set; wherein the first matrix comprises a first dimension representing the sequence number of each data, and a second dimension representing each preset dimension.

[0054] For example, the original data set S can be represented based on the following formula:

[0055]

[0056] wherein i can represent the sequence number of the data in the original data set, K can represent the total number of data in the original data set (i.e. the maximum value of i), q i may represent the i-th data in the original data set; wherein M can represent the total number of preset dimensions, and D can represent the preset distribution corresponding to each preset dimension. On this basis, the matrix representation of the original data set can be as follows:

[0057] Q = [q ij ] KM ; (formula 2)

[0058] wherein Q can represent the first matrix, and the row dimension i can represent the sequence number of each data, and the column dimension j can represent each preset dimension; wherein the element q ij in the first matrix can represent the feature value of the i-th data in the original data set in the j-th preset dimension; wherein the maximum value of i can be K, and the maximum value of j can be M.

[0059] S220, sampling the first matrix according to the first dimension to obtain a second matrix.

[0060] On the basis of the above (formula 2), the first matrix Q can be sampled according to the row dimension to obtain a second matrix wherein the maximum value of i in the second matrix can be N, and N << K.

[0061] S230, determining the distribution distance between the distribution of the second dimension in the second matrix and the corresponding preset distribution.

[0062] In this embodiment, determining the distribution distance between the distribution of the second dimension in the second matrix and the corresponding preset distribution can include: determining the distribution distance between the distribution of the feature value of each preset dimension under the second dimension in the second matrix and the corresponding preset distribution.

[0063] For example, the distribution distance O j between the feature value of the j-th preset dimension and the corresponding preset distribution can be determined based on the following formula:

[0064] O j = ||B j xQ-ND *j||1; (formula 3)

[0065] Wherein, x can represent a sampling matrix, and the sampling matrix can include elements 0 and 1; xQ can represent a second matrix after sampling the first matrix Q, and when the element in x is 0, it can represent that the data at the corresponding position in Q is not sampled, and when the element in x is 1, it can represent that the data at the corresponding position in Q is sampled. Wherein, B j may represent an extraction matrix of a predetermined dimension, and the jth element in the extraction matrix can be 1, and other elements can be 0; accordingly, B j xQ can represent the sum of the eigenvalues of the jth predetermined dimension in the second matrix. Wherein, N can represent the total number of samples, D *j may represent the jth predetermined dimension of the predetermined distribution, ND *j that is, the expected sum of N values under the D *j distribution. Wherein, ||·||1 can represent the absolute difference.

[0066] That is, the above (formula 3), by calculating the difference between the sum of the eigenvalues of the jth predetermined dimension in the second matrix and the expected sum corresponding to the predetermined distribution, the distribution distance of the eigenvalues of the predetermined dimension under the second dimension in the second matrix and the corresponding predetermined distribution can be determined.

[0067] S240, in response to the sum of the distribution distances being minimum, constructing a target data set according to the data corresponding to the second matrix.

[0068] On the basis of the above (formula 3), the sampling matrix x can be determined based on the following formula:

[0069]

[0070] Wherein, s.t. ||x||1=N is a constraint condition, which can be understood as the number of elements of 1 in the sampling matrix x needs to be N, that is, sampling N data from the first matrix. Wherein, is the objective function, and may represent the sum of the distribution distances; that is, the objective function can be understood as the sampling matrix x when the sum of the distribution distances is minimum.

[0071] Wherein, (formula 4) can be solved based on existing linear programming algorithm, for example, the sampling mixed integer linear programming algorithm (Mixed-integer linear programming, MILP) can be used to solve the sampling matrix x. Thus, the first matrix can be sampled based on the solved sampling matrix x, and the target data set s can be constructed according to the data corresponding to the second matrix after sampling At this time, the target data set s can satisfy the following formula That is, the feature values of each pre-dimension in the target data set conform to the corresponding preset distribution.

[0072] In addition to the above examples, each pre-set dimension can also be represented by a column in the first matrix, and each row represents the serial number of the data. At this time, the second matrix can be obtained by sampling the first matrix by column dimension; the distribution of each row in the second matrix is determined, and the distribution distance of the corresponding preset distribution is determined; in response to the sum of each distribution distance being minimum, the target data set is constructed according to the data corresponding to the second matrix.

[0073] The technical scheme of the embodiment of the present disclosure is described in detail. By making the feature value distribution of the sampled data in each pre-set dimension minimum, the sampling strategy can be determined to achieve the construction of the target data set. In addition, the data set generation method provided by the embodiment of the present disclosure belongs to the same disclosure concept as the data set generation method provided by the above embodiment. The technical details not described in detail in this embodiment can be referred to the above embodiment, and the same technical features have the same beneficial effects in this embodiment and the above embodiment.

[0074] Figure 3 A flowchart of an image evaluation method provided by the embodiment of the present disclosure. The embodiment of the present disclosure is applicable to the case of image evaluation, for example, applicable to the case of face image evaluation. The method can be executed by an image evaluation device, which can be realized in the form of software and / or hardware, and the device can be configured in an electronic device, such as a computer.

[0075] As shown in Figure 3 The image evaluation method provided by the embodiment can include:

[0076] S310, obtaining an image to be evaluated;

[0077] S320, performing quality evaluation on the image to be evaluated by a preset evaluation model; wherein the preset evaluation model is constructed based on a target image set, and the target image set is generated based on the data set generation method of any embodiment of the present disclosure.

[0078] In the field of image evaluation, the data set used to construct the image evaluation model usually has a long tail phenomenon, which leads to poor perception of the image evaluation model to the small data amount of the category.

[0079] In the embodiment of the present disclosure, the target image set is generated based on the data set generation method of any embodiment of the present disclosure, so that the target image set can meet the expected preset distribution in each pre-set dimension. Further, by constructing the preset evaluation model based on the target image set, the image evaluation effect of the preset evaluation model can be improved.

[0080] In this embodiment, the preset evaluation model can include an existing image quality evaluation model, for example, a model with a Transformer architecture as the backbone network. The preset evaluation model can extract image features at different levels, and the image features at a high level have more semantic information, and the image features at a low level have more spatial information. The image features at different levels can be extracted and fused, thereby greatly increasing the information contained in the features, which is beneficial to improving the image evaluation accuracy.

[0081] Exemplarily, Figure 4 An exemplary schematic block diagram of generating a target image set in the image evaluation method provided by the embodiments of the present disclosure is shown in FIG. 1. Figure 4 The process of generating the target image set can include:

[0082] First, an original data set can be obtained, wherein the original data set can include sample images under multiple scenes. For example, Figure 4 The original data set can include sample images under scene 1 to scene N. Each scene can be understood as an image under different environmental conditions, for example, scene 1 can be a scene with bright indoor light, scene N can be a scene with dim indoor light, and the like.

[0083] Secondly, the sample images under each scene can be calculated for each preset dimension of the feature value, wherein the preset dimensions can include but are not limited to exposure, evaluation true value, sharpness, and spatial perceptual information (SI). The evaluation true value can be provided by the original data set, and the feature values of other dimensions can be calculated according to existing algorithms. Before sampling, the feature value distribution of each preset dimension under each scene in the original data set is not balanced. For example, Figure 4 Exemplarily, the feature distribution of each scene under the exposure dimension is not balanced.

[0084] Thirdly, the constraint condition and the objective function can be constructed by referring to (formula 3) and (formula 4), and the sampling matrix of the sample images under each scene can be solved based on MILP. The preset distribution corresponding to each preset dimension can be, for example, a uniform distribution.

[0085] Finally, the sample images under the corresponding scene can be sampled based on the sampling matrix under each scene, and the sampling results can be integrated into the target image set. As shown in FIG. 2, Figure 4 In the sampled target image set, the feature values of each preset dimension under each scene can be more balanced.

[0086] The technical scheme of the embodiment of the present disclosure, the image evaluation method can comprise: acquiring an image to be evaluated; performing quality evaluation on the image to be evaluated through a preset evaluation model; wherein the preset evaluation model is constructed based on a target image set, and the target image set is generated based on the dataset generation method of any of the embodiments of the present disclosure. By generating the target image set based on the dataset generation method of any of the embodiments of the present disclosure, the target image set can meet the expected preset distribution in each preset dimension. Further, by constructing the preset evaluation model based on the target image set, the image evaluation effect of the preset evaluation model can be improved.

[0087] The embodiments of the present disclosure can be combined with the various optional schemes of the image evaluation method provided in the above embodiments. The image evaluation method provided in the present embodiment describes in detail the construction process of the preset evaluation model. By combining the rank loss and the evaluation value loss to construct the preset evaluation model, the evaluation effect of the preset evaluation model can be further improved.

[0088] Figure 5 A schematic block diagram of constructing a preset evaluation model in an image evaluation method provided by the embodiments of the present disclosure is shown in FIG. 6. Figure 5 As shown in FIG. 6, the construction process of the preset evaluation model in the image evaluation method provided in the present embodiment can comprise:

[0089] S510, acquiring a sample image pair comprising a first sample image and a second sample image.

[0090] In the target image set, in addition to the sample images, the evaluation true values corresponding to the sample images can also be included. In the present embodiment, the sample image pair is included in the target image set, and the first evaluation true value of the first sample image is different from the second evaluation true value of the second sample image. Since the preset evaluation model is expected to have the ability to evaluate the quality of the image, two sample images with different evaluation true values (i.e., the first sample image and the second sample image) can be acquired each time to be input into the preset evaluation model for contrast learning.

[0091] S520, performing quality evaluation on the first sample image and the second sample image through the preset evaluation model to obtain a first evaluation prediction value and a second evaluation prediction value.

[0092] Referring to Figure 5 The first sample image and the second sample image can be input into two preset evaluation models sharing the same parameters to perform evaluation in the manner of a twin network, and the first evaluation prediction value and the second evaluation prediction value can be obtained, respectively. In addition, the first sample image and the second sample image can be input into the preset evaluation model in sequence to obtain the first evaluation prediction value and the second evaluation prediction value.

[0093] S530, constructing a rank loss according to the first evaluation prediction value and the second evaluation prediction value.

[0094] In this embodiment, rank loss can be understood as the qualitative loss of the evaluation result. In this embodiment, the first evaluation truth value of the first sample image is different from the second evaluation truth value of the second sample image, that is, the image quality of the first sample image can be greater than the image quality of the second sample image, or it can be less than the image quality of the second sample image.

[0095] Accordingly, the rank loss can be constructed based on whether the comparison between the first and second predicted values ​​matches the comparison between the first and second true values. For example, when the first true value is less than the second true value, the rank loss can be constructed using the following formula:

[0096]

[0097] in, These can represent the first and second predicted values, respectively. When the comparison between the first and second predicted values ​​matches the comparison between the first and second true values, that is... At that time, l rank Smaller; when the comparison between the first and second predicted values ​​does not match the comparison between the true first and second predicted values, i.e. At that time, l rank The rank loss increases exponentially with the difference between the two images. Therefore, this rank loss can guide the pre-defined evaluation model, enabling it to compare the merits of two images.

[0098] S540. Construct an assessment value loss based on the first assessment prediction value and the first assessment true value, and / or based on the second assessment prediction value and the second assessment true value.

[0099] In this embodiment, the evaluation value loss can be understood as the quantitative loss of the evaluation result. Specifically, the evaluation value loss can be constructed based on the deviation between the first predicted evaluation value and the first true evaluation value; it can also be constructed based on the deviation between the second predicted evaluation value and the second true evaluation value; or it can be constructed based on the deviation between the first predicted evaluation value and the first true evaluation value, as well as the deviation between the second predicted evaluation value and the second true evaluation value.

[0100] The evaluation loss can be determined based on existing numerical loss algorithms. For example, the mean-square error (MSE) algorithm can be used to determine the evaluation loss.

[0101] S550. Construct the preset evaluation model based on rank loss and evaluation value loss.

[0102] Exemplarily, based on the above (formula 5), the rank loss and the evaluation value loss can be combined based on the following formula to obtain the total loss:

[0103]

[0104] wherein y i , y i+1 may respectively represent the first evaluation true value and the second evaluation true value. It can be known from (formula 6) that the mean square loss of the first evaluation predicted value and the first evaluation true value is adopted as the evaluation value loss in this example. In this example, the first evaluation true value needs to be less than the second evaluation true value, and when the input two sample images meet this requirement, the total loss can be constructed according to the weighted sum of the rank loss and the evaluation value loss (for example, both weights of formula 6 are 1); when the input two sample images do not meet this requirement, only the evaluation value loss can be used to construct the total loss. Wherein N can represent the total number of sample images in the target image set. Since the evaluation value loss is determined only according to the first evaluation predicted value and the first evaluation true value, after the total loss is obtained, it needs to be divided by N2 to determine the total loss.

[0105] In this embodiment, after the total loss is determined, the total loss can be back propagated to adjust the parameters in the preset evaluation model, so as to realize the construction of the preset evaluation model.

[0106] In some implementations, the sample images in the target image set can include sample images of different scenes; wherein the scenes may, for example, include outdoor, indoor, low brightness and night, etc. And the evaluation true value of the sample images in the target image set can be scene-specific labeled, that is, the quality evaluation ranking of the sample images between the same scenes. In this implementation, only the sample images in the same scene can be used to determine the total loss. That is, if the input two images come from different scenes, the loss obtained can not participate in the parameter update of the preset evaluation model.

[0107] In some optional implementations, the construction process of the sample image pair can include: grouping each sample image in the target image set in pairs according to the size order of the evaluation true value to obtain each sample image pair.

[0108] Wherein each sample image in the target image set can be sorted in order of the evaluation true value from small to large, and the sample images can be input into the preset evaluation model in pairs according to the sorting. At this time, it can be considered that the first evaluation true value is less than the second evaluation true value, and the corresponding constructed rank loss can be as shown in (formula 5).

[0109] In addition, each sample image in the target image set can also be sorted in order of the evaluation true value from large to small, and the sample images can be input into the preset evaluation model in pairs according to the sorting. At this time, it can be considered that the first evaluation true value is greater than the second evaluation true value, and the corresponding constructed rank loss can be adaptively adjusted as

[0110] In these optional implementations, the two sample images adjacent in the true value can be input into the preset evaluation model to construct the preset evaluation model. Since the quality difference between the two sample images is small, the model is constructed based on this, which can make the preset evaluation model have more accurate quality evaluation capability.

[0111] In some optional implementations, the image evaluation method can be applied to face image evaluation, and the target image set includes a face image set.

[0112] Correspondingly, referring again to Figure 5 After obtaining the sample image pair including the first sample image and the second sample image, the first sample image and the second sample image can be respectively cropped into at least one local image of a preset size; in response to the local image not completely containing the face region, the sample image to which the local image belongs is recropped; wherein the local image is used for data enhancement of the sample image to which the local image belongs.

[0113] The preset size can be pre-set based on the input layer of the preset evaluation model, for example, it can be 448x448. The first sample image / second sample image can be randomly cropped for a certain number of times to obtain at least one corresponding local image. Before cropping the two sample images, they can also be scaled down in proportion to ensure that the cropped images can contain complete face regions.

[0114] The existing face recognition algorithm can be used to determine the position of the face region; then the position of the face region can be used to determine whether the cropped image contains a complete face region. When each local image corresponding to the first sample image / second sample image contains a face region, each local image can be input into the preset evaluation model for quality evaluation. When there is a local image that does not completely contain a face region in each local image corresponding to the first sample image / second sample image, the sample image to which the local image belongs can be recropped until each local image corresponding to the first sample image / second sample image contains a face region, and then each local image is input into the preset evaluation model for quality evaluation.

[0115] In these optional implementations, the data enhancement of the first sample image and the second sample image can be realized based on the content-aware random cropping method. This data enhancement method not only preserves the integrity of the face region, but also increases the richness of the background region, which can further improve the construction effect of the preset evaluation model.

[0116] The technical solution of this disclosure describes in detail the construction process of the preset evaluation model. By combining rank loss and evaluation value loss to construct the preset evaluation model, the evaluation effect of the preset evaluation model can be further improved. The image evaluation method provided in this disclosure belongs to the same concept as the image evaluation method provided in the above embodiments. Technical details not described in detail in this embodiment can be found in the above embodiments, and the same technical features have the same beneficial effects in this embodiment and the above embodiments.

[0117] Figure 6 This is a schematic diagram of a dataset generation apparatus provided in an embodiment of the present disclosure. The dataset generation apparatus provided in this embodiment is applicable to situations where datasets are generated, such as generating datasets for building models.

[0118] like Figure 6 As shown, the dataset generation apparatus provided in this embodiment may include:

[0119] The first acquisition module 610 is used to acquire the original dataset;

[0120] The feature calculation module 620 is used to calculate the feature values ​​of each preset dimension for each data in the original dataset;

[0121] The sampling module 630 is used to sample the original dataset with the goal of obtaining the target dataset by having the feature values ​​of each preset dimension follow the corresponding preset distribution.

[0122] In some alternative implementations, the sampling module can be used for:

[0123] Obtain a first matrix to represent the original dataset; wherein the first matrix includes a first dimension representing the ordinal number of each data point, and a second dimension representing each preset dimension;

[0124] The first matrix is ​​sampled along its first dimension to obtain the second matrix;

[0125] Determine the distribution of the second dimension in the second matrix and its distance from the corresponding preset distribution;

[0126] To minimize the sum of the distances between all distributions, the target dataset is constructed based on the data corresponding to the second matrix.

[0127] The dataset generation apparatus provided in this disclosure can execute the dataset generation method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0128] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this disclosure.

[0129] Figure 7 This is a schematic diagram of the structure of an image evaluation apparatus provided in an embodiment of this disclosure. The image evaluation apparatus provided in this embodiment is applicable to image evaluation scenarios, such as facial image evaluation scenarios.

[0130] like Figure 7 As shown, the image evaluation apparatus provided in this embodiment may include:

[0131] The second acquisition module 710 is used to acquire the image to be evaluated;

[0132] The evaluation module 720 is used to evaluate the quality of the image to be evaluated by using a preset evaluation model; wherein the preset evaluation model is constructed based on a target image set, and the target image set is generated based on any of the dataset generation methods in the embodiments of this disclosure.

[0133] In some alternative implementations, the image evaluation apparatus may further include:

[0134] The model building module is used to build a pre-defined evaluation model according to the following process:

[0135] Obtain a sample image pair containing a first sample image and a second sample image; wherein the sample image pair is contained in the target image set, and the first evaluation ground truth value of the first sample image is different from the second evaluation ground truth value of the second sample image;

[0136] The quality of the first sample image and the second sample image is evaluated by a preset evaluation model to obtain the first evaluation prediction value and the second evaluation prediction value.

[0137] Construct a rank loss based on the first and second assessment predictions;

[0138] Based on the first assessment prediction value and the first assessment true value, and / or based on the second assessment prediction value and the second assessment true value, construct the assessment value loss;

[0139] The preset evaluation model is constructed based on rank loss and evaluation value loss.

[0140] In some alternative implementations, the image evaluation apparatus may further include:

[0141] The sample pair construction module is used to construct sample image pairs based on the following process:

[0142] The sample images in the target image set are grouped into pairs according to the order of the true values ​​of the evaluation, resulting in sample image pairs.

[0143] In some optional implementations, it is applied to facial image evaluation, where the target image set includes a set of facial images;

[0144] Correspondingly, the model building module can also be used for:

[0145] After obtaining a sample image pair containing a first sample image and a second sample image, the first sample image and the second sample image are respectively cropped into at least one local image of a preset size;

[0146] In response to a local image not fully containing the facial region, the sample image to which the local image belongs is re-cropped;

[0147] Local images are used to perform data augmentation on their respective sample images.

[0148] The image evaluation apparatus provided in this disclosure can execute the image evaluation method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of executing the method.

[0149] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this disclosure.

[0150] The following is for reference. Figure 8 It illustrates an electronic device suitable for implementing embodiments of the present disclosure (e.g., Figure 8 The diagram below shows the structure of the terminal device or server 800. The terminal device in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 8 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0151] like Figure 8As shown, the electronic device 800 may include a processing unit (e.g., a central processing unit, a graphics processor, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 into a random access memory (RAM) 803. The RAM 803 also stores various programs and data required for the operation of the electronic device 800. The processing unit 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0152] Typically, the following devices can be connected to I / O interface 805: input devices 806 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 807 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 808 including, for example, magnetic tapes, hard disks, etc.; and communication devices 809. Communication device 809 allows electronic device 800 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 8 An electronic device 800 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0153] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 809, or installed from a storage device 808, or installed from a ROM 802. When the computer program is executed by a processing device 801, it performs the functions defined in the dataset generation or image evaluation methods of embodiments of this disclosure.

[0154] The electronic device provided in this embodiment belongs to the same concept as the dataset generation or image evaluation method provided in the above embodiments. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0155] This disclosure provides a storage medium for computer-executable instructions, which, when executed by a computer processor, can be used to perform the dataset generation or image evaluation methods provided in the above embodiments.

[0156] It should be noted that the storage medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory (FLASH), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium that contains or stores executable instructions that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable executable instructions. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit executable instructions for use by or in connection with an instruction execution system, apparatus, or device. Executable instructions contained on the storage medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0157] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0158] The aforementioned storage medium may be included in the aforementioned electronic device; or it may exist independently and not be assembled into the electronic device.

[0159] The aforementioned storage medium carries one or more executable instructions. When the aforementioned one or more executable instructions are executed by the electronic device, the electronic device causes the electronic device to: acquire the original dataset; calculate the feature values ​​of each data in the original dataset for each preset dimension; and sample the original dataset with the goal that the feature values ​​of each preset dimension conform to the corresponding preset distribution, thereby obtaining the target dataset.

[0160] Alternatively, the aforementioned computer-readable medium carries one or more executable instructions, which, when executed by the electronic device, cause the electronic device to: acquire an image to be evaluated; and perform a quality evaluation on the image to be evaluated using a preset evaluation model; wherein the preset evaluation model is constructed based on a target image set, and the target image set is generated based on any dataset generation method of the embodiments of this disclosure.

[0161] Executable instructions for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The executable instructions can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0162] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, can implement the dataset generation or image evaluation method provided in any embodiment of this disclosure.

[0163] In implementing a computer program product, computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0164] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0165] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units and modules do not, in certain circumstances, constitute a limitation on the unit or module itself.

[0166] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Array (FPGA), Application Specific Integrated Circuit (ASIC), Application Specific Standard Parts (ASSP), System on Chip (SOC), Complex Programmable Logic Device (CPLD), and so on.

[0167] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0168] According to one or more embodiments of this disclosure, a dataset generation method is provided, the method comprising:

[0169] Obtain the original dataset;

[0170] For each data point in the original dataset, feature values ​​for each preset dimension are calculated.

[0171] The original dataset is sampled to obtain the target dataset, with the goal that the feature values ​​of each preset dimension follow the corresponding preset distribution.

[0172] According to one or more embodiments of this disclosure, a dataset generation method is provided, further comprising:

[0173] In some optional implementations, the step of sampling the original dataset to obtain the target dataset with the goal of ensuring that the feature values ​​of each preset dimension conform to a corresponding preset distribution includes:

[0174] Obtain a first matrix for representing the original dataset; wherein the first matrix includes a first dimension representing the ordinal number of each data point, and a second dimension representing each preset dimension;

[0175] The first matrix is ​​sampled along the first dimension to obtain the second matrix;

[0176] Determine the distribution of the second dimension in the second matrix and the distribution distance between it and the corresponding preset distribution;

[0177] In response to minimizing the sum of the distances of all the aforementioned distributions, the target dataset is constructed based on the data corresponding to the second matrix.

[0178] According to one or more embodiments of this disclosure, an image evaluation method is provided, the method comprising:

[0179] Acquire the image to be evaluated;

[0180] The quality of the image to be evaluated is assessed using a preset evaluation model; wherein the preset evaluation model is constructed based on a target image set, and the target image set is generated based on any of the dataset generation methods described in the embodiments of this disclosure.

[0181] According to one or more embodiments of this disclosure, an image evaluation method is provided, further comprising:

[0182] In some optional implementations, the construction process of the preset evaluation model includes:

[0183] Obtain a sample image pair containing a first sample image and a second sample image; wherein the sample image pair is included in the target image set, and the first evaluation truth value of the first sample image is different from the second evaluation truth value of the second sample image;

[0184] The first sample image and the second sample image are evaluated using the preset evaluation model to obtain a first evaluation prediction value and a second evaluation prediction value.

[0185] Construct a rank loss based on the first and second evaluation prediction values;

[0186] Based on the first predicted evaluation value and the first true evaluation value, and / or based on the second predicted evaluation value and the second true evaluation value, construct the evaluation value loss;

[0187] The preset evaluation model is constructed based on the rank loss and the evaluation value loss.

[0188] According to one or more embodiments of this disclosure, an image evaluation method is provided, further comprising:

[0189] In some optional implementations, the process of constructing the sample image pairs includes:

[0190] The sample images in the target image set are grouped into pairs according to the order of their true values ​​to obtain the sample image pairs.

[0191] According to one or more embodiments of this disclosure, an image evaluation method is provided, further comprising:

[0192] In some alternative implementations, this is applied to facial image evaluation, where the target image set includes a set of facial images;

[0193] Accordingly, after obtaining the sample image pair containing the first sample image and the second sample image, the method further includes:

[0194] The first sample image and the second sample image are each cropped into at least one partial image of a preset size;

[0195] In response to the fact that the local image does not completely contain the facial region, the sample image to which the local image belongs is re-cropped;

[0196] The local image is used to perform data augmentation on the corresponding sample image.

[0197] According to one or more embodiments of this disclosure, a dataset generation apparatus is provided, the apparatus comprising:

[0198] The first acquisition module is used to acquire the original dataset;

[0199] The feature calculation module is used to calculate the feature values ​​of each preset dimension for each data in the original dataset;

[0200] The sampling module is used to sample the original dataset with the goal of obtaining the target dataset by sampling the feature values ​​of each preset dimension according to the corresponding preset distribution.

[0201] According to one or more embodiments of the present disclosure, an image evaluation apparatus is provided, the apparatus comprising:

[0202] The second acquisition module is used to acquire the image to be evaluated;

[0203] An evaluation module is used to evaluate the quality of the image to be evaluated using a preset evaluation model; wherein the preset evaluation model is constructed based on a target image set, and the target image set is generated based on any of the dataset generation methods described in the embodiments of this disclosure.

[0204] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0205] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0206] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for generating a dataset, characterized in that, include: Obtain the original dataset; For each data point in the original dataset, feature values ​​for each preset dimension are calculated. The original dataset is sampled to obtain the target dataset, with the goal that the feature values ​​of each preset dimension follow the corresponding preset distribution.

2. The method according to claim 1, characterized in that, The step of sampling the original dataset to obtain the target dataset, with the goal of ensuring that the feature values ​​of each preset dimension conform to a corresponding preset distribution, includes: Obtain a first matrix for representing the original dataset; wherein the first matrix includes a first dimension representing the ordinal number of each data point, and a second dimension representing each preset dimension; The first matrix is ​​sampled along the first dimension to obtain the second matrix; Determine the distribution of the second dimension in the second matrix and the distribution distance between it and the corresponding preset distribution; In response to minimizing the sum of the distances of all the aforementioned distributions, the target dataset is constructed based on the data corresponding to the second matrix.

3. An image evaluation method, characterized in that, include: Acquire the image to be evaluated; The quality of the image to be evaluated is assessed using a preset evaluation model; wherein the preset evaluation model is constructed based on a target image set, and the target image set is generated based on the dataset generation method described in claim 1.

4. The method according to claim 3, characterized in that, The process of constructing the preset evaluation model includes: Obtain a sample image pair containing a first sample image and a second sample image; wherein the sample image pair is included in the target image set, and the first evaluation truth value of the first sample image is different from the second evaluation truth value of the second sample image; The first sample image and the second sample image are evaluated using the preset evaluation model to obtain a first evaluation prediction value and a second evaluation prediction value. Construct a rank loss based on the first and second evaluation prediction values; Based on the first predicted evaluation value and the first true evaluation value, and / or based on the second predicted evaluation value and the second true evaluation value, construct the evaluation value loss; The preset evaluation model is constructed based on the rank loss and the evaluation value loss.

5. The method according to claim 4, characterized in that, The process of constructing the sample image pairs includes: The sample images in the target image set are grouped into pairs according to the order of their true values ​​to obtain the sample image pairs.

6. The method according to claim 4, characterized in that, Applied to facial image evaluation, wherein the target image set includes a set of facial images; Accordingly, after obtaining the sample image pair containing the first sample image and the second sample image, the method further includes: The first sample image and the second sample image are each cropped into at least one partial image of a preset size; In response to the fact that the local image does not completely contain the facial region, the sample image to which the local image belongs is re-cropped; The local image is used to perform data augmentation on the corresponding sample image.

7. A dataset generation device, characterized in that, include: The first acquisition module is used to acquire the original dataset; The feature calculation module is used to calculate the feature values ​​of each preset dimension for each data in the original dataset; The sampling module is used to sample the original dataset with the goal of obtaining the target dataset by sampling the feature values ​​of each preset dimension according to the corresponding preset distribution.

8. An image evaluation device, characterized in that, include: The second acquisition module is used to acquire the image to be evaluated; An evaluation module is used to evaluate the quality of the image to be evaluated using a preset evaluation model; wherein the preset evaluation model is constructed based on a target image set, and the target image set is generated based on the dataset generation method of claim 1.

9. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the dataset generation method as described in any one of claims 1-2, or the image evaluation method as described in any one of claims 3-6.

10. A storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the dataset generation method as described in any one of claims 1-2, or to implement the image evaluation method as described in any one of claims 3-6.

11. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements a dataset generation method as described in any one of claims 1-2, or an image evaluation method as described in any one of claims 3-6.

Images