Image data augmentation method and device based on local linear embedding

Abstract

The invention provides an image data augmentation method and device based on local linear embedding, which can be applied to the technical field of image processing. The method comprises the following steps: performing amplification processing on an initial image sample of an initial training set to obtain an intermediate training set; for the ith image sample of the intermediate training set, determining a neighbor image sample set corresponding to the ith image sample according to the initial label of the ith image sample; according to the neighbor image sample set corresponding to the i-th image sample, performing local linear embedding processing on the i-th image sample to obtain a target linear combination coefficient corresponding to the i-th image sample; performing local linear embedding processing on the initial label of the ith image sample by using the target linear combination coefficient corresponding to the ith image sample to obtain a target label of the ith image sample; and updating the intermediate training set according to the target label corresponding to each image sample to obtain a target training set.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and more specifically to an augmentation method and device for image data based on local linear embedding. Background Art

[0002] In machine learning, tasks such as age estimation, temperature prediction, and depth estimation have received increasing attention. The labels of deep regression are continuous, there is ambiguous information between the labels of some training samples, and the training data of most deep regression tasks is unbalanced.

[0003] In related technologies, in order to address the problem of imbalanced training data faced by deep regression tasks, continuous labels are first discretized, and corresponding interval samples are generated based on the discrete labels. This causes the continuity of the labels to be destroyed, and the generated samples are also discontinuous, making it difficult to assist the model in learning the intervals where ambiguous information exists between labels. Summary of the invention

[0004] In view of the above problems, the present disclosure provides a method and device for augmenting image data based on local linear embedding.

[0005] According to a first aspect of the present disclosure, a method for augmenting image data based on local linear embedding is provided, comprising: performing augmentation processing on initial image samples of an initial training set to obtain an intermediate training set, wherein the intermediate training set includes multiple image samples, the image samples include initial image samples and noise image samples obtained by augmentation, and each image sample has a corresponding initial label; for an i-th image sample in the intermediate training set, according to the initial label of the i-th image sample, determining a set of neighboring image samples corresponding to the i-th image sample, i≥0; performing local linear embedding processing on the i-th image sample according to the set of neighboring image samples corresponding to the i-th image sample to obtain a target linear combination coefficient corresponding to the i-th image sample, wherein the target linear combination coefficient corresponding to the i-th image sample is a coefficient of the i-th image sample linearly reconstructed by the set of neighboring image samples corresponding to the i-th image sample; performing local linear embedding processing on the initial label of the i-th image sample using the target linear combination coefficient corresponding to the i-th image sample to obtain a target label of the i-th image sample; updating the intermediate training set according to the target label corresponding to each image sample to obtain a target training set.

[0006] According to an embodiment of the present disclosure, an initial image sample of an initial training set is augmented to obtain an intermediate training set, including: dividing the initial training set into regions according to the initial labels of the initial image samples to obtain multiple sub-regions; among the multiple sub-regions, a sub-region that does not meet a preset region threshold is taken as a target sub-region; Gaussian noise is added to the initial image sample of the target sub-region to obtain a noise image sample; and an intermediate training set is generated according to the initial image sample and the noise image sample.

[0007] According to an embodiment of the present disclosure, the initial training set is divided into regions according to the initial labels of the initial image samples to obtain multiple sub-regions, including: determining the division length corresponding to the initial label according to the type of the initial label; and dividing the initial training set according to the division length to obtain multiple sub-regions.

[0008] According to an embodiment of the present disclosure, for the ith image sample of the intermediate training set, according to the initial label of the ith image sample, a set of neighboring image samples corresponding to the ith image sample is determined, including: according to the initial label of the ith image sample, a neighboring interval of the ith image sample is determined; the sample features of the ith image sample are respectively calculated with the sample features of n initial neighboring image samples in the neighboring interval of the ith image sample to obtain n distance values, n≥0; according to a preset selection range, m target distance values ​​are confirmed from the n distance values, 0≤m≤n; and the m initial neighboring image samples corresponding to the m target distance values ​​are used as the set of neighboring image samples corresponding to the ith image sample.

[0009] According to an embodiment of the present disclosure, a local linear embedding process is performed on the ith image sample according to a set of neighboring image samples corresponding to the ith image sample to obtain a target linear combination coefficient corresponding to the ith image sample, including: constructing a linear relationship between sample features of the ith image sample and sample features of m initial neighboring image samples of the set of neighboring image samples corresponding to the ith image sample; and determining the target linear combination coefficient corresponding to the ith image sample according to the linear relationship.

[0010] According to an embodiment of the present disclosure, a local linear embedding process is performed on the initial label of the ith image sample using the target linear combination coefficient corresponding to the ith image sample to obtain the target label of the ith image sample, including: format conversion of the initial labels of m neighboring image samples respectively to obtain intermediate labels corresponding to the m neighboring image samples, wherein the intermediate labels are in vector form; according to a preset relationship, the target linear combination coefficient corresponding to the ith image sample and the intermediate labels of the m neighboring image samples are embedded to obtain the target label of the ith image sample.

[0011] According to an embodiment of the present disclosure, the above method also includes: normalizing the initial image samples and the noise image samples to obtain processed initial image samples and processed noise image samples; generating an intermediate training set based on the initial image samples and the noise image samples includes: generating an intermediate training set based on the processed initial image samples and the processed noise image samples.

[0012] According to an embodiment of the present disclosure, an intermediate training set is updated according to the target label corresponding to each image sample to obtain a target training set, including: updating the initial label of the ith image sample according to the target label of the ith image sample to obtain an updated ith image sample; determining an updated training set according to the updated ith image sample; and merging the image samples of the intermediate training set into the updated training set to obtain a target training set.

[0013] A second aspect of the present disclosure provides a method for training an image regression model, comprising: training the image regression model using the target training set to obtain a trained image regression model.

[0014] The third aspect of the present disclosure provides an augmentation device for image data based on local linear embedding, comprising: an augmentation module, configured to perform augmentation processing on an initial image sample of an initial training set to obtain an intermediate training set, wherein the intermediate training set includes a plurality of image samples, the image samples include an initial image sample and a noise image sample obtained by augmentation, and each image sample has a corresponding initial label; a first determination module, configured to determine, for an i-th image sample in the intermediate training set, a set of neighboring image samples corresponding to the i-th image sample according to the initial label of the i-th image sample, where i≥0; a first processing module, configured to determine, according to the initial label of the i-th image sample, a set of neighboring image samples corresponding to the i-th image sample A set of neighboring image samples is used to perform local linear embedding processing on the ith image sample to obtain a target linear combination coefficient corresponding to the ith image sample, wherein the target linear combination coefficient corresponding to the ith image sample is a coefficient of the ith image sample linearly reconstructed by the set of neighboring image samples corresponding to the ith image sample; a second processing module is used to perform local linear embedding processing on the initial label of the ith image sample using the target linear combination coefficient corresponding to the ith image sample to obtain a target label of the ith image sample; and an updating module is used to update the intermediate training set according to the target label corresponding to each image sample to obtain a target training set.

[0015] A fourth aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0016] The fifth aspect of the present disclosure further provides a computer-readable storage medium on which a computer program or instruction is stored, and the steps of the above method are implemented when the above computer program or instruction is executed by a processor.

[0017] The sixth aspect of the present disclosure further provides a computer program product, including a computer program or instructions, which implement the steps of the above method when the above computer program or instructions are executed by a processor.

[0018] According to an embodiment of the present disclosure, for the i-th image sample in the intermediate training set, according to the initial label of the i-th image sample, a set of neighboring image samples corresponding to the i-th image sample is determined; according to the set of neighboring image samples corresponding to the i-th image sample, a local linear embedding process is performed on the i-th image sample to obtain a target linear combination coefficient corresponding to the i-th image sample; and the initial label is locally linearly embedded using the target linear combination coefficient to obtain a target label of the i-th image sample. Since the i-th image sample is locally linearly embedded according to the i-th set of neighboring image samples, and the initial label of the i-th sample is locally linearly embedded based on the obtained target linear combination coefficient, the problem of the continuity of the label generated on the basis of the discrete label being destroyed is avoided, and the continuity of the label in the deep regression is retained while the image sample is augmented, which helps to enhance the model's prediction ability for samples in the fuzzy label interval. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0020] Figure 1 A diagram schematically shows an application scenario of a method and device for augmenting image data based on local linear embedding according to an embodiment of the present disclosure;

[0021] Figure 2 Schematically shows a flow chart of an augmentation method for image data based on local linear embedding according to an embodiment of the present disclosure;

[0022] Figure 3 A schematic diagram schematically shows an augmentation method for image data based on local linear embedding according to an embodiment of the present disclosure;

[0023] Figure 4 A flowchart of a method for training an image regression model according to an embodiment of the present disclosure is schematically shown;

[0024] Figure 5 A structural block diagram of an image data augmentation device based on local linear embedding according to an embodiment of the present disclosure is schematically shown;

[0025] Figure 6 A structural block diagram schematically shows a training device for an image regression model according to an embodiment of the present disclosure; and

[0026] Figure 7 A block diagram of an electronic device suitable for implementing an augmentation method for image data based on local linear embedding according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0027] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0028] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0029] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0030] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0031] In the technical solution of the present disclosure, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0032] An embodiment of the present disclosure provides an augmentation method for image data based on local linear embedding, wherein an initial image sample of an initial training set is augmented to obtain an intermediate training set, wherein the intermediate training set includes multiple image samples, and the image samples include initial image samples and noise image samples obtained by augmentation, and each image sample has a corresponding initial label; for the i-th image sample of the intermediate training set, according to the initial label of the i-th image sample, a set of neighboring image samples corresponding to the i-th image sample is determined, i≥0; according to the set of neighboring image samples corresponding to the i-th image sample, a local linear embedding process is performed on the i-th image sample to obtain a target linear combination coefficient corresponding to the i-th image sample, wherein the target linear combination coefficient corresponding to the i-th image sample is a coefficient of the i-th image sample linearly reconstructed by the set of neighboring image samples corresponding to the i-th image sample; the initial label of the i-th image sample is locally linearly embedded using the target linear combination coefficient corresponding to the i-th image sample to obtain a target label of the i-th image sample; the intermediate training set is updated according to the target label corresponding to each image sample to obtain a target training set.

[0033] Figure 1 The application scenario diagram of the method and device for augmenting image data based on local linear embedding according to an embodiment of the present disclosure is schematically shown.

[0034] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0035] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0036] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0037] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0038] It should be noted that the method for augmenting image data provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the device for augmenting image data provided in the embodiment of the present disclosure can generally be set in the server 105. The method for augmenting image data provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the device for augmenting image data provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0039] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0040] Figure 2 The flowchart of the method for augmenting image data based on local linear embedding according to an embodiment of the present disclosure is schematically shown.

[0041] like Figure 2 As shown, the method for augmenting image data based on local linear embedding in this embodiment includes operations S210 to S250.

[0042] In operation S210, the initial image samples of the initial training set are augmented to obtain an intermediate training set, wherein the intermediate training set includes a plurality of image samples, the image samples include the initial image samples and the noise image samples obtained through augmentation, and each image sample has a corresponding initial label.

[0043] According to an embodiment of the present disclosure, the initial training set includes multiple initial image samples, each initial image sample corresponds to an initial label. For example, the initial training set can be an age training set, each initial image sample is an image associated with a person's age (such as a face image), and the initial label corresponding to the image is an age value.

[0044] According to an embodiment of the present disclosure, the augmentation process may be to add noise to the initial image sample to obtain a noisy image sample.

[0045] According to an embodiment of the present disclosure, the augmentation process may also be operations such as translating, flipping or scaling the initial image sample.

[0046] In operation S220 , for the i-th image sample in the intermediate training set, a set of neighboring image samples corresponding to the i-th image sample is determined according to the initial label of the i-th image sample, where i≥0.

[0047] According to an embodiment of the present disclosure, each image sample is processed separately using the K nearest neighbor algorithm to determine a set of neighboring image samples corresponding to each image sample. For the i-th image sample, the set of neighboring image samples corresponding to it is obtained based on the initial label of the i-th image sample, for example, based on the initial label division interval, the set of neighboring image samples corresponding to the i-th image sample is confirmed in the divided interval.

[0048] In operation S230, local linear embedding processing is performed on the ith image sample according to the set of neighboring image samples corresponding to the ith image sample to obtain a target linear combination coefficient corresponding to the ith image sample, wherein the target linear combination coefficient corresponding to the ith image sample is a coefficient of the ith image sample linearly reconstructed by the set of neighboring image samples corresponding to the ith image sample.

[0049] According to an embodiment of the present disclosure, Locally Linear Embedding (LLE) is a nonlinear dimensionality reduction technology that calculates the weight between each point and its neighboring points so that each point can be best reconstructed by a linear combination of its neighbors, and can capture and maintain the local linear structure of the data.

[0050] According to an embodiment of the present disclosure, the combination coefficient may be a coefficient for linearly reconstructing the ith image sample and each image sample in a set of neighboring image samples corresponding to the ith image sample.

[0051] According to an embodiment of the present disclosure, local linear embedding processing is performed on the sample features of the ith image sample to obtain a target linear combination coefficient corresponding to the ith image sample.

[0052] In operation S240, a local linear embedding process is performed on the initial label of the i-th image sample using the target linear combination coefficient corresponding to the i-th image sample to obtain a target label of the i-th image sample.

[0053] According to an embodiment of the present disclosure, based on the target linear combination coefficient, the continuity of the sample features of the ith image sample is embedded into the initial label of the ith image sample to obtain the target label of the ith image sample, which has continuity.

[0054] In operation S250, the intermediate training set is updated according to the target label corresponding to each image sample to obtain a target training set.

[0055] According to an embodiment of the present disclosure, the intermediate training set is updated based on the target label, so that the obtained target training set helps to enhance the prediction ability of the model for samples in the fuzzy label interval.

[0056] According to an embodiment of the present disclosure, for the i-th image sample in the intermediate training set, according to the initial label of the i-th image sample, a set of neighboring image samples corresponding to the i-th image sample is determined; according to the set of neighboring image samples corresponding to the i-th image sample, a local linear embedding process is performed on the i-th image sample to obtain a target linear combination coefficient corresponding to the i-th image sample; and the initial label is locally linearly embedded using the target linear combination coefficient to obtain a target label of the i-th image sample. Since the i-th image sample is locally linearly embedded according to the i-th set of neighboring image samples, and the initial label of the i-th sample is locally linearly embedded based on the obtained target linear combination coefficient, the problem of the continuity of the label generated on the basis of the discrete label being destroyed is avoided, and the continuity of the label in the deep regression is retained while the image sample is augmented, which helps to enhance the model's prediction ability for samples in the fuzzy label interval.

[0057] According to an embodiment of the present disclosure, an initial image sample of an initial training set is augmented to obtain an intermediate training set, including: dividing the initial training set into regions according to the initial labels of the initial image samples to obtain multiple sub-regions; among the multiple sub-regions, a sub-region that does not meet a preset region threshold is taken as a target sub-region; Gaussian noise is added to the initial image sample of the target sub-region to obtain a noise image sample; and an intermediate training set is generated according to the initial image sample and the noise image sample.

[0058] According to an embodiment of the present disclosure, the initial label of each initial image sample is discretized and divided into multiple intervals, ie, sub-regions, and each sub-region may include at least one initial image sample corresponding to the initial label.

[0059] According to an embodiment of the present disclosure, the preset area threshold may be a ratio of the number of initial image samples in any sub-area to the number of all initial image samples. The ratio may be set according to actual needs and is not limited here.

[0060] According to an embodiment of the present disclosure, the number of initial image samples in each sub-region is counted, and the sub-region that does not meet the preset region threshold is taken as the target sub-region.

[0061] According to an embodiment of the present disclosure, for each initial image sample in the target sub-region, random Gaussian noise is added to obtain a noisy image sample.

[0062] According to an embodiment of the present disclosure, noise samples are inserted into an initial training set to generate an intermediate training set.

[0063] According to an embodiment of the present disclosure, a target sub-region with a small number of samples is identified in multiple sub-regions using a preset region threshold, and noise addition processing is performed on initial image samples of the target sub-region to achieve sample augmentation.

[0064] According to an embodiment of the present disclosure, the initial training set is divided into regions according to the initial labels of the initial image samples to obtain multiple sub-regions, including: determining the division length corresponding to the initial label according to the type of the initial label; and dividing the initial training set according to the division length to obtain multiple sub-regions.

[0065] According to an embodiment of the present disclosure, the type of the initial label may be age, temperature, or the like.

[0066] According to an embodiment of the present disclosure, the division length can be determined based on the minimum granularity corresponding to different types of samples. For example, when the type of the initial label is age, the corresponding division length is 1 year old, and the obtained sub-areas can be 58 to 59 years old, 59 to 60 years old, etc.

[0067] According to an embodiment of the present disclosure, for the ith image sample of the intermediate training set, according to the initial label of the ith image sample, a set of neighboring image samples corresponding to the ith image sample is determined, including: according to the initial label of the ith image sample, a neighboring interval of the ith image sample is determined; the sample features of the ith image sample are respectively calculated with the sample features of n initial neighboring image samples in the neighboring interval of the ith image sample to obtain n distance values, n≥0; according to a preset selection range, m target distance values ​​are confirmed from the n distance values, 0≤m≤n; and the m initial neighboring image samples corresponding to the m target distance values ​​are used as the set of neighboring image samples corresponding to the ith image sample.

[0068] According to an embodiment of the present disclosure, the initial label of the i-th image sample can be , then the neighboring interval of the i-th image sample , where the value of b can be set according to actual needs and is not limited here.

[0069] According to an embodiment of the present disclosure, a sample feature of an i-th image sample and a sample feature of each of n initial neighboring image samples in a neighborhood of the i-th image sample are determined.

[0070] According to an embodiment of the present disclosure, the Euclidean distance between the sample feature of the i-th image sample and the sample features of n initial neighboring image samples is calculated based on formula (1) to obtain n distance values.

[0071] (1);

[0072] in, is the sample feature of the i-th image sample, is the sample feature of the jth sample among the n initial neighbor image samples.

[0073] According to an embodiment of the present disclosure, the preset selection range may be m initial neighbor image samples with the smallest distance values, wherein the value of m may be set according to actual needs and is not limited here.

[0074] According to an embodiment of the present disclosure, m initial neighbor image samples are used as a set of neighbor image samples corresponding to the i-th image sample.

[0075] According to the embodiments of the present disclosure, searching is performed only within the sample range contained in the neighboring interval, so that the model does not need to traverse all samples in the intermediate training set, thereby improving the confirmation efficiency of the m initial neighbor image samples.

[0076] According to an embodiment of the present disclosure, a local linear embedding process is performed on the ith image sample according to a set of neighboring image samples corresponding to the ith image sample to obtain a target linear combination coefficient corresponding to the ith image sample, including: constructing a linear relationship between sample features of the ith image sample and sample features of m initial neighboring image samples of the set of neighboring image samples corresponding to the ith image sample; and determining the target linear combination coefficient corresponding to the ith image sample according to the linear relationship.

[0077] According to an embodiment of the present disclosure, the linear relationship between the sample features of the ith image sample and the sample features of the m initial neighboring image samples of the neighboring image sample set corresponding to the ith image sample may be as shown in formula (2).

[0078] (2);

[0079] in, The target linear combination coefficient corresponding to the i-th image sample.

[0080] According to an embodiment of the present disclosure, is the reconstruction error between each image sample and its respective m initial neighbor image samples The coefficients of the minimum solution can be shown as formula (3) and formula (4).

[0081] (3);

[0082] (4);

[0083] According to an embodiment of the present disclosure, In matrix form, Can be The i-th row and j-th column in .

[0084] According to an embodiment of the present disclosure, by confirming a target linear combination coefficient corresponding to the ith image sample, an optimal linear relationship is confirmed between sample features of the ith image sample and sample features of a set of neighboring image samples of the ith image sample.

[0085] According to an embodiment of the present disclosure, a local linear embedding process is performed on the initial label of the ith image sample using the target linear combination coefficient corresponding to the ith image sample to obtain the target label of the ith image sample, including: format conversion of the initial labels of m neighboring image samples respectively to obtain intermediate labels corresponding to the m neighboring image samples, wherein the intermediate labels are in vector form; according to a preset relationship, the target linear combination coefficient corresponding to the ith image sample and the intermediate labels of the m neighboring image samples are embedded to obtain the target label of the ith image sample.

[0086] According to an embodiment of the present disclosure, the initial labels of the m neighboring image samples are formatted and converted into binary vector form. For example, the initial label of the jth sample of the m neighboring image samples is , then convert it to an intermediate label , where the a-1th element is 1 and the rest are 0.

[0087] According to an embodiment of the present disclosure, based on a preset relationship, the target linear combination coefficient corresponding to the ith image sample and the intermediate labels of the m neighboring image samples are embedded to obtain the target label of the ith image sample, which can be shown as formula (5).

[0088] (5);

[0089] in, is the target label of the i-th image sample, is the middle label of the jth neighbor image sample among the m neighbor image samples.

[0090] According to an embodiment of the present disclosure, by mapping the linear relationship between the sample features of the ith image sample and the sample features of m neighboring image samples to the initial label of the ith image sample, a label similar to a probability distribution form is given to the initial label of the ith image sample, thereby avoiding the loss of initial label information of the fuzzy interval after discretization and retaining the label continuity in deep regression, which helps to enhance the model's prediction ability for image samples with fuzzy interval labels.

[0091] According to an embodiment of the present disclosure, the above method also includes: normalizing the initial image samples and the noise image samples to obtain processed initial image samples and processed noise image samples; generating an intermediate training set based on the initial image samples and the noise image samples includes: generating an intermediate training set based on the processed initial image samples and the processed noise image samples.

[0092] According to an embodiment of the present disclosure, the initial image samples and the noise image samples are normalized, as specifically shown in formula (6).

[0093] (6);

[0094] in, is the ith component of the i-th image sample in the initial image sample and the noise image sample, is the i-th image sample in the normalized initial image sample and noise image sample The sample characteristics.

[0095] According to an embodiment of the present disclosure, an intermediate training set is formed using processed initial image samples and processed noise image samples.

[0096] According to an embodiment of the present disclosure, an intermediate training set is updated according to the target label corresponding to each image sample to obtain a target training set, including: updating the initial label of the ith image sample according to the target label of the ith image sample to obtain an updated ith image sample; determining an updated training set according to the updated ith image sample; and merging the image samples of the intermediate training set into the updated training set to obtain a target training set.

[0097] According to an embodiment of the present disclosure, image samples of the intermediate training set are incorporated into the updated training set to obtain a target training set, so as to enhance the diversity of data and thereby improve the training effect of the model.

[0098] Figure 3 A schematic diagram of an augmentation method for image data based on local linear embedding according to an embodiment of the present disclosure is schematically shown.

[0099] According to the embodiments of the present disclosure, Figure 3As shown, the method for augmenting image data based on local linear embedding includes operations S310 to S360.

[0100] In operation S310, an augmentation process is performed on initial image samples of an initial training set to obtain an intermediate training set.

[0101] In operation S320, the intermediate training set is processed to determine a set of neighboring image samples corresponding to the i-th image sample.

[0102] In operation S330 , a linear relationship is constructed between a sample feature of the i th image sample and sample features of m initial neighbor image samples of a set of neighbor image samples corresponding to the i th image sample.

[0103] In operation S340, a target linear combination coefficient corresponding to the i-th image sample is determined.

[0104] In operation S350, according to a preset relationship, embedding processing is performed on the target linear combination coefficient corresponding to the ith image sample and the intermediate labels of the m neighboring image samples to obtain a target label of the ith image sample.

[0105] In operation S360, the intermediate training set is updated according to the target label corresponding to each image sample to obtain a target training set.

[0106] Figure 4 The flowchart of the training method of the image regression model according to the embodiment of the present disclosure is schematically shown.

[0107] The second aspect of the present disclosure provides a method for training an image regression model, such as Figure 4 The training method of the image regression model includes operation S410, training the image regression model using a target training set to obtain a trained image regression model.

[0108] According to the embodiments of the present disclosure, the image regression model is trained using the target training set, which avoids data imbalance while maintaining the continuity of sample data, thereby achieving significant model training effects.

[0109] Based on the above-mentioned method for augmenting image data, the present disclosure also provides an apparatus for augmenting image data based on local linear embedding. Figure 5 The device is described in detail.

[0110] Figure 5 The structural block diagram of the device for augmenting image data based on local linear embedding according to an embodiment of the present disclosure is schematically shown.

[0111] like Figure 5As shown, the apparatus 500 for augmenting image data based on local linear embedding of this embodiment includes an augmenting module 510 , a first determining module 520 , a first processing module 530 , a second processing module 540 and an updating module 550 .

[0112] The augmentation module 510 is used to perform augmentation processing on the initial image samples of the initial training set to obtain an intermediate training set, wherein the intermediate training set includes multiple image samples, the image samples include the initial image samples and the noise image samples obtained by augmentation, and each image sample has a corresponding initial label. In one embodiment, the augmentation module 510 can be used to perform the operation S210 described above, which will not be repeated here.

[0113] The first determination module 520 is used to determine, for the i-th image sample of the intermediate training set, a set of neighboring image samples corresponding to the i-th image sample according to the initial label of the i-th image sample, where i≥0. In one embodiment, the first determination module 520 can be used to perform the operation S220 described above, which will not be described in detail herein.

[0114] The first processing module 530 is used to perform local linear embedding processing on the ith image sample according to the set of neighboring image samples corresponding to the ith image sample, to obtain a target linear combination coefficient corresponding to the ith image sample, wherein the target linear combination coefficient corresponding to the ith image sample is a coefficient of the ith image sample linearly reconstructed by the set of neighboring image samples corresponding to the ith image sample. In one embodiment, the first processing module 530 can be used to perform the operation S230 described above, which will not be repeated here.

[0115] The second processing module 540 is used to perform local linear embedding processing on the initial label of the i-th image sample using the target linear combination coefficient corresponding to the i-th image sample to obtain the target label of the i-th image sample. In one embodiment, the second processing module 540 can be used to perform the operation S240 described above, which will not be repeated here.

[0116] The updating module 550 is used to update the intermediate training set according to the target label corresponding to each image sample to obtain the target training set. In one embodiment, the updating module 550 can be used to perform the operation S250 described above, which will not be described in detail here.

[0117] According to an embodiment of the present disclosure, for the i-th image sample in the intermediate training set, according to the initial label of the i-th image sample, a set of neighboring image samples corresponding to the i-th image sample is determined; according to the set of neighboring image samples corresponding to the i-th image sample, a local linear embedding process is performed on the i-th image sample to obtain a target linear combination coefficient corresponding to the i-th image sample; and the initial label is locally linearly embedded using the target linear combination coefficient to obtain a target label of the i-th image sample. Since the i-th image sample is locally linearly embedded according to the i-th set of neighboring image samples, and the initial label of the i-th sample is locally linearly embedded based on the obtained target linear combination coefficient, the problem of the continuity of the label generated on the basis of the discrete label being destroyed is avoided, and the continuity of the label in the deep regression is retained while the image sample is augmented, which helps to enhance the model's prediction ability for samples in the fuzzy label interval.

[0118] According to an embodiment of the present disclosure, the augmentation module 510 includes a division submodule, a first as submodule, an addition submodule, and a generation submodule.

[0119] The division submodule is used to divide the initial training set into regions according to the initial labels of the initial image samples to obtain multiple sub-regions.

[0120] The first is a submodule, which is used to take a subregion that does not meet a preset region threshold as a target subregion among multiple subregions.

[0121] The adding submodule is used to add Gaussian noise to the initial image samples of the target sub-region to obtain noisy image samples.

[0122] The generation submodule is used to generate an intermediate training set according to the initial image samples and the noise image samples.

[0123] According to an embodiment of the present disclosure, the division submodule includes a first determining unit and a division unit.

[0124] The first determining unit is used to determine the division length corresponding to the initial label according to the type of the initial label.

[0125] The partitioning unit is used to partition the initial training set according to the partition length to obtain multiple sub-regions.

[0126] According to an embodiment of the present disclosure, the first determining module 520 includes a first determining submodule, a calculating submodule, a second determining submodule, and a second calculating submodule.

[0127] The first determination submodule determines the neighboring interval of the i-th image sample according to the initial label of the i-th image sample.

[0128] The calculation submodule is used to calculate the distance values ​​between the sample features of the i-th image sample and the sample features of n initial neighboring image samples in the neighboring interval of the i-th image sample, and obtain n distance values, where n≥0.

[0129] The second determination submodule confirms m target distance values ​​from n distance values ​​according to a preset selection range, 0≤m≤n.

[0130] The second is a submodule for taking the m initial neighbor image samples corresponding to the m target distance values ​​as the neighbor image sample set corresponding to the i-th image sample.

[0131] According to an embodiment of the present disclosure, the first processing module 530 includes a construction submodule and a third determination submodule.

[0132] The construction submodule is used to construct a linear relationship between the sample features of the i-th image sample and the sample features of the m initial neighbor image samples of the neighbor image sample set corresponding to the i-th image sample.

[0133] The third determination submodule is used to determine the target linear combination coefficient corresponding to the i-th image sample according to the linear relationship.

[0134] According to an embodiment of the present disclosure, the second processing module 540 includes a conversion submodule and a first processing submodule.

[0135] The conversion submodule is used to convert the formats of the initial labels of the m neighboring image samples respectively to obtain the intermediate labels corresponding to the m neighboring image samples respectively, wherein the intermediate labels are in vector form.

[0136] The first processing submodule is used to embed the target linear combination coefficient corresponding to the ith image sample and the intermediate labels of the m neighboring image samples according to a preset relationship to obtain the target label of the ith image sample.

[0137] According to an embodiment of the present disclosure, the apparatus 500 for augmenting image data based on locally linear embedding further includes a third processing module.

[0138] The third processing module is used to perform normalization processing on the initial image samples and the noise image samples to obtain processed initial image samples and processed noise image samples.

[0139] Among them, the generation submodule includes a generation unit.

[0140] The generating unit is used to generate an intermediate training set according to the processed initial image samples and the processed noise image samples.

[0141] According to an embodiment of the present disclosure, the updating module 550 includes an updating submodule, a fourth determining submodule, and an incorporating submodule.

[0142] The updating submodule is used to update the initial label of the ith image sample according to the target label of the ith image sample to obtain the updated ith image sample.

[0143] The fourth determination submodule is used to determine the updated training set according to the updated i-th image sample.

[0144] The merging submodule is used to merge the image samples of the intermediate training set into the updated training set to obtain the target training set.

[0145] According to an embodiment of the present disclosure, any multiple modules of the augmentation module 510, the first determination module 520, the first processing module 530, the second processing module 540 and the update module 550 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the augmentation module 510, the first determination module 520, the first processing module 530, the second processing module 540 and the update module 550 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in a suitable combination of any of them. Alternatively, at least one of the augmentation module 510, the first determination module 520, the first processing module 530, the second processing module 540 and the update module 550 may be at least partially implemented as a computer program module, which may perform corresponding functions when executed.

[0146] Figure 6 The structural block diagram of the image regression model training device according to the embodiment of the present disclosure is schematically shown.

[0147] like Figure 6 As shown, the training device 600 of the image regression model includes a training module 610.

[0148] The training module 610 is used to train the image regression model using the target training set to obtain a trained image regression model.

[0149] Figure 7A block diagram of an electronic device suitable for implementing an augmentation method for image data according to an embodiment of the present disclosure is schematically shown.

[0150] like Figure 7 As shown, the electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 to a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include an onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0151] In RAM 703, various programs and data required for the operation of electronic device 700 are stored. Processor 701, ROM 702 and RAM 703 are connected to each other via bus 704. Processor 701 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 702 and / or RAM 703. It should be noted that the program can also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0152] According to an embodiment of the present disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to the bus 704. The electronic device 700 may further include one or more of the following components connected to the input / output (I / O) interface 705: an input portion 706 including a keyboard, a mouse, etc.; an output portion 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 708 including a hard disk, etc.; and a communication portion 709 including a network interface card such as a LAN card, a modem, etc. The communication portion 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage portion 708 as needed.

[0153] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0154] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 702 and / or RAM 703 described above and / or one or more memories other than ROM 702 and RAM 703.

[0155] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the above method provided by the embodiment of the present disclosure.

[0156] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 701. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0157] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 709, and / or installed from the removable medium 711. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0158] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0159] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0160] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0161] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0162] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for augmenting image data based on locally linear embedding, characterized in that: The method comprises: Performing augmentation processing on the initial image samples of the initial training set to obtain an intermediate training set, wherein the intermediate training set includes a plurality of image samples, the image samples include the initial image samples and the noise image samples obtained through augmentation, and each of the image samples has a corresponding initial label; For the i-th image sample of the intermediate training set, determine a set of neighboring image samples corresponding to the i-th image sample according to the initial label of the i-th image sample, where i≥0; According to the set of neighboring image samples corresponding to the ith image sample, perform local linear embedding processing on the ith image sample to obtain a target linear combination coefficient corresponding to the ith image sample, wherein the target linear combination coefficient corresponding to the ith image sample is a coefficient of the ith image sample linearly reconstructed by the set of neighboring image samples corresponding to the ith image sample; Performing local linear embedding processing on the initial label of the i-th image sample using the target linear combination coefficient corresponding to the i-th image sample to obtain the target label of the i-th image sample; The intermediate training set is updated according to the target label corresponding to each of the image samples to obtain a target training set.

2. The method according to claim 1, characterized in that: The initial image samples of the initial training set are augmented to obtain an intermediate training set, including: According to the initial labels of the initial image samples, the initial training set is divided into regions to obtain a plurality of sub-regions; Among the multiple sub-regions, taking the sub-region that does not meet the preset region threshold as the target sub-region; Adding Gaussian noise to the initial image sample of the target sub-region to obtain the noisy image sample; The intermediate training set is generated according to the initial image samples and the noise image samples.

3. The method according to claim 2, characterized in that The initial training set is divided into regions according to the initial labels of the initial image samples to obtain a plurality of sub-regions, including: Determining, according to the type of the initial label, a division length corresponding to the initial label; The initial training set is divided according to the division length to obtain a plurality of sub-regions.

4. The method according to claim 1, characterized in that The step of determining, for the i-th image sample of the intermediate training set, according to the initial label of the i-th image sample, a set of neighboring image samples corresponding to the i-th image sample comprises: Determining a neighboring interval of the i-th image sample according to the initial label of the i-th image sample; Calculate the distance between the sample features of the i-th image sample and the sample features of n initial neighboring image samples in the neighboring interval of the i-th image sample to obtain n distance values, where n≥0; According to the preset selection range, m target distance values ​​are confirmed from the n distance values, 0≤m≤n; The m initial neighbor image samples corresponding to the m target distance values ​​are used as a neighbor image sample set corresponding to the i-th image sample.

5. The method according to claim 4, characterized in that The performing local linear embedding processing on the i-th image sample according to the set of neighboring image samples corresponding to the i-th image sample to obtain the target linear combination coefficient corresponding to the i-th image sample includes: Constructing a linear relationship between the sample feature of the i-th image sample and the sample features of the m initial neighboring image samples of the neighboring image sample set corresponding to the i-th image sample; According to the linear relationship, a target linear combination coefficient corresponding to the i-th image sample is determined.

6. The method according to claim 5, characterized in that The method of performing local linear embedding processing on the initial label of the i-th image sample by using the target linear combination coefficient corresponding to the i-th image sample to obtain the target label of the i-th image sample includes: Performing format conversion on the initial labels of the m neighboring image samples respectively to obtain intermediate labels corresponding to the m neighboring image samples respectively, wherein the intermediate labels are in vector form; According to a preset relationship, the target linear combination coefficient corresponding to the ith image sample and the intermediate labels of the m neighboring image samples are embedded to obtain the target label of the ith image sample.

7. The method according to claim 2, characterized in that: The method further comprises: Normalizing the initial image samples and the noise image samples to obtain processed initial image samples and processed noise image samples; The generating the intermediate training set according to the initial image sample and the noise image sample comprises: The intermediate training set is generated according to the processed initial image samples and the processed noise image samples.

8. The method according to claim 1, characterized in that The updating of the intermediate training set according to the target label corresponding to each of the image samples to obtain the target training set includes: According to the target label of the i-th image sample, the initial label of the i-th image sample is updated to obtain the updated i-th image sample; Determine an updated training set according to the updated i-th image sample; The image samples of the intermediate training set are merged into the updated training set to obtain the target training set.

9. A method for training an image regression model, characterized in that: The method comprises: The image regression model is trained using the target training set as described in any one of claims 1 to 8 to obtain the trained image regression model.

10. An image data augmentation device based on local linear embedding, characterized in that: The device comprises: An augmentation module, used to perform augmentation processing on the initial image samples of the initial training set to obtain an intermediate training set, wherein the intermediate training set includes a plurality of image samples, the image samples include the initial image samples and the noise image samples obtained by augmentation, and each of the image samples has a corresponding initial label; A first determination module is used to determine, for an i-th image sample in the intermediate training set, a set of neighboring image samples corresponding to the i-th image sample according to an initial label of the i-th image sample, where i≥0; A first processing module is used to perform local linear embedding processing on the i-th image sample according to the set of neighboring image samples corresponding to the i-th image sample, so as to obtain a target linear combination coefficient corresponding to the i-th image sample, wherein the target linear combination coefficient corresponding to the i-th image sample is a coefficient of the i-th image sample linearly reconstructed by the set of neighboring image samples corresponding to the i-th image sample; a second processing module, configured to perform local linear embedding processing on the initial label of the i-th image sample using the target linear combination coefficient corresponding to the i-th image sample to obtain a target label of the i-th image sample; and The updating module is used to update the intermediate training set according to the target label corresponding to each of the image samples to obtain a target training set.

Images