A multi-group sample construction method and device

Abstract

An embodiment of the present invention provides a method and device for constructing a multi-tuple sample, wherein the method includes: obtaining a first feature set composed of first sample features; obtaining a first multi-tuple sample required for the current training of a preset model to be trained according to the first feature set; obtaining a second sample feature by forward calculation according to the first multi-tuple sample by the model to be trained; in the first feature set, updating the first sample feature corresponding to the first multi-tuple sample to the second sample feature to obtain a second feature set; obtaining a second multi-tuple sample required for the next training of the model to be trained according to the second feature set. The present invention improves the quality of multi-tuple sample construction, and the multi-tuple sample constructed using this method can adapt to the learning of different stages of the model to be trained, thereby improving the convergence speed and effect of the model to be trained.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning and image recognition, and in particular to a multi-tuple sample construction method and device. Background Art

[0002] In metric learning tasks, the network is usually supervised to learn the similarity between images by constructing a multi-tuple consisting of similar images and dissimilar images. The multi-tuple samples include reference samples (anchors), positive samples (positive), and negative samples (negative). The reference samples are similar to the positive samples, but not similar to the negative samples.

[0003] In the current multi-group sample construction strategy, samples are usually traversed and calculated in a fixed sample training set. Due to low computational efficiency, multi-group samples can only be constructed in a small range of the sample training set. The quality of the samples constructed in this way is low and cannot effectively guide the learning of different stages of the model to be trained. Summary of the invention

[0004] In view of this, the purpose of an embodiment of the present invention is to provide a method and device for constructing a multi-group sample. The present invention improves the quality of multi-group sample construction; the multi-group sample constructed using this method can adapt to the learning of different stages of the model to be trained, thereby improving the convergence speed and effect of the model to be trained.

[0005] In the first aspect, the present application provides the following technical solutions through an embodiment of the present application:

[0006] A multivariate sample construction method, comprising:

[0007] Acquire a first feature set consisting of first sample features; obtain a first multi-group sample required for the current training of a preset model to be trained based on the first feature set; perform forward calculation based on the first multi-group sample through the model to be trained to obtain second sample features; in the first feature set, update the first sample features corresponding to the first multi-group sample to the second sample features to obtain a second feature set; obtain a second multi-group sample required for the next training of the model to be trained based on the second feature set.

[0008] Preferably, the step of obtaining a first multi-tuple sample required for the current training of the preset model to be trained according to the first feature set includes:

[0009] According to the random feature, a reference feature is extracted from the first feature set, and a reference sample is obtained based on the reference feature, wherein the random feature is randomly selected from the first feature set, and the first sample features similar to the reference feature in the first feature set are all marked with an identification mark; according to the reference sample and the first sample feature marked with the identification mark, a positive sample corresponding to the reference sample is obtained; according to the reference sample and the first sample feature not marked with the identification mark, a negative sample corresponding to the reference sample is obtained; according to the reference sample, the positive sample corresponding to the reference sample, and the negative sample corresponding to the reference sample, a first multi-group sample is obtained, and each reference sample corresponds to a multi-group sample.

[0010] Preferably, the step of extracting a reference feature from the first feature set according to the random feature and obtaining a reference sample based on the reference feature comprises:

[0011] Obtain a first similarity between the random feature and each first sample feature; when the first similarity belongs to a preset first range, extract the first sample feature corresponding to the first similarity belonging to the first range as a reference feature; and use the sample corresponding to the reference feature as the reference sample.

[0012] Preferably, the step of obtaining a positive sample corresponding to the reference sample according to the reference sample and the first sample feature marked with the identification mark comprises:

[0013] For the same reference sample, obtain the second similarity between the reference feature corresponding to the reference sample and the first sample feature marked with the identification mark; when the second similarity belongs to a preset second range, extract the first sample feature corresponding to the second similarity belonging to the second range as the feature of the positive sample corresponding to the reference sample; based on the feature of the positive sample, obtain the positive sample corresponding to the reference sample.

[0014] Preferably, the step of obtaining a negative sample corresponding to the reference sample based on the reference sample and the first sample feature without the identification mark includes:

[0015] For the same reference sample, obtain the third similarity between the reference feature corresponding to the reference sample and the first sample feature not marked with the identification mark; when the third similarity belongs to a preset third range, extract the first sample feature corresponding to the third similarity belonging to the third range as the feature of the negative sample corresponding to the reference sample; according to the feature of the negative sample, obtain the negative sample corresponding to the reference sample.

[0016] Preferably, the first sample feature and the second sample feature are both features of a picture.

[0017] In the second aspect, based on the same inventive concept, the present application provides the following technical solution through an embodiment of the present application:

[0018] A multi-group sample construction device, comprising:

[0019] A first feature set acquisition module is used to acquire a first feature set composed of first sample features; a first sample acquisition module is used to obtain, based on the first feature set, a first multi-group sample required for the current training of a preset model to be trained; a forward calculation module is used to perform forward calculation based on the first multi-group sample through the model to be trained to obtain second sample features; an update module is used to update the first sample features corresponding to the first multi-group sample in the first feature set to the second sample features to obtain a second feature set; a second sample acquisition module is used to obtain, based on the second feature set, a second multi-group sample required for the next training of the model to be trained.

[0020] Preferably, the first sample acquisition module is further used for:

[0021] According to the random feature, a reference feature is extracted from the first feature set, and a reference sample is obtained based on the reference feature, wherein the random feature is randomly selected from the first feature set, and the first sample features similar to the reference feature in the first feature set are all marked with an identification mark; according to the reference sample and the first sample feature marked with the identification mark, a positive sample corresponding to the reference sample is obtained; according to the reference sample and the first sample feature not marked with the identification mark, a negative sample corresponding to the reference sample is obtained; according to the reference sample, the positive sample corresponding to the reference sample, and the negative sample corresponding to the reference sample, a first multi-group sample is obtained, and each reference sample corresponds to a multi-group sample.

[0022] Preferably, the first sample acquisition module is further used for:

[0023] Obtain a first similarity between the random feature and each first sample feature; when the first similarity belongs to a preset first range, extract the first sample feature corresponding to the first similarity belonging to the first range as a reference feature; and use the sample corresponding to the reference feature as the reference sample.

[0024] Preferably, the first sample acquisition module is further used for:

[0025] For the same reference sample, obtain the second similarity between the reference feature corresponding to the reference sample and the first sample feature marked with the identification mark; when the second similarity belongs to a preset second range, extract the first sample feature corresponding to the second similarity belonging to the second range as the feature of the positive sample corresponding to the reference sample; based on the feature of the positive sample, obtain the positive sample corresponding to the reference sample.

[0026] Preferably, the first sample acquisition module is further used for:

[0027] For the same reference sample, obtain the third similarity between the reference feature corresponding to the reference sample and the first sample feature not marked with the identification mark; when the third similarity belongs to a preset third range, extract the first sample feature corresponding to the third similarity belonging to the third range as the feature of the negative sample corresponding to the reference sample; according to the feature of the negative sample, obtain the negative sample corresponding to the reference sample.

[0028] Preferably, the first sample feature and the second sample feature are both features of a picture.

[0029] In the third aspect, based on the same inventive concept, the present application provides the following technical solution through an embodiment of the present application:

[0030] A multi-tuple sample construction device includes a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors. The one or more programs include instructions for performing the following operations:

[0031] Acquire a first feature set consisting of first sample features; obtain a first multi-group sample required for the current training of a preset model to be trained based on the first feature set; perform forward calculation based on the first multi-group sample through the model to be trained to obtain second sample features; in the first feature set, update the first sample features corresponding to the first multi-group sample to the second sample features to obtain a second feature set; obtain a second multi-group sample required for the next training of the model to be trained based on the second feature set.

[0032] In the fourth aspect, based on the same inventive concept, the present application provides the following technical solution through an embodiment of the present application:

[0033] A machine-readable medium stores instructions thereon, which, when executed by one or more processors, enable a device to execute the multi-tuple sample construction method as described in any one of the first aspects above.

[0034] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0035] The present invention obtains a second feature set by updating the first sample features in the first feature set using the second sample features obtained during the training of the first multi-group sample in the training process of the model to be trained. This ensures that when the model to be trained mines learning samples at different training stages, the distance distribution of the sample features is constantly updated and changed. When mining the second multi-group sample, it will be more targeted, and multi-group samples with higher quality and more suitable for the current training stage can be mined. The multi-group samples obtained using the method of the present invention can adapt to the learning of the model to be trained at different stages, guide the model to be trained to reach a better solution more quickly, and improve the convergence speed and effect of the model to be trained.

[0036] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0038] Figure 1 A flowchart of a multi-group sample construction method provided by the first embodiment of the present invention;

[0039] Figure 2 Based Figure 1 Specific flow chart of step S20;

[0040] Figure 3 A functional module block diagram of a multi-tuple sample construction device provided by the second embodiment of the present invention;

[0041] Figure 4 A structural block diagram of a multi-tuple sample construction device provided by the third embodiment of the present invention;

[0042] Figure 5 A structural block diagram of an exemplary server provided for the fourth embodiment of the present invention. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0044] For ease of explanation and understanding, embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0045] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0046] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as herein.

[0047] First embodiment

[0048] Please refer to Figure 1 In this embodiment, a multi-group sample construction method is provided. Figure 1 The specific process of the method in this embodiment is shown. In this embodiment, each step of the method will be described in detail with reference to the accompanying drawings. The multi-group sample construction method specifically includes:

[0049] Step S10: Acquire a first feature set consisting of first sample features.

[0050] Step S20: According to the first feature set, a first multi-tuple sample required for the current training of the preset model to be trained is obtained.

[0051] Step S30: performing forward calculation according to the first multi-tuple sample by the model to be trained to obtain second sample features.

[0052] Step S40: In the first feature set, the first sample feature corresponding to the first multi-group sample is updated to the second sample feature to obtain a second feature set.

[0053] Step S50: obtaining, according to the second feature set, a second multi-tuple sample required for the next training of the model to be trained.

[0054] In step S10, a first feature set consisting of first sample features is obtained.

[0055] Each first sample feature corresponds to a sample. The first sample feature can be obtained during the forward calculation of the network model, or can be obtained by vectorizing the sample. The first sample feature can be expressed as a feature vector. The first feature set corresponds to a sample set.

[0056] In this embodiment, the network model includes, but is not limited to, convolutional neural network models of various architectures, such as: VGG16, VGG19 (Visual Geometry Group, VGG), ResNet50 model, Inception V3 model, Xception model, etc. The training and use of the above models can be directly implemented by ordinary technicians in the relevant technical field and will not be repeated here.

[0057] In step S20, a first multi-tuple group of samples required for the current training of the preset model to be trained is obtained according to the first feature set.

[0058] In step S20, the model to be trained can be any of the above network models. During the training process of the model to be trained, training samples can be constructed in batches for different training stages of the model to be trained, so that the model to be trained can quickly converge to a better value. Therefore, the training process of the model to be trained can be decomposed into multiple times, and a batch of multi-tuple samples is used for each training.

[0059] The current time mentioned in step S20 may be any training time among the multiple trainings which is not the last time.

[0060] Specifically, this embodiment provides a specific implementation method for obtaining the first multi-tuple sample of the current time. Figure 2 , which includes:

[0061] Step S21: extracting reference features from the first feature set according to random features, and obtaining reference samples based on the reference features, wherein the random features are randomly selected from the first feature set, and first sample features similar to the reference features in the first feature set are all marked with identification marks.

[0062] Step S22: obtaining a positive sample corresponding to the reference sample according to the reference sample and the first sample feature marked with the identification mark;

[0063] Step S23: obtaining a negative sample corresponding to the reference sample according to the reference sample and the first sample feature without the identification mark;

[0064] Step S24: obtaining a first multi-tuple sample according to the reference sample, the positive sample corresponding to the reference sample, and the negative sample corresponding to the reference sample, wherein each reference sample corresponds to a multi-tuple sample.

[0065] In step S21, the random feature is randomly selected from the first feature set. In this embodiment, the first feature set includes a corresponding reference set, and the random feature may be a feature randomly selected from the reference set. Similarly, a sample may be randomly extracted from the sample set as a random sample, and then the random sample may correspond to the first sample feature in the first feature set as a random feature. In addition, in some embodiments, random features may also be obtained from outside the first feature set. For example: a picture is randomly determined as a random sample, and then the picture is vectorized and learned to obtain a feature vector of the picture. The feature vector may be used for subsequent calculations with the first feature set, that is, the feature vector of the picture is a random feature.

[0066] The first sample features in the first feature set that are similar to the reference features are all marked with identification marks to facilitate more accurate selection of positive samples and negative samples corresponding to the reference samples; in the first feature set, similar first sample features of the same series can be marked with an identification mark that can be distinguished from other first sample features.

[0067] A specific implementation method of obtaining the reference sample is:

[0068] First, obtain the first similarity between the random feature and each first sample feature. Generally speaking, the first similarity between the random feature and each first sample feature in the reference set can be directly calculated. If the reference set is large, a subset can be randomly sampled from the reference set to calculate the first similarity, and the subset size can be 50-100 times, or 100-200 times, the size of a batch of multi-group samples constructed. The specific representation of the first similarity (the second similarity and the third similarity described later are the same) may include: cosine distance, Euclidean distance, Mahalanobis distance, etc., without limitation. Preferably, in this embodiment, the representation of the first sample feature can use a feature vector, and the first similarity, the second similarity, and the third similarity can all be measured and represented by cosine distance.

[0069] Then, when the first similarity belongs to a preset first range, the first sample feature corresponding to the first similarity belonging to the first range is extracted as a reference feature. The size interval of the first range can be customized. The extraction can be a full extraction or a partial extraction, for example, a certain number of first sample features are randomly extracted as reference features within the specified first range. In a preferred embodiment, in order to ensure that the reference samples of the same batch are similar, the first range can be determined in an interval where the first similarity represents a relatively similar or close distance.

[0070] Finally, the sample corresponding to the reference feature is used as the reference sample.

[0071] An example is used to illustrate step S21:

[0072] Taking the image sample as an example, first randomly select a random image (corresponding to a random feature) from the sample set. Randomly sample the reference set in the first feature set to construct a subset of the first feature set that is 50 times the batch size (sample batch size). Then calculate the distance between the random feature and all the first sample features in the sampled subset (that is, the distance between the random image and the corresponding sample in the sample set), and recall the batch size set with the smallest distance from the subset. The samples corresponding to the recalled first sample features are used as all reference samples of the current batch, which satisfies the requirement that all reference samples in the current batch are relatively similar, and batch acquisition of reference samples is achieved.

[0073] After obtaining the reference sample, you also need to obtain the positive and negative samples corresponding to the reference sample to construct a multi-tuple sample. Further:

[0074] Step S22: obtaining a positive sample corresponding to the reference sample according to the reference sample and the first sample feature marked with the identification mark.

[0075] In the specific implementation process of step S22, for each reference sample, it is necessary to select a sample similar to the reference sample as a positive sample, and a sample not similar to the reference sample as a negative sample. The specific similarity can be measured by the second similarity or the third similarity. A reference sample can correspond to one positive sample or negative sample, or to multiple positive samples or negative samples.

[0076] Specifically, the acquisition of positive samples:

[0077] First, for the same reference sample, the second similarity between the reference feature corresponding to the reference sample and the first sample feature marked with the identification mark is obtained. If there are a large number of first sample features marked with the identification mark corresponding to a reference sample, a subset can be randomly selected from the first sample features marked with the identification mark corresponding to the reference sample to calculate the second similarity, and the size of the subset can be 50-100 times, or 100-200 times, the size of a batch of multi-tuple samples constructed.

[0078] Then, when the second similarity belongs to a preset second range, the first sample feature corresponding to the second similarity belonging to the second range is extracted as the feature of the positive sample corresponding to the reference sample.

[0079] Finally, according to the characteristics of the positive sample, a positive sample corresponding to the reference sample is obtained.

[0080] In step S22, the extraction may be complete extraction or partial extraction, for example, a certain number of first sample features are randomly extracted within a specified second range as positive samples corresponding to the reference sample. The second range is not limited and can be determined according to the needs of the application scenario.

[0081] If the similarity between similar samples and dissimilar samples in the multi-group sample is too small, the constructed multi-group will be too simple for the network to learn, and the network will not be able to learn features with strong discriminative power. In a preferred embodiment, in order to improve the training effect, it is necessary to construct difficult samples in the multi-group sample. The difficult samples include reference samples, positive samples that are not very similar to the reference samples, and negative samples that are similar to the reference samples. At this time, the second range can be defined as an interval where the second similarity represents dissimilarity or a long distance, so as to obtain positive samples that are not very similar to the reference samples. In general, in a preferred embodiment of this embodiment, samples that are not very similar to the reference samples are selected as positive samples.

[0082] Step S23: obtaining a negative sample corresponding to the reference sample according to the reference sample and the first sample feature not marked with the identification mark.

[0083] In step S23, the following implementation steps are specifically included:

[0084] First, for the same reference sample, the third similarity between the reference feature corresponding to the reference sample and the first sample feature not marked with the identification mark is obtained. If there are a large number of first sample features not marked with identification marks corresponding to a reference sample, a subset can be randomly selected from the first sample features not marked with identification marks corresponding to the reference sample to calculate the third similarity, and the size of the subset can be 50-100 times, or 100-200 times, the size of a batch of multi-tuple samples constructed.

[0085] Then, when the third similarity belongs to a preset third range, the first sample feature corresponding to the third similarity belonging to the third range is extracted as the feature of the negative sample corresponding to the reference sample.

[0086] Finally, according to the characteristics of the negative sample, a negative sample corresponding to the reference sample is obtained.

[0087] In step S23, the extraction may be complete extraction or partial extraction, for example, a certain number of first sample features are randomly extracted within a specified third range as negative samples corresponding to the reference sample. The determination of the third range is not limited and can be determined according to the needs of the application scenario.

[0088] In a preferred implementation of this embodiment, when constructing difficult samples, the third range can be defined in the interval where the third similarity represents similarity or close distance, so as to obtain negative samples that are similar to the reference sample. In general, samples that are similar to the reference sample are selected as negative samples from samples that are not similar to the reference sample.

[0089] The difficult samples constructed by step S22 and step S23 avoid the problem that the similarity between similar samples and dissimilar samples is too high, which makes the multi-tuple too difficult for the network to learn and makes it difficult for the network to converge.

[0090] For the convenience of calculation, the first sample feature, the second sample feature and / or the third sample feature (as described below) may be normalized before calculating the first similarity, the second similarity and the third similarity.

[0091] In this embodiment, the first range, the second range and the third range can be one or more segmented ranges. For example, when sampling, samples are taken from multiple segmented, non-adjacent first ranges. By using this segmentation method, samples that are not too adjacent are selected, thereby avoiding the problem of local defects of samples.

[0092] Step S24: Obtain a first multi-tuple sample according to the reference sample, the positive sample corresponding to the reference sample, and the negative sample corresponding to the reference sample. Each reference sample corresponds to a first multi-tuple sample. In the current multi-tuple sample construction, all the first multi-tuple samples constitute the batch sample set required for the current training of the model to be trained.

[0093] In the calculation of step S20, due to the existence of the first feature set, the corresponding features can be directly selected from the first feature set to calculate the similarity, without the need to obtain the features through the forward calculation of the network model again, thereby improving the calculation efficiency.

[0094] Step S30: performing forward calculation according to the first multi-tuple sample by the model to be trained to obtain second sample features.

[0095] In step S30, the forward calculation of the first multi-tuple sample includes the reference sample, the positive sample and the negative sample corresponding to the reference sample. In the forward calculation of the model to be trained, new features of the sample can be learned, which are the second sample features.

[0096] Step S40: In the first feature set, the first sample feature corresponding to the first multi-group sample is updated to the second sample feature to obtain a second feature set.

[0097] In step S40, the second feature set is used to replace the first feature set. Since the first sample feature and the second sample feature are obtained from the same sample in the forward calculation, the first sample feature corresponding to the second sample feature can be found in the first feature set. For example:

[0098] Table 1 Before update

[0099] sample The first feature set Sample 1 Feature A Sample 2 Feature B ... ... Sample 5 Feature E Sample 6 Feature F Sample 7 Feature G ... ... Sample N Feature X

[0100] If in Table 1, sample 2, sample 5, and sample 6 are selected multi-tuple samples, and feature b, feature e, and feature f, i.e., second sample features, are obtained after forward calculation of the model to be trained; then step S40 is executed to update the features B, feature E, and feature F corresponding to sample 2, sample 5, and sample 6 in the first feature set to feature b, feature e, and feature f, to obtain the second feature set:

[0101] Table 2 After Update

[0102] sample The second feature set Sample 1 Feature A Sample 2 Feature b ... ... Sample 5 Features Sample 6 Feature Sample 7 Feature G ... ... Sample N Feature X

[0103] Therefore, the distance or distribution between sample 2, sample 5, and sample 6 corresponding to feature b, feature e, and feature f relative to other samples has changed. The above Table 1 and Table 2 are only for illustrative purposes and do not limit the storage form, size, etc. of the feature set.

[0104] Step S50: obtaining, according to the second feature set, a second multi-tuple sample required for the next training of the model to be trained.

[0105] In step S50, the method for obtaining the second multi-group sample can refer to the method for obtaining the first multi-group sample, determine a random feature, extract a reference feature from the second feature set according to the random feature, obtain a reference sample according to the reference feature, and obtain positive and negative samples corresponding to the reference sample according to the reference feature and the sample features in the second feature set, so as to form a second multi-group sample.

[0106] Similarly, when the second multi-tuple sample is used to train the model to be trained, the corresponding third sample features can be obtained in the forward calculation, and then the corresponding sample features in the second feature set are replaced / updated to obtain a new set, which can be used as the third feature set.

[0107] The above is executed in a loop, and new sample features are obtained in each forward calculation of the model to be trained, and the feature set is continuously updated.

[0108] During the entire training process of the model to be trained, including the forward calculation process and the back propagation process, the training is stopped until the convergence condition is met.

[0109] Since the method in this embodiment uses the first sample feature for calculation in the process of constructing the first multi-tuple sample, the calculated distance / similarity between the sample corresponding to the first sample feature / second sample feature and other samples can be changed after the first sample feature is replaced and updated, that is, the sample distribution is indirectly changed. Therefore, the update of the feature set can be maintained during each training process of the model to be trained. The obtained second multi-tuple sample can guide the convergence of the model to be trained more quickly.

[0110] It should be noted that, corresponding to different application scenarios, the samples mentioned in this embodiment include but are not limited to various forms of pictures and texts. In the entire training phase of the model to be trained, steps S10-S50 are executed in a loop, and the feature set can be updated once for each training.

[0111] In summary, the present invention obtains the second feature set by updating the first sample features in the first feature set using the second sample features obtained during the training of the first multi-group sample in the training process of the model to be trained. This ensures that when the model to be trained mines learning samples at different training stages, the distance distribution of the sample features is constantly updated and changed. When mining the second multi-group samples, it will be more targeted, and multi-group samples with higher quality and more suitable for the current training stage can be mined. The multi-group samples obtained using the method of the present invention can adapt to the learning of the model to be trained at different stages, guide the model to be trained to reach a better solution more quickly, and improve the convergence speed and effect of the model to be trained.

[0112] Second embodiment

[0113] See also Figure 3 Based on the same inventive concept, in this embodiment, a multi-group sample construction device 400 is provided, and the device 400 includes:

[0114] The first feature set acquisition module 401 is used to acquire a first feature set consisting of first sample features.

[0115] The first sample acquisition module 402 is used to obtain a first multi-tuple sample required for the current training of the preset model to be trained according to the first feature set.

[0116] The forward calculation module 403 is used to perform forward calculation according to the first multi-tuple sample through the model to be trained to obtain the second sample feature.

[0117] The updating module 404 is configured to update the first sample feature corresponding to the first multi-group sample in the first feature set to the second sample feature to obtain a second feature set.

[0118] The second sample acquisition module 405 is used to obtain a second multi-tuple sample required for the next training of the to-be-trained model according to the second feature set.

[0119] As an optional implementation manner, the first sample acquisition module 402 is further configured to:

[0120] According to the random feature, a reference feature is extracted from the first feature set, and a reference sample is obtained based on the reference feature, wherein the random feature is randomly selected from the first feature set, and the first sample features similar to the reference feature in the first feature set are all marked with an identification mark; according to the reference sample and the first sample feature marked with the identification mark, a positive sample corresponding to the reference sample is obtained; according to the reference sample and the first sample feature not marked with the identification mark, a negative sample corresponding to the reference sample is obtained; according to the reference sample, the positive sample corresponding to the reference sample, and the negative sample corresponding to the reference sample, a first multi-group sample is obtained, and each reference sample corresponds to a multi-group sample.

[0121] As an optional implementation manner, the first sample acquisition module 402 is further configured to:

[0122] Obtain a first similarity between the random feature and each first sample feature; when the first similarity belongs to a preset first range, extract the first sample feature corresponding to the first similarity belonging to the first range as a reference feature; and use the sample corresponding to the reference feature as the reference sample.

[0123] As an optional implementation manner, the first sample acquisition module 402 is further configured to:

[0124] For the same reference sample, obtain the second similarity between the reference feature corresponding to the reference sample and the first sample feature marked with the identification mark; when the second similarity belongs to a preset second range, extract the first sample feature corresponding to the second similarity belonging to the second range as the feature of the positive sample corresponding to the reference sample; based on the feature of the positive sample, obtain the positive sample corresponding to the reference sample.

[0125] As an optional implementation manner, the first sample acquisition module 402 is further configured to:

[0126] For the same reference sample, obtain the third similarity between the reference feature corresponding to the reference sample and the first sample feature not marked with the identification mark; when the third similarity belongs to a preset third range, extract the first sample feature corresponding to the third similarity belonging to the third range as the feature of the negative sample corresponding to the reference sample; according to the feature of the negative sample, obtain the negative sample corresponding to the reference sample.

[0127] As an optional implementation manner, the first sample feature and the second sample feature are both features of a picture.

[0128] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0129] Third embodiment

[0130] Figure 4 8 is a block diagram of a multi-tuple sample construction device 800 according to an exemplary embodiment. For example, the device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0131] Reference Figure 4 , the device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0132] The processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0133] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0134] The power component 806 provides power to the various components of the device 800. The power component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 800.

[0135] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user.

[0136] The touch panel includes one or more touch sensors to sense touch, slide and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0137] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the device 800 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0138] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0139] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the device 800, and the sensor assembly 814 can also detect the position change of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and the temperature change of the device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0140] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0141] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to execute the method described in the first embodiment above.

[0142] Fourth embodiment

[0143] In an exemplary embodiment, a non-transitory machine-readable medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by the processor 820 of the device 800 to perform the above method. For example, the non-transitory machine-readable medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0144] A machine-readable medium, specifically a non-transitory machine-readable medium, stores instructions thereon, which, when executed by one or more processors, enable a device to execute a multi-tuple sample construction method as described in any of the aforementioned embodiments.

[0145] Figure 5 1 is a schematic diagram of the structure of the server in an embodiment of the present invention. The server 1900 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 1922 (for example, one or more processors) and memory 1932, and one or more storage media 1930 (for example, one or more mass storage devices) storing application programs 1942 or data 1944. Among them, the memory 1932 and the storage medium 1930 can be short-term storage or permanent storage. The program stored in the storage medium 1930 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the server. Furthermore, the central processing unit 1922 can be configured to communicate with the storage medium 1930 and execute a series of instruction operations in the storage medium 1930 on the server 1900.

[0146] The server 1900 may also include one or more power supplies 1926, one or more wired or wireless network interfaces 1950, one or more input and output interfaces 1958, one or more keyboards 1956, and / or, one or more operating systems 1941, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0147] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this disclosure. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

[0148] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

[0149] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A multi-group sample construction method, characterized in that: include: Obtaining a first feature set consisting of first sample features; Obtaining, according to the first feature set, a first multi-tuple sample required for the current training of the preset model to be trained; Performing forward calculation according to the first multi-tuple sample by the model to be trained to obtain second sample features; In the first feature set, the first sample feature corresponding to the first multi-tuple sample is updated to the second sample feature to obtain a second feature set; the first sample feature and the second sample feature are both features of the picture; Obtaining, according to the second feature set, a second multi-group sample required for the next training of the to-be-trained model; The step of obtaining a first multi-tuple sample required for the current training of the preset model to be trained according to the first feature set includes: Extracting a reference feature from the first feature set according to the random feature, and obtaining a reference sample based on the reference feature, wherein the random feature is randomly selected from the first feature set, and first sample features similar to the reference feature in the first feature set are all marked with identification marks; Obtaining a positive sample corresponding to the reference sample according to the reference sample and the first sample feature marked with the identification mark; Obtaining a negative sample corresponding to the reference sample according to the reference sample and a first sample feature not marked with the identification mark; A first multi-tuple group of samples is obtained according to the reference sample, the positive sample corresponding to the reference sample, and the negative sample corresponding to the reference sample, wherein each reference sample corresponds to a multi-tuple group of samples.

2. The method according to claim 1, characterized in that The step of extracting a reference feature from the first feature set according to the random feature and obtaining a reference sample based on the reference feature comprises: Obtaining a first similarity between the random feature and each first sample feature; When the first similarity belongs to a preset first range, extracting a first sample feature corresponding to the first similarity belonging to the first range as a reference feature; The sample corresponding to the reference feature is used as the reference sample.

3. The method according to claim 2, characterized in that The step of obtaining a positive sample corresponding to the reference sample according to the reference sample and the first sample feature marked with the identification mark comprises: For the same reference sample, obtaining a second similarity between a reference feature corresponding to the reference sample and a first sample feature marked with the identification mark; When the second similarity belongs to a preset second range, extracting a first sample feature corresponding to the second similarity belonging to the second range as a feature of a positive sample corresponding to the reference sample; According to the characteristics of the positive sample, a positive sample corresponding to the reference sample is obtained.

4. The method according to claim 2, characterized in that: The step of obtaining a negative sample corresponding to the reference sample according to the reference sample and the first sample feature not marked with the identification mark comprises: For the same reference sample, obtaining a third similarity between a reference feature corresponding to the reference sample and a first sample feature not marked with the identification mark; When the third similarity belongs to a preset third range, extracting a first sample feature corresponding to the third similarity belonging to the third range as a feature of a negative sample corresponding to the reference sample; According to the characteristics of the negative sample, a negative sample corresponding to the reference sample is obtained.

5. A multi-group sample construction device, characterized in that: include: A first feature set acquisition module, used to acquire a first feature set consisting of first sample features; A first sample acquisition module, used for obtaining a first multi-tuple sample required for the current training of the preset model to be trained according to the first feature set; The first sample acquisition module is further used for: Extracting a reference feature from the first feature set according to the random feature, and obtaining a reference sample based on the reference feature, wherein the random feature is randomly selected from the first feature set, and first sample features similar to the reference feature in the first feature set are all marked with identification marks; Obtaining a positive sample corresponding to the reference sample according to the reference sample and the first sample feature marked with the identification mark; Obtaining a negative sample corresponding to the reference sample according to the reference sample and a first sample feature not marked with the identification mark; Obtain a first multi-tuple sample according to the reference sample, the positive sample corresponding to the reference sample, and the negative sample corresponding to the reference sample, wherein each reference sample corresponds to a multi-tuple sample; A forward calculation module, used to perform forward calculation according to the first multi-tuple sample through the model to be trained to obtain a second sample feature; the first sample feature and the second sample feature are both features of the picture; An updating module, configured to update, in the first feature set, the first sample feature corresponding to the first multi-group sample to the second sample feature, so as to obtain a second feature set; The second sample acquisition module is used to obtain a second multi-group sample required for the next training of the model to be trained according to the second feature set.

6. The device according to claim 5, characterized in that The first sample acquisition module is further used for: Obtaining a first similarity between the random feature and each first sample feature; When the first similarity belongs to a preset first range, extracting a first sample feature corresponding to the first similarity belonging to the first range as a reference feature; The sample corresponding to the reference feature is used as the reference sample.

7. The device according to claim 5, characterized in that The first sample acquisition module is further used for: For the same reference sample, obtaining a second similarity between a reference feature corresponding to the reference sample and a first sample feature marked with the identification mark; When the second similarity belongs to a preset second range, extracting a first sample feature corresponding to the second similarity belonging to the second range as a feature of a positive sample corresponding to the reference sample; According to the characteristics of the positive sample, a positive sample corresponding to the reference sample is obtained.

8. The device according to claim 5, characterized in that The first sample acquisition module is further used for: For the same reference sample, obtaining a third similarity between a reference feature corresponding to the reference sample and a first sample feature not marked with the identification mark; When the third similarity belongs to a preset third range, extracting a first sample feature corresponding to the third similarity belonging to the third range as a feature of a negative sample corresponding to the reference sample; According to the characteristics of the negative sample, a negative sample corresponding to the reference sample is obtained.

9. A multi-group sample construction device, characterized in that: The system comprises a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the following operations: Acquire a first feature set consisting of first sample features; obtain a first multi-group sample required for the current training of a preset model to be trained according to the first feature set; perform forward calculation according to the first multi-group sample through the model to be trained to obtain a second sample feature; the first sample feature and the second sample feature are both features of the image; in the first feature set, update the first sample feature corresponding to the first multi-group sample to the second sample feature to obtain a second feature set; obtain a second multi-group sample required for the next training of the model to be trained according to the second feature set; The step of obtaining a first multi-tuple sample required for the current training of the preset model to be trained according to the first feature set includes: Extracting a reference feature from the first feature set according to the random feature, and obtaining a reference sample based on the reference feature, wherein the random feature is randomly selected from the first feature set, and first sample features similar to the reference feature in the first feature set are all marked with identification marks; Obtaining a positive sample corresponding to the reference sample according to the reference sample and the first sample feature marked with the identification mark; Obtaining a negative sample corresponding to the reference sample according to the reference sample and a first sample feature not marked with the identification mark; A first multi-tuple group of samples is obtained according to the reference sample, the positive sample corresponding to the reference sample, and the negative sample corresponding to the reference sample, wherein each reference sample corresponds to a multi-tuple group of samples.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by the processor, the following steps are implemented: Acquire a first feature set consisting of first sample features; obtain a first multi-group sample required for the current training of a preset model to be trained according to the first feature set; perform forward calculation according to the first multi-group sample through the model to be trained to obtain a second sample feature, wherein the first sample feature and the second sample feature are both features of the image; in the first feature set, update the first sample feature corresponding to the first multi-group sample to the second sample feature to obtain a second feature set; obtain a second multi-group sample required for the next training of the model to be trained according to the second feature set; The step of obtaining a first multi-tuple sample required for the current training of the preset model to be trained according to the first feature set includes: Extracting a reference feature from the first feature set according to the random feature, and obtaining a reference sample based on the reference feature, wherein the random feature is randomly selected from the first feature set, and first sample features similar to the reference feature in the first feature set are all marked with identification marks; Obtaining a positive sample corresponding to the reference sample according to the reference sample and the first sample feature marked with the identification mark; Obtaining a negative sample corresponding to the reference sample according to the reference sample and a first sample feature not marked with the identification mark; A first multi-tuple group of samples is obtained according to the reference sample, the positive sample corresponding to the reference sample, and the negative sample corresponding to the reference sample, wherein each reference sample corresponds to a multi-tuple group of samples.

Images