Robot target recognition method, system and device based on online continual learning

Abstract

The application provides a robot target recognition method, system and device based on online continuous learning, the method comprises the following steps: obtaining a preset number of unlabeled image sample data; calculating a pseudo-loss value of the image sample data based on a historical deep neural network model; obtaining a query probability according to the pseudo-loss value, selecting sample data with large information quantity and diversity in the image sample data based on the query probability, and obtaining artificial labeled samples; calculating the mean and covariance of each class sample in the deep feature space of the artificial labeled samples; selecting a class sample that does not appear in this round of iteration in the replay buffer set, sampling in the class sample according to the covariance, and obtaining a sampling sample; using the historical sample in the replay buffer set, the artificial labeled sample and the sampling sample as an update training set to train the historical deep neural network model, and obtaining an updated deep neural network model, thereby effectively improving the efficiency of the robot in online data acquisition and continuous target recognition.

Description

Technical Field

[0001] The present invention relates to the field of machine learning and robot intelligent perception technology, and in particular to a robot target recognition method, system and device based on online continuous learning. Background Art

[0002] With the continuous development and advancement of machine intelligence technology, robotics are increasingly being used. In practical applications, robots operate in open and dynamically changing environments. Robotic visual perception systems continuously acquire new data over time, which may include new objects. This requires the system to possess lifelong object recognition capabilities—that is, the ability to continuously learn new objects online while retaining previously learned objects to adapt to the dynamic environment.

[0003] Currently, most robotic object recognition technologies rely on offline learning methods, which assume a static environment and cannot dynamically update models in real time with data streams. To cope with a constant flow of new data, the models must be retrained offline using all the data, resulting in significant computational overhead. Continuous learning techniques, on the other hand, can continuously learn new knowledge from data streams, incrementally update models, and prevent previously learned knowledge from being forgotten while retaining only a small portion of historical data. Continuous learning can effectively reduce the consumption of computing resources, attracting the attention of a growing number of researchers, who have proposed a variety of methods, including those based on sample replay, regularization, and parameter isolation.

[0004] However, online continuous learning methods are all supervised passive learning models, which requires that the labels of all samples in the learning process are known. However, manually labeling a large number of samples is costly, resulting in relatively low efficiency for robots to obtain data online and perform continuous target recognition. Summary of the Invention

[0005] The present invention provides a robot target recognition method, system and device based on online continuous learning, which is used to solve the defect of relatively low efficiency of robots in online data acquisition and continuous target recognition in the prior art, and realize the function of efficiently completing online data acquisition and continuous target recognition.

[0006] The present invention provides a robot target recognition method based on online continuous learning, comprising:

[0007] Obtaining a preset number of unlabeled image sample data;

[0008] Calculating a pseudo loss value of the unlabeled image sample data based on a historical deep neural network model;

[0009] Obtaining a query probability according to the pseudo loss value, and selecting sample data with a large amount of information and maintaining diversity from the unlabeled image sample data based on the query probability for labeling, thereby obtaining manually labeled samples;

[0010] Calculating the mean of each type of samples in the manually labeled samples in the deep feature space, and calculating the covariance of each type of samples in the manually labeled samples based on the mean;

[0011] Selecting category samples that have not appeared in the iteration in the playback buffer set, sampling the category samples according to the covariance to obtain sampled samples;

[0012] The historical samples in the playback buffer set, the manually labeled samples and the sampling samples are used as an updated training set to train the historical deep neural network model to obtain an updated deep neural network model.

[0013] According to a robot target recognition method based on online continuous learning provided by the present invention, the pseudo loss value of the unlabeled image sample data is calculated based on the historical deep neural network model, including:

[0014] Based on the historical deep neural network model, predict the category label and the label corresponding to the second largest score of each image sample in the unlabeled image sample data;

[0015] According to the category label and the label corresponding to the second largest score, a pseudo loss value of the unlabeled image sample data is calculated through a preset relationship.

[0016] According to a robot target recognition method based on online continuous learning provided by the present invention, obtaining the query probability according to the pseudo loss value includes:

[0017] Based on a preset method, a target number of query sample data are selected from the unlabeled image sample data;

[0018] The query probability of the query sample data is calculated according to the pseudo loss value.

[0019] According to a robot target recognition method based on online continuous learning provided by the present invention, sample data with large information content and diversity are selected from the unlabeled image sample data based on the query probability for labeling to obtain manually labeled samples, including:

[0020] Generate a random variable that obeys Bernoulli distribution according to the query probability;

[0021] The query sample data whose value of the random variable is 1 is labeled to obtain manually labeled samples.

[0022] According to a robot target recognition method based on online continuous learning provided by the present invention, the method comprises selecting category samples that do not appear in the current iteration in the playback buffer set, sampling the category samples according to the covariance, and obtaining the sampled samples, including:

[0023] Select the category samples that did not appear in this round of iteration in the playback buffer set;

[0024] Taking the category sample as the center of the feature extraction function, and constructing a multivariate Gaussian distribution according to the center of the feature extraction function and the covariance;

[0025] The sampling samples are obtained by sampling from the multivariate Gaussian distribution in a reparameterized manner.

[0026] According to a robot target recognition method based on online continuous learning provided by the present invention, the historical samples in the playback buffer set, the manually labeled samples, and the sampled samples are used as an updated training set to train the historical deep neural network model to obtain an updated deep neural network model, including:

[0027] Merging the historical samples in the playback buffer, the manually annotated samples, and the sampled samples to form an updated training set;

[0028] When the manually labeled samples contain new category data, the same number of neurons as the new categories in the new data are added to the last layer of the historical deep neural network model to obtain a deep neural network model to be updated;

[0029] Based on minimizing the loss function, the deep neural network model to be updated is trained using the updated training set to obtain an updated deep neural network model.

[0030] According to a robot target recognition method based on online continuous learning provided by the present invention, after obtaining and updating the deep neural network model, the method further includes:

[0031] The historical samples in the playback buffer are updated using the manually labeled samples.

[0032] According to a robot target recognition method based on online continuous learning provided by the present invention, the updating of historical samples in the playback buffer set by the manually labeled samples includes:

[0033] Merging the manually annotated samples with the historical samples in the playback buffer set to obtain a candidate set;

[0034] Initializing the playback buffer set and calculating the sample capacity of the playback buffer set;

[0035] Selecting updated samples that are a preset multiple of the sample size from the candidate set;

[0036] Calculate the loss value of each type of updated samples;

[0037] Based on a preset rule, an update sample with the same capacity as the sample is selected from the update samples according to the loss value and stored in a replay buffer set to update the replay buffer set.

[0038] The present invention also provides a robot target recognition system based on online continuous learning, comprising:

[0039] A data acquisition module is used to acquire a preset number of unlabeled image sample data;

[0040] A label prediction module, configured to calculate a pseudo loss value of the unlabeled image sample data based on a historical deep neural network model;

[0041] A manual labeling module is configured to obtain a query probability according to the pseudo loss value, and select sample data with a large amount of information and maintaining diversity from the unlabeled image sample data based on the query probability for labeling to obtain manually labeled samples;

[0042] A feature sampling module is configured to calculate the mean of each class of samples in the manually labeled samples in the deep feature space, and calculate the covariance of each class of samples in the manually labeled samples based on the mean; select class samples that did not appear in the current iteration from the playback buffer set, and sample the class samples based on the covariance to obtain sampled samples;

[0043] The model updating module is used to train the historical deep neural network model using the historical samples in the playback buffer set, the manually labeled samples and the sampling samples as the updated training set to obtain an updated deep neural network model.

[0044] The present invention also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the robot target recognition method based on online continuous learning as described above are implemented.

[0045] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described methods for robot target recognition based on online continuous learning.

[0046] The present invention provides a robot target recognition method, system and device based on online continuous learning. The method obtains a preset number of unlabeled image sample data; calculates the pseudo loss value of the unlabeled image sample data based on a historical deep neural network model; obtains a query probability according to the pseudo loss value, selects sample data with large information content and diversity from the unlabeled image sample data based on the query probability for labeling, and obtains manually labeled samples; calculates the mean of each type of sample in the deep feature space in the manually labeled samples, and calculates the covariance of each type of sample in the manually labeled samples based on the mean; selects category samples that did not appear in the current iteration in the playback buffer set, and samples the category samples based on the covariance. , and obtain sampling samples; use historical samples, manually labeled samples and sampling samples in the playback buffer as updated training sets to train the historical deep neural network model to obtain an updated deep neural network model. By selecting samples that take into account both diversity and richness for manual labeling, the problem of high cost of manually labeling a large number of samples is solved. Moreover, due to continuous learning of real-time data streams with unknown task boundaries, the deep network model is updated in real time, which can reduce the data imbalance between new and old class samples, effectively overcome the forgetting of learned targets, and ultimately achieve high-precision lifelong target recognition with a small amount of sample labeling, effectively improving the efficiency of robots in obtaining data online and performing continuous target recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0048] Figure 1 This is one of the flow charts of the robot target recognition method based on online continuous learning provided by an embodiment of the present invention;

[0049] Figure 2 This is the second flow chart of the robot target recognition method based on online continuous learning provided by an embodiment of the present invention;

[0050] Figure 3 Schematic diagram of the structure of a robot target recognition system based on online continuous learning provided by an embodiment of the present invention;

[0051] Figure 4 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention;

[0052] Figure 5 It is a schematic diagram of the principle of a robot target recognition method based on online continuous learning provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0053] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0054] The following combination Figure 1-Figure 5 The present invention describes a robot target recognition method, system and device based on online continuous learning.

[0055] Figure 1 This is one of the flow charts of the robot target recognition method based on online continuous learning provided by an embodiment of the present invention.

[0056] like Figure 1 As shown, an embodiment of the present invention provides a robot target recognition method based on online continuous learning, comprising the following steps:

[0057] 101. Obtain a preset amount of unlabeled image sample data.

[0058] Specifically, the robot vision system obtains a batch of unlabeled image sample data at time t and records it as: Among them, n t Indicates the total sample data volume of the batch of data. Image sample data includes visible light images, infrared images, or depth of field images obtained by depth sensors collected from the robot's visual sensing system.

[0059] 102. Calculate the pseudo loss value of unlabeled image sample data based on the historical deep neural network model.

[0060] Specifically, the historical deep neural network model f t-1 Including any of VGG, EfficientNet, ResNet, ResNext, MobileNet. Based on the historical deep neural network model f t-1 , calculate the pseudo loss value q of the unlabeled image sample data i , including: predicting the category label of each image sample in the unlabeled image sample data based on the historical deep neural network model The label corresponding to the second largest score The specific calculation formulas are as follows (1) and (2);

[0061] Category tags:

[0062] The label corresponding to the second largest score:

[0063] Then according to the category label The label corresponding to the second largest score The pseudo loss value of the unlabeled image sample data is calculated through the preset relationship, such as (3):

[0064] Pseudo loss value:

[0065] Among them, x i Indicates the i-th data in the current batch of data i=1,…,n t ; Y represents the set of category labels; k is a variable whose value range is an integer from 1 to Y; f k (x i ) represents the sample x at the current time t i The value of the kth element of the prediction vector after passing through the softmax layer of the deep network model; The first The value of the element, The first The value of the element; argmax represents the variable value corresponding to the maximum function value.

[0066] 103. A query probability is obtained according to the pseudo loss value, and sample data with large information content and diversity are selected from the unlabeled image sample data based on the query probability for labeling to obtain manually labeled samples.

[0067] Specifically, according to the pseudo loss value q i Obtaining the query probability includes: selecting a target number of query sample data from the unlabeled image sample data based on a preset method; first, to ensure data diversity, randomly selecting a portion of samples from the unlabeled image sample data according to a uniform distribution, with a sample selection ratio of r; and then calculating the query probability p of the query sample data according to the pseudo loss value using formula (4). i .

[0068]

[0069] Among them, δ is a non-negative real number, is an exponential function, q i is the pseudo loss value, and γ is the parameter that controls the slope of the exponential function.

[0070] Then, based on the query probability, sample data with large information content and diversity are selected from the unlabeled image sample data for labeling, and artificially labeled samples are obtained, wherein the artificial sample data is sample data that takes into account both information content and diversity. Specifically, the following steps are performed: i, generates a random variable Z that follows a Bernoulli distribution i ; For the random variable Z i The query sample data with a value of 1 is labeled to obtain manually labeled samples. If it is not 1, no labeling is performed.

[0071] 104. Calculate the mean of each type of samples in the deep feature space in the manually labeled samples, and calculate the covariance of each type of samples in the manually labeled samples based on the mean.

[0072] Specifically, first calculate the mean of samples belonging to each category in the current manually labeled samples and covariance The calculation process is as follows (5):

[0073]

[0074] in, and Respectively represent the mean and covariance of the samples of category c in the unlabeled image sample data, N c is the number of samples in the cth class, g θ (·) is the feature extraction function of the neural network, θ is the network parameter of the feature extraction part, and the superscript T represents the transpose.

[0075] Then, using the mean calculated on the manually labeled samples and covariance Update the mean of each class and covariance matrix If the category is new, then If the category is an old category that has been seen, then and Updates like (6) and (7):

[0076]

[0077]

[0078] Among them, N c is the number of samples belonging to the cth category in the unlabeled image sample data at time t, M c is the total number of samples belonging to the cth class accumulated up to time t-1. c =M c +N c .

[0079] 105. Select category samples that have not appeared in this round of iteration from the playback buffer set, and sample from the category samples according to the covariance to obtain sampled samples.

[0080] Specifically, to ensure the significance of the semantic features of the sampling, select the category samples x that did not appear in this round of iteration in the playback buffer set i ; Take the category sample as the feature extraction function g θ (x i ), according to the center and covariance of the feature extraction function Constructing a multivariate Gaussian distribution λ is a positive constant that controls the covariance scale, and then n0 sampling samples z are obtained from the multivariate Gaussian distribution by reparameterization. i In order to ensure that the sampled data can update the feature layer parameters, the reparameterization method is used to sample, first randomly sample the original ξ from the standard Gaussian distribution N(0,I), and then obtain the sample z through the following formula (8): i .

[0081] z i =g θ (x i )+ξλ∑ i (8)

[0082] 106. Use the historical samples, manually labeled samples and sampling samples in the playback buffer set as the updated training set to train the historical deep neural network model to obtain an updated deep neural network model.

[0083] Specifically, replay buffer set B t The historical samples, manually labeled samples and sampled samples in the dataset are merged to form the updated training set T t ; Determine whether the current batch of data contains new category data. When the manually labeled samples contain new category data and the new data contains d new categories, the historical deep neural network model f t-1 The last layer of d neurons with the same number as the new categories in the new data is added to obtain the deep neural network model to be updated; if no new category data is included, the structure of the deep neural network model remains unchanged.

[0084] In the historical deep neural network model f t-1 Update and get the deep neural network model f t Afterwards, based on minimizing the loss function L, the training set T is updated t The updated deep neural network model is trained to obtain the updated deep neural network model f t , update its structure and parameters. Among them, the minimization loss function L consists of two parts. The first part is the cross entropy loss function L ce , the second part is the hinge loss function L based on the classification interval hin .

[0085] The first part is the cross entropy loss function Lce , defined as (9):

[0086]

[0087] Among them, C t is the total number of sample classes up to the current moment; is an indicator function, if y=y i ,but is equal to 1, otherwise equal to 0; similarly, if y≠y i ,but is equal to 1, otherwise equal to 0; f y (x i ) is the sample x i The value of the yth element of the probability prediction vector output after the softmax of the deep neural network model f, h y (z i ) is the value of the yth element of the probability prediction vector output after the sample sampled in the feature space passes through the classification layer and softmax, defined as

[0088] The second part is the hinge loss function based on the classification interval, which is defined as (10):

[0089]

[0090] Where y′=argmax k≠y f k (x i ) is the largest irrelevant category, that is, f(x i ) except the value of the y-th element; f y (x i )-f y′ (x i ) represents the sample x i The classification interval between the predicted values ​​of the true class y and the maximum irrelevant class y′; h y (z i )-h y′ (z i ) represents the feature sample z i The classification interval between the predicted values ​​of the true class y and the maximally irrelevant class y′; this loss function helps to alleviate confusion between similar classes.

[0091] The specific loss function calculation formula is (11):

[0092] L=L ce +βL hin (11)

[0093] The robot target recognition method based on online continuous learning provided by the embodiment of the present invention obtains a preset number of unlabeled image sample data; calculates the pseudo loss value of the unlabeled image sample data based on the historical deep neural network model; obtains the query probability according to the pseudo loss value, selects sample data with large information content and diversity from the unlabeled image sample data based on the query probability for labeling, and obtains manually labeled samples; calculates the mean of each type of samples in the deep feature space in the manually labeled samples, and calculates the covariance of each type of samples in the manually labeled samples based on the mean; selects category samples that do not appear in the round of iteration in the playback buffer set, samples the category samples according to the covariance, and obtains Sampling samples; using historical samples, manually labeled samples and sampling samples in the playback buffer as updated training sets, the historical deep neural network model is trained to obtain an updated deep neural network model. By selecting samples that take into account both diversity and richness for manual labeling, the problem of high cost of manually labeling a large number of samples is solved. In addition, by continuously learning the real-time data stream with unknown task boundaries and updating the deep network model in real time, the data imbalance between new and old class samples can be reduced, and the forgetting of learned targets can be effectively overcome. Finally, high-precision lifelong target recognition with a small amount of sample labeling is achieved, which effectively improves the efficiency of the robot in acquiring data online and performing continuous target recognition.

[0094] Figure 2 This is the second flow chart of the robot target recognition method based on online continuous learning provided by an embodiment of the present invention.

[0095] like Figure 2 As shown, this embodiment provides a robot target recognition method based on online continuous learning, including the following steps:

[0096] 201. Obtain a preset amount of unlabeled image sample data.

[0097] 202. Calculate pseudo loss values ​​for unlabeled image sample data based on historical deep neural network models.

[0098] 203. A query probability is obtained according to the pseudo loss value, and sample data with large information content and diversity are selected from the unlabeled image sample data based on the query probability for labeling to obtain manually labeled samples.

[0099] 204. Calculate the mean of each type of samples in the deep feature space in the manually labeled samples, and calculate the covariance of each type of samples in the manually labeled samples based on the mean.

[0100] 205. Select category samples that have not appeared in this round of iteration from the playback buffer set, and sample from the category samples according to the covariance to obtain sampled samples.

[0101] 206. Using the historical samples, manually labeled samples and sampling samples in the playback buffer set as the updated training set, the historical deep neural network model is trained to obtain an updated deep neural network model.

[0102] Among them, steps 201-206 have been described in detail in 101-106 in the above embodiment, and therefore will not be further elaborated in this embodiment.

[0103] 207. Update historical samples in the replay buffer set through manually labeled samples.

[0104] Specifically, the manually labeled samples are compared with the playback buffer set B t Merge the historical samples in to get the candidate set S t ; Initialize playback buffer set B t , and calculate the playback buffer set B t The sample capacity; assuming the playback buffer set B t The maximum capacity is M, and the total number of sample classes up to the current moment is C t Based on the idea of ​​average, each category can be calculated and saved in the buffer set B t The number of samples in is in Indicates that the data is rounded down. In the candidate set, update samples that are multiples of the preset sample size are selected. For example, for the candidate set S t For each class of samples in , in order to ensure diversity, we first randomly select 2m samples based on uniform distribution and calculate the loss value of each class of updated samples: L i =L ce (x i ,y i )+βL hin (x i ,y i ), based on the preset rules, select the update samples with the same sample capacity from the update samples according to the loss value and store them in the replay buffer set to update the replay buffer set. Specifically, it can be sorted from large to small according to the loss function value, and the first v*m samples with the largest loss function value are selected and stored in the buffer set B t , where 0≤v≤1; from the remaining samples, (1-v)*m samples are randomly selected based on uniform distribution and stored in the playback buffer to complete B t Updates.

[0105] Furthermore, the present invention also includes calculating the classification accuracy of each category on the test set using the model learned on the training set at each stage during the learning process. Finally, the effectiveness of the algorithm is evaluated using four metrics: accuracy, forward transfer value, backward transfer value, and overall accuracy. Specifically, the training-test accuracy matrix during the learning process is shown in Table 1:

[0106]

[0107] Table 1

[0108] Among them, A i,j represents the accuracy on the test set of the jth task after completing training on the training set of the i-th task, and N represents the total number of tasks learned. The calculation method of the above four metrics is as follows:

[0109]

[0110]

[0111] The accuracy is calculated as the mean accuracy of the model learned during the online learning process on all learned categories. The backward transfer value is the mean accuracy of the model on all learned categories excluding the current learning category. The forward transfer value is the mean accuracy of the model on all unlearned categories. The overall accuracy is the mean accuracy of the model on all categories.

[0112] For example, the test experiments used real-world visual images captured by a robot, the OpenLORIS-IROS2019 dataset of visible light RGB-D images acquired by a D435i sensor. Different levels of illumination and occlusion were selected from this dataset. The illumination dataset includes target images acquired under high-light, normal-light, and low-light conditions, while the occlusion dataset includes target images acquired under no occlusion, 25% occlusion, and 50% occlusion. Thirty-six target classes were selected from this dataset. To evaluate the proposed method's class-incremental learning capabilities in dynamically changing environments, the dataset was divided into nine tasks. Tasks 1-3, 4-6, and 7-9 correspond to target images under high-light, normal-light, and low-light conditions, or no occlusion, 25% occlusion, and 50% occlusion, respectively. Tasks 1, 4, and 7 belong to classes 1-12, tasks 2, 5, and 8 belong to classes 13-24, and tasks 3, 6, and 9 belong to classes 25-36. The training and test sets contain approximately 22,000 and 2,600 samples, respectively. The software used in the experiment is PyTorch 1.7.0, the processor is E5-2698 v4, the memory is 30.0GB, and the operating system is 64-bit Ubuntu 16.04.

[0113] Experimental content and result analysis:

[0114] To evaluate the effectiveness of the proposed method, experimental comparisons were conducted using fine-tuning and the iCaRL algorithm. All methods used the same EfficientNet-B0 backbone model, with a replay buffer capacity of 1500, five training rounds per batch, and an initial learning rate of 0.01. The following parameters were used for this method: when the sample query ratio was less than 50%, r was set to 50%, otherwise r was set to the expected query ratio + 10%; the parameter for the data query probability, γ, was 0.01; the covariance scale parameter, λ, was t / T, where t represents the current epoch number and T represents the total epoch number; the number of samples stored in the replay buffer, M, was 1500; and the scale parameter, v, was 0.25.

[0115] Table 1: Comparison of lifetime target recognition performance under different sample query rates

[0116]

[0117]

[0118] Table 1 shows the accuracy comparison results of fine-tuning, iCaRL, and this method under four indicators: the lifelong target recognition results of the method proposed in this invention are significantly better than those of fine-tuning and iCaRL in most indicators. In addition, according to Table 1, the results of the model learned by this method when only 25% of the sample labels are queried are similar to the results when 100% of the sample labels are queried. The accuracy, backward transfer, and overall accuracy corresponding to the learning results of this method when the sample query rate is 25% all exceed the learning results of iCaRL when queried 100% of the sample labels, and the forward transfer indicators of the two are not much different. From Table 1, it can be concluded that the method proposed in this invention can use a small number of labeled samples to continuously update the model online to achieve higher-precision lifelong target recognition performance.

[0119] The present invention can select a small number of samples that take into account both diversity and information richness for manual labeling, solving the problem of the high cost of labeling a large number of samples in the prior art. For the unlabeled target image acquired in real time by the robot vision system, the image label can be predicted by a deep neural network and the pseudo-loss value can be calculated. The probability of the Bernoulli distribution is further generated according to the pseudo-loss value, and samples are selected for manual labeling based on the probability, which overcomes the high cost problem of the prior art that requires labeling a large number of samples for supervised learning, and achieves high-precision target recognition when a small number of samples are labeled. It can continuously learn real-time data streams with unknown task boundaries and effectively overcome the forgetting of learned targets. It can continuously learn real-time data streams with fuzzy and unknown task boundaries, solving the problem of the prior art that requires clear task boundaries; in addition, by storing representative data in a buffer set and statistically modeling and sampling the features of the learned targets, the data imbalance between new and old class samples can be reduced, effectively overcoming the forgetting of learned targets.

[0120] Based on the same general inventive concept, this application also protects a robot target recognition system based on online continuous learning. The following describes the robot target recognition system based on online continuous learning provided by the present invention. The robot target recognition system based on online continuous learning described below and the robot target recognition method based on online continuous learning described above can be used in conjunction with each other.

[0121] Figure 3 It is a structural diagram of a robot target recognition system based on online continuous learning provided by an embodiment of the present invention.

[0122] like Figure 3 As shown, this embodiment provides a robot target recognition system based on online continuous learning, including:

[0123] A data acquisition module 31 is used to acquire a preset number of unlabeled image sample data;

[0124] A label prediction module 32 is used to calculate the pseudo loss value of the unlabeled image sample data based on the historical deep neural network model;

[0125] A manual labeling module 33 is configured to obtain a query probability based on the pseudo loss value, and select sample data with large information content and diversity from the unlabeled image sample data based on the query probability for labeling to obtain manually labeled samples;

[0126] The feature sampling module 34 is used to calculate the mean of each class of samples in the manually labeled samples in the deep feature space, and calculate the covariance of each class of samples in the manually labeled samples based on the mean; select class samples that did not appear in the iteration in the playback buffer set, and sample from the class samples based on the covariance to obtain sampled samples;

[0127] The model updating module 35 is used to train the historical deep neural network model using the historical samples, manually labeled samples and sampled samples in the playback buffer set as the updated training set to obtain an updated deep neural network model.

[0128] The robot target recognition system based on online continuous learning provided by the embodiment of the present invention obtains a preset number of unlabeled image sample data; calculates the pseudo loss value of the unlabeled image sample data based on the historical deep neural network model; obtains the query probability according to the pseudo loss value, selects sample data with large information content and diversity from the unlabeled image sample data based on the query probability for labeling, and obtains manually labeled samples; calculates the mean of each type of samples in the deep feature space in the manually labeled samples, and calculates the covariance of each type of samples in the manually labeled samples based on the mean; selects category samples that do not appear in the round of iteration in the playback buffer set, and samples the category samples based on the covariance to obtain Sampling samples; using historical samples, manually labeled samples and sampling samples in the playback buffer as updated training sets, the historical deep neural network model is trained to obtain an updated deep neural network model. By selecting samples that take into account both diversity and richness for manual labeling, the problem of high cost of manually labeling a large number of samples is solved. In addition, by continuously learning the real-time data stream with unknown task boundaries and updating the deep network model in real time, the data imbalance between new and old class samples can be reduced, and the forgetting of learned targets can be effectively overcome. Finally, high-precision lifelong target recognition with a small amount of sample labeling is achieved, which effectively improves the efficiency of the robot in acquiring data online and performing continuous target recognition.

[0129] Furthermore, the label prediction module 32 in this embodiment is specifically configured to:

[0130] Based on the historical deep neural network model, predict the category label and the label corresponding to the second largest score for each image sample in the unlabeled image sample data;

[0131] According to the category label and the label corresponding to the second largest score, the pseudo loss value of the unlabeled image sample data is calculated through the preset relationship.

[0132] Furthermore, the manual labeling module 33 in this embodiment is specifically used to:

[0133] Based on a preset method, a target number of query sample data is selected from the unlabeled image sample data;

[0134] According to the pseudo loss value, the query probability of the query sample data is calculated.

[0135] Furthermore, the manual labeling module 33 in this embodiment is further configured to:

[0136] Generate a random variable that obeys the Bernoulli distribution according to the query probability;

[0137] The query sample data whose random variable value is 1 is labeled to obtain manually labeled samples.

[0138] Furthermore, the feature sampling module 34 in this embodiment is specifically configured to:

[0139] Select the category samples that did not appear in this round of iteration in the playback buffer set;

[0140] Taking the category sample as the center of the feature extraction function, a multivariate Gaussian distribution is constructed based on the center and covariance of the feature extraction function;

[0141] The sampling samples are obtained by sampling from the multivariate Gaussian distribution in a reparameterized manner.

[0142] Furthermore, the model updating module 35 in this embodiment is specifically configured to:

[0143] Merge historical samples, manually labeled samples, and sampled samples in the playback buffer to form an updated training set;

[0144] When the manually labeled samples contain new category data, the same number of neurons as the new categories in the new data are added to the last layer of the historical deep neural network model to obtain the deep neural network model to be updated;

[0145] Based on minimizing the loss function, the deep neural network model to be updated is trained by updating the training set to obtain an updated deep neural network model.

[0146] Furthermore, this embodiment also includes a buffer set update module, which is used to:

[0147] Update historical samples in the replay buffer using manually labeled samples.

[0148] Furthermore, this embodiment also includes a buffer set update module, which is specifically used to:

[0149] Merge the manually annotated samples with the historical samples in the replay buffer to obtain the candidate set;

[0150] Initialize the playback buffer set and calculate the sample capacity of the playback buffer set;

[0151] Selecting updated samples that are multiples of the preset sample size from the candidate set;

[0152] Calculate the loss value of each type of updated sample;

[0153] Based on a preset rule, an update sample with the same sample capacity as the loss value is selected from the update sample and stored in the replay buffer set to update the replay buffer set.

[0154] Figure 4 It is a structural diagram of an electronic device provided by an embodiment of the present invention.

[0155] like Figure 4 As shown, the electronic device may include: a processor (processor) 410, a communication interface (Communications Interface) 420, a memory (memory) 430 and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call the logic instructions in the memory 430 to execute a robot target recognition method based on online continuous learning, which includes: obtaining a preset number of unlabeled image sample data; calculating the pseudo loss value of the unlabeled image sample data based on the historical deep neural network model; obtaining a query probability according to the pseudo loss value, and selecting sample data with large information content and diversity from the unlabeled image sample data based on the query probability for labeling to obtain manually labeled samples; calculating the mean of each type of samples in the manually labeled samples in the deep feature space, and calculating the covariance of each type of samples in the manually labeled samples based on the mean; selecting category samples that did not appear in this round of iteration in the replay buffer set, and sampling from the category samples according to the covariance to obtain sampled samples; using the historical samples, manually labeled samples and sampled samples in the replay buffer set as updated training sets to train the historical deep neural network model to obtain an updated deep neural network model.

[0156] Figure 5 It is a schematic diagram of the principle of a robot target recognition method based on online continuous learning provided by an embodiment of the present invention.

[0157] like Figure 5 As shown in the figure, the update process from obtaining unlabeled data to the final playback buffer set is as follows Figure 5 The arrow shown points to completion, Figure 5 The schematic diagram and Figure 2 The flowchart corresponds to the flowchart, which more clearly and intuitively shows the detailed process of robot target recognition based on online continuous learning.

[0158] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0159] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the robot target recognition method based on online continuous learning provided by the above methods, and the method includes: obtaining a preset number of unlabeled image sample data; calculating the pseudo loss value of the unlabeled image sample data based on a historical deep neural network model; obtaining a query probability according to the pseudo loss value, and selecting sample data with large information content and diversity from the unlabeled image sample data based on the query probability for labeling to obtain manually labeled samples; calculating the mean of each type of samples in the manually labeled samples in the deep feature space, and calculating the covariance of each type of samples in the manually labeled samples based on the mean; selecting category samples that did not appear in this round of iteration in the replay buffer set, and sampling from the category samples according to the covariance to obtain sampled samples; using the historical samples, manually labeled samples and sampled samples in the replay buffer set as updated training sets, training the historical deep neural network model to obtain an updated deep neural network model.

[0160] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the above-mentioned robot target recognition method based on online continuous learning, the method comprising: obtaining a preset number of unlabeled image sample data; calculating the pseudo-loss value of the unlabeled image sample data based on a historical deep neural network model; obtaining a query probability according to the pseudo-loss value, and selecting sample data with large information content and diversity from the unlabeled image sample data based on the query probability for labeling to obtain manually labeled samples; calculating the mean of each type of samples in the manually labeled samples in the deep feature space, and calculating the covariance of each type of samples in the manually labeled samples based on the mean; selecting category samples that did not appear in this round of iteration in the replay buffer set, and sampling from the category samples according to the covariance to obtain sampled samples; using the historical samples, manually labeled samples and sampled samples in the replay buffer set as updated training sets to train the historical deep neural network model to obtain an updated deep neural network model.

[0161] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0162] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A robot target recognition method based on online continuous learning, characterized in that: include: Obtaining a preset number of unlabeled image sample data; Calculating a pseudo loss value of the unlabeled image sample data based on a historical deep neural network model; Obtaining a query probability according to the pseudo loss value, and selecting sample data with a large amount of information and maintaining diversity from the unlabeled image sample data based on the query probability for labeling to obtain manually labeled samples; the sample data with a large amount of information and maintaining diversity is the sample data with a value of 1 in the random variable obeying the Bernoulli distribution; Calculating the mean of each type of samples in the manually labeled samples in the deep feature space, and calculating the covariance of each type of samples in the manually labeled samples based on the mean; Selecting category samples that have not appeared in the iteration in the playback buffer set, sampling the category samples according to the covariance to obtain sampled samples; The historical samples in the playback buffer set, the manually labeled samples and the sampling samples are used as an updated training set to train the historical deep neural network model to obtain an updated deep neural network model.

2. The robot target recognition method based on online continuous learning according to claim 1 is characterized in that: The calculating of the pseudo loss value of the unlabeled image sample data based on the historical deep neural network model includes: Based on the historical deep neural network model, predict the category label and the label corresponding to the second largest score of each image sample in the unlabeled image sample data; According to the category label and the label corresponding to the second largest score, a pseudo loss value of the unlabeled image sample data is calculated through a preset relationship.

3. The robot target recognition method based on online continuous learning according to claim 1 is characterized in that: Obtaining a query probability according to the pseudo loss value includes: Based on a preset method, a target number of query sample data are selected from the unlabeled image sample data; The query probability of the query sample data is calculated according to the pseudo loss value.

4. The robot target recognition method based on online continuous learning according to claim 3 is characterized in that: The step of selecting sample data with large information content and diversity from the unlabeled image sample data based on the query probability for labeling to obtain manually labeled samples includes: Generate a random variable that obeys Bernoulli distribution according to the query probability; The query sample data whose value of the random variable is 1 is labeled to obtain manually labeled samples.

5. The robot target recognition method based on online continuous learning according to claim 1 is characterized in that: The selecting, from the playback buffer set, category samples that do not appear in the iteration, and sampling the category samples according to the covariance to obtain sampled samples includes: Select the category samples that did not appear in this round of iteration in the playback buffer set; Taking the category sample as the center of the feature extraction function, and constructing a multivariate Gaussian distribution according to the center of the feature extraction function and the covariance; The sampling samples are obtained by sampling from the multivariate Gaussian distribution in a reparameterized manner.

6. The robot target recognition method based on online continuous learning according to claim 1 is characterized in that: The method of training the historical deep neural network model using the historical samples in the playback buffer set, the manually annotated samples, and the sampled samples as an updated training set to obtain an updated deep neural network model includes: Merging the historical samples in the playback buffer, the manually annotated samples, and the sampled samples to form an updated training set; When the manually labeled samples contain new category data, the same number of neurons as the new categories in the new category data are added to the last layer of the historical deep neural network model to obtain a deep neural network model to be updated; Based on minimizing the loss function, the deep neural network model to be updated is trained using the updated training set to obtain an updated deep neural network model.

7. The robot target recognition method based on online continuous learning according to claim 1 is characterized in that: After obtaining and updating the deep neural network model, the method further includes: The historical samples in the playback buffer are updated using the manually labeled samples.

8. The robot target recognition method based on online continuous learning according to claim 7 is characterized in that: The updating of the historical samples in the playback buffer set by using the manually labeled samples includes: Merging the manually annotated samples with the historical samples in the playback buffer set to obtain a candidate set; Initializing the playback buffer set and calculating the sample capacity of the playback buffer set; Selecting updated samples that are a preset multiple of the sample size from the candidate set; Calculate the loss value of each type of updated samples; Based on a preset rule, an update sample with the same capacity as the sample is selected from the update samples according to the loss value and stored in a replay buffer set to update the replay buffer set.

9. A robot target recognition system based on online continuous learning, characterized in that: include: A data acquisition module is used to acquire a preset number of unlabeled image sample data; A label prediction module, configured to calculate a pseudo loss value of the unlabeled image sample data based on a historical deep neural network model; A manual labeling module is configured to obtain a query probability based on the pseudo-loss value, and select sample data with a large amount of information and a large diversity from the unlabeled image sample data based on the query probability for labeling to obtain manually labeled samples; the sample data with a large amount of information and a large diversity is the sample data with a value of 1 in a random variable that follows a Bernoulli distribution; A feature sampling module, configured to calculate the mean of each type of samples in the manually labeled samples in the deep feature space, and calculate the covariance of each type of samples in the manually labeled samples based on the mean; Selecting category samples that have not appeared in the iteration in the playback buffer set, sampling the category samples according to the covariance to obtain sampled samples; The model updating module is used to train the historical deep neural network model using the historical samples in the playback buffer set, the manually labeled samples and the sampling samples as the updated training set to obtain an updated deep neural network model.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the robot target recognition method based on online continuous learning as described in any one of claims 1 to 8 are implemented.

Images