Apparatus and method for training a model for identifying persons

Abstract

A device (1) and a method for training a model for identifying people, comprising a primary training module (11) configured to primarily train the model with respect to a prepared source data set, a target subset generation module (12) configured to generate a target subset by selecting some cameras from a plurality of cameras included in a service robot, a feature vector extraction module (13) configured to extract feature vectors of the target subset using the model, a labeling module (15) configured to perform labeling of the feature vectors, and a secondary training module (16) configured to secondarily train the model with respect to a target data set using results of the labeling.

Description

Technical field

[0001] The disclosure relates to an apparatus and a method for training a model for identifying persons. background

[0002] Service robots are robots designed to perform specific tasks or provide services. They differ from industrial robots used in factories and can be used in a wide variety of fields. For example, service robots can include household robots that perform various services such as cleaning at home; medical service robots that assist patients with medical treatment; service robots that serve food; guidance and counseling robots that guide visitors or provide information; security robots that manage specific areas or building security; and assistance robots that facilitate the daily lives of people with disabilities or the elderly.Service robots operate autonomously without direct human control, but interaction with a person is often important, and it is often necessary to detect and locate people.

[0003] For this purpose, a multi-camera system can be used in service robots. The multi-camera system can enable service robots to recognize an environment in three dimensions and detect a person from different angles. In particular, the multi-camera system can also detect areas that may not be detected with a single camera and can integrate and process multiple data collected from multiple cameras, which can improve recognition accuracy. It can also recognize the environment and a person in three dimensions, which can enable more accurate localization and interaction. Although the multi-camera system improves the person recognition capability of service robots, an efficient algorithm is required because the amount of data to be processed and the computational complexity increase.In addition, person re-identification (ReID), which recognizes the person captured in the field of view of one camera in the field of view of another camera, is indispensable in multi-camera systems. Brief explanation

[0004] The disclosure relates to an apparatus and a method for training a model for identifying persons or humans.

[0005] Some embodiments of the present disclosure may provide an apparatus and a method for training a person identification model capable of improving the person recognition and person re-identification performance (hereinafter also referred to as re-recognition performance) of a service robot equipped with a multi-camera system.

[0006] A device for training a model or a model training device for identifying persons according to an embodiment may comprise: one or more processors and a storage medium storing computer-readable instructions that, when executed by the one or more processors, enable the one or more processors to provide: a primary training module configured to primarily (e.g.as the first or first time) with respect to a prepared source data set, a target subset generation module configured to generate a target subset by selecting some cameras from a plurality of cameras mounted on a service robot, a feature vector extraction module configured to extract feature vectors of the target subset using the model, a labeling module configured to perform labeling on the feature vectors, and a secondary training module configured to secondarily train the model (e.g. for the second time) with respect to a target data set using results of the labeling.

[0007] For example, in some embodiments, the instructions further enable the one or more processors to include a feature vector grouping module configured to determine similarities of the feature vectors and group the feature vectors according to the similarities.

[0008] For example, in some embodiments, the feature vector clustering module may further cluster the feature vectors using a DBSCAN (Density-Based Spatial Clustering of Applications with Noise) technique.

[0009] For example, in some embodiments, the labeling module may further perform pseudo-labeling for each cluster classified by the feature vector clustering module.

[0010] For example, in some embodiments, the instructions further enable the one or more processors to include a weight assignment module configured to assign a weight to each group classified by the feature vector grouping module.

[0011] For example, in some embodiments, the weight assignment module may further assign a relatively higher weight to a given group that has been determined to have relatively higher diversity.

[0012] For example, in some embodiments, the secondary training module may further perform progressive training by increasing a reflection ratio of a given group assigned a relatively higher weight.

[0013] For example, in some embodiments, the instructions further enable the one or more processors to include a retry module configured to repeat the target subset generation module, the feature vector extraction module, the feature vector grouping module, and the labeling module (or their functions) until selected, specified, or determined conditions are met.

[0014] For example, in some embodiments, the instructions further enable the one or more processors to have a learning sequence generation module configured to generate a learning sequence for training the model using repeated labeling results generated by the repeating module, and the secondary training module may further perform learning sequence learning on the model according to the learning sequence.

[0015] For example, in some embodiments, the target subset generation module may further generate the target subset by preferentially selecting a given camera from the plurality of cameras that generates values ​​with a relatively smaller difference from the source data set.

[0016] A method for training a model for identifying people according to an embodiment may comprise: primary (e.g., first) training of the model with respect to or with a prepared source data set, generating a target subset by selecting some cameras from a plurality of cameras included in a service robot, extracting feature vectors of the target subset using the model, labeling the feature vectors, and secondary (e.g., second) training of the model with respect to or with the target subset using the results of the labeling.

[0017] For example, in some embodiments, the method may further comprise determining similarities of the feature vectors and grouping the feature vectors according to the similarities.

[0018] For example, in some embodiments, clustering the feature vectors may comprise clustering the feature vectors using a density-based spatial clustering of applications with noise (DBSCAN) technique.

[0019] For example, in some embodiments, the labeling may include performing a pseudo-labeling for each categorized group.

[0020] For example, in some embodiments, the method may further comprise assigning a weight to each classified group.

[0021] For example, in some embodiments, assigning the weight may include assigning a relatively higher weight to a given group that has been determined to have relatively higher diversity.

[0022] For example, in some embodiments, the secondary training may comprise progressively performing the training by increasing a reflection ratio of a given group assigned the relatively higher weight.

[0023] For example, in some embodiments, the method may further comprise repeating the steps of generating the target subset, extracting the feature vectors, grouping the feature vectors, and labeling until (e.g., pre-specified) conditions are met.

[0024] For example, in some embodiments, the method may further comprise generating a learning sequence for training the model using results of the label repeatedly generated by the iteration, and the secondary training may comprise performing a learning sequence learning on the model according to the learning sequence.

[0025] For example, in some embodiments, generating the target subset may comprise generating the target subset by preferentially selecting, from the plurality of cameras, a given camera that generates values ​​with a relatively smaller difference from the source data set. Short description of the drawings Fig. 1 and Fig. 2 are diagrams illustrating an apparatus for training a person identification model according to an embodiment of the present disclosure. Fig.3 is a diagram illustrating a method for training a person identification model according to an embodiment of the present disclosure. Fig. 4 and Fig. 5 are diagrams illustrating an apparatus for training a person identification model according to an embodiment of the present disclosure. Fig. 6 and Fig. 7 are diagrams illustrating an apparatus for training a person identification model according to an embodiment of the present disclosure. Fig. 8 is a diagram for explaining a data processing apparatus according to an embodiment of the present disclosure. Detailed description of the embodiments

[0026] With reference to the accompanying drawings, embodiments of the present disclosure will be described in detail below so that a person skilled in the art can easily implement the present disclosure. However, embodiments of the present disclosure may be implemented in many different forms and are not limited to the embodiments described herein. In order to clearly explain the present disclosure in the drawings, parts irrelevant to the description may be omitted, and like reference numerals may designate like elements throughout the description.

[0027] Throughout the specification and claims, the word "comprise" and variations such as "comprising" or "has" may be understood to imply the inclusion of the recited elements, but not the exclusion of others, unless expressly described otherwise. Terms that include ordinal numbers such as "first," "second," etc., may be used to describe various elements, and those elements are not necessarily limited by such terms. Such terms may be used merely to distinguish one element from another.

[0028] Terms such as "..part", "...group" and "...module" described in the description may refer to an entity that performs at least one operation or function described in the description and that may be implemented as hardware or software or a combination of hardware and software.

[0029] The Fig. 1 and Fig. 2 are diagrams showing an apparatus for training a person identification model according to an embodiment of the present disclosure.

[0030] Human recognition through a camera is an important technical factor for a service robot that has many continuous interactions with a person, and is attracting increasing attention with the development of computer vision and deep learning technology. Human recognition can involve a variety of detailed techniques, such as face recognition, which identifies a person by analyzing facial features; pose and motion recognition, which detects and analyzes a person's posture or movement; human re-recognition, which identifies the same person in the field of view of multiple cameras; behavior recognition, which classifies or predicts a person's behavior in a video sequence; and human segmentation, which separates a human silhouette from an image or video.The performance of such techniques depends heavily on training data or a learning method, and therefore, a learning technique is important for a model used for human recognition. The apparatus 1 for training the human identification model according to an embodiment may have a configuration described below in various embodiments to improve the performance of a service robot equipped with a multi-camera system in human recognition and re-identification. Referring to FIG. Fig.1, the apparatus 1 for training a model for identifying persons may comprise a primary training module 11, a target subset generation module 12, a feature vector extraction module 13, a feature vector grouping module 14, a labeling module 15, and a secondary training module 16, wherein any combination or all of them may be plural or comprise multiple components thereof.

[0031] The primary training module 11 can primarily train a model with respect to a prepared source dataset or can train it for the first time with respect to a prepared source dataset. The model can be a model used to perform human detection and recognition of a service robot equipped with a multi-camera system. A source dataset can be data primarily used to train a model and can generally be configured as data containing a large and diverse amount of information. For example, a large image dataset such as ImageNet can be used as the source dataset. A model can learn relatively rich features by learning with respect to the source dataset. In other words, the primary training module 11 can pre-train (or pre-train) the model. Features learned from the source dataset can also be useful for an actual target task, i.e.for the detection and re-identification of people by a service robot equipped with a multi-camera system and can enable faster secondary learning with respect to a target dataset that is relatively small and contains task-specific information.

[0032] The target subset generation module 12 may generate a target subset by selecting some cameras from a plurality of cameras included in the multi-camera system mounted on the service robot. A target data set may be data for using the model pre-trained by the primary training module 11 for the actual target task, such as human recognition and re-identification tasks of the service robot equipped with the multi-camera system, and the target subset may refer to a data set indicating a specific part or category within the target data set. Specifically, the target subset in the description may refer to an image (data) set obtained primarily from one camera to ultimately generate a target data set used by the secondary training module 16 described below.The target subset may be determined as the target dataset used by the secondary training module 16 through subsequent operations such as the extraction and labeling of feature vectors described below.

[0033] When target subsets are generated simultaneously with respect to all cameras included in the multi-camera system, it may be difficult to find generality due to the high complexity, and when target subsets are generated with respect to all cameras without considering a difference from a source dataset, the learning performance may be insufficient. On the other hand, when a target subset is generated with respect to a camera with a large difference from the source dataset, the learning performance may also deteriorate. To solve such a performance problem, in some embodiments, the target subset generation module 12 may generate the target subset by preferentially selecting a camera with a smaller difference from the source dataset from the plurality of cameras of the multi-camera system.

[0034] The multi-camera system may comprise a first camera Cam 0, a second camera Cam 1, a third camera Cam 2 and a fourth camera Cam 3. With reference to (a) of Fig.2, feature vectors can be expressed in an embedding space, where feature vectors named “Source” can specify source datasets, and feature vectors named “Cam 0”, “Cam 1”, “Cam 2”, and “Cam 3” can specify target datasets corresponding to the first camera Cam 0, the second camera Cam 1, the third camera Cam 2, and the fourth camera Cam 3, respectively.In the human identification model training apparatus 1 according to an embodiment, domain shifting may be performed from the source data set “Source” to the target data sets “Cam 0”, “Cam 1”, “Cam 2”, and “Cam 3”, and when data corresponding to a camera with a smaller difference from the source data set is preferentially selected as a target subset from among the first camera Cam 0, the second camera Cam 1, the third camera Cam 2, and the fourth camera Cam 3, the target data set may also be set as data that can train a model to ensure better or excellent learning performance.

[0035] In some embodiments, differences between the source dataset and the data from the camera to be generated as the target subset can be quantified by measuring distribution differences. For example, the target subset generation module 12 can measure a difference between a distribution of the source dataset and a distribution of the camera data to be generated as the target subset using a maximum mean discrepancy (MMD) technique and, for example, calculate the difference between the two distributions by comparing the means between samples of the two distributions.A specific calculation equation may include an equation that uses a distance between expected values ​​of the two distributions in a space expressed, for example, as a function of the Reproducing Kernel Hilbert Space (RKHS). However, the scope of the present disclosure is not limited to a specific equation and may include the use of various equations that can measure the difference between the two distributions. The target subset generation module 12 may calculate an MMD value for each of the first camera Cam 0, the second camera Cam 1, the third camera Cam 2, and the fourth camera Cam 3 and data, and may consider the data of the camera corresponding to the smallest MMD value as the data with the smallest difference from the source data set and preferentially select the data as the target subset.For example, the target subset generation module 12 may calculate a distribution difference between the source dataset and each of the first camera Cam 0, the second camera Cam 1, the third camera Cam 2, and the fourth camera Cam 3, and generate the target subset corresponding to the first camera Cam 0 with at least the smallest distribution difference as the target subset. Then, feature vector extraction and labeling may be performed on the generated target subset.

[0036] In some embodiments, the target subset generation module 12 may repeat the generation of the target subset for progressive learning over multiple stages. Referring to (b) of Fig.2, the target subset generation module 12 may, as a result of calculating the distribution difference between the source data set and each of the first camera Cam 0, the second camera Cam 1, the third camera Cam 2, and the fourth camera Cam 3, generate, in a first stage, a first target subset corresponding to the first camera Cam 0 with at least the smallest distribution difference from the source data set, and may perform feature vector extraction and labeling on the generated first target subset. In a second stage following the first stage, the target subset generation module 12 may then generate a second target subset corresponding to the second camera Cam 1 with a small distribution difference from the source data set, and may perform feature vector extraction and labeling on the first target subset and the generated second target subset.In a third stage following the second stage, the target subset generation module 12 may then generate a third target subset corresponding to the third camera Cam 2 with a small distribution difference from the source dataset, and may perform feature vector extraction and labeling on the first and second target subsets and the generated third target subset. In a fourth stage following the third stage, the target subset generation module 12 may generate a fourth target subset corresponding to the fourth camera Cam 3 with the largest distribution difference from the source dataset, and may perform feature vector extraction and labeling on the first to third target subsets and the generated fourth target subset.

[0037] The feature vector extraction module 13 can extract feature vectors of the target subset generated by the target subset generation module 12 using the model. The feature vector extraction module 13 can extract information in the form of a vector representing data contained in the target subset. The extraction of feature vectors can be performed manually, but a deep learning method can also be used to directly learn features from data.

[0038] The feature vector grouping module 14 can determine similarities between the feature vectors extracted by the feature vector extraction module 13 and group the feature vectors according to the similarities. The feature vector grouping module 14 can group the feature vectors, for example, using a density-based spatial clustering with noise technique (Density-Based Spatial Clustering of Applications with Noise; also referred to herein as DBSCAN). DBSCAN is a density-based clustering algorithm that can operate on the principle of considering high-density areas, i.e., areas where data points are close to one another, as a group and processing low-density areas using noise. When DBSCAN is applied to the feature vectors extracted by the feature vector extraction module 13, similar feature vectors can be close to one another, and thus these feature vectors can be grouped.classified and considered as a group.

[0039] For example, the feature vector grouping module 14 may specify the maximum distance that another feature vector may have from a feature vector and the minimum number of feature vectors that exist within the maximum distance with respect to a feature vector. The feature vector grouping module 14 may select a feature vector and generate a new group if feature vectors that exceed the minimum number exist within the maximum distance from the selected feature vector.In addition, the feature vector grouping module 14 may form a group based on the similarities between the feature vectors by repeatedly adding all neighboring feature vectors within the maximum distance from the selected feature vector to the group if the selected feature vector has more than the minimum number of neighboring feature vectors within the maximum distance, or adding the selected feature vector to the group if the selected feature vector is within the maximum distance but has fewer feature vectors than the minimum number.

[0040] With reference to (c) of Fig.2, the feature vector grouping module 14 may generate a first group “Cluster 0” that originates from data corresponding to the first camera Cam 0, the second camera Cam 1 and the third camera Cam 2 and corresponds to a set of feature vectors that are estimated to be a person, a second group “Cluster 1” that originates from data corresponding to the third camera Cam 2 and the fourth camera Cam 3 and corresponds to a set of feature vectors that are estimated to be another person, and a third group “Cluster 2” that originates from data corresponding to the second camera Cam 1 and corresponds to a set of feature vectors that are estimated to be another person.The first camera Cam 0, the second camera Cam 1, the third camera Cam 2 and the fourth camera Cam 3 may be factors included in the multi-camera system of the service robot and record the same person from different positions and angles, and therefore the first group “Cluster 0”, which corresponds to the first camera Cam 0, the second camera Cam 1 and the third camera Cam 2, may have the highest diversity, the second group “Cluster 1”, which corresponds to the third camera Cam 2 and the fourth camera Cam 3, may have the next highest diversity, and the third group “Cluster 2”, which corresponds only to the second camera Cam 1, may have the lowest diversity.In other words, the first group, "Cluster 0," encompasses a variety of backgrounds, postures, and clothing and may have relatively extensive learning content, while the third group, "Cluster 2," most likely has the same backgrounds, postures, and clothing and may have relatively little learning content. With regard to learning or training the model for identifying people, the diversity of such groups can be additionally taken into account, which is described below with reference to the . Fig. 4 and Fig. 5 is described.

[0041] The labeling module 15 can perform pseudo-labeling for each feature vector extracted by the feature vector extraction module 13 or for each group classified by the feature vector grouping module 14. The pseudo-labeling can assign a label predicted by the model (also called a "label") to unlabeled data as a "pseudo-label," and pseudo-labeled data generated as above can optionally be used for model training along with the originally labeled data.

[0042] The secondary training module 16 may secondarily or repeatedly train the model on the target dataset using the results of the labeling performed by the labeling module 15.

[0043] As described above, a dataset designed to maximize the training effect by considering detailed features of the data obtained from the plurality of cameras included in the multi-camera system of the service robot can be expected to achieve excellent training effects by applying the training technique described above, while reducing (e.g., conventional) enormous computation time and costs that may occur in unsupervised domain adaptation. Furthermore, the human detection and recognition performance of the service robot can be improved. In particular, as an example implementation, when the model trained according to an embodiment of the present disclosure is used as a dataset of Market-1501 and DukeMTMC, and the mean average precision (MAT) isWhen mean average precision (mAP) and rank-1 accuracy are used as evaluation indicators, a performance improvement effect of about 30 %p (percentage point) is achieved as follows. Table 1 data set Evaluation indicators Conventional procedure This disclosure Market-1501 mAP 32.4% 67.5% Rank 1 51.1% 80.5% DukeMTMC mAP 32.3% 78.1% Rank 1 63.9% 92.0%

[0044] Fig. 3 is a diagram illustrating a method for training a person identification model according to an embodiment of the present disclosure.

[0045] In relation to Fig.3, a method for training a model for identifying people according to an embodiment may include: primary training of a model with respect to a prepared source dataset (operation S301), generating a target subset by selecting some cameras from a plurality of cameras included in a multi-camera mounted on a service robot (operation S302), extracting feature vectors of the target subset using the model (operation S303), performing labeling of the feature vectors (operation S304), and secondary training of the model with respect to the target dataset using the labeling results (operation S305). For more detailed information about the Fig. 3 can be used to describe the example method described here with reference to Fig.1, 2 and 4 to 8, so that a redundant description can be omitted here. For example, the embodiments described in Fig. 3 described methods using embodiments of the Fig. 1, 2 and 4 to 8 are implemented.

[0046] The Fig. 4 and Fig. 5 are diagrams illustrating an apparatus for training a person identification model according to an embodiment of the present disclosure.

[0047] With reference to Fig.4, a model training device 2 for identifying people according to an embodiment may include a primary training module 21, a target subset generation module 22, a feature vector extraction module 23, a feature vector grouping module 24, a labeling module 25, a weight assignment module 26, and a secondary training module 27, wherein any combination or all of them may be plural or may include multiple components thereof.For the primary training module 21, the target subset generation module 22, the feature vector extraction module 23, the feature vector grouping module 24, the labeling module 25 and the secondary training module 27, unless contradictory, the descriptions of the primary training module 11, the target subset generation module 12, the feature vector extraction module 13, the feature vector grouping module 14, the labeling module 15 and the secondary training module 16 described above with reference to . Fig. 1 can be used or applied, and thus the weight assignment module 26 is mainly used with reference to the Fig. 4 and Fig. 5 described.

[0048] The weighting module 26 may assign a weight to each group grouped or classified by the feature vector grouping module 24. In particular, the weighting module 26 may assign a high weight to a group for which high diversity has been determined. The secondary training module 27 may perform training incrementally by increasing the reflection ratio of the group assigned the higher weight.

[0049] With reference to Fig.5 together, the feature vector grouping module 24 can generate a first group “cluster a” that comes from data corresponding to the first camera Cam 0, the second camera Cam 1, the third camera Cam 2 and the fourth camera Cam 3 and corresponds to a set of feature vectors that are estimated to be a person, and a second group “cluster b” that comes from data corresponding to the second camera Cam 1 and the third camera Cam 2 and corresponds to a set of feature vectors that are estimated to be another person.The first camera Cam 0, the second camera Cam 1, the third camera Cam 2, and the fourth camera Cam 3 may be factors included in the service robot's multi-camera system, capturing the same person from different positions and angles. Therefore, the diversity of the first group "Cluster a," which corresponds to the first camera Cam 0, the second camera Cam 1, the third camera Cam 2, and the fourth camera Cam 3, may be higher than the diversity of the second group "Cluster b," which corresponds to the second camera Cam 1 and the third camera Cam 2. In other words, in this example, the first group "Cluster a" has more diverse backgrounds, postures, and clothing than the second group "Cluster b," and may contain relatively more learning content.

[0050] The weighting module 26 may assign a relatively higher weight (e.g., wa = 1.11) to the first group "cluster a" and a relatively lower weight (e.g., wb = 0.25) to the second group "cluster b." The secondary training module 27 may perform training incrementally by increasing the reflection ratio of the first group "cluster a" assigned a relatively higher weight compared to the second group "cluster b." According to the human identification model, model training can be performed primarily based on data with more learning content, and thus, better or excellent training effects can be expected, while training is possible with a smaller amount of data.

[0051] In some embodiments, the weight assignment module 26 may, for example, calculate a weight from an entropy value of an information theory. The entropy of the information theory may be used to measure the uncertainty of a random variable and may be calculated by considering the probability of a possible event and an amount of information of the corresponding event. A weight may be calculated from the entropy value according to a selected, fixed, or predetermined equation. In a simple method, the weight may, for example, be calculated by dividing the entropy value of each of the first group "cluster a" and the second group "cluster b" by the entropy values ​​of all groups. For example, the weight w a be calculated by using an entropy value H a= 2.03 of the first group “Cluster a” is divided by the entropy values ​​of all groups, and the weighting w b can be calculated by using an entropy value H b = 0.28 of the second group "Cluster b" is divided by the entropy values ​​of all groups. A method for calculating a weight using an entropy value is not limited to the example method described above.

[0052] The Fig. 6 and Fig. 7 are diagrams illustrating an apparatus for training a person identification model according to an embodiment of the present disclosure.

[0053] With reference to Fig.6, a model training device 3 for identifying people according to an embodiment may include a primary training module 31, a target subset generation module 32, a feature vector extraction module 33, a feature vector grouping module 34, a labeling module 35, a weight assignment module 36, a repetition module 37, a learning sequence generation module 38, and a secondary training module 39, wherein any combination or all of them may be plural or may include multiple components thereof.For the primary training module 31, the target subset generation module 32, the feature vector extraction module 33, the feature vector grouping module 34, the labeling module 35, the weight assignment module 36 and the secondary training module 39, unless contradictory, the descriptions of the primary training modules 11 and 21, the target subset generation modules 12 and 22, the feature vector extraction modules 13 and 23, the feature vector grouping modules 14 and 24, the labeling modules 15 and 25, the weight assignment module 26 and the secondary training modules 16 and 27 described above with reference to FIG. Fig. 1 and Fig. 4 are referenced or applied, and thus mainly the repetition module 37 and the learning sequence generation module 38 are described with reference to the Fig. 6 and Fig. 7 described.

[0054] The repetition module 37 may repeat (e.g., re-execute) the target subset generation module 32, the feature vector extraction module 33, the feature vector grouping module 34, and the labeling module 35 until selected, specified, or predetermined conditions are met. Additionally, the learning sequence generation module 38 may generate a learning sequence for training a model using labeling results repeatedly generated by the repetition module 37. The secondary training module 39 may perform learning sequence learning on the model(s) according to the learning sequence. Learning sequence learning may be an artificial intelligence learning strategy for solving complex problems and refers to a technique that starts with simple problems and gradually progresses to more difficult problems.In particular, in the embodiment of the example, sequence learning may refer to starting learning with data that has a high similarity to a source dataset and gradually progressing to learning with more complex and generalized data when learning has stabilized.

[0055] With reference to Fig.7, the repetition module 37 may repeat the generation of a target subset for progressive training. In a first iteration in this example, the target subset generation module 32 may generate, as a result of calculating distribution differences between a source data set and each of a plurality of cameras Cam 0 to Cam C (where C is a non-negative integer), a first target subset corresponding to a camera Cam 4 with at least the smallest distribution difference from the source data set, and the results labeled by the feature vector extraction module 33, the feature vector grouping module 34, and the labeling module 35 may be added to the learning sequence. Next, in a second iteration in this example,Execution, the target subset generation module 32 may generate a second target subset corresponding to the camera Cam 1 with a relatively smaller distribution difference from the source data set, and the results labeled by the feature vector extraction module 33, the feature vector grouping module 34, and the labeling module 35 may be added to the learning sequence. .

[0056] Next, in this example, in a third iteration or execution, the target subset generation module 32 may then generate a third target subset corresponding to the camera Cam 0 with a relatively smaller (e.g., even smaller) distribution difference from the source dataset, and results labeled by the feature vector extraction module 33, the feature vector grouping module 34, and the labeling module 35 may be added to the learning sequence. Finally, in an n-th iteration (n is a natural number), the target subset generation module 32 may generate the n-th target subset corresponding to the camera Cam C with the relatively largest distribution difference from the source dataset, and the results labeled by the feature vector extraction module 33, the feature vector grouping module 34, and the labeling module 35 may be added to the learning sequence.The secondary training module 39 may perform learning sequence learning on the model according to the learning sequence generated based on the first to n-th target subsets.

[0057] Fig. 8 is a diagram for explaining a computing device according to an embodiment of the present disclosure.

[0058] With reference to Fig. 8, an exemplary model training apparatus and method for identifying individuals according to exemplary embodiments may be implemented using a computing device 50.

[0059] Computing device 50 may include at least one of a processor 510, a memory 530, a user interface input device 540, a user interface output device 550, and a storage device 560 communicating via a bus 520, wherein any combination or all of these may be plural or comprise multiple components thereof. Computing device 50 may also include a network interface 570 electrically connected to a network 40. Network interface 570 may send or receive signals to and from other devices via network 40.

[0060] The processor 510 may be implemented in various forms, such as a microcontroller (MCU), application processor (AP), central processing unit (CPU), graphics processing unit (GPU), neural processor (NPU), and quantum processor (QPU), and may be any semiconductor device that executes instructions stored in the memory 530 or the storage device 560. The processor 510 may be configured to perform the functions and methods described above with respect to the Fig. 1 to 7 to be implemented.

[0061] A storage medium may include memory 530 and storage device 560, which may include various types of volatile or non-volatile storage media. For example, memory 530 may include read-only memory (ROM) 531 and random access memory (RAM) 532. In the embodiment, memory 530 may be located internally or externally of processor 510, and memory 530 may be coupled to processor 510 via various known implementations.

[0062] In some embodiments, at least some components or functions of the model training device and the method for identifying persons according to the embodiments may be implemented as a program or software executing on the computing device 50, and the program or software may be stored on a computer-readable medium. In particular, according to one embodiment, the computer-readable medium may be a computer having the processor 510 executing a program or instruction stored in the memory 530 or storage device 560, and having a program recorded thereon for performing steps included in the model training device and the method for identifying persons according to the embodiments.

[0063] In some embodiments, at least some components or functions of the model training device and the method for person identification according to the embodiments may be implemented using hardware or circuitry of the computing device 50 or may also be implemented as separate hardware or circuitry that may be electrically connected to the computing device 50.

[0064] According to the exemplary embodiments described above, as a dataset designed to maximize the training effect by considering detailed features of data obtained from a plurality of cameras included in a multi-camera system of a service robot, better or excellent training effects can be expected by applying the training technique described above, while reducing the (e.g., conventional) enormous computation time and cost that may occur in unsupervised domain adaptation. Furthermore, by using an embodiment of the present disclosure, the person detection and recognition performance of the service robot can be improved.

[0065] Although embodiments of the present disclosure have been described in detail above, the scope of the present disclosure is not limited thereto, and various modifications and improvements, including variations thereof, made by those skilled in the art to which the present disclosure relates may also be within the scope of the present disclosure.

Claims

[1] A device (1, 2, 3) for training a model for identifying persons, the device (1, 2, 3) comprising: one or more processors (510) and a storage medium storing computer-readable instructions that, when executed by the one or more processors (510), enable the one or more processors (510) to perform: a primary training module (11, 21, 31) configured to train the model primarily with respect to a prepared source data set, a target subset generation module (12, 22, 32) configured to generate a target subset by selecting some cameras from a plurality of cameras mounted on a service robot, a feature vector extraction module (13, 23, 33) configured to extract feature vectors of the target subset using the model, a labeling module (15, 25, 35) configured to perform labeling of the feature vectors, and a secondary training module (16, 27, 39) configured to secondarily train the model with respect to a target dataset using results of the labeling. [2] The apparatus (1, 2, 3) of claim 1, wherein the instructions further enable the one or more processors (510) to have a feature vector grouping module (14, 24, 24) configured to determine similarities of the feature vectors and group the feature vectors according to the similarities. [3] The apparatus (1, 2, 3) of claim 2, wherein the feature vector grouping module (14, 24, 34) is further configured to group the feature vectors using a density-based spatial grouping of applications with noise, DBSCAN, technique. [4] Device (1, 2, 3) according to claim 2 or 3, wherein the labeling module (15, 25, 35) is further configured to perform a pseudo-labeling for each group classified by the feature vector grouping module (14, 24, 34). [5] The apparatus (2, 3) of any one of claims 2 to 4, wherein the instructions further enable the one or more processors (510) to comprise a weight assignment module (26, 36) configured to assign a weight to each group classified by the feature vector grouping module (14, 24, 34). [6] The device (2, 3) according to claim 5, wherein the weight assignment module (26, 36) is further configured to assign a relatively higher weight to a given group for which a relatively higher diversity is determined. [7] The apparatus (2, 3) according to claim 5 or 6, wherein the secondary training module (27, 39) is further configured to perform training step by step by increasing a reflection ratio of a given group assigned a relatively higher weight. [8] Apparatus (3) according to any one of claims 2 to 7, wherein the instructions further enable the one or more processors (510) to have a repetition module (37) configured to repeat the target subset generation module (32), the feature vector extraction module (33), the feature vector grouping module (34), and the labeling module (35) until specified conditions are met. [9] The apparatus (3) of claim 8, wherein the instructions further enable the one or more processors (510) to have a learning sequence generation module (38) configured to generate a learning sequence for training the model using results of the repeated labeling generated by the repetition module (37), and wherein the secondary training module (16) is further configured to perform learning sequence learning on the model according to the learning sequence. [10] Apparatus (1, 2, 3) according to any one of the preceding claims, wherein the target subset generation module (12, 22, 32) is further configured to generate the target subset by preferentially selecting from the plurality of cameras a given camera that generates values ​​with a relatively smaller difference to the source data set. [11] A method for training a model for identifying people, comprising: primary training (S301) of the model with respect to a prepared source dataset, Creating (S302) a target subset by selecting some cameras from a plurality of cameras provided on a service robot, Extracting (S303) feature vectors of the target subset using the model, Identifying (S304) the feature vectors and secondary training (S305) of the model with respect to the target subset using labeling results. [12] The method of claim 11, further comprising: Determining similarities of the feature vectors and Grouping the feature vectors according to similarities. [13] The method of claim 12, wherein clustering the feature vectors comprises clustering the feature vectors using a density-based spatial clustering of applications with noise, DBSCAN, technique. [14] The method according to claim 12 or 13, wherein the labeling (S304) comprises performing a pseudo-labeling for each classified group. [15] The method of any one of claims 12 to 14, further comprising assigning a weight to each classified group. [16] The method of claim 15, wherein assigning a weight comprises assigning a relatively higher weight to a given group determined to have a relatively higher diversity. [17] The method according to claim 15 or 16, wherein the secondary training (S305) comprises gradually performing training by increasing a reflection ratio of a given group to which a relatively higher weight is assigned. [18] The method of any one of claims 12 to 17, further comprising repeating the generation of the target subset, the extraction of the feature vectors, the clustering of the feature vectors, and the labeling until specified conditions are met. [19] The method of claim 18, further comprising generating a learning sequence for training the model using results of the label repeatedly generated by the iteration, wherein the secondary training comprises performing a learning sequence learning on the model according to the learning sequence. [20] The method of any one of claims 11 to 19, wherein generating the target subset comprises generating the target subset by preferentially selecting, from the plurality of cameras, a given camera that generates values ​​with a relatively smaller difference from the source data set.

Images