Structured planar object recognition model training and recognition method

Abstract

The application discloses a structured planar object identification model training and identification method, and the method comprises the following steps: acquiring image data of a structured planar object to be identified; inputting the image data into a structured planar object identification model, and outputting a category identification result of the structured planar object; wherein the structured planar object identification model is obtained by pre-training, and comprises a feature extractor and a classifier; and a loss function used when training the feature extractor is obtained based on a Center loss function and a Triplet loss function. The application can further improve the discrimination ability of the structured planar object category, especially the discrimination ability of the structured planar object of a small sample category.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer technology, in particular to a structured planar object recognition model training and recognition method. BACKGROUND

[0002] An automatic structured planar object recognition system is of great significance to improve work efficiency. For example, an automatic tablet recognition system can improve hospital work efficiency, help the visually impaired, and avoid cross-infection. However, most of the existing structured planar object (such as tablets) recognition systems based on deep learning algorithms can only classify the structured planar object categories learned by the algorithm under the condition of having enough training data. In actual application, due to the expensive cost of data labeling and the increasing of new category structured planar objects, it is of great significance to develop a small sample category incremental structured planar object (such as tablets) recognition system.

[0003] The prior art proposes a structured planar object (such as tablets) recognition system based on small sample learning (FSL), which combines multiple artificial features and two-stage training based on triplet loss. The system can recognize different structured planar objects (such as tablets) through a small amount of labeled training data.

[0004] The system includes a feature extractor and a classifier, and adopts a decoupling strategy to separate the learning of feature representation and the classifier to avoid the problems of catastrophic forgetting and overfitting. Specifically, the backbone trained on the base class data is fixed as the feature extractor to learn the feature representation of the base class and the new class. Only the classifier will be updated for the new class.

[0005] However, in actual application, the inventors of the present application found that the existing small sample learning based structured planar object recognition system still needs to be further improved in the discrimination ability of structured planar object categories. SUMMARY

[0006] Therefore, the purpose of the present application is to provide a structured planar object recognition model training and recognition method, which can further improve the discrimination ability of structured planar object categories, especially the discrimination ability of small sample category structured planar objects.

[0007] In order to achieve the above purpose, the present application provides a structured planar object recognition method, which comprises:

[0008] Obtaining image data of a structured planar object to be recognized;

[0009] Inputting the image data into a structured planar object recognition model, and outputting a category recognition result of the structured planar object;

[0010] The structured planar object recognition model is pre-trained and includes a feature extractor and a classifier; the loss function used when training the feature extractor is based on a combination of the Center loss function and the Triplet loss function.

[0011] Preferably, image data of the structured planar object to be identified is acquired;

[0012] The image data is input into a structured planar object recognition model, and the category recognition result of the structured planar object is output.

[0013] The structured planar object recognition model is pre-trained and includes a feature extractor and a classifier; the loss function used when training the feature extractor is based on a combination of the Center loss function and the Triplet loss function.

[0014] Furthermore, the structured planar object recognition model also includes: a GAT adaptive model; and

[0015] The structured planar object recognition model is trained using the following method:

[0016] In the first training phase, when training the feature extractor using the base class session, the loss function used is based on the combination of the Center loss function and the Triplet loss function.

[0017] In the second training phase, the GAT adaptive model is trained using pseudo-incremental sessions;

[0018] In the third training phase, the classifier is trained using incremental sessions.

[0019] Preferably, the loss function used when training the feature extractor includes the CT loss function. L ct Specifically, as shown in Formula 1:

[0020] (Formula 1)

[0021] in, Indicates training samples The feature representation output by the feature extractor after being input into the structured planar object recognition model; for The category center of the tag category; To and The category center of the label category is the category center of the nearest category; The set critical value; N This indicates the number of training samples.

[0022] Further, the loss function used when training the feature extractor also includes a SoftMax loss function, specifically as shown in Equation 2:

[0023] (Equation 2)

[0024] wherein, represents the SoftMax loss function, is a hyperparameter set.

[0025] The application also provides a structured planar object recognition model training method, comprising:

[0026] When training the feature extractor in the structured planar object recognition model using the base session, the loss function used is obtained based on the Center loss function combined with the Triplet loss function;

[0027] The classifier in the structured planar object recognition model is trained using the incremental session.

[0028] The application also provides an electronic device comprising a central processing unit, a signal processing and storage unit, and a computer program stored on the signal processing and storage unit and executable on the central processing unit, wherein the central processing unit executes the program to implement the structured planar object recognition method or the structured planar object recognition model training method as described above.

[0029] The application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is executable by at least one processor to enable the at least one processor to execute the steps of the structured planar object recognition method or the structured planar object recognition model training method as described above.

[0030] In the technical solution of the application, image data of a structured planar object to be recognized is obtained; the image data is input into a structured planar object recognition model to output a class recognition result of the structured planar object; wherein the structured planar object recognition model is obtained by pre-training and comprises a feature extractor and a classifier; and a loss function used when training the feature extractor is obtained based on a Center loss function combined with a Triplet loss function; since the Center-Triplet (CT) loss function is obtained based on the Center loss function combined with the Triplet loss function L ctThe learning of the feature representation with high discriminability is realized by selecting an anchor point category and limiting the distance between the feature representation of all samples in the category and the corresponding category center to be less than the distance between the category center and another nearest category center by a certain interval value, so as to improve the discriminability of the structured planar object category, especially the discriminability of the structured planar object of the small sample category. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0032] Figure 1 An internal structure block diagram of a structured planar object recognition model provided for the first embodiment of the present application is provided.

[0033] Figure 2 A structured planar object recognition model training method flowchart provided for the first embodiment of the present application is provided.

[0034] Figure 3 An internal structure block diagram of a structured planar object recognition model provided for the second embodiment of the present application is provided.

[0035] Figure 4 A structured planar object recognition model training method flowchart provided for the second embodiment of the present application is provided.

[0036] Figure 5 A structured planar object recognition method flowchart provided for the third embodiment of the present application is provided.

[0037] Figure 6 An electronic device hardware structure schematic diagram provided for the embodiments of the present application is provided. DETAILED DESCRIPTION

[0038] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to specific embodiments and drawings.

[0039] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0040] In fact, few-shot class-incremental learning is a novel machine learning problem. Its goal is to design a machine learning algorithm that can continuously learn new knowledge from new classes with only a few labeled training data while retaining knowledge learned from previous classes. In the current FSCIL (few-shot class-incremental learning) setting, the data flow typically consists of a base class session and a series of incremental sessions. The base class session has sufficient training data, while the training set in the incremental sessions is presented in an N-way K-shot format. Once the algorithm's learning process enters session i, the training data from previous sessions becomes unavailable. The test data used during the testing process in session i consists of the union of all previously seen sessions and the test dataset from the current session. Specifically, let... and and represent the training set and test set respectively in the few-shot incremental learning session, where n represents the number of incremental sessions in the few-shot incremental learning. This represents the training dataset in the underlying session, which contains a large amount of labeled training data. For any integer... , The training data contained in it is N-way K-shot, that is, the training set in session i contains N categories, and each category has K labeled samples. This represents the test set in session i. For any integer... , represent and The corresponding label space. There is no overlap between data categories in different sessions; that is, for any integer... and , When the training process enters the session i, only the training data in is available, and the training data set in the previous sessions will no longer be available. For the performance test on the session i, the test data used is composed of the test set data in the current session and all previous sessions, that is, .

[0041] The inventors of the present application analyzed the existing structured planar object recognition system and found that in the prior art, when training the feature extractor with base class data, the main goal is to learn a robust backbone from the base class data and fix its parameters as the feature extractor to generate a powerful discriminative feature representation for images from base class sessions and new sessions to avoid catastrophic forgetting and overfitting. However, many existing small sample class incremental methods use traditional loss functions to train the backbone network, such as using a cross-entropy (CE) loss function to train the backbone network. However, the feature representation learned by the CE loss function is usually not discriminative enough because its purpose is to find a decision boundary between different classes. Unlike these small sample class incremental learning methods, many small sample learning methods focus on generating discriminative feature representations to perform recognition tasks with very little training data. Among these methods, the Triplet loss function is widely used to learn a strongly discriminative feature representation. This loss function can make the distance between feature representations from the same class smaller than the distance between feature representations from different classes by a certain interval value. However, in practical applications, the inventors of the present application found that although the Triplet loss function can make the distance between the same class smaller than the distance between different classes, it cannot consider the inter-class distance between feature representations within the same class. In addition, the mining of triplets also has an important impact on the training process. Simple or too difficult triplets have different degrees of negative impact on the training process.

[0042] Based on the above analysis, in the technical scheme of the present application, the Center-Triplet (CT) loss function is obtained based on the Center loss function combined with the Triplet loss function L ct ; the CT loss function achieves the learning of a highly discriminative feature representation by selecting an anchor class and limiting the distance between the feature representations of all samples in this class to their corresponding class center to be less than a certain interval value between the distance between the class center and another nearest class center, thereby improving the discriminative ability for structured planar object classes, especially for small sample classes of structured planar objects; experiments have shown that the CT loss function performs better than other loss functions.

[0043] The technical scheme of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0044] Embodiment one

[0045] The internal structure of a structured planar object recognition model (such as a tablet recognition model) provided by the embodiment one of the present application, as shown in the figure, comprises a feature extractor and a classifier. Figure 1

[0046] For the structured planar object recognition model as shown in the figure, the training method of a structured planar object recognition model provided by the embodiment one of the present application, the flow is as shown in the figure, comprising the following steps: Figure 1 Figure 2

[0047] Step S201: In the first training stage, the feature extractor in the structured planar object recognition model is trained using the base class session.

[0048] Specifically, for the structured planar object (such as a tablet) with a large number of samples, the training data generated according to the image data and the label of the structured planar object (such as a tablet) is used as the base class session.

[0049] In the first training stage of this step, when training the feature extractor in the structured planar object recognition model using the base class session, the image data of one structured planar object (such as a tablet) in the base class session is input as a training sample into the structured planar object recognition model (such as a tablet recognition model).

[0050] According to the recognition result output by the structured planar object recognition model (such as a tablet recognition model), the label of the input training sample, and the loss value calculated by the loss function, the parameters in the feature extractor in the structured planar object recognition model (such as a tablet recognition model) are adjusted according to the calculated loss value to realize the training of the feature extractor. After the training of the feature extractor is completed, the parameters of the feature extractor are fixed.

[0051] In the present application, when training the feature extractor in the structured planar object recognition model (such as a tablet recognition model) using the base class session, the loss function used is based on the Center loss function combined with the Triplet loss function, which can specifically include the Center-Triplet (CT) loss function L ct The CT loss function focuses on learning an efficient backbone model to perform discriminative representation learning by simultaneously consolidating intra-class compactness and inter-class separation in the feature space. The form of the triplet in the CT loss function is similar to that in the TC loss function, which contains a class center , of the label category of a single training sample , and​​​ The category center of the nearest category of the tag category These triples in each mini-batch are used to calculate the CT loss. Specifically, for each valid triple in the mini-batch... The goal of CT loss is to update the backbone model through a predefined critical value. m To constrain feature representation Its corresponding category center The distance between them, that is The value is less than With another nearest sample center The distance between them, that is The value of . The definition of the CT loss function can be shown in Equation 1:

[0052] (Formula 1)

[0053] in, Indicates training samples The feature representation output by the feature extractor after being input into a structured planar object recognition model (such as a pill recognition model); for The category center of the tag category; To and The category center of the label category is the category center of the nearest category; The threshold value is set; N represents the number of training samples.

[0054] Compared to the Triplet loss, the CT loss function proposed in this invention can fully utilize the intra-class distance between a sample and its corresponding analog center to improve intra-class compactness. This is because the CT loss adopts the same triplet form as the TC loss and an intra-class distance constraint strategy based on the Center loss. Moreover, compared to the TC loss, our proposed CT loss has strong noise resistance during gradient backpropagation because the negative sample pairs of the triples in our method are composed of two different class centers, which is generally less noisy than the estimated sample feature representation. In addition, compared to the Triplet loss, our proposed CT loss directly considers the distance between different class centers rather than the distance between one sample and another class center. This difference allows our proposed CT loss to have better inter-class separation than the TC loss. Thanks to the stronger discriminative power of our proposed CT loss, it has great potential in small-sample incremental structured planar object (e.g., pill) recognition tasks.

[0055] Considering that the class centers in the CT loss are randomly initialized and the update is based on a small batch rather than the entire dataset, while the Softmax loss function helps to better find the class centers, further improving the discriminability of the learning of the feature representation, thereby further improving the discriminability of the structured planar object (such as a tablet) class, especially the discriminability of the structured planar object (such as a tablet) of a small sample class;

[0056] Therefore, as a more optimal implementation, the SoftMax loss function is also included in the loss function used when training the feature extractor, that is, a joint supervision method based on the Softmax loss function and the CT loss function is adopted, as shown in formula 2:

[0057] (Formula 2)

[0058] wherein, the SoftMax (cross-entropy) loss function, is a hyperparameter set, L ct The calculation method of is shown in formula 1 above.

[0059] Step S202: The second training phase, using the incremental session to train the classifier in the structured planar object recognition model;

[0060] Specifically, for the structured planar object (such as a tablet) of a small sample class, that is, the structured planar object (such as a tablet) of a small sample class, the training data generated according to the image data and the label of the structured planar object (such as a tablet) is used as an incremental session.

[0061] The second training phase of this step, when training the classifier in the structured planar object recognition model (such as a tablet recognition model) using the incremental session, the image data of one structured planar object (such as a tablet) in the incremental session is input to the structured planar object recognition model (such as a tablet recognition model) as a training sample.

[0062] According to the recognition result output by the structured planar object recognition model (such as a tablet recognition model), the label of the input training sample, and the loss value calculated by the loss function, the parameters in the classifier of the structured planar object recognition model (such as a tablet recognition model) are adjusted according to the calculated loss value, so as to realize the training of the classifier.

[0063] In the technical solution of the embodiment one of the application, when training the feature extractor in the structured planar object recognition model (such as a tablet recognition model), the loss function used is a Center-Triplet (CT) loss function based on a Center loss function combined with a Triplet loss function L ct The CT loss function is used to learn a high-discriminative feature representation by selecting an anchor class and limiting the distance between the feature representation of all samples in the class and the corresponding class center to be less than a certain interval value of the distance between the class center and another nearest class center.

[0064] More preferably, when training the feature extractor in the structured planar object recognition model (such as a tablet recognition model), the loss function used also includes a Softmax loss function to help better find the class center and further improve the discriminability of the feature representation learning, thereby further improving the discrimination ability of the structured planar object (such as a tablet) class, especially the discrimination ability of the structured planar object (such as a tablet) of a small sample class.

[0065] Embodiment two

[0066] The internal structure of the structured planar object recognition model provided by the embodiment two of the application is shown in Figure 3 , which includes a feature extractor, a GAT adaptive model, and a classifier.

[0067] For the structured planar object recognition model (such as a tablet recognition model) shown in Figure 3 , the training method of the structured planar object recognition model (such as a tablet recognition model) provided by the embodiment two of the application is shown in Figure 4 , which includes the following steps:

[0068] Step S401: In the first training phase, the feature extractor in the structured planar object recognition model is trained using the base class session.

[0069] Specifically, for the structured planar object (such as a tablet) of a class with a large number of samples, the training data generated according to the image data and labels of these structured planar objects (such as tablets) is used as the base class session.

[0070] In the first training phase of this step, when training the feature extractor in the structured planar object recognition model using the base class session, the image data of one structured planar object (such as a tablet) in the base class session is input into the structured planar object recognition model (such as a tablet recognition model) as a training sample.

[0071] According to the recognition result output by the structured planar object recognition model (such as a tablet recognition model), the label of the input training sample, and the loss value calculated by the loss function, the parameters in the feature extractor in the structured planar object recognition model (such as a tablet recognition model) are adjusted according to the calculated loss value to realize the training of the feature extractor. After the training of the feature extractor is completed, the parameters of the feature extractor are fixed.

[0072] Preferably, when training the feature extractor in the structured planar object recognition model (such as a tablet recognition model) using the base class conversation, the loss function used is based on the Center loss function combined with the Triplet loss function, which can specifically include the above-mentioned Center-Triplet (CT) loss function L ct .

[0073] More preferably, when training the feature extractor in the structured planar object recognition model (such as a tablet recognition model) using the base class conversation, the loss function used can also include a Softmax loss function, as shown in the above formula 2.

[0074] Step S402: In the second training phase, the GAT (graph attention network) adaptive model in the structured planar object recognition model is trained using pseudo-incremental conversations;

[0075] Specifically, the traditional deep learning method aims to learn an effective classifier to determine the appropriate decision boundary between learned classes. Adding new classes can directly lead to the invalidation of existing decision boundaries. Therefore, in order to find a balance between old and new classes, a GAT adaptive model is proposed to solve this problem, which has great potential to obtain better decision boundaries between old and new classes due to its scalable topology and self-attention-based update mechanism. In detail, the prototypes of previous classes and new classes in the classifier can be regarded as nodes in a graph, and the self-attention mechanism in the GAT adaptive model can expand the context information between nodes in the graph and update them. In order to enable the GAT adaptive model to be used to update the classifier in the actual incremental scenario, it is necessary to create a pseudo-incremental learning step to train it.

[0076] When generating a pseudo-incremental conversation, the image data of the structured planar object (such as a tablet) of the existing class is transformed to generate pseudo image data of the structured planar object (such as a tablet) of the new class, and the training data generated according to the pseudo image data of the structured planar object (such as a tablet) of the new class and its label is generated as a pseudo-incremental conversation;

[0077] The prior art usually obtains pseudo image data of pseudo incremental sessions by rotating real image data; however, the inventors of the present application find that pseudo image data generated in a random rotation manner cannot achieve a better training purpose, and thus the inventors carefully analyze the differences between different structured plane objects, and find that the differences are mainly related to factors such as color and printed characters on the surface of a structured plane object (such as a tablet). Therefore, in order to make the generated pseudo data image more real, the present application proposes a new strategy of creating pseudo image data of a new class of structured plane objects by randomly adding printed characters and changing the color distribution on the surface of a real structured plane object;

[0078] That is, as a more optimal implementation, when generating a pseudo incremental session, a plurality of structured plane objects (such as tablets) are sampled from a base class session, pseudo image data of structured plane objects are generated by randomly adding printed characters and changing the color distribution on the surface of the sampled structured plane objects (such as tablets), and thus training data is generated as pseudo incremental sessions according to the generated pseudo image data of structured plane objects and the pseudo class labels labeled for the pseudo image data;

[0079] The second training phase of the present step is to input the image data of a pseudo structured plane object of the pseudo incremental session as a training sample into the structured plane object recognition model when training the GAT adaptive model in the structured plane object recognition model using the pseudo incremental session;

[0080] According to the recognition result output by the structured plane object recognition model, the label of the input training sample, the loss value is calculated through a loss function, and then the parameters in the GAT adaptive model in the structured plane object recognition model are adjusted according to the calculated loss value, so as to realize the training of the GAT adaptive model.

[0081] In fact, the goal of the pseudo incremental learning proposed by the present application is to transfer the knowledge learned in the previous task to the new task. The pseudo incremental learning strategy proposed by the present application uses the base training data set in the FSCIL task to create a set of pseudo incremental task sequences (pseudo incremental sessions) for learning the GAT adaptive model. In order to enable the GAT adaptive model to perform a real class incremental structured plane object recognition task, our pseudo incremental task needs to be close to the real scene. Therefore, each pseudo incremental task contains a pseudo base class and an incremental class. The pseudo base class is directly sampled from , and each class contains a support set and a query set . Unlike the direct construction process of the pseudo base class, we create the pseudo incremental task by randomly adding printed characters and changing the color distribution on the surface of the structured plane object The rest of the sampled image data is randomly added with printed text on the surface of the structured planar object or the color distribution is changed to create pseudo-incremental classes. This method can make the pseudo-incremental data closer to the real incremental data. For example Each pseudo-incremental class is also composed of a support set and a query set. It can be represented as After creating the pseudo-base class and the incremental class, the backbone model trained by the CT loss is used to learn the classifiers in and respectively. and These classifiers are integrated and fed into the original GAT model for updating. Then, the updated classifiers are combined with the backbone model to predict the query set . The Softmax loss is used to calculate the loss value between the true label and the prediction result to optimize the GAT adaptive model. Once the GAT adaptive model is learned, the internal parameters will be fixed, and the real incremental task can be performed.

[0082] Step S403: a third training phase, using an incremental session to train the classifier in the structured planar object recognition model;

[0083] Specifically, for the structured planar object (such as a tablet) with a small number of samples, i.e., the structured planar object (such as a tablet) with a small sample class, the training data generated according to the image data and the label of the structured planar object (such as a tablet) is used as an incremental session;

[0084] In the third training phase of this step, when training the classifier in the structured planar object recognition model (such as a tablet recognition model) using an incremental session, the image data of one structured planar object (such as a tablet) in the incremental session is input into the structured planar object recognition model (such as a tablet recognition model) as a training sample;

[0085] According to the recognition result output by the structured planar object recognition model (such as a tablet recognition model), the label of the input training sample, and the loss value calculated by the loss function, the parameters in the classifier of the structured planar object recognition model (such as a tablet recognition model) are adjusted according to the calculated loss value to realize the training of the classifier.

[0086] In fact, in real-world class-incremental tasks, the classifier needs to continuously accept new classes without forgetting the knowledge learned from previous classes. To achieve this goal, the present invention adopts a decoupled strategy that separates the learning of feature representation and classifier. The backbone model trained by CT loss is fixed to generate discriminative representations, which can avoid catastrophic forgetting of previously learned knowledge. In addition, the learning of the classifier also significantly affects the performance of the small-sample class-incremental structured plane object recognition task. If the new class prototype generated by the average feature representation is directly connected with the previous classifier, it may cause the decision boundary to fail. Therefore, a GAT adaptive model trained by a pseudo-incremental learning method proposed by the present invention is used in the previous step S402 to solve this bottleneck.

[0087] Specifically, since the dataset used in the small-sample class-incremental learning structured plane object recognition task is composed of a base session and some incremental sessions, we represent the original classifier learned in the session I as a matrix , where denotes the number of classes in the session I , C denotes the number of feature channels. The classifier can be expressed as formula 1:

[0088] (Formula 1)

[0089] In this formula, denotes the prototype of class I in the session . The pre-trained GAT adaptive model is used to update the feature prototype to find the decision boundary between old classes and new classes. The GAT adaptive model can model the classifier as a graph structure. The prototype in can be regarded as a node in the graph, and the GAT adaptive model can update them using the relationship between different nodes and self-attention mechanism. To illustrate the updating process in detail, we take the node as an example. The attention coefficient between node and all nodes in the graph is first calculated. The calculation process is shown in formula 2:

[0090] (Formula 2)

[0091] In this formula, and are linear functions that can project the initial prototype and to a new metric space, denotes the function that computes the inner product of two nodes. After all attention coefficients are computed, a Softmax function is used to regularize them. The final attention coefficient of can be obtained by formula 3:

[0092] (Formula 3)

[0093] Based on all the regularized attention coefficients, the update process of the prototype can be represented as formula 4:

[0094] (Formula 4)

[0095] In the formula, U represents a weight matrix of a linear transformation. The classifier The update of all prototypes in the classifier follows the same update process as The updated classifier can be represented as formula 5:

[0096] (Formula 5)

[0097] In each incremental session, the GAT adaptive model is used to expand the context information between the prototypes learned in the current session and the previous session. After updating, these prototypes will be aggregated into a new classifier to make predictions on the test data of all visible classes.

[0098] Embodiment three

[0099] Based on the structured planar object recognition model pre-trained according to the technical solutions in the above embodiment one or embodiment two, the structured planar object recognition method provided by the embodiment three has the flow as shown in Figure 5 , and includes the following steps:

[0100] Step S501: Obtain image data of a structured planar object to be recognized.

[0101] Step S502: Input the image data into the structured planar object recognition model, and output a class recognition result of the structured planar object.

[0102] The structured planar object recognition model can be specifically the structured planar object recognition model as shown in Figure 1 or Figure 3 , and the structured planar object recognition model can be specifically trained by the method flow as shown in Figure 2 or Figure 4 .

[0103] Figure 6 ​A more specific electronic device hardware structure schematic diagram provided by the embodiment is shown. The device can include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 for internal communication.

[0104] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, and is configured to execute related programs to implement the structured planar object recognition method or the structured planar object recognition model training method provided by the embodiments of the present specification.

[0105] The memory 1020 can be implemented by a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, etc. The memory 1020 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present specification are implemented by software or firmware, the related program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0106] The input / output interface 1030 is configured to connect an input / output module and can be connected to a nonlinear receiver to receive information from the nonlinear receiver and realize information input and output. The input / output module can be configured as a component in the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. The input device can include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device can include a display, a speaker, a vibrator, an indicator light, etc.

[0107] The communication interface 1040 is configured to connect a communication module (not shown in the figure) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired manner (such as USB, network cable, etc.) or through a wireless manner (such as mobile network, WIFI, Bluetooth, etc.).

[0108] The bus 1050 includes a channel to transmit information between various components (such as the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040) of the device.

[0109] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device can also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device can also only contain components necessary to implement the embodiment scheme of the present specification, and does not necessarily contain all the components shown in the figure.

[0110] The embodiment of the present application also provides a computer readable storage medium, the computer readable storage medium has a computer program stored thereon, and the computer program is executed by a processor to implement the steps of the structured planar object identification method or the structured planar object identification model training method.

[0111] In the embodiment, the computer readable storage medium includes a flash memory, a hard disk, a multimedia card, a card type memory (for example, an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read only memory (ROM), an electrically erasable programmable read only memory (EEPROM), a programmable read only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the computer readable storage medium can be an internal storage unit of a computer device, for example, a hard disk or a memory of the computer device. In other embodiments, the computer readable storage medium can also be an external storage device of the computer device, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Of course, the computer readable storage medium can also include both the internal storage unit and the external storage device of the computer device. In the embodiment, the computer readable storage medium is usually used to store the operating system and various application software installed on the computer device, for example, the program code of the structured planar object identification method or the structured planar object identification model training method in the embodiment, etc. In addition, the computer readable storage medium can also be used to temporarily store various data that have been output or will be output.

[0112] In the technical solution of the present application, image data of a structured planar object to be identified is acquired; the image data is input into a structured planar object identification model, and a category identification result of the structured planar object is output; wherein the structured planar object identification model is obtained by pre-training and includes a feature extractor and a classifier; and a loss function used when training the feature extractor is obtained based on a Center loss function combined with a Triplet loss function; since the Center-Triplet (CT) loss function is obtained based on the Center loss function combined with the Triplet loss function L ct is obtained by selecting an anchor category and limiting the distance between the feature representation of all samples in the category and the corresponding category center to be less than the distance between the category center and another nearest category center by a certain interval value, thereby realizing the learning of the feature representation with high discriminability, thereby improving the discriminability of the structured planar object category, especially the discriminability of the structured planar object of a small sample category.

[0113] The computer readable medium of the embodiment includes permanent and non-permanent, removable and non-removable media, which can realize information storage by any method or technology. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0114] Those skilled in the art should understand that the above discussion of any embodiment is only exemplary and is not intended to imply that the scope (including claims) of the present disclosure is limited to these examples; under the idea of the present application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes of different aspects of the present application as described above. In order to be brief, they are not provided in detail.

[0115] Additionally, to simplify the description and discussion, and so as not to obscure the disclosure of the present application with details that are well known to those skilled in the art, some conventional attributes of integrated circuit (IC) chips and other components can or can not be shown in the drawings and can not be explicitly described. Furthermore, devices can be shown in block diagram form in order to avoid obscuring the present application, and this also acknowledges the fact that the details in regard to implementing such block devices are highly dependent on the platform within which the present application is to be implemented (i.e., these details should be well within the purview of one of ordinary skill in the art). Where specific details are set forth in order to describe an illustrative embodiment of the present application, it should be apparent to those skilled in the art that the overall scope of the present application can be practiced without such specific details. In other instances, detailed descriptions of well-known methods, devices, and materials can be omitted so as not to obscure the disclosure of the present application. Likewise, the description just given as to the software implementation should not be taken as indicative of the only manner in which the present application can be implemented but should be taken as illustrative of the functionalities involved.

[0116] While the present application has been described in connection with a specific embodiment thereof, many alternatives, modifications, and variations will be apparent to those skilled in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) can use the embodiments discussed.

[0117] Embodiments of the present application are intended to cover all such alternatives, modifications, and variations as falling within the scope of the appended claims. Accordingly, any omission, modification, substitution, improvement, etc. made in the spirit and principle of the present application should be included within the scope of the present application.

Claims

1. A structured planar object recognition method, comprising: Acquire image data of the structured planar object to be identified; The image data is input into a structured planar object recognition model, and the category recognition result of the structured planar object is output. The structured planar object recognition model is pre-trained and includes a feature extractor and a classifier; and the loss function used when training the feature extractor is based on the Center loss function combined with the Triplet loss function. The loss function used when training the feature extractor includes the CT loss function. L ct Specifically, as shown in Formula 1: (Formula 1) in, Indicates training samples The feature representation output by the feature extractor after being input into the structured planar object recognition model; for The category center of the tag category; To and Category center of the tag category The category center of the nearest category; The set critical value; N This indicates the number of training samples.

2. The method according to claim 1, characterized in that, The structured planar object recognition model is trained using the following method: In the first training phase, when training the feature extractor using the base class session, the loss function used is based on the combination of the Center loss function and the Triplet loss function. In the second training phase, the classifier is trained using incremental sessions.

3. The method according to claim 2, characterized in that, The structured planar object recognition model further includes: a GAT adaptive model, wherein the GAT adaptive model is a graph attention network adaptive model; and The structured planar object recognition model is trained using the following method: In the first training phase, when training the feature extractor using the base class session, the loss function used is based on the combination of the Center loss function and the Triplet loss function. In the second training phase, the GAT adaptive model is trained using pseudo-incremental sessions; wherein, when generating pseudo-incremental sessions, the image data of structured planar objects of the existing categories are transformed to generate image data of pseudo-new categories of structured planar objects, and training data is generated based on the generated image data of pseudo-new categories of structured planar objects and their labels as pseudo-incremental sessions. In the third training phase, the classifier is trained using incremental sessions.

4. The method according to claim 1, characterized in that, The loss function used when training the feature extractor also includes the SoftMax loss function, as shown in Formula 2: (Formula 2) in, Represents the SoftMax loss function. These are the hyperparameters that are set.

5. A method for training a structured planar object recognition model, characterized in that, include: When training the feature extractor in the structured planar object recognition model using the base class session, the loss function used is obtained by combining the Center loss function with the Triplet loss function; The classifier in the structured planar object recognition model is trained using incremental sessions; The loss function used when training the feature extractor includes the CT loss function. L ct Specifically, as shown in Formula 1: (Formula 1) in, Indicates training samples The feature representation output by the feature extractor after being input into the structured planar object recognition model; for The category center of the tag category; To and The category center of the label category is the category center of the nearest category; The set critical value; N This indicates the number of training samples.

6. The method according to claim 5, characterized in that, The loss function used when training the feature extractor also includes the SoftMax loss function, as shown in Formula 2: (Formula 2) in, Represents the SoftMax loss function. These are the hyperparameters that are set.

7. An electronic device comprising a central processing unit, a signal processing and storage unit, and a computer program stored on the signal processing and storage unit and executable on the central processing unit, characterized in that, When the central processing unit executes the program, it implements the method as described in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that can be executed by at least one processor to cause the at least one processor to perform the steps of the method according to any one of claims 1-6.

Images