Structured planar object recognition model training and recognition method, electronic device

Abstract

The application discloses a structured planar object identification model training and identification method and electronic equipment. The identification method comprises the following steps: acquiring image data of a structured planar object to be identified; inputting the image data into a structured planar object identification model to output a category identification result of the structured planar object; wherein the structured planar object identification model comprises a feature extractor, a GAT adaptive model and a classifier; and the structured planar object identification model is obtained by pre-training according to the following method: in a first training stage, the feature extractor is trained using a base class session; in a second training stage, the GAT adaptive model is trained using a pseudo-incremental session; and in a third training stage, the classifier is trained using an incremental session. The application can further improve the discrimination ability of the structured planar object category, especially the discrimination ability of the structured planar object of a small sample category.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer technology, in particular to a structured planar object recognition model training and recognition method and electronic equipment. BACKGROUND

[0002] An automatic structured planar object recognition system is of great significance to improve work efficiency. For example, an automatic tablet recognition system can improve hospital work efficiency, help the visually impaired, and avoid cross-infection. However, most existing structured planar object (such as tablets) recognition systems based on deep learning algorithms can only classify the structured planar object categories learned by the algorithm under the condition of sufficient training data. In actual application, due to the expensive cost of data labeling and the increasing of new category structured planar objects, it is of great significance to develop a small sample category incremental structured planar object (such as tablets) recognition system.

[0003] The prior art proposes a structured planar object (such as tablets) recognition system based on small sample learning (FSL), which combines multiple artificial features and two-stage training based on triplet loss. The system can recognize different structured planar objects (such as tablets) through a small amount of labeled training data.

[0004] The system includes a feature extractor and a classifier, and adopts a decoupling strategy to separate the learning of feature representation and the classifier to avoid the problems of catastrophic forgetting and overfitting. Specifically, the backbone trained on base class data is fixed as the feature extractor to learn the feature representation of the base class and the new class. Only the classifier will be updated for the new class.

[0005] However, in actual application, the inventors of the present application found that the existing small sample learning based structured planar object recognition system still needs to be further improved in the discrimination ability of structured planar object categories. SUMMARY

[0006] Therefore, the purpose of the present application is to provide a structured planar object recognition model training and recognition method and electronic equipment, which can further improve the discrimination ability of structured planar object categories, especially the discrimination ability of small sample category structured planar objects.

[0007] In order to achieve the above purpose, the present application provides a structured planar object recognition method, comprising:

[0008] Obtaining image data of a structured planar object to be recognized;

[0009] Inputting the image data into a structured planar object recognition model, and outputting a category recognition result of the structured planar object;

[0010] The structured planar object recognition model includes a feature extractor, a GAT adaptive model, and a classifier; and the structured planar object recognition model is pre-trained using the following method:

[0011] In the first training phase, the feature extractor is trained using a base class session;

[0012] In the second training phase, the GAT adaptive model is trained using pseudo-incremental sessions;

[0013] In the third training phase, the classifier is trained using incremental sessions.

[0014] Preferably, the pseudo-incremental session is generated according to the following method:

[0015] Image data of pseudo-new categories of structured planar objects were created by randomly adding imprinted text and changing the color distribution to the surface of real structured planar objects as pseudo-incremental sessions.

[0016] Preferably, the pseudo-incremental session consists of a set of pseudo-incremental task sequences, wherein each pseudo-incremental task contains a pseudo-base class and an increment class.

[0017] The pseudo-base class is obtained by directly sampling from the base class session;

[0018] The incremental class is created by randomly adding imprinted text and changing the color distribution to the surface of a real structured planar object in the base class session.

[0019] Preferably, the loss function used when training the feature extractor includes the CT loss function. L ct Specifically, as shown in Formula 1:

[0020] (Formula 1)

[0021] in, Indicates training samples The feature representation output by the feature extractor after being input into the structured planar object recognition model; for The category center of the tag category; To and The category center of the label category is the category center of the nearest category; The set critical value; N This indicates the number of training samples.

[0022] Preferably, the loss function used when training the feature extractor further comprises a SoftMax loss function, specifically as shown in Equation 2:

[0023] (Equation 2)

[0024] wherein, denotes the SoftMax loss function, is a hyperparameter set.

[0025] The present application also provides a structured planar object recognition model training method, comprising:

[0026] training a feature extractor in the structured planar object recognition model using a base session;

[0027] training a GAT adaptive model in the structured planar object recognition model using a pseudo-incremental session;

[0028] training a classifier in the structured planar object recognition model using an incremental session.

[0029] The present application also provides an electronic device comprising a central processing unit, a signal processing and storage unit, and a computer program stored in the signal processing and storage unit and executable on the central processing unit, wherein the central processing unit implements the structured planar object recognition method or the structured planar object recognition model training method as described above when executing the program.

[0030] The present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is executable by at least one processor to cause the at least one processor to perform the steps of the structured planar object recognition method or the structured planar object recognition model training method as described above.

[0031] In the technical solution of the present application, image data of a structured planar object to be identified is acquired; the image data is input into a structured planar object identification model, and a category identification result of the structured planar object is output; wherein the structured planar object identification model comprises a feature extractor, a GAT adaptive model and a classifier; and the structured planar object identification model is obtained by pre-training according to the following method: in a first training stage, the feature extractor is trained using a base class session; in a second training stage, the GAT adaptive model is trained using a pseudo-incremental session; and in a third training stage, the classifier is trained using an incremental session. Since the GAT adaptive model has great potential to obtain a better decision boundary between old classes and new classes, as well as an extensible topology and a self-attention-based updating mechanism. That is, the previous classes and new classes in the classifier can be regarded as nodes in a graph, and the self-attention mechanism in the GAT adaptive model can expand the context information between the nodes in the graph and update them; so that the GAT adaptive model can be used to update the classifier in the actual incremental scenario, thereby further improving the classification ability of the classifier for the categories of structured planar objects, especially for small sample categories of structured planar objects. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0033] Figure 1 An internal structure block diagram of a structured planar object identification model provided for Embodiment Two of the present application;

[0034] Figure 2 A structured planar object identification model training method flowchart provided for Embodiment Two of the present application;

[0035] Figure 3 A structured planar object identification method flowchart provided for Embodiment Three of the present application;

[0036] Figure 4 An electronic device hardware structure schematic diagram provided for the embodiments of the present application. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solutions and advantages of the present application more clear, the present application will be further described in detail below with reference to specific embodiments and drawings.

[0038] It should be noted that the technical terms or scientific terms used in the embodiments of the present application should be understood as the common meanings understood by those skilled in the art to which the present disclosure belongs, unless otherwise defined. The terms "first", "second", and the like used in the present disclosure do not represent any order, number, or importance, but are only used to distinguish different components. The terms "include", "contain", and the like mean that the elements or objects before the terms encompass the elements or objects listed after the terms and their equivalents, and do not exclude other elements or objects. The terms "connect" or "connected" and the like are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms "upper", "lower", "left", "right", and the like are only used to represent relative positional relationships, and when the absolute positions of the described objects are changed, the relative positional relationships can also be changed accordingly.

[0039] In fact, few-shot class-incremental learning is a new machine learning problem. The purpose is to design a machine learning algorithm that can continuously learn new knowledge from new classes with only a few labeled training data while retaining the knowledge learned from previous classes. In the current FSCIL (few-shot class-incremental learning) setting, the data stream is usually composed of a base session and a series of incremental sessions, where the base session has sufficient training data, and the training set in the incremental session is in the form of N-way K-shot. Once the learning process of the algorithm enters session i, the training data in the previous sessions will no longer be available. The test data used in the test process of session i is the union of the test data sets in all previously seen sessions and the current session. Specifically, let and represent the training set and test set in the few-shot class-incremental learning session, respectively, where n represents the number of incremental sessions in the few-shot class-incremental learning. represents the training data set in the base session, which contains a large amount of labeled training data. For any integer , The training data contained in represents the test set in session i. For any integer , represents and correspond to the label space. There is no intersection between the data classes in different sessions, i.e., for any integer and , When the training process enters session i, only the training data in is available, and the training data sets in previous sessions will no longer be available. For performance testing on session i, the test data used is composed of the test set data in the current session and all previous sessions, that is, .

[0040] The inventors of the present application analyzed the existing structured planar object recognition system and found that the traditional deep learning method aims to learn an effective classifier to determine the appropriate decision boundary between learned classes, and then perform structured planar object recognition through the trained classifier; however, adding new classes may directly cause the existing decision boundary to fail. Therefore, in order to find the balance between old and new classes, the technical solution of the present application proposes a GAT (graph attention network) adaptive model to solve this problem, which has great potential to obtain a better decision boundary between old and new classes due to its scalable topology and self-attention-based update mechanism. That is, the prototypes of previous classes and new classes in the classifier can all be regarded as nodes in a graph, and the self-attention mechanism in the GAT adaptive model can expand the context information between the nodes in the graph and update them; so that the GAT adaptive model can be used to update the classifier in the actual incremental scenario, thereby further improving the discrimination ability of the classifier for structured planar object classes, especially the discrimination ability for small sample classes of structured planar objects.

[0041] Further, when training the feature extractor with base class data, the main goal is to learn a robust backbone with base class data and fix its parameters as the feature extractor to generate a strong discriminative feature representation for images from base class sessions and new sessions to avoid catastrophic forgetting and overfitting. However, many existing few-shot class-incremental methods employ traditional loss functions to train the backbone network, such as using a cross-entropy (CE) loss function to train the backbone network. However, the feature representation learned by the CE loss function is usually not discriminative enough because its purpose is to find a decision boundary between different classes. Unlike these few-shot class-incremental learning methods, many few-shot learning methods focus on generating discriminative feature representations to perform recognition tasks with very few training data. Among these methods, the Triplet loss function is widely used to learn a strong discriminative feature representation. This loss function can make the distance between feature representations from the same class smaller than the distance between feature representations from different classes by a certain interval value. However, in practical applications, the inventors of the present application have found that although the Triplet loss function can make the distance between the same class smaller than the distance between different classes, it cannot consider the inter-class distance between feature representations within the same class. In addition, the mining of triplets also has an important impact on the training process. Simple or too difficult triplets have different degrees of negative impact on the training process.

[0042] Based on the above analysis, in the technical scheme of the present application, the Center-Triplet (CT) loss function is obtained based on the Center loss function combined with the Triplet loss function L ct The CT loss function achieves the learning of a highly discriminative feature representation by selecting an anchor class and limiting the distance between the feature representations of all samples in this class to their corresponding class center to be less than a certain interval value between the distance between this class center and another nearest class center, thereby improving the discriminative ability for structured planar object classes, especially for small sample classes of structured planar objects. Experimental results show that the CT loss function performs better than other loss functions.

[0043] The technical scheme of the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0044] Embodiment one

[0045] The internal structure of a structured planar object recognition model (such as a tablet recognition model) provided by the embodiment of the present application is shown in Figure 1 , which includes a feature extractor, a GAT adaptive model, and a classifier.

[0046] For Figure 1The structured planar object recognition model shown in Embodiment 2 of this invention provides a training method for a structured planar object recognition model, the process of which is as follows: Figure 2 As shown, it includes the following steps:

[0047] Step S201: First training phase, using base class sessions to train the feature extractor in the structured planar object recognition model;

[0048] Specifically, for structured planar objects (such as pills) with a large number of samples, training data generated from the image data and labels of these structured planar objects (such as pills) is used as the base class session;

[0049] In the first training phase of this step, when training the feature extractor in the structured planar object recognition model using the base class session, the image data of a structured planar object (such as a pill) from the base class session is input as a training sample into the structured planar object recognition model (such as the pill recognition model).

[0050] Based on the recognition results output by the structured planar object recognition model (such as a pill recognition model), the labels of the input training samples are used. After calculating the loss value using a loss function, the parameters in the feature extractor of the structured planar object recognition model (such as a pill recognition model) are adjusted according to the calculated loss value to achieve feature extractor training. After the feature extractor training is completed, the parameters of the feature extractor are fixed.

[0051] In the technical solution of this invention, when training the feature extractor in the structured planar object recognition model (such as a pill recognition model) using a base class session, the loss function used is based on a combination of the Center loss function and the Triplet loss function, specifically including the Center-Triplet (CT) loss function. L ct The CT loss function focuses on learning an efficient backbone model for discriminative representation learning by simultaneously consolidating intra-class compactness and inter-class separation in the feature space. The triplet form in the CT loss function is similar to that in the TC loss function, containing a single training sample. , Category center of the tag category ,and The category center of the nearest category of the tag category These triples in each mini-batch are used to calculate the CT loss. Specifically, for each valid triple in the mini-batch... The goal of CT loss is to update the backbone model through a predefined critical value. m To constrain feature representation the distance between it and the nearest class center the value of the distance between it and another nearest sample center the value of

[0052] (Formula 1)

[0053] wherein, denotes the feature representation output by the feature extractor after the training sample is input to the structured planar object recognition model (such as the tablet recognition model); is the class center of the label class of is the class center of the nearest class to the class center of is the threshold value set; N represents the number of training samples.

[0054] Compared with the Triplet loss, the CT loss function proposed in the present application can make full use of the intra-class distance between the sample and the corresponding class center to improve the intra-class compactness, because the CT loss adopts the same triplet form as the TC loss and the intra-class distance constraint strategy based on the Center loss. Moreover, compared with the TC loss, the CT loss proposed in the present application has strong anti-noise ability in the process of gradient back propagation, because the negative sample pair of the triplet in the method is composed of two different class centers, which is usually smaller than the noise contained in the estimated sample feature representation. In addition, compared with the Triplet loss, the CT loss proposed in the present application directly considers the distance between different class centers instead of the distance between a sample and another class center. This difference makes the CT loss proposed in the present application have better inter-class separation than the TC loss. Thanks to the stronger discriminative ability of the CT loss proposed in the present application, it has great potential in the small sample class incremental structured planar object (such as tablet) recognition task.

[0055] Considering that the class centers in the CT loss are randomly initialized and the update is based on the small batch instead of the entire data set, and the Softmax loss function helps to better find the class center and further improve the discriminative ability of the feature representation learning, thereby further improving the discriminative ability of the structured planar object (such as tablet) class, especially the discriminative ability of the structured planar object (such as tablet) of the small sample class;

[0056] ​​​​Therefore, as a better implementation, the loss function used when training the feature extractor also includes a SoftMax loss function, that is, a joint supervision method based on the Softmax loss function and the CT loss function is adopted, as shown in formula 2:

[0057] (Formula 2)

[0058] wherein, denotes the SoftMax (cross-entropy) loss function, is a hyperparameter set, L ct The calculation method of is shown in formula 1 above.

[0059] Step S202: In the second training stage, the GAT (graph attention network) adaptive model in the structured planar object recognition model is trained using pseudo incremental sessions.

[0060] Specifically, the traditional deep learning method aims to learn an effective classifier to determine the appropriate decision boundary between learned classes. Adding new classes may directly lead to the invalidation of existing decision boundaries. Therefore, in order to find a balance between old and new classes, a GAT adaptive model is proposed to solve this problem, which has great potential to obtain better decision boundaries between old and new classes due to its scalable topology and self-attention-based update mechanism. In detail, the prototypes of previous classes and new classes in the classifier can be regarded as nodes in a graph, and the self-attention mechanism in the GAT adaptive model can expand the context information between nodes in the graph and update them. In order to enable the GAT adaptive model to be used to update the classifier in the actual incremental scenario, it is necessary to create a pseudo incremental learning step to train it.

[0061] In generating the pseudo incremental session, the image data of the structured planar object (such as a tablet) of the existing class is transformed to generate pseudo image data of the structured planar object (such as a tablet) of the new class, and the training data is generated as the pseudo incremental session according to the generated pseudo image data of the structured planar object (such as a tablet) of the new class and its label;

[0062] The prior art usually obtains pseudo image data of pseudo incremental conversation by rotating real image data; however, the inventors of the present application find that pseudo image data generated in a random rotation manner cannot achieve a better training purpose, and thus the inventors carefully analyze the differences between different structured planar objects and find that the differences are mainly related to factors such as color and marked text on the surface of the structured planar object. Therefore, in order to make the generated pseudo data image more real, the present application proposes a new strategy of creating pseudo image data of a new class of structured planar object by randomly adding marked text and changing the color distribution on the surface of the real structured planar object.

[0063] That is, as a more optimal implementation, when generating a pseudo incremental conversation, a plurality of structured planar objects are sampled from a base class conversation, pseudo image data of a structured planar object is generated by randomly adding marked text and changing the color distribution on the surface of the sampled structured planar object, and thus training data is generated as a pseudo incremental conversation according to the generated pseudo image data of the structured planar object and the pseudo class label annotated for the pseudo image data.

[0064] The second training phase of the present step is that when training the GAT adaptive model in the structured planar object recognition model (such as a tablet recognition model) using the pseudo incremental conversation, the image data of a pseudo structured planar object (such as a tablet) of the pseudo incremental conversation is input into the structured planar object recognition model as a training sample.

[0065] According to the recognition result output by the structured planar object recognition model (such as a tablet recognition model), the label of the input training sample, the loss value is calculated through a loss function, and then the parameters in the GAT adaptive model in the structured planar object recognition model (such as a tablet recognition model) are adjusted according to the calculated loss value, so as to realize the training of the GAT adaptive model.

[0066] In fact, the goal of the pseudo incremental learning proposed by the present application is to transfer the knowledge learned in the previous task to the new task. The pseudo incremental learning strategy proposed by the present application uses the base training data set in the FSCIL task to create a set of pseudo incremental task sequences (pseudo incremental conversations) for learning the GAT adaptive model. In order to enable the GAT adaptive model to perform a real class incremental structured planar object recognition task, our pseudo incremental task needs to be close to the real scene. Therefore, each pseudo incremental task contains a pseudo base class and an incremental class. The pseudo base class is directly sampled from , and each class contains a support set and a query set . Unlike the direct construction process of the pseudo base class, we create a pseudo incremental task by randomly adding marked text and changing the color distribution on the surface of the structured planar object in the base class The rest of the sampled image data is randomly added with printed text on the surface of the structured planar object or the color distribution is changed to create pseudo-incremental classes. This method can make the pseudo-incremental data closer to the real incremental data. For example Each pseudo-incremental class is also composed of a support set and a query set. It can be represented as After creating the pseudo-base class and the incremental class, the backbone model trained by the CT loss is used to learn the classifiers in and respectively. and These classifiers are integrated and fed into the original GAT model for updating. Then, the updated classifiers are combined with the backbone model to predict the query set . The Softmax loss is used to calculate the loss value between the true label and the prediction result to optimize the GAT adaptive model. Once the GAT adaptive model is learned, the internal parameters will be fixed, and the real incremental task can be performed.

[0067] Step S203: a third training phase, using an incremental session to train the classifier in the structured planar object recognition model;

[0068] Specifically, for the structured planar object (such as a tablet) with a small number of samples, i.e. the structured planar object of the small sample class, the training data generated according to the image data and the label of the structured planar object (such as a tablet) is used as an incremental session.

[0069] In the third training phase of this step, when training the classifier in the structured planar object recognition model (such as a tablet recognition model) using an incremental session, the image data of one structured planar object (such as a tablet) in the incremental session is input into the structured planar object recognition model (such as a tablet recognition model) as a training sample.

[0070] According to the recognition result output by the structured planar object recognition model, the label of the input training sample, and the loss value calculated by the loss function, the parameters in the classifier of the structured planar object recognition model are adjusted according to the calculated loss value to realize the training of the classifier.

[0071] In fact, in real-world incremental class recognition tasks, classifiers need to continuously accept new classes without forgetting knowledge learned from previous classes. To achieve this, this invention employs a decoupling strategy of learning feature representations and classifiers separately. The backbone model trained using CT loss is fixed to generate discriminative representations, which avoids catastrophic forgetting of previously learned knowledge. Furthermore, classifier learning significantly impacts the performance of few-sample incremental structured planar object recognition tasks. Directly connecting new class prototypes generated through average feature representations to previous classifiers might cause decision boundary failures. Therefore, in the previous step S202, a GAT adaptive model trained using the pseudo-incremental learning method proposed in this invention is used. To solve this bottleneck.

[0072] Specifically, since the dataset used in the few-shot category incremental learning structured planar object recognition task consists of a base session and several incremental sessions, we will... I The original classifier learned above is represented as a matrix. ,in Representing a session I The number of categories in C Indicates the number of feature channels. (Classifier) It can be expressed as in Formula 3:

[0073] (Formula 3)

[0074] In this formula Representing a session I Medium category The prototype is then used. A pre-trained GAT adaptive model is employed to update the feature prototype in order to find the decision boundary between the old and new classes. The GAT adaptive model can improve the classifier... The model is a graph structure. The prototypes in the graph can be viewed as nodes, and the GAT adaptive model can update them using the relationships between different nodes and the self-attention mechanism. To illustrate the update process in detail, we will use the nodes in the graph... As an example. Node. The attention coefficients between the node and all nodes in the graph are calculated first. The calculation process is shown in Equation 4:

[0075] (Formula 4)

[0076] In this formula and It is able to convert the initial prototype and A linear function projected onto a new metric space. denotes the function that computes the inner product of two nodes. After all the attention coefficients are computed, a Softmax function is used to regularize them. with the last attention coefficient of can be obtained by formula 5:

[0077] (formula 5)

[0078] Based on all the regularized attention coefficients, the update process of the prototype can be represented as formula 6:

[0079] (formula 6)

[0080] In this formula, U represents a weight matrix of a linear transformation. The updated classifier can be represented as formula 7:

[0081] (formula 7)

[0082] In each incremental session, the GAT adaptive model is used to expand the context information between the prototypes learned in the current session and the previous session. After updating, these prototypes will be aggregated into a new classifier to make predictions on the test data of all visible classes.

[0083] Embodiment two

[0084] Based on the technical solution of the above embodiment one, the structured planar object recognition model is pre-trained, and the structured planar object recognition method provided by the embodiment two of the application has a process as shown in Figure 3 , and includes the following steps:

[0085] Step S301: Obtain image data of a structured planar object to be identified.

[0086] Step S302: Input the image data into the structured planar object recognition model, and output a class recognition result of the structured planar object.

[0087] Among them, the structured planar object recognition model can be specifically the structured planar object recognition model as shown in Figure 1 , and the structured planar object recognition model can be specifically trained by the method process as shown in Figure 2 .

[0088] Figure 4 ​​A more specific electronic device hardware structure schematic diagram provided by the embodiment is shown. The device can include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 for internal communication.

[0089] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, and is configured to execute related programs to implement the structured planar object recognition method or the structured planar object recognition model training method provided by the embodiments of the present specification.

[0090] The memory 1020 can be implemented by a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, etc. The memory 1020 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present specification are implemented by software or firmware, the related program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0091] The input / output interface 1030 is configured to connect an input / output module and can be connected to a nonlinear receiver to receive information from the nonlinear receiver and realize information input and output. The input / output module can be configured as a component in the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. The input device can include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device can include a display, a speaker, a vibrator, an indicator light, etc.

[0092] The communication interface 1040 is configured to connect a communication module (not shown in the figure) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired manner (such as USB, network cable, etc.) or through a wireless manner (such as mobile network, WIFI, Bluetooth, etc.).

[0093] The bus 1050 includes a channel to transmit information between various components (such as the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040) of the device.

[0094] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device can also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device can also only contain components necessary to implement the embodiment scheme of the present specification, and does not necessarily contain all the components shown in the figure.

[0095] The embodiment of the present application also provides a computer readable storage medium, the computer readable storage medium has a computer program stored thereon, and the computer program is executed by a processor to implement the steps of the structured planar object identification method or the structured planar object identification model training method.

[0096] In the embodiment, the computer readable storage medium includes a flash memory, a hard disk, a multimedia card, a card type memory (for example, an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read only memory (ROM), an electrically erasable programmable read only memory (EEPROM), a programmable read only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the computer readable storage medium can be an internal storage unit of a computer device, for example, a hard disk or a memory of the computer device. In other embodiments, the computer readable storage medium can also be an external storage device of the computer device, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Of course, the computer readable storage medium can also include both the internal storage unit and the external storage device of the computer device. In the embodiment, the computer readable storage medium is usually used to store the operating system and various application software installed on the computer device, for example, the program code of the structured planar object identification method or the structured planar object identification model training method in the embodiment, etc. In addition, the computer readable storage medium can also be used to temporarily store various data that have been output or will be output.

[0097] In the technical scheme of the present application, image data of a structured planar object to be identified is acquired; the image data is input into a structured planar object identification model, and a category identification result of the structured planar object is output; wherein the structured planar object identification model comprises a feature extractor, a GAT adaptive model and a classifier; and the structured planar object identification model is obtained by pre-training according to the following method: in a first training stage, the feature extractor is trained using a base class session; in a second training stage, the GAT adaptive model is trained using a pseudo-incremental session; and in a third training stage, the classifier is trained using an incremental session. Since the GAT adaptive model has great potential to obtain a better decision boundary between old classes and new classes, and an extensible topology and a self-attention-based updating mechanism. That is, the previous classes and the new class prototypes in the classifier can be regarded as nodes in a graph, and the self-attention mechanism in the GAT adaptive model can expand the context information between the nodes in the graph and update them; so that the GAT adaptive model can be used to update the classifier in the actual incremental scene, thereby further improving the discrimination ability of the classifier for the categories of structured planar objects, especially the discrimination ability for small sample categories of structured planar objects.

[0098] The computer readable medium of the present embodiment includes permanent and non-permanent, removable and non-removable media, which can realize information storage by any method or technology. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0099] It should be understood by those of ordinary skill in the art that the above discussion of any embodiment is only exemplary and is not intended to imply that the scope (including claims) of the present disclosure is limited to these examples; under the idea of the present application, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the present application as described above. In order to be brief, they are not provided in detail.

[0100] Additionally, to simplify the description and discussion, and so as not to obscure the disclosure with details that will be readily apparent to those skilled in the art, known power / ground connections to integrated circuit (IC) chips and other components can or can not be shown in the drawings. Moreover, devices can be shown in block diagram form in order to avoid obscuring the present disclosure, and this also is intended to include all such detail within the scope of the present disclosure. To the extent that specific details of certain embodiments of the present disclosure are set forth in this detailed description, it should be understood that such is for illustration only and should not limit the scope of the disclosure.

[0101] While the present disclosure has been described with respect to a limited number of embodiments, it will be apparent to those skilled in the art that many alternate, modifications, and variations will be apparent. For instance, other memory architectures (e.g., dynamic RAM (DRAM)) can use the embodiments discussed.

[0102] Embodiments of the present disclosure are intended to encompass all such alternatives, modifications, and variations as falling within the broad scope of the appended claims. Accordingly, any and all such modifications, variations, or equivalents that fall within the spirit and scope of the present disclosure are intended to be embraced herein.

Claims

1. A method for structured planar object recognition, comprising: obtaining image data of a structured planar object to be recognized; inputting the image data into a structured planar object recognition model to output a class recognition result of the structured planar object; wherein the structured planar object recognition model comprises a feature extractor, a GAT adaptive model, and a classifier, and the structured planar object recognition model is pre-trained according to the following method: in a first training phase, training the feature extractor using a base session; in a second training phase, training the GAT adaptive model using a pseudo-incremental session; in a third training phase, training the classifier using an incremental session; the pseudo-incremental session is generated according to the following method: creating image data of a pseudo new class of structured planar object by randomly adding marked text and changing color distribution on the surface of a real structured planar object as a pseudo-incremental session; the pseudo-incremental session is composed of a group of pseudo-incremental task sequences, wherein each pseudo-incremental task contains a pseudo-base class and an incremental class; wherein the pseudo-base class is directly sampled from the base session; the incremental class is created by randomly adding marked text and changing color distribution on the surface of a real structured planar object in the base session; The loss function employed in training the feature extractor includes a CT loss function L ct as shown in Equation 1 : (Formula 1) wherein, represents a training sample a feature representation output by the feature extractor upon input to the structured planar object recognition model; is a class center of the label class is a class center of the class closest to the class center of the label class is a class center of the class closest to the class center of the label class is a threshold value set N represents a number of training samples; the loss function used when training the feature extractor further comprises a SoftMax loss function, specifically as shown in formula 2: (Formula 2) wherein, denotes a SoftMax loss function, is a hyperparameter set.

2. A structured planar object identification model training method applied to the structured planar object identification method according to claim 1, characterized by, comprising: training the feature extractor in the structured planar object recognition model using a base session; training the GAT adaptive model in the structured planar object recognition model using a pseudo-incremental session; training the classifier in the structured planar object recognition model using an incremental session.

3. The method of claim 2, wherein, the pseudo-incremental session is generated according to the following method: creating image data of a pseudo new class of structured planar object by randomly adding marked text and changing color distribution on the surface of a real structured planar object as a pseudo-incremental session.

4. The method of claim 2, wherein, the pseudo-incremental session is composed of a group of pseudo-incremental task sequences, wherein each pseudo-incremental task contains a pseudo-base class and an incremental class; wherein the pseudo-base class is directly sampled from the base session; the incremental class is created by randomly adding marked text and changing color distribution on the surface of a real structured planar object in the base session.

5. An electronic device comprising a central processing unit, a signal processing and storage unit, and a computer program stored on the signal processing and storage unit and executable on the central processing unit, characterized in that, The central processing unit implements the method of any one of claims 1-4 when executing the program.

6. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program which can be executed by at least one processor to cause the at least one processor to perform the steps of the method of any one of claims 1-4.

Images