Method for constructing brand proxy network knowledge graph

Abstract

The invention relates to the technical field of artificial intelligence and knowledge maps, and discloses a method for constructing a brand proxy network knowledge map, comprising the following steps: S1, performing multi-modal data preprocessing and OCR enhancement, eliminating interference information in a document image, improving the OCR recognition precision, and obtaining an OCR recognition result; a deep separable convolutional architecture model is adopted as a feature extraction backbone network to train in a MaskR-CNN model, a seal area in a document image is positioned, and seal removal and font enhancement are performed on the area; s2, performing OCR character recognition, and performing text extraction by using a character recognition model after processing and enhancement in the S1 are completed. According to the method, the application of the OCR and the knowledge graph on the brand proxy network is combined for the first time, measures of eliminating seals and enhancing data are taken, entity and entity relationship extraction is performed on brand proxy documents, the brand proxy knowledge graph is established, and the method is used for achieving dynamic management and visual query of brand proxies.

Description

Technical Field

[0001] The present invention relates to the technical fields of artificial intelligence and knowledge graph, and specifically provides a method for constructing a knowledge graph of a brand agency network. Background Art

[0002] In the global business ecosystem, the brand agency network presents a multi-level, time-sensitive, and cross-regional complex structure, and its management relies on the agency relationship information hidden in a large number of documents such as contracts and authorization letters. The traditional methods have the following problems:

[0003] Low data extraction efficiency: Brand agency information usually exists in the form of unstructured text (such as scanned documents) or images, which need to be manually entered, are error-prone and cannot be scaled up.

[0004] Serious seal interference: The agency documents often contain seals covering key texts, such as the name of the agent, the name of the brand, the authorization date, etc., resulting in a significant decline in the OCR recognition accuracy and affecting the accuracy of subsequent information extraction.

[0005] Insufficient semantic understanding: The existing entity relationship extraction models rely on domain-annotated data, and the data in the brand agency field is scarce. General models are difficult to recognize complex relationships such as "authorizing party" and "hierarchical authorization".

[0006] Loose knowledge modeling: There is a lack of a unified ontology model to describe elements such as brands, agents, and authorization relationships, resulting in knowledge redundancy and limited reasoning ability.

[0007] In the prior art, the combination of OCR and knowledge graph is mostly used for simple structured data (invoices, tables), but the processing methods for complex relationships and multi-modal data in the brand agency field are not yet mature, and an end-to-end automated solution is urgently needed. Summary of the Invention

[0008] Aiming at the deficiencies of the prior art, the present invention provides a method for constructing a knowledge graph of a brand agency network, which solves the problems of fragmented processing of multi-modal data and OCR failure caused by seal interference in the prior art.

[0009] To achieve the above object, the present invention is realized through the following technical solutions: A method for constructing a knowledge graph of a brand agency network includes the following steps:

[0010] S1. For the purpose of preprocessing multi-modal data and enhancing OCR, eliminate the interference information in the document image, improve the OCR recognition accuracy, use a depthwise separable convolutional architecture model as the feature extraction backbone network to train in the MaskR-CNN model, locate the seal area in the document image, and perform seal removal and font enhancement on this area;

[0011] S2. OCR text recognition. After processing and enhancement in S1, a text recognition model is used for text extraction.

[0012] S3. Based on the general pre-trained model Paddlenlp-uie-base, in the text recognized in S2, incremental training is carried out using the corpus in the brand agency field to learn the exclusive semantic features in the brand agency field.

[0013] S4. After completing the training of the transfer learning model, entity extraction and entity relationships are carried out.

[0014] S5. Brand agency ontology modeling design, constructing the node type and relationship edge model of the brand agency knowledge graph, covering ontology naming and attribute field design.

[0015] S6. Map the result relationships extracted by the transfer learning model into RDF format data, detect logical conflicts based on the rule engine, and store them in the brand agency knowledge graph using a graph database.

[0016] S7. Use the Cypher query language to develop a visualization interface to dynamically display and manage the brand agency network topology structure.

[0017] Preferably, for the multi-modal data preprocessing and OCR enhancement in S1, combined with the MaskR-CNN instance segmentation model, the depthwise separable convolutional architecture model MobileNetV2 is used to replace the backbone network of MaskR-CNN to train and locate the seal area.

[0018] MaskR-CNN includes: feature extraction, region proposal network (RPN), region of interest alignment (RoIAlign), mask branch, and total loss function.

[0019] (1) The input feature map X passes through depthwise separable convolution to obtain the output feature map, and the residual block calculation for feature extraction is as follows:

[0020] Y = Convpointwise(ReLU(BN(Convdepthwise(X))))

[0021] Convpointwise: Depth convolution, operating independently on each channel.

[0022] Convdepthwise: Point convolution, fusing channel information through 1×1 convolution.

[0023] BN: Normalization.

[0024] ReLU: Activation function.

[0025] (2) Region proposal network (RPN):

[0026] A = {(xi, yi, wk, hk) | k ∈ {32, 64, 128, 256, 512}, aspect ratio ∈ {0.5, 1.0, 2.0}}

[0027] The Region Proposal Network (RPN) extracts candidate bounding box regions containing seals through the set of anchor boxes A, and the anchor boxes can adapt to seals of different sizes;

[0028] RPN loss calculation:

[0029] Lrpn = CrossEntropy(pi, pi*) + λ · SmoothL1(Δi, Δi*)

[0030] pi: Probability of the predicted class, pi*: Label of the predicted class, Δi: Predicted bounding box offset, Δi*: Ground truth bounding box offset, λ: Hyperparameter balancing classification and regression losses.

[0031] (3) Region of Interest (ROI) for candidate recognition:

[0032] RoIAlign(F, R) = BilinearInterpolation(F, R)

[0033] Output a fixed-size feature F ROI ∈R 7×7×C , where F is the feature map and R is the coordinate of the candidate region.

[0034] (4) Predict a binary mask for each candidate region:

[0035]

[0036] yi: Ground truth mask value (0 or 1), pi: Predicted mask value.

[0037] (5) Calculate the total loss and select the optimizer:

[0038] Total loss:

[0039] Ltotal = Lrpn + Lmask

[0040] Lrpn includes the classification cross-entropy loss and the Smooth L1 loss, and Lmask is the binary cross-entropy loss.

[0041] Optimizer: Use the Stochastic Gradient Descent (SGD) optimizer to update the model parameters:

[0042]

[0043] θ: Model parameters, η: Learning rate is 0.001, Gradient of the loss function.

[0044] Preferably, in S2, the processed image is used for OCR recognition, and an OCR text recognition model is used to extract text from the agency document.

[0045] Preferably, in S3, the general pre-trained model Paddlenlp-uie-base model shares the underlying Transformer encoder parameters, and the training goal is to minimize the loss, which includes two parts: entity extraction and relationship extraction:

[0046] Training entity and entity relationship extraction: Using the Span classification method, calculate whether the labeled text segment belongs to a certain entity type, and jointly adopt the cross-entropy loss:

[0047]

[0048] N is the number of candidate Spans or the number of candidate entity relationship pairs, yi is the true entity label, and pi is the probability value predicted by the model. During the training process, predict the position where the entity appears, combine the learned context relationship between entities, predict the relationship between entities, and obtain the extraction result.

[0049] Preferably, in S4, the extracted entities and entity relationships include agency merchants, agency brands, agency time, and agency area entities, and accurately extract the agent name and agent relationship triple from the unstructured text.

[0050] Preferably, in S5, the construction of the relationship edge model includes the construction of relationship edges and the design of edge attribute fields.

[0051] Preferably, in S6, the RDF format data detects logical conflicts based on the rule engine, and uses a graph database to store the knowledge graph.

[0052] Preferably, in S7, a mature front-end framework is combined with a graph database to develop a visualization interface for the brand agency knowledge graph, which can manage brand agency and visualize it in a chronological and multi-dimensional manner.

[0053] The present invention provides a method for constructing a brand agency network knowledge graph. It has the following beneficial effects:

[0054] 1. The present invention combines the applications of OCR and knowledge graph in the brand agency network for the first time. When recognizing text by OCR, due to the influence of the seal, some key words are blocked, reducing the accuracy of extracting and recognizing text. The present invention takes measures to eliminate the seal and enhance the data, and well solves the problem of low recognition rate.

[0055] 2. Based on the learning of transfer learning in the agency field, the present invention completes the entity relationship joint extraction technology, accurately extracts the brand, agent, and agency relationship triples, calculates the similarity in combination with the brand dictionary and the agent merchant dictionary, marks the results with low matching rates as anomalies, and then conducts audits or discards them to enhance the accuracy of the data.

[0056] 3. The present invention detects logical conflicts (multiple exclusive agents exist in the same area) through the RDF rule engine, stores the knowledge graph in combination with the graph database, and uses the Cypher query language and dynamic visualization display to realize the intelligent management and display of the agency network. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 is a flowchart of the present invention;

[0058] Figure 2 is a schematic structural diagram of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0059] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0060] Embodiment 1:

[0061] Please refer to the attached Figure 1 - attached Figure 2 , the embodiment of the present invention provides a method for constructing a brand agency network knowledge graph, including:

[0062] S1. The depthwise separable convolutional architecture model MobileNetV2 is used as the feature extraction backbone network to train the dataset with the marked seal positions. The model automatically recognizes the seal positions, locates the seal positions, extracts the red channel of the located area, processes the red imprints within the seal area, and replaces the fused mask area with white.

[0063] In a possible embodiment, there is a set of images of agency qualification documents, which contain red seals of different sizes. The Labelme image labeling tool is selected to label the position information and label information of these seals. The labeled json file stores the position information of the red seals in the image. The original image data and the labeled data are used as the training input data of the MaskR-CNN model. During the training process, the model first extracts image features through the backbone network MobileNetV2, then uses the Region Proposal Network (RPN) to generate candidate regions, and extracts region features through RoIAlign. By continuously optimizing the bounding box regression loss and cross-entropy loss, using the SGD optimizer, combined with an appropriate learning rate, after multiple rounds of training iterations, the task training for red seal recognition is completed. Finally, the model is used to recognize the agency documents in the test set. By combining the extraction of the red channel and processing the red marks in the seal area, the fused mask area is replaced with white, and finally the elimination of the agency document seal is completed.

[0064] S2. OCR Recognition: After the processing and enhancement in Step 1 are completed, use the OCR text recognition model to extract text from the agency documents.

[0065] S3. Based on the general pre-trained model (Paddlenlp-uie-base), in the text recognized in Step 2, use the corpus in the brand agency field for incremental training to learn the exclusive semantic features in the brand agency field.

[0066] S4. After completing the training of the transfer learning model, extract entities and entity relationships, including agency merchants, agency brands, agency time, and agency area entities, and accurately extract the agent name and agent relationship triples from the unstructured text.

[0067] S5. Ontology Modeling of Brand Agency Knowledge Graph: Building a unified brand agency ontology model through the knowledge graph plays a key role in the management and query of brand agencies; the modeling of the brand agency knowledge graph includes but is not limited to the following: the design of agent name, brand name, agent level, and agent area nodes and the construction of relationship edges.

[0068] S6. Map the extracted triples to the RDF (Resource Description Framework) format according to the ontology model; detect logical conflicts based on the rule engine and store the knowledge graph using a graph database.

[0069] S7. Use a mature front-end framework combined with a graph database to develop a visualization interface for the brand agency knowledge graph, which can manage and visually display brand agencies in a time-series and multi-dimensional manner.

[0070] Embodiment 2:

[0071] S1. The depthwise separable convolutional architecture model MobileNetV2 is used as the feature extraction backbone network to train the dataset with the marked seal positions. The model automatically identifies the seal positions, locates the seal positions, extracts the red channel of the located area, processes the red imprints within the seal area, and replaces the fused mask area with white.

[0072] S2. Use the paddle-ocr model to identify and extract the original proxy documents, and save the text information on the qualification documents locally.

[0073] S3. Synchronously train the pre-trained model paddlenlp-uie-base model for entity relationship extraction in the brand agency field. It is necessary to manually label the authorized merchants, agent merchants, agent brands, agent regions, and agency times. The input of the model needs to include the position indexes of the entities in the text, as well as the corresponding entity labels.

[0074] As an option, perform relationship extraction from the brand agency field. Assume that the qualification document contains the text: "Hereby authorize a certain A Electronic Co., Ltd. to be the exclusive agent of a certain B Technology Co., Ltd. of our company for a certain brand". Mark the entity labels from the qualification text: authorizer, hereby authorizer, authorization relationship. The training data contains the position annotations of the entities in the text, including the start and end position indexes of each entity and relationship.

[0075] Train entity and entity relationship extraction: Use the Span classification method to calculate whether the marked text segment belongs to a certain entity type, and jointly adopt the cross-entropy loss:

[0076]

[0077] N is the number of candidate Spans or the number of candidate entity relationship pairs, yi is the true entity label, and pi is the probability value predicted by the model. During the training process, predict the positions where the entities appear, combine the learned context relationships between the entities, predict the relationships between the entities, and obtain the extraction results. The final output result of the model is: {Authorizer: a certain B Technology Co., Ltd., Hereby authorizer: a certain A Electronic Co., Ltd., Relationship: Authorization}.

[0078] S4. After completing the training of the transfer learning model, extract entities and entity relationships, including agent merchants, agent brands, agency times, and agent region entities, and accurately extract the agent name and agent relationship triple from the unstructured text.

[0079] Example 3:

[0080] S1. Use MobileNetV2 as the feature extraction backbone network, train the dataset with the marked seal positions, complete the model saving for seal position recognition, locate the seal position. Next, only extract the red channel from the located area, process the red imprint within the seal area, and replace the fused mask area with white;

[0081] S2. Use the paddle-ocr model to recognize and extract the agency documents, and save the text information on the agency documents locally.

[0082] S3. Synchronously train the pre-trained model paddlenlp-uie-base model for entity relationship extraction in the brand agency field. It is necessary to manually label the authorized merchants and agent merchants, agent brands, agent regions, and agent times. The input of the model needs to include the position indexes of the entities in the text and the corresponding entity labels.

[0083] S4. After completing the training of the transfer learning model, extract entities and entity relationships, including agent merchants, agent brands, agent times, and agent region entities, and accurately extract the agent name and agent relationship triples from the unstructured text.

[0084] S5. Ontology modeling of the brand agency knowledge graph: Construct a unified brand agency ontology model through the knowledge graph, which plays a key role in the management and query of brand agencies; The modeling of the brand agency knowledge graph includes but is not limited to the following: the design of agent name, brand name, agent level, and agent region nodes and the construction of relationship edges.

[0085] (1) Construction of the ontology: Authorizer, Hereby Authorizer, Authorization Period, Business Scope, Main Business Category.

[0086] (2) Define edge attributes: Authorization edge attribute (Authorizer -> Hereby Authorizer); Agency period edge (Hereby Authorizer -> Authorization Period); Agency region edge (Hereby Authorizer -> Authorization Region); Main business edge (Hereby Authorizer -> Main Business Category). The authorization edge needs to add the brand name and agency level attribute to reflect the hierarchical result of the agency.

[0087] (3) Define ontology attributes: Attributes of the authorizer and the hereby authorizer (merchant name), Authorization period (start time, end time), Agency region (sales region), Main business category (business category).

[0088] S6. Map the extracted triples to the RDF (Resource Description Framework) format according to the ontology model; Detect logical conflicts based on the rule engine, and perform edit distance similarity matching with the existing agent name and brand name dictionaries, which can effectively enhance the final extraction effect and data quality. Finally, store them in the brand agency knowledge graph using a graph database.

[0089] In some embodiments, during the merchant name processing, an edit distance threshold is set. For example, the following entities are extracted: the authorized party (B Technology Co., Ltd.) and the authorized party (A Electronics Co., Ltd.). First, the edit distance matching is performed in the agent merchant dictionary. The merchant with the smallest edit distance is found in the merchant dictionary as 'B Technology Co., Ltd.'. Here, OCR recognizes '技' as '枝'. At this time, the distance score is less than the threshold, and a replacement is performed, and the dictionary content is used to replace the recognized content; if the final calculated distance score is greater than the threshold, it is marked, and then it is decided whether to retain or modify.

[0090] In the process of brand name processing, the brand name edit distance threshold is also set, requiring that the brand name must be completely matched or consistent with the aggregated brand name. This strict matching rule is necessary because the differences between brand names are usually small and easy to cause confusion. If we allow fuzzy matching in data processing, it is easy to cause misclassification, thus affecting data quality.

[0091] S7. Visualize the brand agency network. Assume that a brand agency network knowledge graph has been built, which contains the following entities and relationships: brand, agent, agency product category, agency region and agency period. Query by limiting conditions in the query interface. From the spatial dimension, you can query the brand agency situation in different regions; from the temporal dimension, you can uniformly manage and standardize the agency period of the agent; support the function of comprehensive query from the spatial dimension (agency region), temporal dimension (agency period) and multiple dimensions, covering the dynamic management of agents, agent early warning mechanism and brand agency data visualization.

[0092] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for constructing a knowledge graph of a brand agency network, characterized in that It includes the following steps: S1. For the purpose of multi-modal data preprocessing and OCR enhancement, eliminating interference information in the document image and improving OCR recognition accuracy, a depthwise separable convolution architecture model is used as the feature extraction backbone network and trained in the MaskR-CNN model to locate the seal area in the document image, and then seal removal and font enhancement are performed on this area; S2. OCR text recognition. After processing and enhancement in S1, a text recognition model is used for text extraction; S3. Based on the general pre-trained model Paddlenlp-uie-base, in the text recognized in S2, incremental training is carried out using the corpus in the brand agency field to learn the exclusive semantic features in the brand agency field; S4. After completing the training of the transfer learning model, entity extraction and entity relationships are carried out; S5. Brand agency ontology modeling design, constructing the node type and relationship edge model of the brand agency knowledge graph, covering ontology naming and attribute field design; S6. Mapping the result relationships extracted by the transfer learning model into RDF format data, detecting logical conflicts based on the rule engine, and storing them in the brand agency knowledge graph using a graph database; S7. Using the Cypher query language, developing a visualization interface to dynamically display and manage the brand agency network topology structure.

2. The method for constructing a brand agency network knowledge graph according to claim 1, wherein In the multi-modal data preprocessing and OCR enhancement in S1, combined with the MaskR-CNN instance segmentation model, the depthwise separable convolution architecture model MobileNetV2 is used to replace the backbone network of MaskR-CNN for training to locate the seal area.

3. The method for constructing a brand agency network knowledge graph according to claim 1, wherein In S2, the processed image is used for OCR recognition, and an OCR text recognition model is used to extract text from the agency document.

4. A method for constructing a knowledge graph of a brand agency network according to claim 1, characterized in that In the general pre-trained model Paddlenlp-uie-base in S3, by sharing the underlying Transformer encoder parameters, the training goal is to minimize the loss, which includes two parts: entity extraction and relationship extraction.

5. A method for constructing a brand agency network knowledge graph according to claim 1, characterized in that, The entity extraction and entity relationships in S4 include agent merchants, agent brands, agency time, and agency area entities, and accurately extract the agent name and agency relationship triples from unstructured text.

6. A method for constructing a brand agency network knowledge graph according to claim 1, characterized in that The construction of the relationship edge model in S5 includes the construction of relationship edges and the design of edge attribute fields.

7. A method for constructing a knowledge graph of a brand agency network according to claim 1, characterized in that In S6, the RDF format data detects logical conflicts based on the rule engine and uses a graph database to store the knowledge graph.

8. A method for constructing a knowledge graph of a brand agency network according to claim 1, characterized in that, In S7, using a mature front-end framework combined with a graph database, a visualization interface for the brand agency knowledge graph is developed, which can manage and visually display brand agencies in a chronological and multi-dimensional manner.

Images