Novel network case unstructured data relation analysis method based on large language model
Through the unstructured data relationship analysis method based on large language model, the problem of complex entity relationship extraction in case investigation is solved, and efficient entity and relationship map construction is achieved, supporting investigation, analysis and reasoning in the public security industry.
Patent Information
- Application Number
- CN202411433489.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-15
- Publication Date
- 2025-01-17
AI Technical Summary
Traditional natural language processing models are difficult to effectively extract entity relationships in unstructured documents in case investigations in public safety industry, and are complex in processing and poor in operability.
A new unstructured data relationship analysis method for network cases based on large language models is adopted, including fine-tuning of large language models, relational pattern design, entity extraction, entity relationship mapping, data annotation, custom model construction, relationship extraction and visualization processing, through LLM -NER and LLM-RE models realize the extraction of entities and relationships and the construction of relationship maps.
With limited computing power, the entities and relationships in the text are efficiently mined, the relationship map is constructed, and the investigation and analysis and reasoning of the public safety industry are supported, interpretable and provide effective clue mining and recommendation directions for case investigation.
Smart Images
Figure CN119322854A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data relationship analysis, and in particular to a novel network case unstructured data relationship analysis method based on a large language model. Background Art
[0002] During the investigation of relevant cases in the public security industry, there are a large number of unstructured documents, which contain many clues.
[0003] Traditional natural language processing models can only extract names, identification numbers, and other named entities. However, there are many ways to extract the relationships between these entities, and the processing is complex. This requires a deep understanding of natural language processing models and machine learning, and has poor operability. Summary of the invention
[0004] The technical problem to be solved by the present invention is to provide a new type of unstructured data relationship analysis method for network cases based on a large language model, which can not only realize the naming of related objects in unstructured documents and the extraction of various relationships, and form a related object relationship map, but also automatically realize the autonomous mining of clue information in various investigation processes based on related objects to find out the relationship between various entities.
[0005] In order to solve the above technical problems, the technical solutions adopted by the present invention are as follows.
[0006] A novel network case unstructured data relationship analysis method based on a large language model includes the following steps:
[0007] S1. Fine-tune the large language model and design the relational model;
[0008] S2. Unstructured data entity extraction;
[0009] S3. Entity-relationship mapping;
[0010] S4. Data annotation and custom model construction;
[0011] S5. Perform relationship extraction based on predefined entity relationship pairs;
[0012] S6. Autonomously mine entities and relationships based on survey results;
[0013] S7. Visualization processing.
[0014] Preferably, the method for fine-tuning the large language model in step S1 is: selecting a BERT model as a base model for the large language model, and defining an LLM-NER model and an LLM-RE model; the training method of the BERT model is: first, marking the sentence, adding a [CLS] tag at the beginning of the sentence, and adding a [SEP] tag at the end of the sentence; then sending these tags to the pre-trained BERT model to obtain the features of each tag; next, sending these tag features to a classifier; finally, the classifier returns the category corresponding to each named entity.
[0015] Preferably, the specific method for performing relational model design in step S1 is: performing model design according to different types of unstructured data, constructing a relational graph of each entity, the relational graph including each entity node extracted from the text, the relationship between each entity node, and the attributes corresponding to each entity node, the extracted literal values will be used as the attribute values of these attributes, and will represent the unique identifier of the corresponding attribute.
[0016] Preferably, the specific method of step S2 is: select and call the LLM-NER model to extract entities starting from the most basic names and place names that match the requirements.
[0017] Preferably, the specific method of step S3 is: first, map the extracted entities with the pre-designed pattern in step S1; then, construct the extracted entities into entity pairs to implement an entity pair constructor; the constructed entity pairs are one-to-one matched by Cartesian products, and a pair of entity types that do not need to be considered are screened out in advance through restriction conditions to reduce the complexity of subsequent relationship reasoning and improve construction speed.
[0018] Preferably, the specific method of step S4 is: manually identifying and recording entities and relationships appearing in the new text; manually annotating and recording entities and relationships in the sample text using the defined entity and relationship categories; after the annotation is completed, performing small sample training on the LLM-NER model and the LLM-RE model, and extracting new entity and relationship information after the LLM-NER model and the LLM-RE model are trained.
[0019] Preferably, the specific method of step S5 is: selectively calling the LLM-RE model to complete the extraction based on the predefined entity relationship, and obtain a triple json file for the specific text.
[0020] Preferably, the specific method of step S6 is: on the basis of autonomously mining entities and relationships of the survey results of relevant objects in different levels of relationship graphs through the above steps, various types of access or imported resources and services are investigated again for the mined related objects, and at the same time, the survey results are used through a large language model to mine the initial investigation result data again to obtain relevant various aspects of relationship information until the mining can no longer be continued.
[0021] Preferably, the specific method of step S7 is: writing the entity relationship json file obtained in step S5 into the local gStore triple database, and using the workbench tool to connect and visualize it.
[0022] Due to the adoption of the above technical scheme, the technical progress achieved by the present invention is as follows.
[0023] Under the condition of limited computing power, the present invention mines various related elements and element relationships from large-scale texts and semi-structured texts, and makes a useful attempt to construct a relationship map in the process of investigation and judgment of new network cases in the public security industry. It lays a foundation for future applications such as reasoning, question and answering based on different types of new network case investigation suggestions and recommended investigation directions, and meets the public security industry's requirement for strong explainability of reasoning. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 is a flow chart of the present invention;
[0025] Figure 2 This is a working principle diagram of the large language model of the present invention;
[0026] Figure 3 It is a training flow chart of the BERT model of the present invention;
[0027] Figure 4 This is a relationship model diagram of the present invention. DETAILED DESCRIPTION
[0028] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0029] A new network case unstructured data relationship analysis method based on a large language model, combined with Figure 1 As shown, the following steps are included:
[0030] S1. Fine-tune large language models and design relational schemas.
[0031] The method for fine-tuning a large language model is:
[0032] The base model selected by the present invention is the BERT model, and the LLM-NER model and the LLM-RE model are defined. Among them, the LLM-NER model is a Python library for implementing zero-sample and small-sample NER (named entity recognition) tasks; by providing an easy-to-use interface, the LLM-NER model can compose prompt information, query models, and parse the results returned by LLM. In addition, the library enables users to test multiple variables by providing a simple interface, thereby performing prompt engineering more efficiently, and can more accurately identify names, place names, organization names, service identification numbers, etc.
[0033] like Figure 2 As shown, user-provided named entity definitions and few-shot examples are compiled into a text prompt using a prompt method; then, the task completed by the large language model (LLM) is parsed by the answer parser, returning an object containing the named entity mention.
[0034] In this method, the first task is to fine-tune the large language model for the public safety industry to improve the model's ability to process texts in the public safety field, to fully understand contextual information, and to more accurately identify entities such as scene means, locations, and other professional data.
[0035] In the named entity recognition task, our goal is to classify named entities into preset categories. For example, in the sentence "Zhang San lives in Beijing.", Zhang San should be classified as a person, and Beijing should be classified as a place. Figure 3 As shown in the figure, first, the sentence is marked, and the [CLS] tag is added at the beginning of the sentence and the [SEP] tag is added at the end of the sentence; then these tags are sent to the pre-trained BERT model to obtain the features of each tag; next, these tag features are sent to a classifier; finally, the classifier returns the category corresponding to each named entity.
[0036] The specific method for relational model design is:
[0037] Design patterns based on different types of unstructured data, such as building a relationship map of various entities from brief case details and transcripts, including where the entities in the case have been, the ID numbers they have, the license plates they use, what actions they have taken, etc.
[0038] In general, the relationship model is Figure 4 As shown, Person 1, Person 2, Phone Number, Location, and Behavior are all entity nodes, and there are Figure 4The relationship shown in the figure; name, latitude and longitude, and behavior content are the attributes corresponding to the above entities. The literal values extracted from the text will be used as the attribute values of these attributes and will represent the unique identifier of the corresponding attribute.
[0039] S2. Entity extraction from unstructured data.
[0040] In order to achieve the preset business goals, we need to do the following: extract characters, regional entities, service identifiers, behaviors in the text, and explore the relationships between the above entities. To this end, we need to build a pipeline that can do these functions and has good effects. First, we need to extract entities. The present invention starts with the most basic names and place names that match the requirements. At this time, we choose to call the "LLM-NER model to realize entity extraction.
[0041] S3. Entity-relationship mapping.
[0042] First, the extracted entities are mapped to the pre-designed pattern in step S1; then, the extracted entities are constructed into entity pairs (the relationship must occur between two entities). An entity pair constructor is implemented here, and the constructed entity pairs are one-to-one matching of Cartesian products. Many pairs composed of entity types are not the relationships we need to consider, so they need to be screened out in advance through restrictions to reduce the complexity of subsequent relationship reasoning and increase the construction speed.
[0043] S4. Data annotation and custom model construction.
[0044] Since the pre-defined patterns cannot handle all texts and extract the relationships among them, it is necessary to annotate data and train models to meet complete business needs. The process is as follows: manually identify and record the entities and relationships that appear in the new text; use the defined entity and relationship categories to manually annotate and record the entities and relationships in the sample text; after the annotation is completed, the LLM-NER model and LLM-RE model are trained with a small sample. After the LLM-NER model and LLM-RE model are trained, new entity and relationship information can be extracted.
[0045] S5. Perform relationship extraction based on predefined entity-relationship pairs.
[0046] After the relationship pairs are labeled, the relationship extraction model can be used to extract the predefined entity relationships. Here, the LLM-RE model is selected. After the information extraction is completed, a triple json file for a specific text is obtained.
[0047] S6. Autonomously mine entities and relationships based on survey results.
[0048] Based on the autonomous mining of entities and relationships of the survey results of relevant objects in relationship graphs of different levels achieved through the above steps, various access or imported resources and services are investigated again for the mined relevant objects. At the same time, the survey results are used through a large language model to mine the initial investigation results data again to obtain relevant person-person relationships, person-number relationships, number-number relationships and other information until mining is no longer possible.
[0049] S7. Visualization processing.
[0050] Use the workbench tool to perform visual analysis on the local gStore triple database. Specifically, write the entity relationship json file obtained in step S5 into the local gStore triple database, and use the workbench tool to connect and visualize it. In the formal application, the visual display component should be fully developed using the B / S architecture so that it can be used in the production project.
[0051] When in use, the present invention utilizes the great advantages of the large language model in context analysis to extract entities and relationships in unstructured text data, and independently conducts investigations on various types of accessed or imported resources and services based on related objects. At the same time, the investigation results data of the disordered data resources obtained through the investigation are mined through the large language model to obtain relevant information such as person-person relationships, person-number relationships, and number-number relationships, thereby solving key problems for the next step of building a complete new type of public security industry network case investigation and analysis guidance and relationship knowledge graph.
Claims
1. A novel network case unstructured data relationship analysis method based on a large language model, characterized by: The following steps are involved: S1. Fine-tune the large language model and design the relational model; S2. Unstructured data entity extraction; S3. Entity-relationship mapping; S4. Data annotation and custom model construction; S5. Perform relationship extraction based on predefined entity relationship pairs; S6. Autonomously mine entities and relationships based on survey results; S7. Visualization processing.
2. According to claim 1, a novel network case unstructured data relationship analysis method based on a large language model is characterized by: The method for fine-tuning the large language model in step S1 is: selecting a BERT model as a base model for the large language model, and defining an LLM-NER model and an LLM-RE model; the training method of the BERT model is: first, marking the sentence, adding a [CLS] tag at the beginning of the sentence, and adding a [SEP] tag at the end of the sentence; then sending these tags to the pre-trained BERT model to obtain the features of each tag; next, sending these tag features to a classifier; finally, the classifier returns the category corresponding to each named entity.
3. According to claim 2, a novel network case unstructured data relationship analysis method based on a large language model is characterized by: The specific method for performing relational model design in step S1 is: perform model design according to different types of unstructured data, and construct a relational graph of each entity. The relational graph includes each entity node extracted from the text, the relationship between each entity node, and the attributes corresponding to each entity node. The extracted literal values will be used as the attribute values of these attributes and will represent the unique identifier of the corresponding attribute.
4. According to claim 3, a novel network case unstructured data relationship analysis method based on a large language model is characterized by: The specific method of step S2 is: select and call the LLM-NER model to extract entities starting from the most basic names and place names that match the requirements.
5. According to the novel network case unstructured data relationship analysis method based on large language model according to claim 4, it is characterized by: The specific method of step S3 is: first, map the extracted entities with the pre-designed pattern in step S1; then, construct the extracted entities into entity pairs to implement an entity pair constructor; the constructed entity pairs are one-to-one matched by Cartesian products, and a pair of entity types that do not need to be considered are screened out in advance through restriction conditions to reduce the complexity of subsequent relationship reasoning and improve the construction speed.
6. According to the novel network case unstructured data relationship analysis method based on large language model according to claim 5, it is characterized by: The specific method of step S4 is: manually identifying and recording entities and relationships appearing in the new text; manually annotating and recording entities and relationships in the sample text using the defined entity and relationship categories; after the annotation is completed, performing small sample training on the LLM-NER model and the LLM-RE model, and extracting new entity and relationship information after the LLM-NER model and the LLM-RE model are trained.
7. The novel network case unstructured data relationship analysis method based on a large language model according to claim 6 is characterized by: The specific method of step S5 is: select and call the LLM-RE model to complete the extraction based on the predefined entity relationship, and obtain a triple json file for the specific text.
8. The novel network case unstructured data relationship analysis method based on a large language model according to claim 7 is characterized by: The specific method of step S6 is: on the basis of autonomously mining entities and relationships of the survey results of relevant objects in different levels of relationship graphs through the above steps, various types of access or imported resources and services are investigated again for the mined related objects, and at the same time, the survey results are used through a large language model to mine the initial investigation result data again to obtain relevant various aspects of relationship information until the mining can no longer be continued.
9. The novel network case unstructured data relationship analysis method based on a large language model according to claim 8 is characterized by: The specific method of step S7 is: writing the entity relationship json file obtained in step S5 into the local gStore triple database, and using the workbench tool to connect and visualize it.