A structured data processing method for intelligent supervision black box

Abstract

The application discloses a structured data processing method for a smart supervision black box. In order to overcome the problems of high limitation of entity extraction in a rule mode and single data collection mode in the prior art, the application comprises the following steps: node data collection, data cleaning, text feature vectorization, key entity identification, entity linking, knowledge alignment and filtering, a knowledge graph and an implementation training process. Through one-stop data collection, key entity identification, entity linking and data storage technologies, the application solves the structured storage of multiple data sources, multiple data types and multiple data forms.

Description

Technical Field

[0001] This invention relates to the field of medical data processing, and more particularly to a structured data processing method for a smart monitoring black box. Background Technology

[0002] Knowledge graphs, which establish representational and inferential relationships between text and objects, and objects and attributes, are a crucial task in natural language processing and have found wide applications in areas such as medical modeling and intelligent customer service. Knowledge graphs place high demands on data collection, model definition, and storage methods. They require not only a large amount of text relevant to a given business domain but also a wealth of readily available key information and corresponding attribute values. Furthermore, their model building and storage methods present significant hardware resource challenges.

[0003] In recent years, with the widespread use of machine learning, diverse models have emerged in the knowledge graph technology stack, primarily focusing on Chinese modeling. These models utilize BERT clusters to build vector representations of text, fuse word information for named entity recognition modeling, employ remote supervision and attention mechanisms in classification models for entity linking and relation extraction, and use a dual-radial mechanism to determine source positions and arc labels in syntactic analysis. Finally, graph databases, such as Neo4j, are used for data storage and retrieval.

[0004] For example, a "Method and Apparatus for Fusing Medical Knowledge Graphs Based on Multiple Data Sources" disclosed in Chinese patent literature, publication number CN110866124A, includes the following method: performing knowledge representation learning based on a first medical knowledge graph and a second medical knowledge graph respectively to obtain each first initial vector and each second initial vector; mapping each first initial vector and each second initial vector to a reference vector space based on a pre-acquired reference vector set to obtain each first mapping vector and each second mapping vector; and fusing the knowledge in the first medical knowledge graph and the second medical knowledge graph according to each first mapping vector and each second mapping vector to obtain the fused medical knowledge graph.

[0005] The shortcomings of the above solution are as follows:

[0006] 1) Rule-based entity extraction has high limitations.

[0007] Existing technologies for entity and keyword extraction have significant limitations. They can only identify relatively general text fragments and entities, such as names and organizations, and are not applicable to most industries. Therefore, these tasks often rely on manually defined rules or keyword matching for entity extraction. This approach is labor-intensive, has poor transferability, and lacks the ability to continuously learn to adapt to the discovery of new entities.

[0008] 2) Data collection methods are limited.

[0009] For sustainable and integrated knowledge graphs, they should be adaptable to multi-source and multi-type data inputs. Existing projects generally require separate functional development and embedding based on the data source. In addition, existing modeling methods are difficult to support streaming text input and knowledge base retrieval or integration.

[0010] 3) The underlying modeling lacks statistical knowledge mining capabilities.

[0011] Existing models lack continuous knowledge discovery and alignment with data inputs from different sources. This means that different original fields may represent the same quantitative feature, which would increase model confusion if used directly for training. Furthermore, entities, relationships, and other knowledge not present in existing models or graphs should be identified using some method.

[0012] 4) Lacks topic cluster segmentation

[0013] Existing technologies typically insert or infer information into a full graph database after mining entities and relationships. However, the same text fragment may have different meanings in different contexts. Therefore, such inference can easily introduce noise and cause confusion in the graph. Summary of the Invention

[0014] This invention primarily addresses the limitations of existing technologies that rely on rule-based entity extraction and the simplistic data collection methods; it provides a structured data processing method for intelligent regulatory black boxes.

[0015] The above-mentioned technical problems of the present invention are mainly solved by the following technical solutions:

[0016] A structured data processing method for intelligent monitoring black boxes includes the following steps:

[0017] S1: Node data acquisition; the open standard interface takes raw text as input data and automatically assigns a unique label to each node;

[0018] S2: Data cleaning; Remove irrelevant characters with structural rules by using preset cleaning rules;

[0019] S3: Vectorize the input text into sentences based on character, word, sentence, tag features, and statistical features of each unit;

[0020] S4: Key Entity Recognition; The text data collected from each node is identified by a pre-trained entity extraction model, and all key entities and their business attribute fields contained in the text data are output.

[0021] S5: Entity linking, extracting the relationships between entities identified in step S4 through semantic recognition;

[0022] S6: Collect all relationship results generated from entity linking in step S5, and perform knowledge alignment and knowledge filtering;

[0023] S7: Knowledge Graph; After obtaining entities and their corresponding relationships, all newly identified relationships of all data nodes are stored in the graph database, and an interface is opened for retrieval and querying;

[0024] S8: Implement training; periodically collect the results after knowledge filtering, and retrain and optimize the neural network for key entity recognition and entity linking.

[0025] This solution utilizes a rich stack of natural language processing technologies and designs a variety of feature extraction and entity linking algorithms for intelligent structured data storage. It can dynamically acquire key entities from node data and identify behavioral relationships between entities, thereby achieving structured data aggregation.

[0026] As a preferred method, specified data is passed in via key-value JSON, and the key name is stored in the graph database as a node along with the parsed data. This solves the problem of multiple data types.

[0027] Preferably, in step S3, the input text is first segmented using a word segmentation algorithm, and then each unit is mapped using a pre-trained vector library. Statistical features are represented using one-hot encoding, embedding, or absolute numbers. These features include historical frequency, stemming, index position within the sentence, whether the unit appears in the past after concatenation, word attributes, and the unit's position within the current word. A "0" indicates the beginning of a word, a "1" indicates the middle of a word, and a "2" indicates the end of a word. Through feature quantization, a high-dimensional vector for each segmented unit is obtained, and each feature of each unit is concatenated to form the final vector representation.

[0028] Preferably, the entity extraction model uses a neural network for modeling, fusing four features: character features, word features, index position, and the position of the unit in the current word. It uses a one-layer bidirectional long short memory model and a self-attention mechanism, and employs a full-path conditional random field for result decoding to output all potential key entities and their corresponding business attributes in the sentence.

[0029] Text modeling typically uses characters or words as units. By constructing features and then quantifying them, textual information can be expressed mathematically. However, entity recognition in business applications using manually defined rules has significant limitations. It can only recognize entities within specific, predefined rules, failing to identify newly emerging entities, and requires substantial manpower to maintain these rule dictionaries. Therefore, this solution employs an entity extraction model for automatic extraction. This model is pre-trained and optimized through the system's real-time learning module.

[0030] Preferably, the semantic recognition includes preset relation links, existing graph retrieval, remote supervised relation connections, and existing syntactic trees;

[0031] Preset relationship links:

[0032] The relationships between entities are identified by classifying the extracted entities into a predefined set of relationships. A neural network is used to extract word and phrase vectors, which are then incorporated into the index position and label features of each pair of entities. A convolutional neural network and an affine attention mechanism layer are used to output the relationship between each pair of entities.

[0033] Existing spectral search:

[0034] For the extracted entity pairs, if the corresponding relationship can be found directly in the existing graph database, then output it directly; otherwise, use other methods.

[0035] Remote supervision relationship connection:

[0036] New relationships between each pair of entities are discovered using a general Chinese knowledge graph. If a link can be found between the pair of entities in the graph, the relationship is output and stored in the final database.

[0037] Existing syntax tree:

[0038] By identifying the grammatical attributes of each word in the text, the semantic relationships between various linguistic units in the sentence are analyzed, and these semantic relationships are presented as a dependency structure.

[0039] By identifying key entity information and potential relationships in the data source, unified data storage and retrieval are performed, resulting in intelligent extraction and rich extraction results.

[0040] Preferably, step S6 includes the following steps:

[0041] S601: Knowledge Alignment;

[0042] The similarity method is used to integrate the relationships generated by the three sub-modules of preset relation links, remote supervised relation links, and dependency syntax trees, and to produce a unified relation representation.

[0043] S602: Knowledge Filtering;

[0044] The extracted relationships are statistically analyzed, and low-frequency relationships are excluded using statistical indicators and confidence tests.

[0045] Stabilize the model's recognition performance to optimize the system's self-learning.

[0046] As a preferred option, topic modeling is also included; after unsupervised topic mining of the collected node text data, topic labeling is performed on each input data, and relevant operations are performed based on the corresponding topic when accessing the database.

[0047] Data collected from different source nodes generally has potential themes. For example, node data from company daily reports and drug production reports have different prior information. Therefore, this information is also needed when extracting relationships or making retrieval predictions through knowledge graphs.

[0048] Preferably, the training input data for key entity recognition includes text, entity location, and corresponding label; the training input data for entity linking includes entity location, corresponding label, and relationship label.

[0049] The system will periodically collect the results after knowledge filtering and retrain and optimize the neural network for key entity recognition and entity linking. This is because the data after knowledge filtering is clean and labeled, and can be directly used in the self-learning process.

[0050] The beneficial effects of this invention are:

[0051] 1. By employing technologies such as one-stop data aggregation, key entity identification, entity linking, and data storage, the structured storage of multiple data sources, data types, and data formats has been solved.

[0052] 2. By identifying key entity information and potential relationships in the data source, unified data storage and retrieval are performed, resulting in intelligent extraction and rich extraction results.

[0053] 3. Based on multi-dimensional relationship extraction, knowledge mining, and relationship filtering, the extraction model is continuously learned to enhance the relationship recognition capability. Attached Figure Description

[0054] Figure 1 This is a flowchart of the structured data processing method of the present invention.

[0055] Figure 2 This is a schematic diagram of the output result of the dependency syntax tree of the present invention. Detailed Implementation

[0056] The technical solution of the present invention will be further described in detail below through embodiments and in conjunction with the accompanying drawings.

[0057] Example:

[0058] This embodiment presents a structured data processing method for a smart monitoring black box, such as... Figure 1 As shown, it includes the following steps:

[0059] S1: Node data acquisition; the open standard interface takes raw text as input data and automatically assigns a unique label to each node.

[0060] To address the issue of multiple data types, specified data is passed in via key-value JSON, and the key name will be stored in the graph database as a node along with the parsed data.

[0061] By assigning a unique label to each node and using that node as the parent node in subsequent processing to create and expand child nodes, the attributes of each node can be traced. This allows for matching and outputting the original data and parsed data of the node during retrieval.

[0062] For example, "raw_text" is a required field used to accept raw text, while "attributes" are additional attributes that can be freely passed in without being limited by length or key name, thus enabling data to be transmitted as long as it is available.

[0063] "raw_text":

[0064] Article 201 of the Contract Law states that if a lender fails to provide a loan on the agreed date or in the agreed amount, causing losses to the borrower, the lender shall compensate for the losses.

[0065] "attributes":

[0066] "pic":" / path / to / pic",

[0067] "desc": "Image of the Contract Law"

[0068] "timestamp":"1625456251",

[0069] "type":"Device 1".

[0070] S2: Data cleaning; Remove irrelevant characters with structural rules by using preset cleaning rules.

[0071] It offers a variety of preset cleaning rules, as well as custom cleaning rules. The cleaning object is the text data of nodes. The purpose of cleaning is to remove irrelevant characters with structural rules, so as to output more regular data for display, and at the same time improve the accuracy and effect of subsequent models.

[0072] S3: Vectorize the input text in terms of word, phrase, sentence, tag features, and statistical feature dimensions of each unit.

[0073] To better adapt to multi-source, multi-type, and multi-modal data, the text is first vectorized. This module calculates all specified features of the input text, facilitating subsequent model selection.

[0074] The input text is first segmented through word and character segmentation algorithms and then undergoes vector mapping of each unit through a pre-trained vector library. For statistical features, the system represents them in the form of one-hot, Embedding, or absolute numbers.

[0075] Statistical features include historical occurrence frequency, stem, index position of the sentence where it is located, whether it has appeared historically after concatenating the previous and next units, word and character attributes, and the position of the unit in the current word. The beginning of a word is represented by "0", the middle of a word by "1", and the end of a word by "2". As shown in Table 1, summarize the quantization methods of each feature.

[0076] Table 1. Quantization methods of each feature

[0077]

[0078] Through this step, the text will obtain a rich high-dimensional representation. For example, for the text "How is the weather in Hangzhou today", after word and sentence segmentation, the segmented text can be obtained, "今 / 天 / 杭 / 州 / 的 / 天 / 气 / 如 / 何 / 今天 / 杭州 / 的 / 天气 / 如何". Through feature quantization, high-dimensional vectors of each segmented unit are obtained, and each feature of each unit forms the final vector representation in a concatenated manner.

[0079] For the first segmented unit "今", by selecting character features, index position, whether it has appeared historically after concatenating the previous and next units, the position of the unit in the current word, and word and character attributes, a vector representation of 100 + 1 + 1 + 1 + 21 dimensions can be obtained.

[0080] Thus, through the above steps, encoding the training data, we can obtain the primary vectorized representation of each sentence.

[0081] S4: Key entity recognition; Identify the text data collected at each node through a pre-trained entity extraction model, and output all key entities and their business attribute fields contained in this piece of text data.

[0082] After data cleaning, the system will collect text data at each collection point, which can be used for training and prediction of downstream models. The input of key entity recognition is a piece of text data, and the output is all key business attribute fields contained in this piece of text.

[0083] Business attributes are predefined by the system and then handed over to the model for recognition. In addition, a hot word function is enabled to quickly add or delete undefined categories to achieve the desired recognition effect.

[0084] Text modeling is generally done on a character or word basis. By constructing features and then quantifying them, text information can be expressed in a mathematical way.

[0085] Entity identification in business operations using manually defined rules has significant limitations. It can only identify entities within specific, predefined rules, failing to recognize newly emerging entities, and requires substantial manpower to maintain these rule dictionaries. This embodiment employs an entity extraction model for automatic extraction. This model is pre-trained and optimized through the system's real-time learning module.

[0086] In this embodiment, business attribute recognition is used to identify whether key entities such as pharmaceutical company names, drug names, and time appear in the text.

[0087] The entity extraction model uses a neural network for modeling, fusing four features: character features, word features, index position, and the unit's position within the current word. The fusion method involves concatenating the vectors of these four features, then passing them through a feedforward network layer. This indirectly assigns weights to each feature, outputting a fixed-length feature vector. For example, if the dimensions of the four features are 100, 300, 1, and 1 respectively, the concatenated vector has a dimension of 402, and the feedforward network output has a dimension of 256. Finally, the fused vector is passed through a bidirectional long short-term memory model, and a full-path conditional random field is used for decoding, outputting all potential key entities and their corresponding business attributes in the sentence.

[0088] For example, given the sentence "Recently, the National Medical Products Administration conducted a surprise inspection of a company in Guangzhou. The production record for medical infrared thermometers with batch number 20200620 was found to be unsigned, which does not comply with the requirement that each batch (unit) of product must have a production record," the model will output the following results.

[0089] {"value":"A company in Guangzhou",

[0090] "entity":"ORG",

[0091] "start":"15",

[0092] "end":"26"};

[0093] {"value":"20200620",

[0094] "entity":"CNUM"

[0095] "start":"38",

[0096] "end":"46"};

[0097] {"value":"Medical infrared thermometer",

[0098] "entity":"CPROD"

[0099] "start":"47",

[0100] "end":"54"};

[0101] {"value":"《Specifications》",

[0102] "entity":"LAW",

[0103] "start":"65",

[0104] "end":"69"}.

[0105] S5: Entity linking, extracting the relationships between entities identified in step S4 through semantic recognition.

[0106] By extracting business attributes, key information of each node in the text can be obtained, while entity links use semantic understanding to extract behavioral relationships between entities, such as which company produced which drug at what time.

[0107] Entity linking is a classification task that identifies relationships by categorizing each extracted pair of entities into a predefined set of relationships. However, given the large number of relationships in a typical domain that are difficult to collect and define all at once, the system employs methods such as existing graph retrieval, remote supervision, and dependency syntax trees to discover potential new relationships, in addition to predefined relationships.

[0108] Preset relationship links:

[0109] The relationships between entities are identified by categorizing the extracted entities into a predefined set of relationships.

[0110] With the help of professional opinions in the field, we can support the extraction of relationship clusters such as enterprise-time, enterprise-drug, drug-ingredient, and literature-event. Each cluster contains multiple sub-relationships. For example, enterprise-time includes enterprise establishment time, enterprise production time, and enterprise behavior time.

[0111] A neural network is used to extract character and word vectors, which are then incorporated into the index position and label features of each entity pair. The method of incorporating features is the same as that of entity recognition in S4, that is, the vectors are concatenated and then input into a feedforward network layer, and a fixed-length fused feature vector is output. Finally, a convolutional neural network and an affine attention mechanism layer are used to fit the feature vectors to the classification model parameters and output the relationship between each entity pair.

[0112] Existing spectral search:

[0113] For the extracted entity pairs, if the corresponding relationship can be found directly in the existing graph database, it is output directly; otherwise, other methods are used.

[0114] Remote supervision relationship connection:

[0115] This module attempts to discover new relationships between each pair of entities. A general Chinese knowledge graph is used to discover new relationships between each pair of entities. If a link can be found between the pair of entities in the graph, the relationship is output and stored in the final database.

[0116] Existing syntax tree:

[0117] Dependency trees identify the grammatical attributes of each word in a text, such as subject and predicate, to form linguistic units. They then analyze the semantic relationships between these units and present them as dependency structures. This module represents the system's ability to extract new relations directly from syntactic construction, enriching the linking relationships and capabilities of entities. Dependency trees are primarily used to analyze the agent, patient, time, and location relationships between the subject and object in a text.

[0118] like Figure 2 As shown, the output of the dependency syntax tree can be identified from the result. The subject is "I", the object is "corn soup" and the predicate is "drink". Therefore, the subject-verb-object relationship can be established. In addition, rich secondary relationships such as "I-restaurant-drink soup" and "I-spoon-drink soup" are also extracted.

[0119] S6: Collect all relation results generated from entity linking in step S5, perform knowledge alignment and knowledge filtering to optimize the system's self-learning.

[0120] S601: Knowledge Alignment;

[0121] Similarity is used to merge the relationships generated by the three sub-modules of preset relation links, remote supervised relation links, and dependency syntax trees to produce a unified relation representation. For example, "producing medicine" and "producing medicine" are the same representation, and will be aligned to "producing medicine". The similarity model is performed using a logistic regression model. The relationships extracted from the preset relation links and remote supervised relation links sub-modules are merged, and the word-level overlap and corresponding overlap rate of the input text and entities, and input text and relations are calculated separately. The overlap amount is incremented by 1 if the latter appears in the former, and the overlap rate is calculated by dividing the overlap amount by the union of the two. This yields 8 numerical features. One numerical feature is obtained by calculating the number of relations generated by the dependency syntax tree. The logistic regression model is used to model and predict whether these 9 features represent the same thing. For example, if the input sentence is "Company A conducted its first batch of...", the similarity model is used to model and predict whether they represent the same thing. The phrase "production trials of related drugs" is segmented into "Company A / at / the beginning of this year / conducted / the / first / production trials of / related / drugs". The preset relation link output relation is "production of drugs", and the remote supervision relation link output relation is "produced drugs". The entities are "company" and "drugs". The character-level features of the input text and entities are [4, 0.174], and the word-level features are [1, 0.091]. The merged relation and word segmentation are ["production", "drugs", "production", "out", "drugs"]. The character-level features of the input text and relations are [5, 0.2], and the word-level features are [3, 0.25]. The number of relations output by the dependency syntax tree is 1. The final obtained features are [4, 0.174, 1, 0.091, 5, 0.2, 3, 0.25, 1]. Logistic regression predicts whether the output is the same feature based on these 9 features.

[0122] S602: Knowledge Filtering;

[0123] The extracted relationships are statistically analyzed, and low-frequency relationships are excluded using statistical indicators and confidence tests. To stabilize the model's recognition performance, low-frequency relationships need to be filtered out. The statistical indicator chosen is the chi-square test of frequency intervals. After grouping the extracted relationships into intervals, the sum of the frequencies of the relationships in each group is statistically analyzed, and a chi-square test is used to test the frequency confidence level under each interval. Given a confidence level of p = 0.05, if the confidence level of a low-frequency interval is higher than the confidence level, the hypothesis that the low-frequency interval is significant is rejected, and all relationships within that interval are excluded.

[0124] Main body modeling;

[0125] Data collected from different source nodes generally has potential themes. For example, node data from company daily reports and drug production reports have different prior information. Therefore, this information is also needed when extracting relationships or making retrieval predictions through knowledge graphs.

[0126] After performing unsupervised topic mining on the collected node text data, each input data is labeled with a topic, and relevant operations are performed based on the corresponding topic when accessing the database.

[0127] S7: Knowledge Graph; After obtaining entities and their corresponding relationships, all newly identified relationships of all data nodes are stored in the graph database, and an interface is opened for retrieval and querying.

[0128] S8: Implement training; periodically collect the results after knowledge filtering, and retrain and optimize the neural network for key entity recognition and entity linking.

[0129] The data filtered through knowledge is clean and labeled, and can be directly used in the self-learning process.

[0130] The training input data for key entity recognition includes text, entity location, and corresponding label. The training input data for entity linking includes entity location, corresponding label, and relationship label.

[0131] This solution provides one-stop data aggregation, key entity identification, entity linking, and data storage technologies, solving the structured storage problem for multiple data sources, data types, and data formats. It achieves unified data storage and retrieval by identifying key entity information and potential relationships from data sources, with intelligent extraction processes and rich results. Based on multi-dimensional relationship extraction, knowledge mining, and relationship filtering, it continuously learns the extraction model to enhance relationship identification capabilities.

[0132] Traditional data aggregation methods either store raw data or employ costly manual or rule-based labeling for simple data relationship extraction, resulting in chaotic storage, difficult retrieval, and limited portability and usability. This embodiment's data aggregation system utilizes a rich stack of natural language processing technologies, employing sophisticated feature extraction and entity linking algorithms for intelligent structured data storage. It dynamically acquires key entities from node data and identifies behavioral relationships between entities, thereby achieving structured data aggregation.

[0133] Traditional key entity recognition and relation extraction require a large amount of data for model training, which is costly. The solution in this embodiment, through topic modeling, pre-trained models, rich text features, remote supervision, and continuous learning capabilities, supports system cold start and exhibits good relation extraction performance. The system uses multi-dimensional capabilities to identify existing entity links and discover new relations, thereby supporting the storage of various relations.

[0134] Conventional entity linking models struggle to eliminate irrelevant relationships, resulting in cluttered extraction results. This embodiment addresses this by using topic modeling, similarity models, and statistical confidence methods to align and filter extracted relationships, thereby contributing to the model's self-learning optimization and a cleaner knowledge graph.

[0135] Traditional knowledge graph construction requires insertion, updating, and reasoning across the entire database. The solution in this embodiment strengthens entity link relationships and background information relationships of data collection nodes through topic modeling, thereby enabling diverse representations within the graph.

[0136] It should be understood that the embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

Claims

1. A structured data processing method for smart supervising black box, characterized in that, Comprise the following steps: S1: node data collection; open standard interface to raw text as incoming data, and automatically assign a unique label to the node; S2: data cleaning; by default cleaning rules, remove irrelevant characters with structural rules; S3: input text is vectorized in words, sentences, label features and statistical features of each unit; S4: key entity recognition; the text data collected by each node is identified by the pre-trained entity extraction model, and all key entities and their business attribute fields contained in the text data are output; S5: entity linking, the entity identified in step S4 is linked by semantic recognition to extract the relationship between entities; S6: collect all relationship results from step S5 entity linking, knowledge alignment and knowledge filtering; after grouping the extracted relationships, the frequency of the group is counted, and the confidence of the frequency in the interval is tested using chi-square test, given the confidence level p=0.05, if the confidence of the low frequency interval is higher than the confidence level, the hypothesis that the low frequency interval is significant is rejected, and all relationships in the interval are excluded; S7: knowledge graph; after obtaining the entity and its corresponding relationship, store the new relationship recognized by all data nodes in the graph database, and open the interface for retrieval and query; S8: training; Collect the results of knowledge filtering regularly, and retrain and optimize the neural network of key entity recognition and entity linking.

2. The structured data processing method of a smart supervisory black box according to claim 1, wherein, The specified data is transmitted by key-value json method, and the key name is parsed with attribute to store the data in the graph database in node mode.

3. The structured data processing method of a smart supervising black box according to claim 1, wherein, In step S3, the input text is first segmented by word segmentation algorithm, and then the vector mapping of each unit is performed by pre-trained vector library; for statistical features, the system is represented in one-hot, Embedding or absolute number mode; Statistical features include historical frequency, word stem, index position of the sentence, whether the unit appears after splicing, word attribute and the position of the unit in the current word, the beginning of the word is represented by "0", the middle of the word is represented by "1", and the end of the word is represented by "2".

4. The structured data processing method of the intelligent supervisory black box according to claim 1 or 3, characterized in that, The entity extraction model uses neural network modeling, fuses four features of word features, word features, index positions, and the position of the unit in the current word, passes through a bidirectional long short memory model, combines with self-attention mechanism, and adopts full path conditional random field for result decoding, to output all potential key entities and corresponding business attributes in the sentence.

5. The structured data processing method of the intelligent supervisory black box according to claim 1 or 3, characterized in that, The semantic recognition includes preset relationship linking, existing graph retrieval, remote supervision relationship connection and existing syntax tree; Preset relationship linking: By classifying the extracted entities into predefined relationship set, the relationship between entities is identified; neural network is used for extraction, word and word vector are obtained, index position and label feature of each pair of extracted entities are fused, convolutional neural network and affine attention mechanism layer are used to output the relationship of each pair of entities; Existing graph retrieval: For the extracted entity pair, if the corresponding relationship can be found directly in the existing graph database, it is directly output; otherwise, other methods are used; Remote supervision relationship connection: A general Chinese knowledge graph is used to explore new relationships between each pair of entities. If the pair of entities can find a link relationship in the graph, output the relationship and store it in the final database. Stored syntax tree: By identifying the syntactic properties of each word in the text, the semantic association between each language unit in the sentence is analyzed, and the semantic association is presented in a dependency structure.

6. The structured data processing method of a smart supervising black box according to claim 5, wherein, The step S6 includes the following steps: S601: knowledge alignment; By similarity, the relationships generated by the three sub-modules of the preset relationship link, the remote supervision relationship link, and the dependency syntax tree are fused to generate a unified relationship representation. S602: knowledge filtering; The extracted relationships are counted to exclude low-frequency relationships based on statistical indicators and confidence tests.

7. The structured data processing method for a smart monitoring black box according to claim 1, characterized in that, It also includes topic modeling; after unsupervised topic mining of the collected node text data, each input data is labeled with a topic, and relevant operations are performed based on the corresponding topic during database access.

8. The structured data processing method of a smart supervising black box according to claim 1, wherein, The training input data of the key entity recognition includes text and entity position and corresponding labels; the training input data of the entity link includes entity position and corresponding labels and relationship labels.

Images