A knowledge joint extraction method, system, storage medium and device for unstructured vehicle model data
By combining the preprocessing of unstructured vehicle model data and deep learning models (BERT, CRF, GRU), the problem of unstructured data processing in the Internet of Vehicles is solved, efficient entity and relationship recognition is achieved, and significantly improved The accuracy and efficiency of knowledge extraction.
Patent Information
- Application Number
- CN202410883512.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-03
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-07-03
AI Technical Summary
In the field of Internet of Vehicles, it is difficult for the prior art to effectively process unstructured vehicle model data, especially due to the scarcity of labeling corpus, the application of deep learning-based methods is limited, resulting in insufficient accuracy and efficiency of entity recognition and relationship extraction.
By preprocessing the unstructured vehicle model data, determining the entity type, using the BERT model for word embedding and sequence annotation, combining the CRF model to capture entity dependencies, and using the GRU neural network for relationship classification, finally building a vehicle model knowledge graph.
The entity recognition accuracy rate is achieved at 95%, and the relationship recognition accuracy is achieved at 85%, which significantly reduces the operation time and manual investment of traditional manual processing methods, and improves the efficiency and accuracy of knowledge extraction.
Smart Images

Figure CN118410154B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of automobile data processing, and in particular relates to a knowledge joint extraction method, system, storage medium and device for unstructured vehicle model data. Background Art
[0002] As an important industry in the intelligent manufacturing planning, the Internet of Vehicles industry has received extensive attention from the intelligent manufacturing research in the industrial community. Natural language processing and deep learning technologies provide important support for knowledge mining. The Internet of Vehicles industry brings together data from various links of the entire life cycle of vehicles and related products, such as design and development, supply chain, manufacturing, maintenance and operation. It has the characteristics of complex industrial chain and multiple business logics. It also has the characteristics of multi-source data and heterogeneous knowledge representation. It can be analyzed as typical data. Designing a general knowledge extraction method suitable for the field of Internet of Vehicles is a major challenge in the digital transformation of industry.
[0003] Knowledge extraction from vehicle model data involves two tasks: named entity recognition and relationship extraction. In the current industrial field, due to the limitations of high data specialization and scarcity of annotated corpora, dictionary and rule-based methods have been widely used. However, such methods are highly dependent on manually set features, which increases R&D and maintenance costs. In recent years, machine learning methods based on deep learning have begun to be applied in the industrial field. However, such methods are highly dependent on large-scale, high-quality data support, and the significant scarcity of annotated corpora in the industrial field still affects the application and development of such methods.
[0004] In view of this, the present invention is proposed. Summary of the invention
[0005] In response to the above problems, the present invention pre-processes the unstructured vehicle model data; determines the entity types that need to be extracted from the vehicle model data; uses the labeled vehicle model data as a training set to optimize the BERT model; uses the BERT model to learn word embedding for the input text and converts the input text information into high-dimensional vector information; uses the CRF model to perform sequence annotation on the high-dimensional vector output by BERT, annotates the head entity and tail entity in the input text to capture the dependency relationship between entities and obtain the optimal entity sequence label; replaces the entity pairs obtained by sequence annotation with the corresponding entity type, and inputs the BERT model again for vector learning to obtain the entity association representation vector M; inputs the entity association representation vector M into the GRU neural network for relationship classification. This method has an entity recognition accuracy of 95% and a relationship recognition accuracy of 85%. Compared with the traditional manual vehicle model data processing method, the relevant operation time is reduced by at least 75%, and the manual input is reduced by at least 50%.
[0006] The technical solution adopted by the present invention to solve the above technical problems is: a knowledge joint extraction method for unstructured vehicle model data includes:
[0007] S1. Preprocess the unstructured vehicle model data, which includes sequentially performing text cleaning, word segmentation, and removing stop words and punctuation marks.
[0008] S2. Determine the entity types to be extracted from the vehicle model data, where the entities include vehicle model name, brand, color, size, mileage, fuel consumption per 100 kilometers, power consumption per 100 kilometers, bus name, and in-vehicle control unit;
[0009] S3. Use the labeled vehicle model data as a training set to fine-tune the BERT model; the labeling method is the BMEO labeling method, where B is the start character of the entity, M is the middle character of the entity, E is the end character of the entity, and O is the character of the non-named entity;
[0010] S4. Use the BERT (Bidirectional Encoder Representation from Transformers) model to learn word embedding for the input text and convert the input text information into high-dimensional vector information;
[0011] S5. Use the Conditional Random Field (CRF) model to perform sequence labeling on the high-dimensional vector output by BERT, label the head entity and tail entity in the input text to capture the dependency between entities and obtain the optimal entity sequence label.
[0012] S6. Replace the entity pairs obtained by sequence annotation with the corresponding entity types and input them into the BERT model again for vector learning to obtain the entity association representation vector M.
[0013] S7, input the entity association representation vector M into the GRU (Gate Recurrent Unit) neural network for relationship classification;
[0014] S8. Integrate the extracted entity and relationship information into a knowledge base and construct a vehicle model knowledge graph for querying, analyzing and displaying vehicle model related knowledge.
[0015] Wherein, the text information is vehicle control flow information text;
[0016] Converting input text information into high-dimensional vector information includes:
[0017] Among them, the BERT text representation layer outputs the control flow information text as a 512-dimensional vector, the control flow information statement , Which represents the i-th character of the sentence:
[0018] S41, the control flow information statement L is converted into three corresponding vectors:
[0019] Word vectors , segment vector and the position vector ,
[0020] S42, adding the word vector, the segment vector, and the position vector to obtain a sum vector ;
[0021] S43. Input the sum vector into the BERT model, and output a hidden state vector after encoding and decoding.
[0022] Among them, using the model to perform sequence labeling on the high-dimensional vector output by BERT includes:
[0023] S51: Obtain the input vector sequence according to the calculation model to obtain the total score of the predicted sequence. The calculation model is:
[0024]
[0025] Where D represents the transfer matrix, Indicates that the label in the transfer matrix is changed from y i Transfer to y i+1 score;
[0026] Indicates that the i-th character is predicted to be y i The score of the label; H represents the input observation sequence, that is, the sum vector, Y represents the corresponding output tag sequence or state sequence, and n is a positive integer;
[0027] S52. Calculate the probability of the input vector sequence being converted into a prediction sequence:
[0028] ; Y h represents all forecast sequences.
[0029] The inputting the entity association representation vector into the GRU (Gate Recurrent Unit) neural network for relationship classification includes:
[0030] The GRU model control includes:
[0031]
[0032] Among them, m t represents the input vector at time t, z t represents the update gate, r t represents the reset gate, ht represents the hidden layer state, Represents the sigmoid function of vector multiplication; represents the dot product operation, represents the matrix product operation, W i , W r , W c represents the weight matrix; is the correction factor.
[0033] The sigmoid function is .
[0034] A knowledge joint extraction system for unstructured vehicle model data, which includes:
[0035] Preprocessing module, used to preprocess unstructured vehicle model data;
[0036] Entity processing module: used to determine the entity types that need to be extracted from the vehicle model data; use the labeled vehicle model data as a training set to optimize the BERT model;
[0037] Entity recognition module: It is used to learn word embedding for input text using the BERT model and convert the input text information into high-dimensional vector information; it uses the CRF model to perform sequence annotation on the high-dimensional vector output by BERT, annotate the head entity and tail entity in the input text to capture the dependency relationship between entities and obtain the optimal entity sequence label;
[0038] The relationship classification module is used to replace the entity pairs obtained by sequence annotation with the corresponding entity types, and input them into the BERT model again for vector learning to obtain the entity association representation vector M; the entity association representation vector M is input into the GRU neural network for relationship classification;
[0039] The graph construction module is used to integrate the extracted entity and relationship information into a knowledge base, build a vehicle model knowledge graph, and query, analyze and display vehicle model related knowledge.
[0040] A computer-readable storage medium stores a computer program, and a processor executes the computer program to implement a knowledge joint extraction method for unstructured vehicle model data.
[0041] A terminal device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a knowledge joint extraction method for unstructured vehicle model data.
[0042] The beneficial effects of the present invention are as follows:
[0043] 1) This invention targets unstructured vehicle model data. The recognition accuracy of this method for entities reaches 95%, and the recognition accuracy for relationships reaches 85%. Compared with the traditional manual vehicle model data processing method, the relevant operation time is reduced by at least 75%, and the manual input is reduced by at least 50%.
[0044] 2) In order to address the problem of lack of professional knowledge when extracting knowledge from vehicle model data in the automotive industry, the integration of multiple professional knowledge bases related to vehicle model data has played a guiding role in entity recognition and improved the accuracy of recognition.
[0045] 3) Using the joint knowledge extraction model of the present invention, by jointly extracting entities and relationships, the hidden information therebetween is fully mined, thereby improving the effect of knowledge extraction.
[0046] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above description and other purposes, features and advantages of the present invention more obvious and easy to understand, the preferred embodiments are specifically cited and described in detail as follows. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] By reading the detailed description of the preferred embodiment below, various other advantages and benefits will become clear to those of ordinary skill in the art. The accompanying drawings are only used for the purpose of illustrating the preferred embodiment and are not considered to be limitations of the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings.
[0048] Figure 1 It is a flowchart of the joint knowledge extraction method for unstructured vehicle model data;
[0049] Figure 2 A schematic diagram of a specific implementation method. DETAILED DESCRIPTION
[0050] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.
[0051] In the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be connected, detachably connected, or integrated; it can be mechanically connected or electrically connected; it can be directly connected or indirectly connected through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0052] Example 1
[0053] See also Figure 1 , a joint knowledge extraction method for unstructured vehicle model data, including:
[0054] S1. Preprocess the unstructured vehicle model data, which includes sequentially performing text cleaning, word segmentation, and removing stop words and punctuation marks.
[0055] S2. Determine the entity types to be extracted from the vehicle model data, where the entities include vehicle model name, brand, color, size, mileage, fuel consumption per 100 kilometers, power consumption per 100 kilometers, bus name, and in-vehicle control unit;
[0056] S3. Use the labeled vehicle model data as a training set to fine-tune the BERT model; the labeling method is the BMEO labeling method, where B is the start character of the entity, M is the middle character of the entity, E is the end character of the entity, and O is the character of the non-named entity;
[0057] S4. Use the BERT (Bidirectional Encoder Representation from Transformers) model to learn word embedding for the input text and convert the input text information into high-dimensional vector information;
[0058] S5. Use the Conditional Random Field (CRF) model to perform sequence labeling on the high-dimensional vector output by BERT, label the head entity and tail entity in the input text to capture the dependency between entities and obtain the optimal entity sequence label.
[0059] S6. Replace the entity pairs obtained by sequence annotation with the corresponding entity types and input them into the BERT model again for vector learning to obtain the entity association representation vector M.
[0060] S7, input the entity association representation vector M into the GRU (Gate Recurrent Unit) neural network for relationship classification;
[0061] S8. Integrate the extracted entity and relationship information into a knowledge base and construct a vehicle model knowledge graph for querying, analyzing and displaying vehicle model related knowledge.
[0062] Wherein, the text information is vehicle control flow information text;
[0063] Converting input text information into high-dimensional vector information includes:
[0064] Among them, the BERT text representation layer outputs the control flow information text as a 512-dimensional vector, the control flow information statement , Which represents the i-th character of the sentence:
[0065] S41, the control flow information statement L is converted into three corresponding vectors:
[0066] Word vectors , segment vector and the position vector ,
[0067] S42, adding the word vector, the segment vector, and the position vector to obtain a sum vector ;
[0068] S43. Input the sum vector into the BERT model, and output a hidden state vector after encoding and decoding.
[0069] Among them, using the model to perform sequence labeling on the high-dimensional vector output by BERT includes:
[0070] S51: Obtain the input vector sequence according to the calculation model to obtain the total score of the predicted sequence. The calculation model is:
[0071] ,
[0072] Where D represents the transfer matrix, Indicates that the label in the transfer matrix is changed from y i Transfer to y i+1 score;
[0073] Indicates that the i-th character is predicted to be y i The score of the label; H represents the input observation sequence, that is, the sum vector, Y represents the corresponding output tag sequence or state sequence, and n is a positive integer;
[0074] S52. Calculate the probability of the input vector sequence being converted into a prediction sequence:
[0075] ; Y h represents all forecast sequences.
[0076] The inputting the entity association representation vector into the GRU (Gate Recurrent Unit) neural network for relationship classification includes:
[0077] The GRU model control includes:
[0078]
[0079] Among them, m t represents the input vector at time t, z t represents the update gate, r t represents the reset gate, h t represents the hidden layer state, Represents the sigmoid function of vector multiplication; represents the dot product operation, represents the matrix product operation, W i , W r , W c represents the weight matrix;
[0080] The sigmoid function is .
[0081] Among them, in the process of model training, knowledge extraction for vehicle model data faces the serious problem of insufficient training data. Under the guidance of automotive industry experts, this method uses the BMEO tagging scheme to perform character-based named entity tagging: B is the start character of the entity, M is the middle character of the entity, E is the end character of the entity, and O is the character of the non-named entity. At the same time, this method further annotates the relationship between entity pairs based on the annotated entity pairs. This method annotated 500 data in total, which effectively improved the training effect of the model.
[0082] Example 2
[0083] A knowledge joint extraction system for unstructured vehicle model data, which includes:
[0084] Preprocessing module, used to preprocess unstructured vehicle model data;
[0085] Entity processing module: used to determine the entity types that need to be extracted from the vehicle model data; use the labeled vehicle model data as a training set to optimize the BERT model;
[0086] Entity recognition module: It is used to learn word embedding for input text using the BERT model and convert the input text information into high-dimensional vector information; it uses the CRF model to perform sequence annotation on the high-dimensional vector output by BERT, annotate the head entity and tail entity in the input text to capture the dependency relationship between entities and obtain the optimal entity sequence label;
[0087] The relationship classification module is used to replace the entity pairs obtained by sequence annotation with the corresponding entity types, and input them into the BERT model again for vector learning to obtain the entity association representation vector M; the entity association representation vector M is input into the GRU neural network for relationship classification;
[0088] The graph construction module is used to integrate the extracted entity and relationship information into a knowledge base, build a vehicle model knowledge graph, and query, analyze and display vehicle model related knowledge.
[0089] A computer-readable storage medium stores a computer program, and a processor executes the computer program to implement a knowledge joint extraction method for unstructured vehicle model data.
[0090] A terminal device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a knowledge joint extraction method for unstructured vehicle model data.
[0091] Example 3
[0092] See also Figure 2 , the present invention provides a specific implementation method.
[0093] For example, the unstructured data "LDCU (P-05-LDCU control logic) is sent through ICAN (How4), and CDCU (P-01-CDCU control logic) receives and displays mileage" is obtained through preprocessing. Through BERT model and CRF model, the LDCUP-05-LDCU control logic is converted into vectors B-1, M-11, M-12,...E-1; ICANHow4 is converted into B-2, M-21, M-22,...E-2; .... and then form a vector M. The GRU neural network is used to classify the relationship of vector M, and the association relationship of "LDCU (P-05-LDCU control logic)"-"ICANHow4"-"CDCU (P-01-CDCU control logic)"-displayed mileage is obtained.
[0094] The beneficial effects of the present invention are as follows:
[0095] 1) This invention targets unstructured vehicle model data. The recognition accuracy of this method for entities reaches 95%, and the recognition accuracy for relationships reaches 85%. Compared with the traditional manual vehicle model data processing method, the relevant operation time is reduced by at least 75%, and the manual input is reduced by at least 50%.
[0096] 2) In order to address the problem of lack of professional knowledge when extracting knowledge from vehicle model data in the automotive industry, the integration of multiple professional knowledge bases related to vehicle model data has played a guiding role in entity recognition and improved the accuracy of recognition.
[0097] 3) Using the joint knowledge extraction model of the present invention, by jointly extracting entities and relationships, the hidden information therebetween is fully mined, thereby improving the effect of knowledge extraction.
[0098] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. A joint knowledge extraction method for unstructured vehicle model data, It is characterized in that The method comprises the following steps: S1. Preprocessing unstructured vehicle model data; S2, determine the entity type that needs to be extracted from the vehicle model data; S3. Use the labeled vehicle model data as a training set to optimize the BERT model. S4. Use the BERT model to learn word embedding for the input text and convert the input text information into high-dimensional vector information; S5. Use the CRF model to perform sequence annotation on the high-dimensional vector output by BERT, annotate the head entity and tail entity in the input text to capture the dependency relationship between entities and obtain the optimal entity sequence label; S6. Replace the entity pairs obtained by sequence annotation with the corresponding entity types and input them into the BERT model again for vector learning to obtain the entity association representation vector M. S7, input the entity association representation vector M into the GRU neural network for relationship classification; S8. Integrate the extracted entity and relationship information into a knowledge base and construct a vehicle model knowledge graph for querying, analyzing and displaying vehicle model related knowledge; Wherein, in S4, the text information is vehicle control flow information text; Converting input text information into high-dimensional vector information includes: S40, BERT text representation layer outputs the control flow information text as a 512-dimensional vector, control flow information statement , Which represents the i-th character of the sentence: S41, the control flow information statement L is converted into three corresponding vectors: Word vectors , segment vector and the position vector ; S42, adding the word vector, the segment vector, and the position vector to obtain a sum vector ; S43, inputting the sum vector into the BERT model, and outputting a hidden state vector after encoding and decoding; Using the CRF model to perform sequence labeling on the high-dimensional vector output by BERT includes: S51: Obtain the input vector sequence according to the calculation model to obtain the total score of the predicted sequence. The calculation model is: Where D represents the transfer matrix, Indicates that the label in the transfer matrix is changed from y i Transfer to y i+1 score; Indicates that the i-th character is predicted to be y i The score of the label; H represents the input observation sequence, that is, the sum vector, Y represents the corresponding output tag sequence or state sequence, and n is a positive integer; S52. Calculate the probability of the input vector sequence being converted into a prediction sequence: Y h represents all forecast sequences; The GRU model control includes: Among them, m t represents the input vector at time t, z t represents the update gate, r t represents the reset gate, h t represents the hidden layer state, Represents the sigmoid function of vector multiplication; represents the dot product operation, represents the matrix product operation, W i , W r , W c represents the weight matrix; is the correction factor; The sigmoid function is .
2. According to the method for joint knowledge extraction of unstructured vehicle model data according to claim 1, It is characterized in that The preprocessing includes sequentially performing text cleaning, word segmentation, and removing stop words and punctuation marks.
3. According to the method for joint knowledge extraction of unstructured vehicle model data according to claim 1, It is characterized in that The entities include vehicle model name, brand, color, size, mileage, fuel consumption per 100 kilometers, power consumption per 100 kilometers, bus name, and in-vehicle control unit.
4. According to the method for joint knowledge extraction of unstructured vehicle model data according to claim 1, It is characterized in that The marking methods in steps S3 and S5 are both BMEO marking methods, where B is the start character of the entity, M is the middle character of the entity, E is the end character of the entity, and O is the character of the non-named entity.
5. A knowledge joint extraction system for unstructured vehicle model data, It is characterized in that The system includes: Preprocessing module, used to preprocess unstructured vehicle model data; Entity processing module: used to determine the entity types that need to be extracted from the vehicle model data; use the labeled vehicle model data as a training set to optimize the BERT model; Entity recognition module: It is used to learn word embedding for input text using the BERT model and convert the input text information into high-dimensional vector information; it uses the CRF model to perform sequence annotation on the high-dimensional vector output by BERT, annotate the head entity and tail entity in the input text to capture the dependency relationship between entities and obtain the optimal entity sequence label; The relationship classification module is used to replace the entity pairs obtained by sequence annotation with the corresponding entity types, and input them into the BERT model again for vector learning to obtain the entity association representation vector M; the entity association representation vector M is input into the GRU neural network for relationship classification; The graph construction module is used to integrate the extracted entity and relationship information into a knowledge base and build a vehicle model knowledge graph for querying, analyzing and displaying vehicle model related knowledge; Among them, the control flow information statement L is converted into three corresponding vectors: Word vectors , segment vector and the position vector ; Add the word vector, the segment vector, and the position vector to obtain a sum vector ; The sum vector is input into the BERT model, and a hidden state vector is output after encoding and decoding; Using the CRF model to perform sequence labeling on the high-dimensional vector output by BERT includes: S51: Obtain the input vector sequence according to the calculation model to obtain the total score of the predicted sequence. The calculation model is: Where D represents the transfer matrix, Indicates that the label in the transfer matrix is changed from y i Transfer to y i+1 score; Indicates that the i-th character is predicted to be y i The score of the label; H represents the input observation sequence, that is, the sum vector, Y represents the corresponding output tag sequence or state sequence, and n is a positive integer; S52. Calculate the probability of the input vector sequence being converted into a prediction sequence: Y h represents all forecast sequences; The GRU model control includes: Among them, m t represents the input vector at time t, z t represents the update gate, r t represents the reset gate, h t represents the hidden layer state, Represents the sigmoid function of vector multiplication; represents the dot product operation, represents the matrix product operation, W i , W r , W c represents the weight matrix; is the correction factor; The sigmoid function is .
6. A computer-readable storage medium storing a computer program, It is characterized in that The processor executes the computer program to implement the knowledge joint extraction method for unstructured vehicle model data as described in any one of claims 1-4.
7. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that The processor executes the computer program to implement the knowledge joint extraction method for unstructured vehicle model data as described in any one of claims 1 to 4.
