Text analysis method, device and computer equipment
The material text extraction and segmentation through pre-trained models solves the problems of low efficiency and insufficient accuracy of engineering text processing in the prior art, and realizes efficient and accurate semantic analysis of engineering text.
Patent Information
- Application Number
- CN202110895620.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-08-05
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2041-08-05
Smart Images

Figure CN113449509B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of semantic recognition, and in particular to a text analysis method, device and computer equipment. Background Art
[0002] In engineering construction, it is necessary to extract and analyze the types, specifications, and usage of various materials in engineering texts. In order to solve this problem, the conventional solution is to analyze the characteristics and business needs of specific engineering texts, and then sort out and construct rules that are suitable for them. However, when faced with a large number of engineering texts to be processed, this rule-based engineering text processing method is time-consuming and labor-intensive, and cannot meet the needs of material extraction and analysis in batches of engineering texts. At the same time, the rule-based processing method is not accurate enough in extracting and analyzing materials in engineering texts, and its adaptability to different engineering texts is also low. Summary of the invention
[0003] In view of the above problems, the present invention provides a text analysis method, apparatus and computer device.
[0004] The specific plan is as follows:
[0005] In a first aspect, an embodiment of the present disclosure provides a text analysis method, the method comprising:
[0006] Get the text to be analyzed;
[0007] Input the text into a pre-trained material text extraction model to extract material text containing material characteristics, wherein the material characteristics include material usage and material information, and the material information includes at least one of material type, material name, and material model;
[0008] Input the material text into a pre-trained material unit segmentation model to obtain the corresponding material units and labeling results in the material text, wherein the material unit includes at least one text block including a material name, a material model, and a material usage;
[0009] Input the material unit into a pre-trained model extraction model to extract the material model of each material in the material unit;
[0010] The material model of each material is input into the pre-trained model analysis model to obtain the material model analysis result.
[0011] According to a specific implementation of the present disclosure, the training step of the material text extraction model includes:
[0012] Acquire a sample text set, wherein the sample text set includes a first type of sample text containing material features and a second type of sample text not containing material features;
[0013] The sample text set is input into a basic neural network model for training to obtain a material text extraction model.
[0014] According to a specific implementation of the present disclosure, the step of inputting the material text into a pre-trained material unit segmentation model to obtain the corresponding material units and marking results in the material text includes:
[0015] Splitting the material text into multiple text blocks according to the separator;
[0016] Inputting the multiple text blocks into the material unit segmentation model for processing, and outputting multiple material units;
[0017] Supplement the missing quantifiers in the material unit.
[0018] According to a specific implementation of the present disclosure, the step of inputting the multiple text blocks into the material unit segmentation model for processing includes:
[0019] Inputting the plurality of text blocks into the material unit segmentation model according to the order in the material text;
[0020] If the text block contains the material name, material model and material usage at the same time, the text block is marked as a material unit;
[0021] If the text block does not contain the material name, material model and material usage at the same time, continue to enter the next text block until all the entered text blocks cumulatively contain the material name, material model and material usage, and mark the combination of all text blocks as a material unit.
[0022] According to a specific implementation of the present disclosure, after the step of inputting the material text into a pre-trained material unit segmentation model to obtain the corresponding material units and marking results in the material text, the method further includes:
[0023] Inputting each of the material units into a pre-trained hierarchical identification model to identify the hierarchy of each of the material units, wherein the hierarchy is an inclusion relationship between the material units;
[0024] Input the material unit into a pre-trained material type prediction model, and output the material type of each material unit;
[0025] According to the hierarchy and the inclusion relationship of material types, the material units are divided into multiple material unit meaning groups, where the material unit meaning group is a combination of material units associated with the material name, material model and material usage.
[0026] According to a specific implementation of the present disclosure, the step of training the material unit segmentation model includes:
[0027] Obtain a material unit sample set and a non-material unit sample set, wherein the material unit sample set includes a first subset containing quantifiers and a second subset not containing quantifiers, and the non-material unit sample set is a set of text blocks that do not simultaneously contain material names, material models, and material quantities, or do not simultaneously contain material names and material quantities;
[0028] Inputting the material unit sample set and the non-material unit sample set into a basic machine learning model;
[0029] The basic machine learning model is trained using a support vector machine algorithm to obtain a material unit segmentation model.
[0030] In a second aspect, an embodiment of the present disclosure further provides a text analysis device, the device comprising:
[0031] A first acquisition module is used to acquire the text to be analyzed;
[0032] A material text extraction module, used for inputting the text into a pre-trained material text extraction model to extract material text containing material characteristics, wherein the material characteristics include material usage and material information, and the material information includes at least one of material type, material name, and material model;
[0033] A material unit acquisition module, used for inputting the material text into a pre-trained material unit segmentation model to obtain the corresponding material units and marking results in the material text, wherein the material unit includes at least one text block including the material name, material model and material usage;
[0034] A material model extraction module, used to input the material unit into a pre-trained model extraction model to extract the material model of each material in the material unit;
[0035] The analysis module is used to input the material model of each material into a pre-trained model analysis model to obtain the material model analysis result.
[0036] According to a specific embodiment of the present disclosure, the device further includes:
[0037] A second acquisition module is used to acquire a sample text set, wherein the sample text set includes a first type of sample text containing material characteristics and a second type of sample text not containing material characteristics;
[0038] The training module is used to input the sample text set into the basic neural network model for training to obtain a material text extraction model.
[0039] In a third aspect, an embodiment of the present disclosure further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is run on the processor, the text analysis method described in any one of the first aspects is executed.
[0040] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium storing a computer program, which executes the text analysis method described in any one of the first aspects when running on a processor.
[0041] The text analysis method, device and computer equipment provided by the embodiments of the present disclosure first extract the material text in the input text, then segment the material units in the material text, then extract the material model in each material unit, and finally analyze the material model to obtain the final material model analysis result. It can realize the accurate segmentation of the material units in the engineering quantity text, the fine analysis of the material units at the semantic level, and the accurate extraction and analysis of the material model; thus improving the efficiency and accuracy of the semantic analysis of the engineering quantity text. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to more clearly illustrate the technical solution of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope of protection of the present invention. In each of the drawings, similar components are numbered similarly.
[0043] Figure 1 A flowchart of a text analysis method provided by an embodiment of the present disclosure is shown;
[0044] Figure 2 A schematic diagram of a text to be analyzed in a text analysis method provided by an embodiment of the present disclosure is shown;
[0045] Figure 3 A schematic diagram of a text to be analyzed in another text analysis method provided by an embodiment of the present disclosure is shown;
[0046] Figure 4 An overall flow chart of a text analysis method provided by an embodiment of the present disclosure is shown;
[0047] Figure 5 A whole module block diagram of a text analysis device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0048] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.
[0049] The components of the embodiments of the present invention generally described and shown in the drawings herein may be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0050] Hereinafter, the terms "including", "having" and their cognates, which may be used in various embodiments of the present invention, are intended only to indicate specific features, numbers, steps, operations, elements, components or combinations of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing items or adding the possibility of one or more features, numbers, steps, operations, elements, components or combinations of the foregoing items.
[0051] Furthermore, the terms “first”, “second”, “third”, etc. are merely used for distinguishing descriptions and are not to be understood as indicating or implying relative importance.
[0052] Unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meanings as those generally understood by those skilled in the art to which the various embodiments of the present invention belong. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meanings as the contextual meanings in the relevant technical field and will not be interpreted as having idealized meanings or overly formal meanings unless clearly defined in the various embodiments of the present invention.
[0053] Example 1
[0054] Figure 1 The following is a flow chart of a text analysis method provided by an embodiment of the present disclosure. Figure 1 As shown, the method includes:
[0055] S101, obtaining a text to be analyzed;
[0056] Specifically, the text to be analyzed can be a text describing the engineering quantity obtained by parsing the design drawings, and the design drawings can be drawings written manually or drawings automatically generated by design software according to a template. It can be flexibly processed according to actual needs and is not limited here.
[0057] The text to be analyzed includes Figure 2 The non-displayed element identification text and Figure 3The scanned drawing description is shown. The text to be analyzed includes material type, material name, material model, material usage, etc., as well as descriptive text unrelated to the material.
[0058] S102, inputting the text into a pre-trained material text extraction model to extract material text containing material characteristics, wherein the material characteristics include material usage and material information, and the material information includes at least one of material type, material name, and material model;
[0059] In the specific implementation, there is noise data in the original text to be analyzed, which will affect the accuracy of subsequent text analysis. Noise data is non-engineering quantity description text that does not contain material characteristics. It mainly includes two parts: Figure 2 The element identification text shown is embedded in the drawing but not displayed, such as: <acdbproxyentity>...[1EC08]; the other is Figure 3 The drawings marked in the middle bar frame do not show text describing the materials and their quantities.
[0060] Before performing text analysis on the text to be analyzed, it is necessary to filter the noise data in the text and extract the material text in the text. In a specific implementation, a material text extraction model is used to extract and integrate the material text in the text to be analyzed. Of course, other methods can also be used to extract the material text, which is not limited here. In the embodiment of the present disclosure, the material text at least includes the material name, material model and material usage. In other embodiments, the material text can be flexibly determined as needed. Figure 3 As shown, the material type can be overhead lines, poles, etc.; the material name can be 10kV overhead lines, new poles, etc.; the material model can be LGJ-120 / 20, LGJ-185-25, etc.; the material quantity can be path length 12754.82 meters, 20 poles, etc.
[0061] S103, inputting the material text into a pre-trained material unit segmentation model to obtain the corresponding material units and marking results in the material text, wherein the material unit includes at least one text block including a material name, a material model, and a material usage;
[0062] Specifically, in order to facilitate the subsequent extraction and analysis of the material model, the material text needs to be segmented into appropriate granularity input models. In this embodiment, the material unit is used as the granularity input model, and the material unit is one or more text blocks including the complete material name, material quantity and material model of a material. The text block is a plurality of parts divided by punctuation marks.
[0063] The extracted material text is input into the material unit segmentation model in paragraphs to obtain all the material units in the material text. Each material unit is marked and segmented, and then output from the material unit segmentation model to facilitate the subsequent hierarchical division of the material units. The material unit can be Figure 2 The text blocks shown as "Towers: 85 in total" and "Spans: 50 in total" contain material quantities and material names.
[0064] S104, inputting the material unit into a pre-trained model extraction model to extract the material model of each material in the material unit;
[0065] In specific implementation, a model extraction model is pre-trained to extract the material model in the material unit. In order to improve the accuracy of the model, the model is trained by combining the rule base and deep learning during the training process. Material models may include two types, one is the model format of a combination of characters and numbers, and the other is the model format of text description. During the model training process, the model will be trained to recognize and extract the model in both formats.
[0066] S105, inputting the material model of each material into a pre-trained model analysis model to obtain a material model analysis result.
[0067] Specifically, the extracted material model is input into the model analysis model, the attributes of each material model are parsed, and finally the meaning of the attributes contained in each material model is analyzed. The model analysis model is trained using semi-supervised learning; of course, other methods can also be used to train the model in actual use, which is not limited here. If the extracted material model is JKLYJ-1-70, the output analysis result is that the conductor model used for the new line is JKLYJ-1-70, the conductor type of this model is JKLYJ, and the conductor cross section is 70.
[0068] Through the above method, it is possible to accurately segment the material units in the engineering quantity text, conduct detailed analysis of the material units at the semantic level, and accurately extract and parse the material models; thus improving the efficiency and accuracy of the semantic analysis of the engineering quantity text.
[0069] According to a specific implementation of the present disclosure, the training step of the material text extraction model includes:
[0070] Acquire a sample text set, wherein the sample text set includes a first type of sample text containing material features and a second type of sample text not containing material features;
[0071] In the specific implementation, in this embodiment, the sample text is first divided into two categories by combining manual and automatic methods. One category is the material description text containing material characteristics, that is, the first category of sample text; the other category is the non-material description text that does not contain material characteristics, that is, the second category of sample text. The two categories of text are stored in different files, and the two files are stored in different folders. The name of the folder is the classification label of the sample text. The two folders are stored in the same folder to form a three-level structure. The folder composed of the three-level structure is the sample text set used to train the material text extraction model.
[0072] The sample text set is input into a basic neural network model for training to obtain a material text extraction model.
[0073] Specifically, the sample text set is input into the basic neural network model, and trained based on the machine learning algorithm to obtain the corresponding material text extraction model. When the material text extraction model is used, the text to be analyzed is input into the model line by line to extract the material text.
[0074] According to a specific implementation of the present disclosure, the step of inputting the material text into a pre-trained material unit segmentation model to obtain the corresponding material units and marking results in the material text includes:
[0075] Splitting the material text into multiple text blocks according to the separator;
[0076] In a specific implementation, the material text is segmented according to punctuation marks. The material text is segmented into multiple text blocks according to punctuation marks, such as commas, semicolons, periods, etc. Each text block may contain one or more of the material name, material quantity, and material model.
[0077] Inputting the multiple text blocks into the material unit segmentation model for processing, and outputting multiple material units;
[0078] Specifically, multiple text blocks are input into the material unit segmentation model in order of their sequence in the material text, and the model determines whether the text blocks are material units. The final output of the material unit segmentation model is multiple material units.
[0079] Supplement the missing quantifiers in the material unit.
[0080] In specific implementation, the end of the output material unit may be missing a quantifier and end with only a number; this will cause interference in the subsequent model extraction and analysis, affecting the accuracy of model extraction and analysis. Therefore, it is necessary to complete the quantifier for the material unit with missing quantifiers at the end. The method of completing the quantifier can be to automatically complete the quantifier through a neural network model, or other methods can be used, which are not limited here.
[0081] According to a specific implementation of the present disclosure, the step of inputting the multiple text blocks into the material unit segmentation model for processing includes:
[0082] Inputting the plurality of text blocks into the material unit segmentation model according to the order in the material text;
[0083] Specifically, in order to ensure that the material name, material model and material usage in the output material unit are consistent, each text block needs to be input into the material unit segmentation model according to the order in the material text.
[0084] If the text block contains the material name, material model and material usage at the same time, the text block is marked as a material unit; if the text block does not contain the material name, material model and material usage at the same time, continue to enter the next text block until all the input text blocks cumulatively contain the material name, material model and material usage, and the combination of all text blocks is marked as a material unit.
[0085] In specific implementation, a text block that contains the material name, material quantity and material model at the same time can be used as a material unit, and a text block that only contains part of the material name, material quantity and material model needs to be combined with other text blocks to form a material unit.
[0086] If the currently processed text block does not contain the material name, material model and material quantity at the same time, continue to input the next text block after the current text block, and determine whether the two input text blocks contain the material name, material model and material quantity at the same time; if not, continue to perform the above operations. If yes, combine all the input text blocks as a material unit and output them.
[0087] In a specific implementation, the set of text blocks to be predicted is S(t), when S(t) i When the material name, material model and material usage are included in S(t), it is output as a material unit. i When the material name, material model and material quantity are not included at the same time, in S(t) i Add a text block S(t) on the basis i+1 , and judge whether it contains the material name, material model and material quantity at the same time. If the judgment result is yes, [S(t) i, S(t) i+1 ] is cut into a material unit for output.
[0088] According to a specific implementation of the present disclosure, after the step of inputting the material text into a pre-trained material unit segmentation model to obtain the corresponding material units and marking results in the material text, the method further includes:
[0089] Inputting each of the material units into a pre-trained hierarchical identification model to identify the hierarchy of each of the material units, wherein the hierarchy is an inclusion relationship between the material units;
[0090] In a specific implementation, there is a containment relationship between each material unit, and the material units need to be hierarchically divided according to the containment relationship between the material units to avoid repeated extraction of material models in related material units, and also to avoid redundant data in the text analysis results. In a specific implementation, the levels are divided into a first level and a second level, wherein the first level contains the second level.
[0091] According to the inclusion relationship in each material unit, the material units are hierarchically identified and divided through the hierarchical identification model.
[0092] In a specific implementation, the hierarchical relationship in the sample is annotated by manual annotation to train the material type prediction model. Of course, the material type prediction model can also be trained by other methods, which are not limited here.
[0093] Input the material unit into a pre-trained material type prediction model, and output the material type of each material unit;
[0094] Specifically, the association relationship between the material units is determined by judging the material type contained in each material unit.
[0095] According to the hierarchy and the inclusion relationship of material types, the material units are divided into multiple material unit meaning groups, where the material unit meaning group is a combination of material units associated with the material name, material model and material usage.
[0096] In specific implementation, for example, the level and material type of the conductor include the level and material type of the copper conductor. Based on the association information of the level and material type, the material unit is grouped, and the correctness of the grouping result is further verified. The grouping logic is that there is an association relationship of material types between continuous material units and the material unit has multiple levels.
[0097] According to a specific implementation of the present disclosure, the step of training the material unit segmentation model includes:
[0098] Obtain a material unit sample set and a non-material unit sample set, wherein the material unit sample set includes a first subset containing quantifiers and a second subset not containing quantifiers, and the non-material unit sample set is a set of text blocks that do not simultaneously contain material names, material models, and material quantities, or do not simultaneously contain material names and material quantities;
[0099] Specifically, when training the material unit segmentation model, the material unit sample set and the non-material unit sample set are used to increase the robustness of the model in identifying material units. At the same time, since the material text includes two types of material units, those with quantifiers at the end and those without quantifiers at the end, the material unit segmentation model needs to identify and segment both types of material units with quantifiers at the end and those without quantifiers at the end. The first subset containing quantifiers and the second subset not containing quantifiers need to be set in the material unit sample set to improve the accuracy of the model.
[0100] Inputting the material unit sample set and the non-material unit sample set into a basic machine learning model;
[0101] The basic machine learning model is trained using a support vector machine algorithm to obtain a material unit segmentation model.
[0102] Specifically, the support vector machine algorithm is used for model training. Similarly, in order to improve the accuracy of model prediction, the weights of the numbers and quantifiers at the end of the material unit are set during the training process, and finally the material unit segmentation model is obtained.
[0103] The text analysis method provided by the embodiment of the present disclosure first extracts the material text in the input text, then segments the material units in the material text, then extracts the material model in each material unit, and finally analyzes the material model to obtain the final material model analysis result. It can realize the accurate segmentation of the material units in the engineering quantity text, the fine analysis of the material units at the semantic level, and the accurate extraction and analysis of the material model; thus improving the efficiency and accuracy of the semantic analysis of the engineering quantity text.
[0104] In a specific embodiment, Figure 4 As shown, the overall process of a text analysis method provided by an embodiment of the present disclosure is as follows:
[0105] After receiving the engineering quantity description text segment, data filtering is first performed to obtain a set of text segments containing engineering quantity description units, namely, material text. Each text segment is marked with a segment number. The specific process of data filtering is as follows:
[0106] Data filtering is based on a classification mechanism, the core of which is to train a material text extraction model. First, the text parsed from the drawings is divided into two categories through a combination of manual and automated methods: one is material text, and the other is non-material text. The two types of text are stored in different files, and the files are stored in different folders. The names of the folders are classification labels. The two folders are stored in the same folder, forming a three-level structure. The folders composed of the three-level structure are the data sets used to train the material description / non-material description prediction model.
[0107] After the data set is constructed, the corresponding material text extraction model is trained based on the machine learning algorithm. Then the text parsed from the drawing is input into the model line by line. The model predicts whether it is a material description text. If it is a non-material description text, it will be filtered out.
[0108] Extract the key-value pairs in the material text in paragraphs, and the extracted key-value pairs are the material units. Get the entire material unit set, and identify the serial number of each material unit in turn.
[0109] The level of each material unit is identified based on the level prediction model and the accuracy of the prediction model is verified based on the rules, and then the material type prediction model is used to predict the material type described by each material unit. In this embodiment, the level is three levels.
[0110] The block is grouped based on the association information of the level and material type and the correctness of the block result is checked, wherein the block logic is that there is an association relationship of material type between consecutive material units and the material unit has multiple levels.
[0111] The material units of the group block are subjected to the intra-block implication relationship. The construction logic is that the material unit of the first level has the implication mark as empty, the material unit of the second level has the implication mark as the serial number of the material unit of the first level, and the material unit of the third level has the implication mark as the serial number of the material unit of the second level. Then, the quantity and unit of the material unit are extracted based on the rules. At this time, the data of each material unit after processing is obtained, and then it is formatted in combination with the predefined fields.
[0112] For texts whose material units are material types, a method based on deep learning and rules is used to extract the model, and then the model is parsed based on the machine learning model precision parsing method to obtain the final parsing result.
[0113] Through the above method, it is possible to accurately segment the material units in the engineering quantity text, conduct detailed analysis of the material units at the semantic level, and accurately extract and parse the material models; thus improving the efficiency and accuracy of the semantic analysis of the engineering quantity text.
[0114] Example 2
[0115] Figure 5 Module block diagram of a text analysis device provided by an embodiment of the present disclosure. Figure 5 As shown, the text analysis device 500 includes:
[0116] A first acquisition module 501 is used to acquire a text to be analyzed;
[0117] A material text extraction module 502 is used to input the text into a pre-trained material text extraction model to extract the material text containing material characteristics, wherein the material characteristics include material usage and material information, and the material information includes at least one of material type, material name, and material model;
[0118] The material unit acquisition module 503 is used to input the material text into a pre-trained material unit segmentation model to obtain the corresponding material units and marking results in the material text, wherein the material unit includes at least one text block including the material name, material model and material usage;
[0119] A material model extraction module 504 is used to input the material unit into a pre-trained model extraction model to extract the material model of each material in the material unit;
[0120] The analysis module 505 is used to input the material model of each material into a pre-trained model analysis model to obtain a material model analysis result.
[0121] According to a specific embodiment of the present disclosure, the device further includes:
[0122] A second acquisition module is used to acquire a sample text set, wherein the sample text set includes a first type of sample text containing material characteristics and a second type of sample text not containing material characteristics;
[0123] The training module is used to input the sample text set into the basic neural network model for training to obtain a material text extraction model.
[0124] In summary, the text analysis device provided in the embodiment of the present disclosure first extracts the material text in the input text, then segments the material units in the material text, then extracts the material model in each material unit, and finally analyzes the material model to obtain the final material model analysis result. It can achieve accurate segmentation of the material units in the engineering quantity text, perform detailed analysis of the material units at the semantic level, and accurately extract and parse the material models; thus improving the efficiency and accuracy of the semantic analysis of the engineering quantity text. The specific implementation process of the provided text analysis device can be found in the above Figure 1 , Figure 2 , Figure 3 and Figure 4 The specific implementation process of the text analysis method provided by the illustrated embodiment will not be described in detail here.
[0125] In addition, the present disclosure also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the computer program is executed when it is executed on the processor. Figures 1 to 4 The text analysis method shown.
[0126] In addition, the present disclosure also provides a computer-readable storage medium storing a computer program, which executes the following when the computer program is executed on a processor: Figures 1 to 4 The text analysis method shown.
[0127] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or the flow chart, and the combination of boxes in the structure diagram and / or the flow chart, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.
[0128] In addition, the functional modules or units in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.
[0129] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0130] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.< / acdbproxyentity>
Claims
1. A text analysis method, It is characterized in that The method comprises: Get the text to be analyzed; Input the text into a pre-trained material text extraction model to extract material text containing material characteristics, wherein the material characteristics include material usage and material information, and the material information includes at least one of material type, material name, and material model; Input the material text into a pre-trained material unit segmentation model to obtain the corresponding material units and labeling results in the material text, wherein the material unit includes at least one text block including a material name, a material model, and a material usage; Input the material unit into a pre-trained model extraction model to extract the material model of each material in the material unit; Input the material model of each material into the pre-trained model analysis model to obtain the material model analysis result; The step of inputting the material text into a pre-trained material unit segmentation model to obtain the corresponding material units and marking results in the material text includes: Splitting the material text into multiple text blocks according to the separator; Inputting the plurality of text blocks into the material unit segmentation model for processing, Output multiple material units; Supplement the missing quantifiers in the material unit; The step of inputting the plurality of text blocks into the material unit segmentation model for processing includes: Inputting the plurality of text blocks into the material unit segmentation model according to the order in the material text; If the text block contains the material name, material model and material usage at the same time, the text block is marked as a material unit; If the text block does not contain the material name, material model and material usage at the same time, continue to enter the next text block until all the entered text blocks cumulatively contain the material name, material model and material usage, and mark the combination of all text blocks as a material unit.
2. The text analysis method according to claim 1, It is characterized in that The training steps of the material text extraction model include: A sample text set is obtained, wherein the sample text set includes a first type of sample text containing material features and a second type of sample text not containing material features; the sample text set is input into a basic neural network model for training to obtain a material text extraction model.
3. The text analysis method according to claim 1, It is characterized in that After the step of inputting the material text into a pre-trained material unit segmentation model to obtain the corresponding material units and marking results in the material text, the method further includes: Inputting each of the material units into a pre-trained hierarchical identification model to identify the hierarchy of each of the material units, wherein the hierarchy is an inclusion relationship between the material units; Input the material unit into a pre-trained material type prediction model, and output the material type of each material unit; According to the hierarchy and the inclusion relationship of material types, the material units are divided into multiple material unit meaning groups, where the material unit meaning group is a combination of material units associated with the material name, material model and material usage.
4. The text analysis method according to claim 1, It is characterized in that The training steps of the material unit segmentation model include: Obtain a material unit sample set and a non-material unit sample set, wherein the material unit sample set includes a first subset containing quantifiers and a second subset not containing quantifiers, and the non-material unit sample set is a set of text blocks that do not simultaneously contain material names, material models, and material quantities, or do not simultaneously contain material names and material quantities; Inputting the material unit sample set and the non-material unit sample set into a basic machine learning model; The basic machine learning model is trained using a support vector machine algorithm to obtain a material unit segmentation model.
5. A text analysis device, It is characterized in that The device comprises: A first acquisition module is used to acquire the text to be analyzed; A material text extraction module, used for inputting the text into a pre-trained material text extraction model to extract material text containing material features, wherein the material features include material usage and material information, and the material information includes at least one of material type, material name, and material model; A material unit acquisition module, used for inputting the material text into a pre-trained material unit segmentation model to obtain the corresponding material units and marking results in the material text, wherein the material unit includes at least one text block including the material name, material model and material usage; A material model extraction module, used to input the material unit into a pre-trained model extraction model to extract the material model of each material in the material unit; The analysis module is used to input the material model of each material into the pre-trained model analysis model to obtain the material model analysis result; The step of inputting the material text into a pre-trained material unit segmentation model to obtain the corresponding material units and marking results in the material text includes: Splitting the material text into multiple text blocks according to the separator; Inputting the plurality of text blocks into the material unit segmentation model for processing, Output multiple material units; Supplement the missing quantifiers in the material unit; The step of inputting the plurality of text blocks into the material unit segmentation model for processing includes: Inputting the plurality of text blocks into the material unit segmentation model according to the order in the material text; If the text block contains the material name, material model and material usage at the same time, the text block is marked as a material unit; If the text block does not contain the material name, material model and material usage at the same time, continue to enter the next text block until all the entered text blocks cumulatively contain the material name, material model and material usage, and mark the combination of all text blocks as a material unit.
6. The text analysis device according to claim 5, It is characterized in that The device also includes: A second acquisition module is used to acquire a sample text set, wherein the sample text set includes a first type of sample text containing material characteristics and a second type of sample text not containing material characteristics; The training module is used to input the sample text set into the basic neural network model for training to obtain a material text extraction model.
7. A computer device, It is characterized in that The invention comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is run on the processor, the text analysis method according to any one of claims 1 to 4 is executed.
8. A computer-readable storage medium, It is characterized in that The computer program stores a computer program, which executes the text analysis method according to any one of claims 1 to 4 when running on a processor.
