Text mining method and system for chemical industry inspection reports

Abstract

This invention provides a text mining method and system for major inspection reports in the chemical industry, belonging to the field of text mining. The method includes: identifying the chapter structure of the major inspection report text; performing sentiment analysis on the text based on the chapter structure to obtain the chapters with the most negative sentiment analysis results; segmenting the chapters with the most negative sentiment analysis results into words to obtain the vocabulary and corresponding part-of-speech tags for each paragraph within the chapters; and integrating and statistically analyzing the vocabulary and corresponding part-of-speech tags to obtain a summary of safety management problems existing in each enterprise. By identifying the chapters with the most negative sentiment analysis results through sentiment analysis, segmenting the most negative chapters into words, and performing word frequency statistics, the problems existing in each enterprise in the major inspection report are obtained, providing information support for regulatory decision-making.

Description

Technical Field

[0001] The present invention relates to the field of text mining, and in particular to a text mining method for chemical industry inspection reports and a text mining system for chemical industry inspection reports. Background Art

[0002] The chemical industry inspection report is stored in an unstructured text data format, mainly including outstanding performance in safety management and equipment management and problems that existed during the inspection that year. With the development of the industry, it has accumulated year by year to form a kind of text big data. Although the problem information contained therein is of guiding significance for safe production and supervision, the process of extracting and organizing information from it is usually done manually, which is very time-consuming and laborious. With the development of computer technology, text mining technology, which can extract potentially useful information from text, has been widely used in information retrieval, public opinion analysis and other fields in recent years, and has gradually developed into text analysis in professional fields.

[0003] General text mining mainly designs methods that can extract information from text data in multiple fields. Meng Tao and Li Jiajing's patent "A general text mining method and system" crawls network text information and annotates concepts and relationships, thereby being able to search for concepts and relationships. Liu Junqi, Su Aijun, Tang Huiming, etc.'s patent "A method for quickly extracting useful data from a document collection" performs word segmentation and word frequency statistics on text data, imports all words into the database, and supports the use of search terms to search for information in the database. General text mining processes text data only at the language level, so the information that can be analyzed is highly general but shallow in depth, and it is also impossible to further process the text data in combination with the structure and entity meaning of the document.

[0004] Text mining in professional fields mainly automates information extraction from a large amount of text data accumulated in the industry, thereby improving efficiency and enabling historical document records to play a greater role. The utility model "95598 complaint work order key information extraction and analysis device and system based on text mining" by Wan Lei, Yan Daobo, Yang Yong, etc. converts customer complaint calls into work order text information, designs work order categories and corresponding feature text sets, matches the work order with the feature text after word segmentation, and divides them into corresponding categories according to the matching results. The patent "A text mining method for reporting cases of distribution line tripping" by Liu Xuan, Shang Yinhui, Che Liang, etc. extracts core vocabulary and main narrative syntax based on the results of word segmentation and part-of-speech tagging to form multiple groups of factual multi-group phrases for reporting cases. The patent "A text mining method and system based on unstructured electronic medical records" by Yang Bo, Wang Rui, and Peng Lizhi stores cases in a structured manner according to time, extracts descriptions of symptoms, examination findings, radiotherapy and chemotherapy regimens, and efficacy evaluation, and realizes the structuring of electronic medical records and prediction of radiotherapy and chemotherapy sensitivity. Huang Xuyong, Wang Zha, Sun Peng and others' patent "A method for extracting circuit breaker fault feature information based on text mining technology" classifies fault information by identifying and extracting representative words of circuit breaker faults, and at the same time establishes the association between fault information and components. Through the analysis of existing professional text mining technologies, it can be seen that this technology has been implemented in some fields. However, due to the design for professional texts, these technologies are difficult to apply across fields.

[0005] The chemical industry inspection reports are long, organized in a special way, and use professional terms. Manual statistics of many years of inspection reports are time-consuming and laborious. General methods cannot effectively extract useful information, and existing professional text mining methods cannot be applied to the automated information extraction of this report. Summary of the invention

[0006] The purpose of the embodiment of the present invention is to provide a text mining method and system for the chemical industry inspection report. Through sentiment analysis, the chapter with the most negative text description in the chemical industry inspection report is analyzed. This chapter reflects the problems existing in the enterprise when it was inspected that year. Through word segmentation and word frequency statistics, the existing problems and objects are extracted and associated, and structured enterprise problem statistics are output, thereby intuitively reflecting the existing problems and providing information support for regulatory decisions.

[0007] In order to achieve the above object, the present invention provides a text mining method for chemical industry inspection reports in a first aspect, the method comprising:

[0008] Identify the chapter structure of the chemical industry inspection report text;

[0009] Perform sentiment analysis on the text of the chemical industry inspection report according to the chapter structure, and obtain the chapter with the most negative sentiment analysis result;

[0010] Segmenting the chapter with the most negative sentiment analysis result to obtain the vocabulary and corresponding part-of-speech tags for each paragraph in the chapter;

[0011] The above words and corresponding part-of-speech tags are integrated and counted to obtain a summary of the safety management problems existing in each enterprise. Through sentiment analysis, the most negative chapters in the sentiment analysis results are identified, and the most negative chapters are segmented and word frequency statistics are performed to obtain the problems existing in each enterprise in the inspection report, providing information support for regulatory decision-making.

[0012] Furthermore, the chapter structure of the identification of the chemical industry inspection report text includes:

[0013] Identify paragraphs in the text of the chemical industry inspection report;

[0014] Read the chapter index of the chemical industry inspection report text;

[0015] According to the title structure in the chapter index, each paragraph is grouped into the corresponding third-level title to obtain the chapter structure of the chemical industry inspection report text. The chapter index includes first-level titles, second-level titles and third-level titles. Generally, the first-level title is used as a chapter, the second-level title is used as a section, and the third-level title is used as a subsection. By grouping each paragraph into the third-level title, the tree-shaped chapter structure of the entire report text is obtained, which is convenient for word frequency statistics in subsequent word segmentation analysis.

[0016] Furthermore, the sentiment analysis of the chemical industry inspection report text is performed according to the chapter structure to obtain the chapter with the most negative sentiment analysis result, including:

[0017] Perform sentiment analysis on each paragraph in the chemical industry inspection report to obtain a sentiment analysis score for each paragraph;

[0018] Calculate the sentiment analysis score k of each chapter based on the sentiment analysis scores of each paragraph j ,

[0019] Where P is the number of chapters in the chemical industry inspection report, n j is the number of paragraphs in the jth chapter, m j，i is the sentiment analysis score of the i-th paragraph in the j-th chapter;

[0020] The chapter with the most negative sentiment analysis result is determined based on the sentiment analysis score of each chapter. The sentiment analysis probability results of each paragraph are obtained through sentiment analysis. The larger the probability result, the more the text content tends to have positive sentiment. Conversely, the smaller the probability result, the more the text content tends to have negative sentiment. The chapter with the most negative sentiment analysis result reflects the description of the problems and shortcomings of the enterprise in the inspection report text. Determine the chapter with the most negative sentiment analysis result, and the subsequent word segmentation process directly processes the most negative chapter, reducing the amount of calculation and mining content quickly and accurately.

[0021] Furthermore, the paragraphs in the chemical industry inspection report are subjected to sentiment analysis one by one to obtain sentiment analysis scores for each paragraph, including:

[0022] Set the scoring rules for sentiment analysis results;

[0023] Perform sentiment analysis on each paragraph in the chemical industry inspection report to obtain sentiment analysis results for each paragraph;

[0024] The sentiment analysis score of each paragraph is obtained according to the sentiment analysis result scoring rule and the sentiment analysis result.

[0025] Optionally, the sentiment analysis scoring rule is:

[0026] If the sentiment analysis result is less than or equal to the preset threshold, the sentiment analysis score is 1; if the sentiment analysis result is greater than the preset threshold, the sentiment analysis score is 0;

[0027] Determining the chapter with the most negative sentiment analysis result according to the sentiment analysis scores of the respective chapters comprises:

[0028] The chapter with the highest sentiment analysis score is regarded as the chapter with the most negative sentiment analysis result.

[0029] Optionally, the sentiment analysis scoring rule is:

[0030] If the sentiment analysis result is less than or equal to the preset threshold, the sentiment analysis score is 0; if the sentiment analysis result is greater than the preset threshold, the sentiment analysis score is 1;

[0031] The step of obtaining a sentiment analysis score for each paragraph according to the sentiment analysis result scoring rule and the sentiment analysis result includes:

[0032] The chapter with the lowest sentiment analysis score is taken as the most negative chapter in the sentiment analysis result. The sentiment analysis result is a probability. By designing a scoring rule to normalize the probabilities, the differences between different probabilities bounded by a preset threshold are highlighted, and then the most negative chapter in the sentiment analysis result is calculated more accurately.

[0033] Furthermore, the chapter with the most negative sentiment analysis result is segmented to obtain the word division and corresponding part-of-speech tagging of each paragraph in the chapter, including:

[0034] Construct a text mining word segmentation library for chemical industry inspection reports as a custom dictionary;

[0035] The word segmentation model is used to segment each paragraph of the chapter with the most negative sentiment analysis results according to the dictionary in the search engine mode to obtain the word division of each paragraph and the corresponding part-of-speech tagging. The mining of the chemical industry inspection report is mainly to dig out the problems and deficiencies of the enterprise, so it is necessary to build a custom dictionary in a targeted manner so that the word segmentation can be correct, so as to obtain the relevant problems and deficiencies of the enterprise from the word segmentation results.

[0036] Optionally, the integrating statistics of the words and the corresponding part-of-speech tags to obtain a summary of the security management problems existing in each enterprise includes:

[0037] Create a paragraph statistics table to count the entity name words and their corresponding frequencies in each paragraph;

[0038] Combine the statistics of all paragraphs under the same third-level heading to obtain a third-level heading statistics table;

[0039] Combine the statistical tables of all the third-level titles contained in the same second-level title to obtain the statistical table of the second-level title;

[0040] A statistical table of all the secondary titles and tertiary titles in the chapters with the most negative sentiment analysis results is formed to form a statistical table with entity names, chapter secondary titles, and chapter tertiary titles as columns, and a summary of the safety management problems existing in each enterprise is obtained. In the general inspection report, there is usually a special chapter for analyzing the problems existing in the general inspection. There are multiple sections in this chapter, involving multiple aspects. Each aspect is divided into multiple subsections for description according to different contents. Therefore, in the statistical process, by counting the entity name vocabulary and the corresponding frequency appearing in each third-level title, the deficiencies of the corresponding entity name vocabulary in the current content can be obtained; similarly, by counting the entity name vocabulary and the corresponding frequency appearing in each second-level title, the total number of problems existing in the corresponding entity name vocabulary in various aspects can be obtained, thereby realizing the mining of existing problems in the enterprise.

[0041] A second aspect of the present invention provides a text mining system for chemical industry inspection reports, the system comprising:

[0042] Chapter structure recognition unit, used to recognize the chapter structure of the chemical industry inspection report text;

[0043] A sentiment analysis unit, used to perform sentiment analysis on the text of the chemical industry inspection report according to the chapter structure, and obtain the chapter with the most negative sentiment analysis result;

[0044] A word segmentation unit is used to segment the chapter with the most negative sentiment analysis result to obtain the words and corresponding part-of-speech tags for each paragraph in the chapter; and

[0045] The result statistics unit is used to integrate and count the words and corresponding part-of-speech tags to obtain a summary of the safety management problems existing in each enterprise. The sentiment analysis unit identifies the chapters with the most negative sentiment analysis results, and the most negative chapters are segmented and word frequency statistics are performed to obtain the problems existing in each enterprise in the inspection report, providing information support for regulatory decisions.

[0046] Furthermore, the chapter structure identification unit includes:

[0047] A chapter index reading module is used to read the chapter index of the chemical industry inspection report text;

[0048] A paragraph recognition module, used to recognize paragraphs in the chemical industry inspection report text; and

[0049] The paragraph attribution division module is used to group each paragraph into its corresponding third-level heading according to the heading in the chapter index, so as to obtain the chapter structure of the chemical industry inspection report text. The chapter index includes first-level headings, second-level headings and third-level headings. Generally, the first-level heading is used as a chapter, the second-level heading is used as a section, and the third-level heading is used as a subsection. By grouping each paragraph into the third-level heading, the tree-shaped chapter structure of the entire report text is obtained, which is convenient for word frequency statistics in subsequent word segmentation analysis.

[0050] Furthermore, the sentiment analysis unit includes:

[0051] A paragraph sentiment analysis module performs sentiment analysis on each paragraph in the chemical industry inspection report to obtain a sentiment analysis score for each paragraph; and

[0052] A chapter score calculation module, used to calculate the sentiment analysis score of each chapter according to the scores of each paragraph;

[0053] The chapter score comparison module is used to determine the chapter with the most negative sentiment analysis results based on the sentiment analysis scores of the chapters. The sentiment analysis probability results of each paragraph are obtained through sentiment analysis. The larger the probability result, the more the text content tends to have positive sentiment. Conversely, the smaller the probability result, the more the text content tends to have negative sentiment. The chapter with the most negative sentiment analysis results reflects the description of the problems and shortcomings of the enterprise in the inspection report text. Determine the chapter with the most negative sentiment analysis results, and the subsequent word segmentation process directly processes the most negative chapter, reducing the amount of calculation and mining content quickly and accurately.

[0054] Furthermore, the paragraph sentiment analysis module includes:

[0055] Scoring rule setting module, used to set scoring rules for sentiment analysis results;

[0056] A sentiment analysis module, used to perform sentiment analysis on each paragraph in the chemical industry inspection report to obtain sentiment analysis results for each paragraph; and

[0057] The sentiment scoring module is used to obtain the sentiment analysis score of each paragraph according to the sentiment analysis result scoring rule and the sentiment analysis result. The sentiment analysis result is a probability. By setting the scoring rule to score the sentiment analysis result, the probability is normalized, and the difference between different probabilities bounded by a preset threshold is highlighted, so that the chapter with the most negative sentiment analysis result is calculated more accurately.

[0058] Furthermore, the word segmentation unit includes:

[0059] Dictionary definition module, used to build a text mining word segmentation library for chemical industry inspection reports as a custom dictionary; and

[0060] The word segmentation module is used to segment each paragraph of the chapter with the most negative sentiment analysis results according to the dictionary in the search engine mode using the word segmentation model to obtain the word division of each paragraph and the corresponding part-of-speech tagging. The mining of the chemical industry inspection report is mainly to dig out the problems and deficiencies of the enterprise, so it is necessary to build a custom dictionary in a targeted manner to obtain the relevant problems and deficiencies of the enterprise through correct word segmentation.

[0061] Furthermore, the result statistics unit includes:

[0062] The paragraph statistics unit is used to create a paragraph statistics table to count the entity name words and their corresponding frequencies that appear in each paragraph;

[0063] The third-level title statistics unit is used to combine the statistics of all paragraphs under the same third-level title to obtain a third-level title statistics table;

[0064] The secondary title statistical unit is used to combine and count the statistical tables of all the third-level titles contained in the same secondary title to obtain the secondary title statistical table;

[0065] The chapter statistics unit is used to count the statistics of all the secondary titles and tertiary titles in the chapter with the most negative sentiment analysis results, forming a statistical table with entity names, chapter secondary titles, and chapter tertiary titles as columns, and obtaining a summary of the safety management problems existing in each enterprise. In the general inspection report, there is usually a special chapter for analyzing the problems existing in the general inspection. There are multiple sections in this chapter, involving multiple aspects. Each aspect is divided into multiple subsections for description according to different contents. Therefore, in the statistical process, by counting the entity name vocabulary and the corresponding frequency appearing in each third-level title, the deficiencies of the corresponding entity name vocabulary in the current content can be obtained; similarly, by counting the entity name vocabulary and the corresponding frequency appearing in each second-level title, the total number of problems existing in the corresponding entity name vocabulary in various aspects can be obtained, thereby realizing the mining of existing problems in the enterprise.

[0066] On the other hand, the present invention provides a machine-readable storage medium having instructions stored thereon, the instructions being used to enable a machine to execute the text mining method for the chemical industry inspection report.

[0067] Through the above technical scheme, the present invention analyzes the chapter with the most negative text description in the chemical industry inspection report through sentiment analysis. This chapter reflects the problems that existed when the enterprise was inspected that year. Through word segmentation and word frequency statistics, the existing problems and objects are extracted and associated, and structured enterprise problem statistics are output, thereby intuitively reflecting the existing problems and providing information support for regulatory decisions.

[0068] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0070] Figure 1 It is a flow chart of a text mining method for chemical industry inspection reports provided by an embodiment of the present invention;

[0071] Figure 2 It is a block diagram of a text mining system for chemical industry inspection reports provided by an embodiment of the present invention;

[0072] Figure 3is a tree structure diagram of document titles in one embodiment of the present invention;

[0073] Figure 4 is a safety inspection report k in one embodiment of the present invention max A schematic diagram summarizing the companies mentioned in each section and subsection of the chapter;

[0074] Figure 5 It is a schematic diagram summarizing various major problems existing in the enterprise during the safety inspection in one embodiment of the present invention. DETAILED DESCRIPTION

[0075] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present invention, and is not used to limit the present invention.

[0076] Figure 1 This is a flow chart of a text mining method for chemical industry inspection reports provided by an embodiment of the present invention. Figure 1 As shown, the method includes:

[0077] Identify the chapter structure of the chemical industry inspection report text;

[0078] Perform sentiment analysis on the text of the chemical industry inspection report according to the chapter structure, and obtain the chapter with the most negative sentiment analysis result;

[0079] Segmenting the chapter with the most negative sentiment analysis result to obtain the vocabulary and corresponding part-of-speech tags for each paragraph in the chapter;

[0080] The above words and corresponding part-of-speech tags are integrated and counted to obtain a summary of the safety management problems existing in each enterprise. Through sentiment analysis, the most negative chapters in the sentiment analysis results are identified, and the most negative chapters are segmented and word frequency statistics are performed to obtain the problems existing in each enterprise in the inspection report, providing information support for regulatory decision-making.

[0081] Furthermore, the chapter structure of the identification of the chemical industry inspection report text includes:

[0082] Identify paragraphs in the text of the chemical industry inspection report;

[0083] Read the chapter index of the chemical industry inspection report text;

[0084] According to the title structure in the chapter index, each paragraph is grouped into the corresponding third-level title to obtain the chapter structure of the chemical industry inspection report text. The chapter index includes first-level titles, second-level titles and third-level titles. Generally, the first-level title is used as a chapter, the second-level title is used as a section, and the third-level title is used as a subsection. By grouping each paragraph into the third-level title, the tree-shaped chapter structure of the entire report text is obtained, which is convenient for word frequency statistics in subsequent word segmentation analysis.

[0085] Furthermore, the sentiment analysis of the chemical industry inspection report text is performed according to the chapter structure to obtain the chapter with the most negative sentiment analysis result, including:

[0086] Perform sentiment analysis on each paragraph in the chemical industry inspection report to obtain a sentiment analysis score for each paragraph;

[0087] Calculate the sentiment analysis score k of each chapter based on the sentiment analysis scores of each paragraph j ,

[0088] Where P is the number of chapters in the chemical industry inspection report, n j is the number of paragraphs in the jth chapter, m j，i is the sentiment analysis score of the i-th paragraph in the j-th chapter;

[0089] The chapter with the most negative sentiment analysis result is determined based on the sentiment analysis score of each chapter. The sentiment analysis probability results of each paragraph are obtained through sentiment analysis. The larger the probability result, the more the text content tends to have positive sentiment. Conversely, the smaller the probability result, the more the text content tends to have negative sentiment. The chapter with the most negative sentiment analysis result reflects the description of the problems and shortcomings of the enterprise in the inspection report text. Determine the chapter with the most negative sentiment analysis result, and the subsequent word segmentation process directly processes the most negative chapter, reducing the amount of calculation and mining content quickly and accurately.

[0090] Furthermore, the paragraphs in the chemical industry inspection report are subjected to sentiment analysis one by one to obtain sentiment analysis scores for each paragraph, including:

[0091] Set the scoring rules for sentiment analysis results;

[0092] Perform sentiment analysis on each paragraph in the chemical industry inspection report to obtain sentiment analysis results for each paragraph;

[0093] The sentiment analysis score of each paragraph is obtained according to the sentiment analysis result scoring rule and the sentiment analysis result.

[0094] The sentiment analysis scoring rule is:

[0095] If the sentiment analysis result is less than or equal to the preset threshold, the sentiment analysis score is 1; if the sentiment analysis result is greater than the preset threshold, the sentiment analysis score is 0;

[0096] Determining the chapter with the most negative sentiment analysis result according to the sentiment analysis scores of the respective chapters comprises:

[0097] The chapter with the highest sentiment analysis score is regarded as the chapter with the most negative sentiment analysis result.

[0098] Optionally, the sentiment analysis scoring rule is:

[0099] If the sentiment analysis result is less than or equal to the preset threshold, the sentiment analysis score is 0; if the sentiment analysis result is greater than the preset threshold, the sentiment analysis score is 1;

[0100] The step of obtaining a sentiment analysis score for each paragraph according to the sentiment analysis result scoring rule and the sentiment analysis result includes:

[0101] The chapter with the lowest sentiment analysis score is taken as the most negative chapter in the sentiment analysis result. The sentiment analysis result is a probability. By designing a scoring rule to normalize the probabilities, the differences between different probabilities bounded by a preset threshold are highlighted, and then the most negative chapter in the sentiment analysis result is calculated more accurately.

[0102] Furthermore, the chapter with the most negative sentiment analysis result is segmented to obtain the word division and corresponding part-of-speech tagging of each paragraph in the chapter, including:

[0103] Construct a text mining word segmentation library for chemical industry inspection reports as a custom dictionary;

[0104] The word segmentation model is used to segment each paragraph of the chapter with the most negative sentiment analysis results according to the dictionary in the search engine mode to obtain the word division of each paragraph and the corresponding part-of-speech tagging. The mining of the chemical industry inspection report is mainly to dig out the problems and deficiencies of the enterprise, so it is necessary to build a custom dictionary in a targeted manner so that the word segmentation can be correct, so as to obtain the relevant problems and deficiencies of the enterprise from the word segmentation results.

[0105] Optionally, the integrating statistics of the words and the corresponding part-of-speech tags to obtain a summary of the security management problems existing in each enterprise includes:

[0106] Create a paragraph statistics table to count the entity name words and their corresponding frequencies in each paragraph;

[0107] Combine the statistics of all paragraphs under the same third-level heading to obtain a third-level heading statistics table;

[0108] Combine the statistical tables of all the third-level titles contained in the same second-level title to obtain the statistical table of the second-level title;

[0109] A statistical table of all the secondary titles and tertiary titles in the chapters with the most negative sentiment analysis results is formed to form a statistical table with entity names, chapter secondary titles, and chapter tertiary titles as columns, and a summary of the safety management problems existing in each enterprise is obtained. In the general inspection report, there is usually a special chapter for analyzing the problems existing in the general inspection. There are multiple sections in this chapter, involving multiple aspects. Each aspect is divided into multiple subsections for description according to different contents. Therefore, in the statistical process, by counting the entity name vocabulary and the corresponding frequency appearing in each third-level title, the deficiencies of the corresponding entity name vocabulary in the current content can be obtained; similarly, by counting the entity name vocabulary and the corresponding frequency appearing in each second-level title, the total number of problems existing in the corresponding entity name vocabulary in various aspects can be obtained, thereby realizing the mining of existing problems in the enterprise.

[0110] Figure 2 This is a block diagram of a text mining system for chemical industry inspection reports provided by an embodiment of the present invention. The system includes:

[0111] Chapter structure recognition unit, used to recognize the chapter structure of the chemical industry inspection report text;

[0112] A sentiment analysis unit, used to perform sentiment analysis on the text of the chemical industry inspection report according to the chapter structure, and obtain the chapter with the most negative sentiment analysis result;

[0113] A word segmentation unit is used to segment the chapter with the most negative sentiment analysis result to obtain the words and corresponding part-of-speech tags for each paragraph in the chapter; and

[0114] The result statistics unit is used to integrate and count the words and corresponding part-of-speech tags to obtain a summary of the safety management problems existing in each enterprise. The sentiment analysis unit identifies the chapters with the most negative sentiment analysis results, and the most negative chapters are segmented and word frequency statistics are performed to obtain the problems existing in each enterprise in the inspection report, providing information support for regulatory decisions.

[0115] Furthermore, the chapter structure identification unit includes:

[0116] A chapter index reading module is used to read the chapter index of the chemical industry inspection report text;

[0117] A paragraph recognition module, used to recognize paragraphs in the chemical industry inspection report text; and

[0118] The paragraph attribution division module is used to group each paragraph into its corresponding third-level heading according to the heading in the chapter index, so as to obtain the chapter structure of the chemical industry inspection report text. The chapter index includes first-level headings, second-level headings and third-level headings. Generally, the first-level heading is used as a chapter, the second-level heading is used as a section, and the third-level heading is used as a subsection. By grouping each paragraph into the third-level heading, the tree-shaped chapter structure of the entire report text is obtained, which is convenient for word frequency statistics in subsequent word segmentation analysis.

[0119] Furthermore, the sentiment analysis unit includes:

[0120] A paragraph sentiment analysis module performs sentiment analysis on each paragraph in the chemical industry inspection report to obtain a sentiment analysis score for each paragraph; and

[0121] A chapter score calculation module, used to calculate the sentiment analysis score of each chapter according to the scores of each paragraph;

[0122] The chapter score comparison module is used to determine the chapter with the most negative sentiment analysis results based on the sentiment analysis scores of the chapters. The sentiment analysis probability results of each paragraph are obtained through sentiment analysis. The larger the probability result, the more the text content tends to have positive sentiment. Conversely, the smaller the probability result, the more the text content tends to have negative sentiment. The chapter with the most negative sentiment analysis results reflects the description of the problems and shortcomings of the enterprise in the inspection report text. Determine the chapter with the most negative sentiment analysis results, and the subsequent word segmentation process directly processes the most negative chapter, reducing the amount of calculation and mining content quickly and accurately.

[0123] Furthermore, the paragraph sentiment analysis module includes:

[0124] Scoring rule setting module, used to set scoring rules for sentiment analysis results;

[0125] A sentiment analysis module, used to perform sentiment analysis on each paragraph in the chemical industry inspection report to obtain sentiment analysis results for each paragraph; and

[0126] The sentiment scoring module is used to obtain the sentiment analysis score of each paragraph according to the sentiment analysis result scoring rule and the sentiment analysis result. The sentiment analysis result is a probability. By setting the scoring rule to score the sentiment analysis result, the probability is normalized, and the difference between different probabilities bounded by a preset threshold is highlighted, so that the chapter with the most negative sentiment analysis result is calculated more accurately.

[0127] Furthermore, the word segmentation unit includes:

[0128] Dictionary definition module, used to build a text mining word segmentation library for chemical industry inspection reports as a custom dictionary; and

[0129] The word segmentation module is used to segment each paragraph of the chapter with the most negative sentiment analysis results according to the dictionary in the search engine mode using the word segmentation model to obtain the word division of each paragraph and the corresponding part-of-speech tagging. The mining of the chemical industry inspection report is mainly to dig out the problems and deficiencies of the enterprise, so it is necessary to build a custom dictionary in a targeted manner to obtain the relevant problems and deficiencies of the enterprise through correct word segmentation.

[0130] Furthermore, the result statistics unit includes:

[0131] The paragraph statistics unit is used to create a paragraph statistics table to count the entity name words and their corresponding frequencies that appear in each paragraph;

[0132] The third-level title statistics unit is used to combine the statistics of all paragraphs under the same third-level title to obtain a third-level title statistics table;

[0133] The secondary title statistical unit is used to combine and count the statistical tables of all the third-level titles contained in the same secondary title to obtain the secondary title statistical table;

[0134] The chapter statistics unit is used to count the statistics of all the secondary titles and tertiary titles in the chapter with the most negative sentiment analysis results, forming a statistical table with entity names, chapter secondary titles, and chapter tertiary titles as columns, and obtaining a summary of the safety management problems existing in each enterprise. In the general inspection report, there is usually a special chapter for analyzing the problems existing in the general inspection. There are multiple sections in this chapter, involving multiple aspects. Each aspect is divided into multiple subsections for description according to different contents. Therefore, in the statistical process, by counting the entity name vocabulary and the corresponding frequency appearing in each third-level title, the deficiencies of the corresponding entity name vocabulary in the current content can be obtained; similarly, by counting the entity name vocabulary and the corresponding frequency appearing in each second-level title, the total number of problems existing in the corresponding entity name vocabulary in various aspects can be obtained, thereby realizing the mining of existing problems in the enterprise.

[0135] In one embodiment of the present invention, the paragraphs method in the docx library is used to read the paragraphs in the document one by one, and store them in a UTF-8 encoded text document for later use. Then the style method in the docx library is used to determine the style of the paragraphs one by one, and the paragraphs with style types of "Heading 1", "Heading 2" and "Heading 3" are extracted. These paragraphs are the chapter indexes in the report text, and then a tree structure is created to store the logical structure of the paragraphs, with the document title as the root node, the first-level title (chapter) as the second-level node, the second-level title (section) as the third-level node, and the third-level title (subsection) as the fourth-level node, forming a tree structure as follows: Figure 3 In some embodiments, the first, second and third level headings of a paragraph are combined to encode the paragraph.

[0136] When performing sentiment analysis, first write a sentiment analysis function, use a loop to read the text documents that contain paragraph text one by one, use the SnowNLP method in the snownlp library to perform sentiment analysis on the read paragraphs, obtain the sentiment analysis probability results of the paragraphs and store them. For the encoded paragraphs, you can create an array to record the title encoding of the paragraph and the sentiment analysis probability results of the paragraph.

[0137] Next, we set the scoring rules to further process the sentiment analysis probability results. Since the SnowNLP method calculates the probability that the input text is a positive sentiment description, and the description of the object problems or deficiencies in the inspection report has a strong negative sentiment. Therefore, only paragraphs with lower scores contain valuable text information. In this embodiment, the preset threshold is 0.4, and the scoring rule used is: if the sentiment analysis result is less than or equal to 0.4, the score is 1, and if it is greater than 0.4, the score is 0. Then the final score k of the jth chapter j for:

[0138]

[0139] m j，i is the sentiment analysis score of the i-th paragraph of the j-th chapter, and the chapter with the highest score is k max , then the chapter with the highest score is k max The most negative chapters in the inspection report are described in the text. Text mining mainly focuses on the chapters with the highest scores as k max Conduct analysis.

[0140] In this embodiment, the jieba library is selected to implement word segmentation. After obtaining the most negative chapters in the text description, a text mining word segmentation library for the chemical industry inspection report is constructed as a custom dictionary, and a text document in UTF-8 encoding format is created to store professional vocabulary. Each professional vocabulary entry occupies one line, and each entry contains a vocabulary text, a part-of-speech mark, and an initial value of word frequency. In the present invention, the vocabulary text in the word segmentation library is the entity name that needs to be identified, such as a company name or an equipment name. In order to facilitate duplication with the default part-of-speech mark of the jieba library and to be unrecognizable, a special part-of-speech mark nz is added to the vocabulary in the dictionary for unified identification.

[0141] Then, we will perform word segmentation. Before word segmentation, we will first create a stop word dictionary, which includes punctuation marks, numbers, and English letters. Next, we will use the jieba library to load the stop word dictionary and read the constructed dictionary at the same time. Set the word segmentation mode of the jieba library to "search engine word segmentation mode" and perform word segmentation on the chapter k. max Each paragraph of the word segmentation is performed, and the word segmentation results of each paragraph obtained include the division of words and the corresponding part-of-speech tags. In some embodiments, by creating a queue structure, the words divided by the jieba library are imported into the word segmentation result queue one by one, and similarly, the part-of-speech tags of the words are imported into the part-of-speech tag queue.

[0142] Then use the DataFrame library to create a table structure with 3 columns and unlimited rows, and the three columns of the table structure record vocabulary, part of speech and word frequency respectively. The division of words in the word segmentation result and the corresponding part of speech tagging are filled in the table one by one, and the word frequency initial value of all professional vocabulary is set to 1. In the embodiment of the queue structure created, the two queues storing the word segmentation result and the part of speech tagging result can be directly imported into the first two columns of the table structure respectively, and at the same time, the word frequency initial value of all professional vocabulary is set to 1. Then filter out the vocabulary with the "nz" mark in the table structure according to the part of speech tagging of the vocabulary. Then use the groupby function of the DateFrame library to merge the rows with the same vocabulary and part of speech in the table structure, and update the word frequency to the word frequency column of the table structure. Finally, the word frequency statistics of each paragraph are sorted in descending order and stored in different files for standby.

[0143] Then, data integration is performed based on the chapter structure of the inspection report and the word segmentation results of each paragraph. The word segmentation results of paragraphs with the same third-level title index are put into the same set, and all words with nz part-of-speech tags are screened to form a list. The word frequency attribute is added, and the word frequency is counted and sorted for exactly the same words. The words with higher word frequency represent entities that appear more times in the paragraphs included in the third-level title. The statistics for the paragraphs included in the first-level and second-level titles are the same as those for the third-level titles.

[0144] In this embodiment, chapter k max The third-level title and entity name are used as the perspective for word frequency statistics. First, the statistics chapter k max The number of occurrences of each entity in each secondary title and tertiary title forms a statistical table with entities and word frequencies as columns. Take the statistical mentions of companies in the safety inspection report as an example. The fourth chapter of the safety inspection report, "Main Existing Problems", contains 9 sections describing nine aspects of the problem, and each section involves several sub-problems. The algorithm first divides the report from top to bottom, splitting the report into chapters, sections, and sub-sections to form independent documents. The document is fragmented, the company names therein are identified, and the frequency of each company name appearing in the text is counted. Then, from bottom to top, the number of times the company is mentioned is summarized by sub-section, section, and chapter level, and finally a layer-by-layer statistical chart is formed, such as Figure 4 shown.

[0145] Then count the entities mentioned in the inspection report in section k. max The number of occurrences of each secondary title and tertiary title in the paragraphs is formed by entity name, chapter k max Secondary title, chapter k max A statistical table with three-level headings as columns. Let’s take the case of companies mentioned in the safety inspection report as an example. The algorithm aggregates all word frequency statistical tables and generates a complete statistical table, and also aggregates statistics by company name. The results are as follows: Figure 5 In this summary table, the problems of the same enterprise in various safety management aspects are summarized in the same row. The value of each column represents the number of times the enterprise is mentioned in the corresponding safety management problem, while the null value indicates that the enterprise is not mentioned in this safety management problem.

[0146] Through the above method, statistical tables of problems existing in various enterprises can be mined and extracted from the large-scale inspection report, which can intuitively reflect the existing problems and provide information support for enterprise regulatory decisions.

[0147] An embodiment of the present invention further provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute the text mining method for the chemical industry inspection report.

[0148] Those skilled in the art will understand that all or part of the steps in the method for implementing the above-mentioned embodiments can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including several instructions for making a single-chip microcomputer, a chip or a processor (processor) perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0149] The optional embodiments of the present invention are described in detail above in conjunction with the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the above embodiments. Within the technical concept of the embodiments of the present invention, the technical scheme of the embodiments of the present invention can be subjected to a variety of simple modifications, and these simple modifications all belong to the protection scope of the embodiments of the present invention. It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the embodiments of the present invention will not further describe various possible combinations.

[0150] In addition, the various embodiments of the present invention may be arbitrarily combined, as long as they do not violate the concept of the embodiments of the present invention, they should also be regarded as the embodiments of the present invention.

[0151] The contents disclosed in the embodiments of the invention.

Claims

1. A text mining method for chemical industry inspection reports, characterized in that: The method comprises: Identify the chapter structure of the chemical industry inspection report text; Perform sentiment analysis on the text of the chemical industry inspection report according to the chapter structure, and obtain the chapter with the most negative sentiment analysis result; The chapter with the most negative sentiment analysis result is segmented to obtain the vocabulary and corresponding part-of-speech tags for each paragraph in the chapter: Construct a text mining word segmentation library for chemical industry inspection reports as a custom dictionary; Using the word segmentation model to segment each paragraph of the chapter with the most negative sentiment analysis result according to the dictionary in the search engine mode, to obtain the word division of each paragraph and the corresponding part-of-speech tagging; The above words and corresponding part-of-speech tags are integrated and counted to obtain a summary of the safety management problems existing in various enterprises, including: Create a paragraph statistics table to count the entity name words and their corresponding frequencies in each paragraph; Combine the statistics of all paragraphs under the same third-level heading to obtain a third-level heading statistics table; Combine the statistical tables of all the third-level titles contained in the same second-level title to obtain the statistical table of the second-level title; A statistical table of all the secondary and tertiary titles in the chapters with the most negative sentiment analysis results is formed to form a statistical table with entity name, chapter secondary title, and chapter tertiary title as columns, and a summary of the security management problems existing in each enterprise is obtained.

2. The text mining method for chemical industry inspection reports according to claim 1 is characterized in that: The chapter structure of the chemical industry inspection report text includes: Identify paragraphs in the text of the chemical industry inspection report; Read the chapter index of the chemical industry inspection report text; According to the title structure in the chapter index, each paragraph is grouped into its corresponding third-level title to obtain the chapter structure of the chemical industry inspection report text.

3. The text mining method for chemical industry inspection reports according to claim 1 is characterized in that: The sentiment analysis of the chemical industry inspection report text is performed according to the chapter structure to obtain the chapter with the most negative sentiment analysis results, including: Perform sentiment analysis on each paragraph in the chemical industry inspection report to obtain a sentiment analysis score for each paragraph; Calculate the sentiment analysis score k of each chapter based on the sentiment analysis scores of each paragraph j , Where P is the number of chapters in the chemical industry inspection report, n j is the number of paragraphs in the jth chapter, m j，i is the sentiment analysis score of the i-th paragraph in the j-th chapter; The chapter with the most negative sentiment analysis result is determined according to the sentiment analysis scores of the respective chapters.

4. The text mining method for chemical industry inspection reports according to claim 3 is characterized in that: The sentiment analysis is performed on each paragraph in the chemical industry inspection report to obtain a sentiment analysis score for each paragraph, including: Set the scoring rules for sentiment analysis results; Perform sentiment analysis on each paragraph in the chemical industry inspection report to obtain sentiment analysis results for each paragraph; The sentiment analysis score of each paragraph is obtained according to the sentiment analysis result scoring rule and the sentiment analysis result.

5. The text mining method for chemical industry inspection reports according to claim 4 is characterized in that: The sentiment analysis scoring rule is: If the sentiment analysis result is less than or equal to the preset threshold, the sentiment analysis score is 1; if the sentiment analysis result is greater than the preset threshold, the sentiment analysis score is 0; Determining the chapter with the most negative sentiment analysis result according to the sentiment analysis scores of the respective chapters comprises: The chapter with the highest sentiment analysis score is regarded as the chapter with the most negative sentiment analysis result.

6. The text mining method for chemical industry inspection reports according to claim 4 is characterized in that: The sentiment analysis scoring rule is: If the sentiment analysis result is less than or equal to the preset threshold, the sentiment analysis score is 0; if the sentiment analysis result is greater than the preset threshold, the sentiment analysis score is 1; The step of obtaining a sentiment analysis score for each paragraph according to the sentiment analysis result scoring rule and the sentiment analysis result includes: The chapter with the lowest sentiment analysis score is regarded as the chapter with the most negative sentiment analysis result.

7. A text mining system for chemical industry inspection reports, characterized in that: The system comprises: Chapter structure recognition unit, used to recognize the chapter structure of the chemical industry inspection report text; A sentiment analysis unit, used to perform sentiment analysis on the text of the chemical industry inspection report according to the chapter structure, and obtain the chapter with the most negative sentiment analysis result; A word segmentation unit is used to segment the chapter with the most negative sentiment analysis result to obtain the words and corresponding part-of-speech tags for each paragraph in the chapter; and A result statistics unit is used to integrate and count the words and corresponding part-of-speech tags to obtain a summary of the safety management problems existing in each enterprise; The word segmentation unit comprises: Dictionary definition module, used to build a text mining word segmentation library for chemical industry inspection reports as a custom dictionary; and A word segmentation module, used to use the word segmentation model to segment each paragraph of the chapter with the most negative sentiment analysis result according to the dictionary in the search engine mode, to obtain the word division of each paragraph and the corresponding part-of-speech tagging; The result statistics unit comprises: The paragraph statistics unit is used to create a paragraph statistics table to count the entity name words and their corresponding frequencies that appear in each paragraph; The third-level title statistics unit is used to combine the statistics of all paragraphs under the same third-level title to obtain the third-level title statistics table; The secondary title statistical unit is used to combine and count the statistical tables of all the third-level titles contained in the same secondary title to obtain the secondary title statistical table; The chapter statistics unit is used to compile a statistical table of all secondary and tertiary titles in the chapter with the most negative sentiment analysis results, forming a statistical table with entity name, chapter secondary title, and chapter tertiary title as columns, and obtaining a summary of the security management problems existing in each enterprise.

8. The text mining system for chemical industry inspection reports according to claim 7 is characterized in that: The chapter structure identification unit comprises: A chapter index reading module is used to read the chapter index of the chemical industry inspection report text; A paragraph recognition module, used to recognize paragraphs in the chemical industry inspection report text; and The paragraph attribution division module is used to group each paragraph into its corresponding third-level heading according to the heading in the chapter index, so as to obtain the chapter structure of the chemical industry inspection report text.

9. The text mining system for chemical industry inspection reports according to claim 7 is characterized in that: The sentiment analysis unit comprises: A paragraph sentiment analysis module performs sentiment analysis on each paragraph in the chemical industry inspection report to obtain a sentiment analysis score for each paragraph; and A chapter score calculation module, used to calculate the sentiment analysis score of each chapter according to the scores of each paragraph; The chapter score comparison module is used to determine the chapter with the most negative sentiment analysis result according to the sentiment analysis scores of the various chapters.

10. The text mining system for chemical industry inspection reports according to claim 9 is characterized in that: The paragraph sentiment analysis module includes: Scoring rule setting module, used to set the scoring rules for sentiment analysis results; A sentiment analysis module is used to perform sentiment analysis on each paragraph in the chemical industry inspection report to obtain sentiment analysis results for each paragraph; and The sentiment scoring module is used to obtain the sentiment analysis score of each paragraph according to the sentiment analysis result scoring rule and the sentiment analysis result.

11. A machine-readable storage medium having instructions stored thereon, the instructions being used to enable a machine to execute the text mining method for chemical industry inspection reports as described in any one of claims 1-6.

Images