A GMP standard dynamic update and management system based on big data

Abstract

The present invention relates to the technical field of data update management, and in particular to a GMP standard dynamic update and management system based on big data, the system comprising: a construction module identifies several historical keywords of historical GMP standard texts as historical cluster centers, and constructs an initial GMP standard text library through clustering division; a text acquisition module obtains a target GMP standard text; a text processing module identifies several actual keywords of the target GMP standard text; an update module compares several actual keywords with several historical keywords to determine whether to update several historical cluster centers, compares the target GMP standard text with the historical GMP standard text, and updates the initial GMP standard text library; a feedback module adjusts a preset acquisition frequency based on the update difference between the target GMP standard text library and the initial GMP standard text library. The present invention improves the timeliness of GMP standard updates.

Description

Technical Field

[0001] The present invention relates to the technical field of data update management, and in particular to a GMP standard dynamic update and management system based on big data. Background Art

[0002] GMP standards are important regulations that must be followed by the pharmaceutical, food and other production industries. Timely updates of their content are crucial to ensuring product quality. However, traditional GMP standard management methods have problems such as delayed updates and incomplete information, making it difficult to adapt to rapidly changing market demands and technological developments. Therefore, it is particularly important to develop a system that can automatically and efficiently dynamically update GMP standards based on big data.

[0003] The patent document with Chinese patent publication number CN111932149A discloses a method, device and asset management system for dynamically updating maintenance standards. The method includes the steps of: receiving maintenance standards input by a user, and storing the maintenance standards in a pre-established maintenance business database; when performing maintenance operations, pulling the maintenance standards from the maintenance business database, and executing maintenance business according to the maintenance standards to obtain update data corresponding to the maintenance standards; and dynamically updating the maintenance standards in the maintenance business database according to the update data.

[0004] Existing technologies rely on regular manual review or fixed update cycles, resulting in a lag in data updates. Summary of the invention

[0005] To this end, the present invention provides a GMP standard dynamic update and management system based on big data, which can solve the problem of delayed data update.

[0006] To achieve the above objectives, the present invention provides a GMP standard dynamic update and management system based on big data, the system comprising:

[0007] A construction module is used to construct an initial GMP standard text library according to a number of historical clustering results, wherein the several historical clustering results are obtained by clustering the historical GMP standard texts based on a number of historical keywords identified by the construction module; the construction module is also used to use the several historical keywords identified based on the historical GMP standard texts as a number of historical clustering centers;

[0008] A text collection module, used to obtain target GMP standard text from the Internet of Things based on a preset collection frequency;

[0009] A text processing module, connected to the text acquisition module, for preprocessing the target GMP standard text and identifying a number of actual keywords in the processed target GMP standard text;

[0010] An updating module, connected to the text processing module, for determining whether to update the plurality of historical cluster centers to obtain updated cluster centers based on the comparison results of the plurality of actual keywords with the plurality of historical keywords, and for updating the initial GMP standard text library to obtain a target GMP standard text library based on the text comparison results of the target GMP standard text with the historical GMP standard text;

[0011] A feedback module is connected to the update module and is used to adjust the preset acquisition frequency based on the update difference between the target GMP standard text library and the initial GMP standard text library to obtain an adjusted acquisition frequency.

[0012] Furthermore, the building blocks include:

[0013] A keyword determination unit, used to determine a number of initial keywords based on a comparison result of a number of historical frequencies and a preset frequency, and to determine a number of said historical keywords according to a similarity calculation result obtained based on each of the initial keywords; wherein the said historical frequency is a frequency of occurrence of historical participles, and the historical participles are obtained by performing a participle processing on the said historical GMP standard text by the keyword determination unit;

[0014] A division unit connected to the keyword determination unit, for dividing the historical GMP standard text according to structural features, obtaining a plurality of historical text segments, respectively determining the frequency of occurrence of a plurality of historical keywords in the plurality of historical text segments, or comparing the similarity between the plurality of historical text segments and the plurality of historical keywords, and associating the plurality of historical text segments with the plurality of historical keywords based on the frequency statistics result or the similarity calculation result, so as to obtain a plurality of the historical clustering results;

[0015] The database construction unit is connected to the division unit and is used to store a number of the historical clustering results in a database according to a preset storage rule to obtain the initial GMP standard text library.

[0016] Furthermore, the keyword determination unit includes:

[0017] A word segmentation subunit is used to perform word segmentation processing on the historical GMP standard text based on a word segmentation tool, identify the parts of speech of a number of processed word segments, and extract a number of word segments corresponding to a noun as a number of historical word segments;

[0018] A frequency comparison subunit, connected to the word segmentation subunit, for counting the historical frequencies corresponding to a number of the historical word segmentations, and taking the historical word segmentations corresponding to the historical frequencies greater than or equal to the preset frequency as the initial keywords;

[0019] A first calculation subunit, connected to the frequency comparison subunit, for calculating the similarity between any initial keyword among the initial keywords and the remaining initial keywords among the initial keywords, to obtain a plurality of first calculated similarities;

[0020] A determination subunit is connected to the first calculation subunit, and is used to compare several of the first calculated similarities with a first preset similarity and a second preset similarity respectively. When the first calculated similarity is less than or equal to the first preset similarity, the two initial keywords corresponding to the first calculated similarity are both used as the historical keywords. When the first calculated similarity is greater than the first preset similarity, the one with a larger historical frequency among the two initial keywords corresponding to the first calculated similarity is selected as the historical keyword.

[0021] Furthermore, the division unit includes:

[0022] A structure recognition subunit, used for recognizing paragraph separators in the historical GMP standard text, and dividing the historical GMP standard text into a plurality of historical text segments based on the paragraph separators;

[0023] A statistical subunit, connected to the structure identification subunit, is used to identify and mark a number of historical keywords in a number of historical text segments, count the occurrence frequency of each historical keyword, and obtain a number of historical frequencies;

[0024] A second calculation subunit is connected to the statistical subunit, and is used to respectively calculate the similarity values ​​corresponding to the historical text segment and the historical keywords when any of the historical text segments does not contain the historical keywords, and obtain a plurality of second calculated similarities;

[0025] The clustering subunit is connected to the statistical subunit and the second calculation subunit, and is used to associate the historical text segment corresponding to the maximum frequency among several historical frequencies with the historical keywords, or to associate the historical text segment corresponding to the maximum similarity among several second calculated similarities with the historical keywords, so as to obtain several historical clustering results.

[0026] Furthermore, the update module includes:

[0027] A center updating unit, used for comparing any of the actual keywords with the plurality of the historical keywords one by one, obtaining a comparison result, and when the plurality of the historical keywords do not contain the actual keyword, calculating the similarity between the actual keyword and the plurality of the historical keywords, obtaining a plurality of third calculated similarities, comparing the third calculated similarities with the second preset similarities, determining whether to update the plurality of the historical cluster centers according to the comparison result, and obtaining an updated cluster center;

[0028] A clustering update unit is used to compare the target GMP standard text with the historical GMP standard text, mark the different parts of the target GMP standard text and the historical GMP standard text, obtain an updated text, and update the historical clustering result based on the updated text to update the initial GMP standard text library and obtain a target GMP standard text library.

[0029] Furthermore, the central updating unit includes:

[0030] a threshold comparison subunit, configured to compare the third calculated similarities with the second preset similarities, and generate a center update instruction when the third calculated similarities are less than the second preset similarities;

[0031] The center updating subunit is connected to the threshold comparison subunit, and is used to receive the center updating instruction, and add the actual keyword corresponding to the third calculated similarity as a new clustering center to the plurality of historical clustering centers to obtain an updated clustering center.

[0032] Furthermore, the cluster updating unit includes:

[0033] A comparison subunit, used to compare the target GMP standard text with the historical GMP standard text one by one, obtain a comparison result, and determine an updated text according to the comparison result, wherein the updated text includes a newly added text, a deleted text, and a modified text;

[0034] The clustering update subunit is connected to the comparison subunit and is used to formulate an update strategy according to the update text to update a number of the historical clustering results, to update the initial GMP standard text library, and to obtain a target GMP standard text library.

[0035] Furthermore, the feedback module includes:

[0036] a difference calculation unit, used for comparing the target GMP standard text library with the initial GMP standard text library, obtaining a text library comparison result, and determining a text difference value based on the text library comparison result;

[0037] The adjustment unit is connected to the difference calculation unit and is used to calculate an adjustment parameter based on the text difference value, and adjust the preset acquisition frequency based on the adjustment parameter to obtain an adjusted acquisition frequency.

[0038] Furthermore, the difference calculation unit includes:

[0039] A library comparison subunit, for comparing the target GMP standard text library with the initial GMP standard text library one by one, obtaining a text library comparison result, and identifying adding clusters, deleting clusters, and changing clusters based on the text library comparison result;

[0040] The difference calculation subunit is connected to the library comparison subunit, and is used to respectively determine the cluster proportions of the added cluster, the deleted cluster and the modified cluster in the initial GMP standard text library, and determine the text difference value based on several cluster proportions and preset weights.

[0041] Furthermore, the adjustment unit includes:

[0042] A parameter calculation subunit, used for comparing the text difference value with a preset difference value, and determining an adjustment parameter based on the comparison result;

[0043] The adjustment subunit is connected to the parameter calculation subunit and is used to adjust the preset acquisition frequency based on the adjustment parameter to obtain the adjusted acquisition frequency.

[0044] Compared with the prior art, the beneficial effects of the present invention are that, by setting up a construction module to obtain historical GMP standard data and identify historical keywords, an initial GMP standard database is constructed, and effective organization and classification of historical data is achieved. By using a cluster analysis method, the efficiency and accuracy of data processing are improved, and a reliable basis is provided for subsequent data updating and management. By setting the text acquisition module based on a preset acquisition frequency, the target GMP standard data is acquired in real time through the Internet of Things, thereby ensuring the timeliness and comprehensiveness of the data and improving the efficiency and accuracy of data acquisition. By setting up the data preprocessing step of the text processing module, noise and redundancy in the data can be eliminated, and data quality can be improved. The recognition of keywords is effective. It helps to quickly locate key information in the data and provide a basis for subsequent comparison and updating. By setting the update module, dynamic updating of GMP standards is realized, ensuring the timeliness and accuracy of the standards. By comparing the updated clustering results with the historical clustering results, the effect of the update is evaluated to provide a basis for subsequent adjustment and optimization. By setting the feedback module based on the update difference between the target GMP standard database and the initial GMP standard database, the preset collection frequency is adjusted, and the frequency of data collection is flexibly adjusted according to actual conditions, which further improves the efficiency and accuracy of data management. Through the collaborative work of various modules, dynamic updating and management of GMP standards are realized, which significantly improves the timeliness and accuracy of the standards.

[0045] In particular, by setting the keyword determination unit to determine the initial keywords based on the comparison results of the preset frequencies, low-frequency words and irrelevant words are effectively filtered, the accuracy and representativeness of the keywords are improved, the similarity between the initial keywords is further calculated, and the final historical keywords are determined accordingly, avoiding the redundancy and repetition of keywords, and providing a more accurate and effective keyword set for subsequent data processing and clustering division. By setting the division unit, it is helpful to decompose complex text content into small units that are easier to process and manage. By judging the frequency of occurrence of historical keywords in historical text segments or comparing the similarity between historical text segments and historical keywords, the effective association between text segments and keywords is achieved, which provides a reliable basis for subsequent clustering division, improves the efficiency of data processing, and ensures the accuracy and rationality of clustering results. By setting the database construction unit, centralized storage and management of data are achieved, which facilitates subsequent data updates and management. By storing data in the form of a database, the security and maintainability of the data are improved, the long-term availability and reliability of the data are ensured, and the accurate processing, effective division and reasonable storage of historical GMP standard data are achieved, providing a solid foundation for subsequent data updates and management.

[0046] In particular, by setting the word segmentation subunit, efficient word segmentation processing of the historical GMP standard text is achieved. Based on the part-of-speech recognition technology, noun participles are accurately screened out as historical participles, and non-noun interference items are effectively eliminated, laying a solid foundation for subsequent frequency statistics and keyword determination. By setting the frequency comparison subunit, high-frequency keywords are screened out as initial keywords by comparing the actual frequency of historical participles with a preset frequency threshold, thereby improving the pertinence of keyword screening. By setting the first calculation subunit, the initial keywords are further calculated for similarity, and by quantifying the degree of association between keywords, keywords with similar semantics are effectively identified and merged, avoiding redundancy and repetition in the keyword set. By setting the determination subunit, the calculated similarity is accurately judged according to the preset similarity threshold, ensuring that the historical keywords finally determined are both representative and non-redundant, providing an accurate keyword set for subsequent data processing.

[0047] In particular, by setting up the structure identification subunit, a detailed division of the structure of the historical GMP standard text is achieved. By identifying paragraph separators, the huge text content is broken down into multiple text segments with clear logic and relatively independent content, which provides convenience for subsequent processing and improves the efficiency and accuracy of the overall processing. The statistical subunit provides basic data for subsequent similarity calculation and clustering analysis. The second calculation subunit identifies text segments that are highly correlated with keywords by calculating similarity, which provides strong support for subsequent clustering analysis. The clustering subunit associates text segments with high similarity with historical keywords to form historical clustering results, thereby achieving efficient division and clustering processing of historical GMP standard texts and providing support for subsequent data analysis and management. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A structural block diagram of a GMP standard dynamic update and management system based on big data provided by an embodiment of the present invention;

[0049] Figure 2 A structural block diagram of the building blocks in the GMP standard dynamic update and management system based on big data provided by an embodiment of the present invention;

[0050] Figure 3 A structural block diagram of an update module in a GMP standard dynamic update and management system based on big data provided by an embodiment of the present invention;

[0051] Figure 4 A structural block diagram of the feedback module in the GMP standard dynamic update and management system based on big data provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0053] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0054] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is merely for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it should not be understood as a limitation on the present invention.

[0055] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0056] See also Figure 1 As shown, an embodiment of the present invention provides a GMP standard dynamic update and management system based on big data, the system comprising:

[0057] A construction module 10 is used to construct an initial GMP standard text library according to a number of historical clustering results, wherein the number of historical clustering results are obtained by clustering the historical GMP standard texts based on a number of historical keywords identified by the construction module; the construction module is also used to use the number of historical keywords identified based on the historical GMP standard texts as a number of historical clustering centers;

[0058] A text collection module 20, used to obtain a target GMP standard text from the Internet of Things based on a preset collection frequency;

[0059] A text processing module 30, connected to the text collection module 20, is used to pre-process the target GMP standard data and identify a number of actual keywords in the processed target GMP standard data;

[0060] An updating module 40, connected to the text processing module 30, for determining whether to update the plurality of historical cluster centers to obtain updated cluster centers based on the comparison results of the plurality of actual keywords with the plurality of historical keywords, and for updating the initial GMP standard text library to obtain a target GMP standard text library based on the text comparison results of the target GMP standard text with the historical GMP standard text;

[0061] The feedback module 50 is connected to the updating module 40, and is used to adjust the preset acquisition frequency based on the update difference between the target GMP standard database and the initial GMP standard database to obtain an adjusted acquisition frequency.

[0062] Specifically, the GMP (Good Manufacturing Practice) in the embodiment of the present invention is a quality management system that aims to ensure that the production processes in the pharmaceutical, medical device, food and other industries meet certain standards and specifications to ensure the quality, safety and effectiveness of the products. Different countries and regions have different GMP regulations and standards. In short, good manufacturing practice or GMP is a system composed of processes, procedures and documents that ensures that food, cosmetics and medicines are consistently produced and controlled according to set quality standards. "Quality Management Standards for Pig Slaughter" is the first quality management standard in the field of livestock and poultry slaughter in my country. It provides basic requirements and guidelines for the quality management of pig slaughter. The GMP standard data in the embodiment of the present invention refers to the "Quality Management Standards for Pig Slaughter".

[0063] Specifically, the preset collection frequency in the embodiment of the present invention is once a year.

[0064] Specifically, the text collection module of the embodiment of the present invention obtains the latest version of the GMP standard text on the Internet of Things based on a preset collection frequency as the target GMP standard text. For example, one year later, the Internet of Things text is retrieved to obtain the "Quality Management Standards for Swine Slaughter" corresponding to the latest release date.

[0065] Specifically, the text processing module of the embodiment of the present invention performs preprocessing on the target GMP standard text, including but not limited to: removing invalid information in the target GMP standard text, such as non-standard content such as advertisements.

[0066] Specifically, the embodiment of the present invention acquires historical GMP standard data and identifies historical keywords by setting up a construction module, constructs an initial GMP standard database, and realizes effective organization and classification of historical data. The efficiency and accuracy of data processing are improved by cluster analysis, and a reliable basis is provided for subsequent data updating and management. The text acquisition module is set to acquire target GMP standard data in real time through the Internet of Things based on a preset acquisition frequency, thereby ensuring the timeliness and comprehensiveness of the data and improving the efficiency and accuracy of data acquisition. The data preprocessing step of the text processing module can eliminate noise and redundancy in the data and improve data quality, and the recognition of keywords helps to quickly The key information in the positioning data provides a basis for subsequent comparison and updating. By setting the update module, the dynamic update of the GMP standard is realized, ensuring the timeliness and accuracy of the standard. By comparing the updated clustering results with the historical clustering results, the effect of the update is evaluated to provide a basis for subsequent adjustment and optimization. By setting the feedback module based on the update difference between the target GMP standard database and the initial GMP standard database, the preset collection frequency is adjusted, and the frequency of data collection is flexibly adjusted according to actual conditions, which further improves the efficiency and accuracy of data management. Through the collaborative work of various modules, the dynamic update and management of the GMP standard is realized, which significantly improves the timeliness and accuracy of the standard.

[0067] See also Figure 2 As shown, the building block 10 includes:

[0068] The keyword determination unit 11 is used to determine a number of initial keywords based on the comparison results of a number of historical frequencies and a preset frequency, and to determine a number of the historical keywords according to the similarity calculation results obtained based on each of the initial keywords; wherein the historical frequency is the frequency of occurrence of historical participles, and the historical participles are obtained by performing a participle processing on the historical GMP standard text by the keyword determination unit;

[0069] The division unit 12 is connected to the keyword determination unit 11, and is used to divide the historical GMP standard text according to the structural features, obtain a number of historical text segments, respectively determine the frequency of occurrence of a number of historical keywords in the number of historical text segments, or compare the similarity between the number of historical text segments and the number of historical keywords, and associate the number of historical text segments with the number of historical keywords based on the frequency statistics result or the similarity calculation result to obtain a number of the historical clustering results;

[0070] The database construction unit 13 is connected to the division unit 12 and is used to store a number of the historical clustering results in a database according to a preset storage rule to obtain the initial GMP standard text library.

[0071] Specifically, the preset storage rule described in the embodiment of the present invention is to take several historical keywords as storage entries, and store several historical text segments contained in their corresponding clustering results under the corresponding storage entries, wherein the several historical text segments are stored sequentially from large to small based on the similarity calculation between them and the storage entries.

[0072] Specifically, the embodiment of the present invention determines the initial keywords based on the comparison results of the preset frequencies by setting the keyword determination unit, effectively filters low-frequency words and irrelevant words, improves the accuracy and representativeness of the keywords, further calculates the similarity between the initial keywords, and determines the final historical keywords based on this, avoids redundancy and repetition of keywords, and provides a more accurate and effective keyword set for subsequent data processing and clustering division. The setting of the division unit helps to decompose complex text content into small units that are easier to process and manage. By judging the frequency of occurrence of historical keywords in historical text segments or comparing the similarity between historical text segments and historical keywords, an effective association between text segments and keywords is achieved, which provides a reliable basis for subsequent clustering division, improves the efficiency of data processing, and ensures the accuracy and rationality of clustering results. The setting of the database construction unit realizes centralized storage and management of data, which facilitates subsequent data updates and management. Storing data in the form of a database also improves the security and maintainability of the data, ensures the long-term availability and reliability of the data, and realizes accurate processing, effective division and reasonable storage of historical GMP standard data, providing a solid foundation for subsequent data updates and management.

[0073] Specifically, the keyword determination unit 11 includes:

[0074] A word segmentation subunit is used to perform word segmentation processing on the historical GMP standard text based on a word segmentation tool, identify the parts of speech of a number of processed word segments, and extract a number of word segments corresponding to a noun as a number of historical word segments;

[0075] A frequency comparison subunit, connected to the word segmentation subunit, for counting the historical frequencies corresponding to a number of the historical word segmentations, and taking the historical word segmentations corresponding to the historical frequencies greater than or equal to the preset frequency as the initial keywords;

[0076] A first calculation subunit, connected to the frequency comparison subunit, for calculating the similarity between any initial keyword among the initial keywords and the remaining initial keywords among the initial keywords, to obtain a plurality of first calculated similarities;

[0077] A determination subunit is connected to the first calculation subunit, and is used to compare several of the first calculated similarities with a first preset similarity and a second preset similarity respectively. When the first calculated similarity is less than or equal to the first preset similarity, the two initial keywords corresponding to the first calculated similarity are both used as the historical keywords. When the first calculated similarity is greater than the first preset similarity, the one with a larger historical frequency among the two initial keywords corresponding to the first calculated similarity is selected as the historical keyword.

[0078] Specifically, the preset frequency in the embodiment of the present invention is 2 / 3 of the maximum frequency among the several historical frequencies;

[0079] The first preset similarity is 30%.

[0080] Specifically, the determining subunit in the embodiment of the present invention further includes repeatedly calculating and determining the similarity between historical keywords until the similarity between any two historical keywords is less than a first preset similarity.

[0081] Specifically, a possible example of an embodiment of the present invention is that if the "Specifications for Quality Management of Swine Slaughter" are analyzed, the initial keywords selected are "live pigs", "slaughterhouse", "slaughterhouse", "designated pig slaughterhouse", and "personnel", among which the similarity between any two of "slaughterhouse", "slaughterhouse", and "designated pig slaughterhouse" is greater than the first preset similarity, then the one with the highest historical frequency among "slaughterhouse", "slaughterhouse", and "designated pig slaughterhouse" is selected as the final historical keyword, that is, "designated pig slaughterhouse", and the similarity between any two of "live pigs", "personnel" and "designated pig slaughterhouse" is less than the first preset similarity, so these three words are retained as historical keywords.

[0082] Specifically, the embodiment of the present invention realizes efficient word segmentation processing of the historical GMP standard text by setting the word segmentation subunit, and based on the part-of-speech recognition technology, accurately screens out noun participles as historical participles, effectively eliminates non-noun interference items, and lays a solid foundation for subsequent frequency statistics and keyword determination. By setting the frequency comparison subunit, high-frequency keywords are screened out as initial keywords by comparing the actual frequency of historical participles with the preset frequency threshold, thereby improving the pertinence of keyword screening. By setting the first calculation subunit, the initial keywords are further calculated for similarity, and by quantifying the degree of association between keywords, keywords with similar semantics are effectively identified and merged, thereby avoiding redundancy and repetition in the keyword set. By setting the determination subunit, the calculated similarity is accurately judged according to the preset similarity threshold, thereby ensuring that the historical keywords finally determined are both representative and non-redundant, thereby providing an accurate keyword set for subsequent data processing.

[0083] Specifically, the embodiment of the present invention also ensures the accuracy and stability of historical word segmentation and keyword screening by introducing a repeated calculation mechanism. This mechanism continuously corrects and optimizes the word segmentation and screening results in multiple iterations until the preset accuracy and stability standards are met. This refined processing method not only improves the accuracy of keyword extraction, but also enhances the system's ability to process complex text data.

[0084] Specifically, the division unit 12 includes:

[0085] A structure recognition subunit, used for recognizing paragraph separators in the historical GMP standard text, and dividing the historical GMP standard text into a plurality of historical text segments based on the paragraph separators;

[0086] A statistical subunit, connected to the structure identification subunit, is used to identify and mark a number of historical keywords in a number of historical text segments, count the occurrence frequency of each historical keyword, and obtain a number of historical frequencies;

[0087] A second calculation subunit is connected to the statistical subunit, and is used to respectively calculate the similarity values ​​corresponding to the historical text segment and the historical keywords when any of the historical text segments does not contain the historical keywords, and obtain a plurality of second calculated similarities;

[0088] The clustering subunit is connected to the statistical subunit and the second calculation subunit, and is used to associate the historical text segment corresponding to the maximum frequency among several historical frequencies with the historical keywords, or to associate the historical text segment corresponding to the maximum similarity among several second calculated similarities with the historical keywords, so as to obtain several historical clustering results.

[0089] Specifically, the embodiment of the present invention can identify the paragraph separators in the historical GMP standard text through regular expressions or specific text tags to ensure the accuracy of the division.

[0090] Specifically, when calculating the similarity between the historical text segment and the historical keyword, the second calculation subunit of the embodiment of the present invention may use a variety of algorithms, such as cosine similarity, Jaccard similarity, etc., to meet the needs of different text characteristics.

[0091] Specifically, in addition to association based on frequency and similarity, the embodiments of the present invention can also consider the logical relationship and semantic coherence between text segments in the clustering sub-units to perform more refined clustering. For example, multiple text segments describing the same topic or process can be grouped into one cluster to facilitate subsequent management and updating.

[0092] Specifically, the embodiment of the present invention realizes the detailed division of the historical GMP standard text structure by setting the structure identification subunit, and decomposes the huge text content into multiple text segments with clear logic and relatively independent content by identifying paragraph separators, which provides convenience for subsequent processing and improves the efficiency and accuracy of the overall processing. The statistical subunit provides basic data for subsequent similarity calculation and clustering analysis. The second calculation subunit identifies text segments that are highly correlated with keywords by calculating similarity, which provides strong support for subsequent clustering analysis. The clustering subunit associates text segments with high similarity with historical keywords to form historical clustering results, thereby realizing efficient division and clustering processing of historical GMP standard texts and providing support for subsequent data analysis and management.

[0093] See also Figure 3 As shown, the update module 40 includes:

[0094] The center updating unit 41 is used to compare any of the actual keywords with the plurality of the historical keywords one by one, obtain the comparison result, and when the plurality of the historical keywords do not contain the actual keyword, calculate the similarity between the actual keyword and the plurality of the historical keywords to obtain a plurality of third calculated similarities, compare the third calculated similarities with the second preset similarities, determine whether to update the plurality of the historical cluster centers according to the comparison result, and obtain the updated cluster center;

[0095] The clustering update unit 42 is used to compare the target GMP standard text with the historical GMP standard text, mark the different parts of the target GMP standard text and the historical GMP standard text, obtain the updated text, and update the historical clustering result based on the updated text to update the initial GMP standard text library and obtain the target GMP standard text library.

[0096] Specifically, in the embodiment of the present invention, the second preset similarity is 30%.

[0097] Specifically, the central update unit in the embodiment of the present invention adopts an efficient string matching algorithm, such as the KMP algorithm or the Rabin-Karp algorithm, to quickly complete the accurate comparison of keywords;

[0098] To calculate the similarity between the actual keyword and several historical keywords, a suitable similarity calculation method can be selected according to actual needs, such as edit distance, TF-IDF weighted similarity, etc., to accurately calculate the similarity between keywords.

[0099] Specifically, the embodiment of the present invention sets the center update unit to compare actual keywords with historical keywords to decide whether to update the historical cluster center, thereby effectively responding to newly emerging keywords or concepts and maintaining the adaptability of the text library. The cluster update unit further integrates the updated text with the historical clustering results, thereby achieving a comprehensive update of the historical GMP standard text library and ensuring the integrity and accuracy of the text library content. The dynamic update mechanism improves the practicality and reliability of the text library and also provides a more solid data foundation for subsequent data analysis and decision support.

[0100] Specifically, the central updating unit 41 includes:

[0101] a threshold comparison subunit, configured to compare the third calculated similarities with the second preset similarities, and generate a center update instruction when the third calculated similarities are less than the second preset similarities;

[0102] The center updating subunit is connected to the threshold comparison subunit, and is used to receive the center updating instruction, and add the actual keyword corresponding to the third calculated similarity as a new clustering center to the plurality of historical clustering centers to obtain an updated clustering center.

[0103] Specifically, the cluster updating unit 42 includes:

[0104] A comparison subunit, used to compare the target GMP standard text with the historical GMP standard text one by one, obtain a comparison result, and determine an updated text according to the comparison result, wherein the updated text includes a newly added text, a deleted text, and a modified text;

[0105] The clustering update subunit is connected to the comparison subunit and is used to formulate an update strategy according to the update text to update a number of the historical clustering results, to update the initial GMP standard text library, and to obtain a target GMP standard text library.

[0106] Specifically, the update strategy of the embodiment of the present invention includes, when the updated text is a newly added text, calculating the similarity between the newly added text and a number of updated cluster centers, associating the updated cluster center corresponding to the maximum similarity value in the calculation result with the newly added text information, and obtaining an updated cluster result;

[0107] When the updated text is a deleted text, the historical clustering result corresponding to the deleted text is removed from the initial GMP standard text library to obtain an updated clustering result;

[0108] When the updated text is a modified text, the corresponding historical clustering results are corrected or re-clustered according to the modified content to obtain an updated clustering result.

[0109] See also Figure 4 As shown, the feedback module 50 includes:

[0110] A difference calculation unit 51 is used to compare the target GMP standard text library with the initial GMP standard text library to obtain a text library comparison result, and determine a text difference value based on the text library comparison result;

[0111] The adjustment unit 52 is connected to the difference calculation unit 51, and is used to calculate an adjustment parameter based on the text difference value, and adjust the preset acquisition frequency based on the adjustment parameter to obtain an adjusted acquisition frequency.

[0112] Specifically, the embodiment of the present invention sets the difference calculation unit to perform a detailed analysis of the changes between the target GMP standard text library and the initial GMP standard text library, thereby quantifying the difference value between the text libraries, providing a basis for subsequent adjustments. The adjustment unit calculates the adjustment parameters based on the difference value through a preset algorithm and adjusts the preset acquisition frequency.

[0113] Specifically, if the difference value is large, it means that the text library has undergone significant changes. At this time, the collection frequency needs to be increased to capture new changes more quickly. On the contrary, if the difference value is small, it means that the text library is relatively stable and the collection frequency does not need to be adjusted. The dynamic adjustment mechanism ensures that the text library is updated in a timely and efficient manner, avoids waste of resources, and improves the flexibility and response speed of data processing.

[0114] Specifically, the difference calculation unit 51 includes:

[0115] A library comparison subunit, for comparing the target GMP standard text library with the initial GMP standard text library one by one, obtaining a text library comparison result, and identifying adding clusters, deleting clusters, and changing clusters based on the text library comparison result;

[0116] The difference calculation subunit is connected to the library comparison subunit, and is used to respectively determine the cluster proportions of the added cluster, the deleted cluster and the modified cluster in the initial GMP standard text library, and determine the text difference value based on several cluster proportions and preset weights.

[0117] Specifically, the embodiment of the present invention increases the proportion of clusters = the number of increased clusters / the total number of several historical clustering results;

[0118] Deleted cluster ratio = number of deleted clusters / total number of historical cluster results;

[0119] Change cluster ratio = number of changed clusters / total number of historical cluster results;

[0120] Text difference value = α×increased cluster proportion + β×deleted cluster proportion + γ×changed cluster proportion, where α, β, and γ are preset proportions, α+β+γ=1, α>β>γ.

[0121] Specifically, the adjustment unit 52 includes:

[0122] A parameter calculation subunit, used for comparing the text difference value with a preset difference value, and determining an adjustment parameter based on the comparison result;

[0123] The adjustment subunit is connected to the parameter calculation subunit and is used to adjust the preset acquisition frequency based on the adjustment parameter to obtain the adjusted acquisition frequency.

[0124] Specifically, the adjustment unit 52 in this embodiment of the present invention includes:

[0125] When the text difference value is greater than or equal to the preset difference value, the adjustment parameter=(text difference value-preset difference value) / preset difference value, and the adjustment acquisition frequency=adjustment parameter×preset acquisition frequency;

[0126] When the text difference value is less than the preset difference value, the adjustment parameter=1, and the adjustment acquisition frequency=adjustment parameter×preset acquisition frequency.

[0127] Specifically, the preset difference value in the embodiment of the present invention is 1 / 3.

[0128] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0129] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A GMP standard dynamic update and management system based on big data, characterized in that: include: A construction module is used to construct an initial GMP standard text library according to a number of historical clustering results, wherein the several historical clustering results are obtained by clustering the historical GMP standard texts based on a number of historical keywords identified by the construction module; the construction module is also used to use the several historical keywords identified based on the historical GMP standard texts as a number of historical clustering centers; A text collection module, used to obtain target GMP standard text from the Internet of Things based on a preset collection frequency; A text processing module, connected to the text acquisition module, for preprocessing the target GMP standard text and identifying a number of actual keywords in the processed target GMP standard text; An updating module, connected to the text processing module, for determining whether to update the plurality of historical cluster centers to obtain updated cluster centers based on the comparison results of the plurality of actual keywords with the plurality of historical keywords, and for updating the initial GMP standard text library to obtain a target GMP standard text library based on the text comparison results of the target GMP standard text with the historical GMP standard text; A feedback module connected to the update module, configured to adjust the preset acquisition frequency based on the update difference between the target GMP standard text library and the initial GMP standard text library to obtain an adjusted acquisition frequency; The update module includes: A center updating unit, used for comparing any of the actual keywords with the plurality of the historical keywords one by one, obtaining a comparison result, and when the plurality of the historical keywords do not contain the actual keyword, calculating the similarity between the actual keyword and the plurality of the historical keywords, obtaining a plurality of third calculated similarities, comparing the third calculated similarities with the second preset similarities, determining whether to update the plurality of the historical cluster centers according to the comparison result, and obtaining an updated cluster center; A clustering update unit, for comparing the target GMP standard text with the historical GMP standard text, marking different parts of the target GMP standard text and the historical GMP standard text, obtaining an updated text, and updating the historical clustering result based on the updated text to update the initial GMP standard text library and obtain a target GMP standard text library; The central updating unit comprises: a threshold comparison subunit, configured to compare the third calculated similarities with the second preset similarities, and generate a center update instruction when the third calculated similarities are less than the second preset similarities; A center updating subunit is connected to the threshold comparison subunit to receive the center updating instruction and add the actual keyword corresponding to the third calculated similarity as a new cluster center to the plurality of historical cluster centers to obtain an updated cluster center. The cluster updating unit comprises: A comparison subunit, used to compare the target GMP standard text with the historical GMP standard text one by one, obtain a comparison result, and determine an updated text according to the comparison result, wherein the updated text includes a newly added text, a deleted text, and a modified text; The clustering update subunit is connected to the comparison subunit and is used to formulate an update strategy according to the update text to update a number of the historical clustering results, to update the initial GMP standard text library, and to obtain a target GMP standard text library.

2. The GMP standard dynamic update and management system based on big data according to claim 1, characterized in that: The building blocks include: A keyword determination unit, used to determine a number of initial keywords based on a comparison result of a number of historical frequencies and a preset frequency, and to determine a number of said historical keywords according to a similarity calculation result obtained based on each of the initial keywords; wherein the said historical frequency is a frequency of occurrence of historical participles, and the historical participles are obtained by performing a participle processing on the said historical GMP standard text by the keyword determination unit; A division unit connected to the keyword determination unit, for dividing the historical GMP standard text according to structural features, obtaining a plurality of historical text segments, respectively determining the frequency of occurrence of a plurality of historical keywords in the plurality of historical text segments, or comparing the similarity between the plurality of historical text segments and the plurality of historical keywords, and associating the plurality of historical text segments with the plurality of historical keywords based on the frequency statistics result or the similarity calculation result, so as to obtain a plurality of the historical clustering results; The database construction unit is connected to the division unit and is used to store a number of the historical clustering results in a database according to a preset storage rule to obtain the initial GMP standard text library.

3. The GMP standard dynamic update and management system based on big data according to claim 2, characterized in that: The keyword determination unit comprises: A word segmentation subunit is used to perform word segmentation processing on the historical GMP standard text based on a word segmentation tool, identify the parts of speech of a number of processed word segments, and extract a number of word segments corresponding to a noun as a number of historical word segments; A frequency comparison subunit, connected to the word segmentation subunit, for counting the historical frequencies corresponding to a number of the historical word segmentations, and taking the historical word segmentations corresponding to the historical frequencies greater than or equal to the preset frequency as the initial keywords; A first calculation subunit, connected to the frequency comparison subunit, for calculating the similarity between any initial keyword among the initial keywords and the remaining initial keywords among the initial keywords, to obtain a plurality of first calculated similarities; A determination subunit is connected to the first calculation subunit, and is used to compare several of the first calculated similarities with a first preset similarity and a second preset similarity respectively. When the first calculated similarity is less than or equal to the first preset similarity, the two initial keywords corresponding to the first calculated similarity are both used as the historical keywords. When the first calculated similarity is greater than the first preset similarity, the one with a larger historical frequency among the two initial keywords corresponding to the first calculated similarity is selected as the historical keyword.

4. The GMP standard dynamic update and management system based on big data according to claim 3, characterized in that: The division unit comprises: A structure recognition subunit, used for recognizing paragraph separators in the historical GMP standard text, and dividing the historical GMP standard text into a plurality of historical text segments based on the paragraph separators; A statistical subunit, connected to the structure identification subunit, is used to identify and mark a number of historical keywords in a number of historical text segments, count the occurrence frequency of each historical keyword, and obtain a number of historical frequencies; A second calculation subunit is connected to the statistical subunit, and is used to respectively calculate the similarity values ​​corresponding to the historical text segment and the historical keywords when any of the historical text segments does not contain the historical keywords, and obtain a plurality of second calculated similarities; The clustering subunit is connected to the statistical subunit and the second calculation subunit, and is used to associate the historical text segment corresponding to the maximum frequency among several historical frequencies with the historical keywords, or to associate the historical text segment corresponding to the maximum similarity among several second calculated similarities with the historical keywords, so as to obtain several historical clustering results.

5. The GMP standard dynamic update and management system based on big data according to claim 1, characterized in that: The feedback module comprises: a difference calculation unit, used to compare the target GMP standard text library with the initial GMP standard text library, obtain a text library comparison result, and determine a text difference value based on the text library comparison result; The adjustment unit is connected to the difference calculation unit and is used to calculate an adjustment parameter based on the text difference value, and adjust the preset acquisition frequency based on the adjustment parameter to obtain an adjusted acquisition frequency.

6. The GMP standard dynamic update and management system based on big data according to claim 5, characterized in that: The difference calculation unit comprises: A library comparison subunit, for comparing the target GMP standard text library with the initial GMP standard text library one by one, obtaining a text library comparison result, and identifying adding clusters, deleting clusters, and changing clusters based on the text library comparison result; The difference calculation subunit is connected to the library comparison subunit, and is used to respectively determine the cluster proportions of the added cluster, the deleted cluster and the modified cluster in the initial GMP standard text library, and determine the text difference value based on several cluster proportions and preset weights.

7. The GMP standard dynamic update and management system based on big data according to claim 6, characterized in that: The adjustment unit comprises: A parameter calculation subunit, used for comparing the text difference value with a preset difference value, and determining an adjustment parameter based on the comparison result; The adjustment subunit is connected to the parameter calculation subunit and is used to adjust the preset acquisition frequency based on the adjustment parameter to obtain the adjusted acquisition frequency.

Images