Information extraction method and system based on cloud computing

By extracting information on the cloud computing platform and using distributed computing and parallel processing methods of subtasks, the problems of low efficiency and poor security of traditional information extraction methods are solved, and efficient and secure information extraction effect is achieved.

CN119475408APending Publication Date: 2025-02-18WUXI YUNJIANTONGYING FILM & TELEVISION TECHNOLOGY CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411248164.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-06
Publication Date
2025-02-18

AI Technical Summary

Technical Problem

The existing information extraction methods have problems such as complex rule design, difficulty in management, poor general use ability, and difficulty in dealing with complex languages. In addition, deep learning methods have high demand for computing resources, resulting in low information extraction efficiency and data security risks.

Method used

Using cloud computing-based information extraction method, data preprocessing and feature extraction are performed through user registration and login, information extraction model is trained using distributed computing, and tasks are split into multiple subtasks for parallel processing, enhancing the security of information extraction sex.

Benefits of technology

It improves the efficiency and security of information extraction, reduces the demand for computing resources, and enhances the data protection capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119475408A_ABST
    Figure CN119475408A_ABST
Patent Text Reader

Abstract

The invention discloses an information extraction method and system based on cloud computing, and the system comprises a user management module, an information management module and a user feedback module, the user management module is responsible for the registration and login of a user, collects the information of the user needing information extraction, stores the information after preprocessing, and the information management module is responsible for the information feedback module. The information extraction process is managed, an existing information extraction data set is used, an information extraction model is established through distributed calculation, information needing information extraction is subjected to feature extraction through a TF-IDF algorithm, features serve as input of the information extraction algorithm for information extraction, an information extraction task is divided into a plurality of sub-tasks, and the sub-tasks are subjected to information extraction. Sequence fragments in a specific sequence and corresponding labels are added for sub-tasks to enhance the security of information extraction, and a user feedback module is responsible for scoring information extraction and uploading collected feedback information to a management end, so that the method has the characteristics of improving the information extraction efficiency and the security of information extraction data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of information extraction, and in particular to an information extraction method and system based on cloud computing. Background Art

[0002] Information extraction is a technology that automatically identifies and extracts useful information from unstructured data, such as text, images, audio, video, etc. It converts raw data into structured or semi-structured data to facilitate subsequent analysis, storage and application. Generally speaking, information extraction technology mainly includes named entity recognition, attribute extraction, and event extraction.

[0003] Traditional information extraction methods are mostly based on rule-based methods, but this method generally has the problems of complex rule design, difficult management, poor generalization ability, and difficulty in processing complex languages. Existing information extraction methods mostly use deep learning methods for information extraction, but the use of deep learning methods has high requirements for computer resources, especially in large-scale pre-trained models. It takes a lot of time to train the information extraction model, resulting in low information extraction efficiency. In addition, during the process of information extraction by users, the results of information extraction are easily stolen and leaked. Therefore, it is very necessary to design a cloud computing-based information extraction method and system to improve information extraction efficiency and information extraction data security. Summary of the invention

[0004] The purpose of the present invention is to provide an information extraction method and system based on cloud computing to solve the problems raised in the above background technology.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: a cloud computing-based information extraction method, the specific method steps include:

[0006] Step S1: The user provides basic information including user name, password and email address to register and obtain the corresponding account. The system verifies the validity of the user's input through the email address and stores the user information in the database. The user logs in by entering the user name and password, and the system verifies the correctness of the password through the hash algorithm;

[0007] Step S2: The user provides the data for information extraction, the data is preprocessed, the features of the data are extracted using the TF-IDF algorithm, the information extraction model is trained using distributed computing, the extracted features are used as the input of the information extraction model, and the information extraction result is obtained;

[0008] Step S3: The extraction results are sent to the user, the user scores the extraction results, and feedback is collected from the user based on the scoring results.

[0009] According to the above technical solution, step S2 further includes the following steps:

[0010] Step S21: collect text data that users need to extract information from, save the data into a database, pre-process the data, divide the data into multiple terms, calculate the TF value Q and IDF value W corresponding to each term through the TF-IDF algorithm, multiply the TF value by the IDF value to obtain the TF-IDF value E=W×Q of the corresponding term, manually set the threshold E0 according to vertical experience, and use the terms that meet the condition E>E0 as the feature value of the data that needs to be extracted information. The system uses the existing public large-scale data set COTE, cleans the data as a data set for pre-training information extraction models, and stores it in a cloud server. Nodes required for distributed computing are arranged through cloud computing, and the distributed computing framework Apache Spark is installed on the nodes. By additionally installing Spark on each node, communication between the master node and the slave node is realized. The GPU arranged on each node through cloud computing realizes the training of the information extraction model. By using the pre-training data set and selecting an appropriate Transformer model, the number of layers, hidden layer size and number of attention heads of the model are configured, and the information extraction model is pre-trained through horizontal model parallelism and batch data parallelism, and the information extraction model is output;

[0011] Step S22: The characteristic value of the information extraction data is used as the input of the information extraction model. The system splits the information extraction task into multiple subtasks, and assigns different subtasks to different nodes for parallel processing. The system collects the GPU utilization A, memory utilization B, network bandwidth C, task, and execution time data D of each node in real time, and stores the data in the cloud server after normalization. The corresponding weight is manually set for each data based on historical experience, where the weight of GPU utilization is 0.4, the weight of memory utilization is 0.4, the weight of network bandwidth is 0.1, and the weight of task execution time is 0.1. The total load score of the calculated number of nodes is δ=0.4A+0.4B+0.1C+0.1D. The thresholds δ1 and δ2 are manually set based on historical experience, and δ1>δ2. When δ>δ1, the system judges the node as high load, δ1>δ>δ2, the system judges it as medium load, and δ<δ2, the system judges it as low load;

[0012] Step S23: When the system determines that the node is medium-loaded or low-loaded, the node performs the information extraction task normally. When the system determines that the node is highly loaded, the task that needs to be transferred is prioritized according to the urgency, importance, and expected completion time of the task to obtain the task priority σ. The threshold σ0 is manually set based on historical experience. When σ>σ0, the task is transferred to the low-load node. When σ>σ0, the task is transferred to the medium-load node. The system counts the number of high-load nodes t in real time. When the number of nodes t is greater than the threshold set by the system, the system expands cloud computing resources and increases the number of nodes. After the task is completed, the nodes that are no longer needed are automatically released. After each computing node completes the task, the system summarizes the extraction results of each part and stores the extracted results in the cloud server to support user access.

[0013] According to the above technical solution, step S23 further includes the following steps:

[0014] Step S231: The system splits the information extraction task into multiple subtasks and randomly generates a sequence S=S1, S2, ... S with a length of n. n , split the sequence into multiple S i , where i = 1, 2, ..., k, k is the number of subtasks, each S i Each corresponds to a unique label T i , label T in a certain order i With S i Assign to each subtask;

[0015] Step S232: When each subtask is completed, a result X with a label is returned. i =(S i ,T i ), the system collects all subtask results X i , according to the label T i The order of each subtask determines the result X i Part S i Correct position, press T i Combine the results of the subtasks in the order of o;

[0016] Step S233: Set the i The sequence fragment S corresponding to the subtasks combined in order i The new sequence S' is obtained by splicing in order, and the recombined S' is verified with the original sequence S. The task is completed only when S'=S. When the user needs to access the information extraction results, the system needs to provide the sequence M for verification. When M=S, the user can access the information extraction results.

[0017] According to the above technical solution, step S3 further includes the following steps:

[0018] Step S31: The system sends a request for scoring the information extraction result to the user, collects information extraction scoring data, and classifies the information extraction result into three levels: excellent, good, and average. The scoring criteria for the information extraction result include the precision rate of information extraction where α is the number of information correctly extracted by the model, β is the number of information wrongly extracted by the model, and the recall rate of information extraction γ is the number of information that should be extracted but not extracted by the model. By calculating the F1 value of information extraction to evaluate the comprehensive performance of information extraction. Thresholds Z1 and Z2 are set manually according to historical experience, and Z1 > Z2. When Z > Z1, the information extraction result is excellent; when Z1 > Z > Z2, the information extraction result is good; when Z < Z2, the information extraction result is average. When the information extraction result is excellent or good, the system works normally. When the information extraction result is average, the system collects samples with average information extraction results, marks the information that is difficult to extract in the sample information, adds the preprocessed sample information to the pre-training set of the information extraction model, retrains the information extraction model, outputs the optimized information extraction model, and uses the optimized information extraction model to perform information extraction again;

[0019] Step S32: The system collects feedback from the user and uploads the collected feedback to the management terminal.

[0020] According to the above technical solution, the system includes:

[0021] A user management module, which is responsible for user registration, login, and processing of information in the information extraction system;

[0022] An information management module, which is used to manage the information extraction process and implement information extraction;

[0023] A user feedback module, which is used to collect feedback information from users.

[0024] According to the above technical solution, the user management module includes:

[0025] A user registration unit, which is used for the user to provide basic information including username, password, and email for registration;

[0026] An information processing unit, which is used to manage the information collected in the information extraction system.

[0027] According to the above technical solution, the information management module includes:

[0028] An information extraction model, which is used to generate an information extraction model through distributed computing;

[0029] An information extraction unit, which is used to implement information extraction.

[0030] According to the above technical solution, the information processing unit:

[0031] A scoring unit, used for scoring the results of information extraction;

[0032] The feedback collection unit sends feedback requests to users, collects feedback information and uploads it to the management end.

[0033] According to the above technical solution, the information processing unit includes:

[0034] An information collection unit, used to collect information uploaded by users that needs to be extracted;

[0035] An information storage unit, used for storing information generated during the information extraction process;

[0036] The information preprocessing unit is used to preprocess the information data.

[0037] According to the above technical solution, the information extraction unit includes:

[0038] A TF-IDF feature extraction unit is used to extract feature values ​​from information data that needs to be extracted through a TF-IDF algorithm;

[0039] A feature processing unit, used for taking the extracted feature value as the input of the information extraction model to perform information extraction;

[0040] The information verification unit is used to increase the security of information extraction by dividing the information extraction task into multiple subtasks and adding characteristic sequence fragments and corresponding labels in a certain order.

[0041] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: first, the user registers the information extraction system, the information to be extracted is preprocessed and then feature extracted by using the TF-IDF algorithm, the extracted features are input into the information extraction model to realize information extraction, and after the information extraction is completed, the information extraction results are scored, and feedback information is collected according to the scoring results. The information extraction efficiency is improved by setting up a distributed computing training information extraction model and the information extraction task is divided into multiple subtasks. The security of information extraction is enhanced by adding sequence fragments of a specific order and corresponding labels to the subtasks, which has the characteristics of improving information extraction efficiency and information extraction data security. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0043] Figure 1 A flowchart of a cloud computing-based information extraction method provided in accordance with the first embodiment of the present invention

[0044] Figure 2 A schematic diagram of the module composition of a cloud computing-based information extraction system provided in Example 2 of the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] Embodiment 1: This embodiment can be applied to the scenario of information extraction. The method can be executed by a cloud computing-based information extraction method and system provided in Embodiment 2. The method specifically includes the following steps:

[0047] Step S1: The user provides basic information including user name, password and email address to register and obtain the corresponding account. The system verifies the validity of the user's input through the email address and stores the user information in the database. The user logs in by entering the user name and password, and the system verifies the correctness of the password through the hash algorithm;

[0048] Step S2: The user provides the data for information extraction, the data is preprocessed, the features of the data are extracted using the TF-IDF algorithm, the information extraction model is trained using distributed computing, the extracted features are used as the input of the information extraction model, and the information extraction result is obtained;

[0049] Step S3: The extraction results are sent to the user, the user scores the extraction results, and feedback is collected from the user based on the scoring results.

[0050] In an embodiment of the present invention, step S2 further includes the following steps:

[0051] Step S21: collect text data that users need to extract information from, save the data into a database, pre-process the data, divide the data into multiple terms, calculate the TF value Q and IDF value W corresponding to each term through the TF-IDF algorithm, multiply the TF value by the IDF value to obtain the TF-IDF value E=W×Q of the corresponding term, manually set the threshold E0 according to vertical experience, and use the terms that meet the condition E>E0 as the feature value of the data that needs to be extracted information. The system uses the existing public large-scale data set COTE, cleans the data as a data set for pre-training information extraction models, and stores it in a cloud server. Nodes required for distributed computing are arranged through cloud computing, and the distributed computing framework Apache Spark is installed on the nodes. By additionally installing Spark on each node, communication between the master node and the slave node is realized. The GPU arranged on each node through cloud computing realizes the training of the information extraction model. By using the pre-training data set and selecting an appropriate Transformer model, the number of layers, hidden layer size and number of attention heads of the model are configured, and the information extraction model is pre-trained through horizontal model parallelism and batch data parallelism, and the information extraction model is output;

[0052] Step S22: The characteristic value of the information extraction data is used as the input of the information extraction model. The system splits the information extraction task into multiple subtasks, and assigns different subtasks to different nodes for parallel processing. The system collects the GPU utilization A, memory utilization B, network bandwidth C, task, and execution time data D of each node in real time, and stores the data in the cloud server after normalization. The corresponding weight is manually set for each data based on historical experience, where the weight of GPU utilization is 0.4, the weight of memory utilization is 0.4, the weight of network bandwidth is 0.1, and the weight of task execution time is 0.1. The total load score of the calculated number of nodes is δ=0.4A+0.4B+0.1C+0.1D. The thresholds δ1 and δ2 are manually set based on historical experience, and δ1>δ2. When δ>δ1, the system judges the node as high load, δ1>δ>δ2, the system judges it as medium load, and δ<δ2, the system judges it as low load;

[0053] Step S23: When the system determines that the node is medium-loaded or low-loaded, the node performs the information extraction task normally. When the system determines that the node is highly loaded, the task that needs to be transferred is prioritized according to the urgency, importance, and expected completion time of the task to obtain the task priority σ. The threshold σ0 is manually set based on historical experience. When σ>σ0, the task is transferred to the low-load node. When σ>σ0, the task is transferred to the medium-load node. The system counts the number of high-load nodes t in real time. When the number of nodes t is greater than the threshold set by the system, the system expands cloud computing resources and increases the number of nodes. After the task is completed, the nodes that are no longer needed are automatically released. After each computing node completes the task, the system summarizes the extraction results of each part and stores the extracted results in the cloud server to support user access.

[0054] In the embodiment of the present invention, step S23 further includes the following steps:

[0055] Step S231: The system splits the information extraction task into multiple subtasks and randomly generates a sequence S=S1, S2, ... S with a length of n. n , split the sequence into multiple S i , where i = 1, 2, ..., k, k is the number of subtasks, each S i Each corresponds to a unique label T i , label T in a certain order i With S i Assign to each subtask;

[0056] Step S232: When each subtask is completed, a result X with a label is returned. i =(S i ,T i ), the system collects all subtask results X i , according to the label T i The order of each subtask determines the result X i Part S i Correct position, press T i Combine the results of the subtasks in the order of

[0057] Step S233: Set the i The sequence fragment S corresponding to the subtasks combined in order i The new sequence S' is obtained by splicing in order, and the recombined S' is verified with the original sequence S. The task is completed only when S'=S. When the user needs to access the information extraction results, the system needs to provide the sequence M for verification. When M=S, the user can access the information extraction results.

[0058] In an embodiment of the present invention, step S3 further includes the following steps:

[0059] Step S31: The system sends a request for scoring the information extraction result to the user, collects information extraction scoring data, and divides the information extraction result into three levels: excellent, good, and average. The scoring criteria for the information extraction result include the precision rate of information extraction where α is the number of information correctly extracted by the model, β is the number of information wrongly extracted by the model, and the recall rate of information extraction γ is the number of information that should be extracted but not extracted by the model. By calculating the F1 value of information extraction to evaluate the comprehensive performance of information extraction. According to historical experience, thresholds Z1 and Z2 are set manually, and Z1 > Z2. When Z > Z1, the information extraction result is excellent; when Z1 > Z > Z2, the information extraction result is good; when Z < Z2, the information extraction result is average. When the information extraction result is excellent or good, the system works normally. When the information extraction result is average, the system collects samples with an average information extraction result, marks the information that is difficult to extract in the sample information, adds the sample information to the pre-training set of the information extraction model after preprocessing, retrains the information extraction model, outputs the optimized information extraction model, and uses the optimized information extraction model to perform information extraction again;

[0060] Step S32: The system collects feedback from the user and uploads the collected feedback to the management end.

[0061] Embodiment 2: Embodiment 2 of the present invention provides an information extraction method and system based on cloud computing, Figure 1 which is a schematic diagram of the module composition of an information extraction method and system based on cloud computing provided in Embodiment 2 of the present invention. As Figure 2 shown, the system includes:

[0062] A user management module, which is responsible for user registration, login, and processing of information in the information extraction system;

[0063] An information management module, which is used to manage the information extraction process and implement information extraction;

[0064] A user feedback module, which is used to collect feedback information from users.

[0065] In some embodiments of the present invention, it includes:

[0066] A user registration unit, which is used for users to provide basic information including username, password, and email for registration;

[0067] An information processing unit, which is used to manage the information collected in the information extraction system.

[0068] In some embodiments of the present invention, it includes:

[0069] An information extraction model, used for generating an information extraction model through distributed computing;

[0070] The information extraction unit is used to realize information extraction.

[0071] In some embodiments of the present invention, comprising:

[0072] A scoring unit, used for scoring the results of information extraction;

[0073] The feedback collection unit sends feedback requests to users, collects feedback information and uploads it to the management end.

[0074] In some embodiments of the present invention, comprising:

[0075] An information collection unit, used to collect information uploaded by users that needs to be extracted;

[0076] An information storage unit, used for storing information generated during the information extraction process;

[0077] Information preprocessing unit, used to preprocess information data

[0078] In some embodiments of the present invention, comprising:

[0079] A TF-IDF feature extraction unit is used to extract feature values ​​from information data that needs to be extracted through a TF-IDF algorithm;

[0080] A feature processing unit, used for taking the extracted feature value as the input of the information extraction model to perform information extraction;

[0081] The information verification unit is used to increase the security of information extraction by dividing the information extraction task into multiple subtasks and adding characteristic sequence fragments and corresponding labels in a certain order.

[0082] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0083] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A cloud computing-based information extraction method, characterized in that: The specific method steps include: Step S1: The user provides basic information including user name, password and email address to register and obtain the corresponding account. The system verifies the validity of the user's input through the email address and stores the user information in the database. The user logs in by entering the user name and password, and the system verifies the correctness of the password through the hash algorithm; Step S2: The user provides the data for information extraction, the data is preprocessed, the features of the data are extracted using the TF-IDF algorithm, the information extraction model is trained using distributed computing, the extracted features are used as the input of the information extraction model, and the information extraction result is obtained; Step S3: The extraction results are sent to the user, the user scores the extraction results, and feedback is collected from the user based on the scoring results.

2. The cloud computing-based information extraction method according to claim 1, characterized in that: The step S2 further comprises the following steps: Step S21: collect text data that users need to extract information from, save the data into a database, pre-process the data, divide the data into multiple terms, calculate the TF value Q and IDF value W corresponding to each term through the TF-IDF algorithm, multiply the TF value by the IDF value to obtain the TF-IDF value E=W×Q of the corresponding term, manually set the threshold E0 according to vertical experience, and use the terms that meet the condition E>E0 as the feature value of the data that needs to be extracted information. The system uses the existing public large-scale data set COTE, cleans the data as a data set for pre-training information extraction models, and stores it in a cloud server. Nodes required for distributed computing are arranged through cloud computing, and the distributed computing framework Apache Spark is installed on the nodes. By additionally installing Spark on each node, communication between the master node and the slave node is realized. The GPU arranged on each node through cloud computing realizes the training of the information extraction model. By using the pre-training data set and selecting an appropriate Transformer model, the number of layers, hidden layer size and number of attention heads of the model are configured, and the information extraction model is pre-trained through horizontal model parallelism and batch data parallelism, and the information extraction model is output; Step S22: The characteristic value of the information extraction data is used as the input of the information extraction model. The system splits the information extraction task into multiple subtasks, and assigns different subtasks to different nodes for parallel processing. The system collects the GPU utilization A, memory utilization B, network bandwidth C, task, and execution time data D of each node in real time, and stores the data in the cloud server after normalization. The corresponding weight is manually set for each data based on historical experience, where the weight of GPU utilization is 0.4, the weight of memory utilization is 0.4, the weight of network bandwidth is 0.1, and the weight of task execution time is 0.

1. The total load score of the calculated number of nodes is δ=0.4A+0.4B+0.1C+0.1D. The thresholds δ1 and δ2 are manually set based on historical experience, and δ1>δ2. When δ>δ1, the system judges the node as high load, δ1>δ>δ2, the system judges it as medium load, and δ<δ2, the system judges it as low load; Step S23: When the system determines that the node is medium-loaded or low-loaded, the node performs the information extraction task normally. When the system determines that the node is highly loaded, the task that needs to be transferred is prioritized according to the urgency, importance, and expected completion time of the task to obtain the task priority σ. The threshold σ0 is manually set based on historical experience. When σ>σ0, the task is transferred to the low-load node. When σ<σ0, the task is transferred to the medium-load node. The system counts the number of high-load nodes t in real time. When the number of nodes t is greater than the threshold set by the system, the system expands the cloud computing resources and increases the number of nodes. After the task is completed, the nodes that are no longer needed are automatically released. After each computing node completes the task, the system summarizes the extraction results of each part and stores the extracted results in the cloud server to support user access.

3. The cloud computing-based information extraction method and system according to claim 2, characterized in that: The step S23 further comprises the following steps: Step S231: The system divides the information extraction task into multiple subtasks and randomly generates a sequence S=S1, S2, ...S with a length of n. n , split the sequence into multiple S i , where i = 1, 2, ..., k, k is the number of subtasks, and each S i Each corresponds to a unique label T i , label T in a certain order i With S i Assign to each subtask; Step S232: When each subtask is completed, a result X with a label is returned. i =(S i , T i ), the system collects all subtask results X i , according to the label T i The order of each subtask determines the result X i Part S i Correct position, press T i Combine the results of the subtasks in the order of Step S233: Set the i The sequence fragment S corresponding to the subtasks combined in order i The new sequence S′ is obtained by splicing in sequence, and the reorganized S′ is verified with the original sequence S. The task is completed only when S′=S. When the user needs to access the information extraction results, the system needs to provide the sequence M for verification. When M=S, the user can access the information extraction results.

4. The cloud computing-based information extraction method and system according to claim 3, characterized in that: The step S3 further comprises the following steps: Step S31: The system sends a request for scoring the information extraction result to the user, collects information extraction scoring data, and divides the information extraction result into three levels: excellent, good, and average. The scoring criteria for the information extraction result include the precision rate of information extraction where α is the number of information correctly extracted by the model, β is the number of information wrongly extracted by the model, and the recall rate of information extraction γ is the number of information that should have been extracted but was not extracted by the model. By calculating the F1 value of information extraction to evaluate the comprehensive performance of information extraction. Thresholds Z1 and Z2 are set manually according to historical experience, and Z1 > Z2. When Z > Z1, the information extraction result is excellent; when Z1 > Z > Z2, the information extraction result is good; when Z < Z2, the information extraction result is average. When the information extraction result is excellent or good, the system works normally. When the information extraction result is average, the system collects samples with average information extraction results, marks the information that is difficult to extract in the sample information, adds the preprocessed sample information to the pre-training set of the information extraction model, retrains the information extraction model, outputs the optimized information extraction model, and uses the optimized information extraction model to perform information extraction again; Step S32: The system collects feedback from users, and uploads the collected feedback to the management end.

5. A cloud computing-based information extraction system, characterized in that: The system comprises: User management module is responsible for user registration, login and information extraction system information processing; Information management module, used to manage the information extraction process and realize information extraction; User feedback module, used to collect user feedback information.

6. The cloud computing-based information extraction method and system according to claim 5, characterized in that: The user management module comprises: User registration unit, used for users to provide basic information including user name, password and email address for registration; An information processing unit is used to manage the information collected in the information extraction system.

7. The cloud computing-based information extraction method and system according to claim 6, characterized in that: The information management module includes: An information extraction model, used for generating an information extraction model through distributed computing; The information extraction unit is used to realize information extraction.

8. The cloud computing-based information extraction method and system according to claim 7, characterized in that: The information processing unit: A scoring unit, used for scoring the results of information extraction; The feedback collection unit sends feedback requests to users, collects feedback information and uploads it to the management end.

9. The cloud computing-based information extraction method and system according to claim 8, characterized in that: The information processing unit comprises: An information collection unit, used to collect information uploaded by users that needs to be extracted; An information storage unit, used for storing information generated during the information extraction process; The information preprocessing unit is used to preprocess the information data.

10. The cloud computing-based information extraction method and system according to claim 9, characterized in that: The information extraction unit comprises: A TF-IDF feature extraction unit is used to extract feature values ​​from information data that needs to be extracted through a TF-IDF algorithm; A feature processing unit, used for taking the extracted feature value as the input of the information extraction model to perform information extraction; The information verification unit is used to increase the security of information extraction by dividing the information extraction task into multiple subtasks and adding characteristic sequence fragments and corresponding labels in a certain order.

Citation Information

Patent Citations

  • Management method for unstructured data based on cloud computing environment

    CN102012912A

  • Model management method and device, storage medium and electronic device

    CN110795529A

  • Task distribution method and device, electronic equipment and storage medium

    CN117742948A

  • Keyword extraction method for unstructured text

    CN117829143A

  • Text keyword extraction method, electronic device, and computer readable storage medium

    US20230136368A1