Method and device for extracting structured information from text

Abstract

The present application provides a method and device for extracting structured information from a text. The method includes determining the attribute value of the text to be extracted based on the content classification label of the text to be extracted, determining the user behavior value of the target data block based on the user operation record of the target data block for the target data block in the text to be extracted, determining the comprehensive value score of the target data block based on the attribute value of the text to be extracted and the user behavior value of the target data block, selecting a regular expression corresponding to the extraction granularity based on the comprehensive value score of the target data block, extracting structured information from the target data block, and storing the extracted structured information in a structured storage system corresponding to the comprehensive value score. Therefore, even if there are multiple data blocks in the text to be extracted, structured information can be extracted from each data block by this method and stored in the corresponding structured storage system, solving the problems in the prior art.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and device for extracting structured information from text. Background Art

[0002] In the daily business processes of enterprise employees, business data is usually recorded in the form of unstructured text. However, when this business data needs to be stored, the storage system is usually a structured storage system. Therefore, in this storage process, it is crucial to extract business data from the unstructured text and convert it into structured information.

[0003] Currently, business data is typically extracted from unstructured text and converted into structured information through neural networks or regular expressions. For example, patent 202111279596.X discloses a method for extracting structured information from text based on a neural network and related equipment. This method provides a method for extracting structured information from text using a neural network. Patent CN202110701621.2 discloses a method, equipment, and storage medium for accurately extracting structured information from complex web pages. This patent discloses a method for extracting structured information using regular expressions.

[0004] However, considering that structured storage systems typically require classified storage of structured information, the various structured information extraction methods mentioned above can only directly extract structured information from text, which can easily lead to difficulties in subsequent storage. For example, it is easy to be difficult to determine the structured storage system that needs to store the extracted structured information. For example, in practical applications, structured information is usually stored separately through multiple structured storage systems. The current extraction method directly extracts structured information from text through neural networks and regular expressions. However, even if the same text usually contains a large number of data blocks, the structured information extracted from these data blocks may also need to be stored in different structured storage systems. The current extraction method obviously makes it difficult to achieve this classified storage. Summary of the Invention

[0005] The present invention provides a method and device for extracting structured information from text, which is used to solve the problem that it is difficult to achieve classified storage when extracting structured information from text in the prior art.

[0006] In one aspect, the present invention provides a method for extracting structured information from text, comprising:

[0007] Determining the attribute value of the text to be extracted based on the content classification label of the text to be extracted;

[0008] For a target data block in the text to be extracted, determining the user behavior value of the target data block according to the user operation record of the target data block; wherein the target data block is specifically any data block in the text to be extracted;

[0009] Determining a comprehensive value score of the target data block based on the attribute value of the text to be extracted and the user behavior value of the target data block;

[0010] According to the comprehensive value score of the target data block, a regular expression corresponding to the extraction granularity is selected to extract structured information from the target data block;

[0011] The extracted structured information is stored in a structured storage system corresponding to the comprehensive value score.

[0012] Preferably, the method further comprises presetting a regular expression for a coarse extraction granularity, a regular expression for a fine extraction granularity, and a preset threshold; and

[0013] According to the comprehensive value score of the target data block, a regular expression corresponding to the extraction granularity is selected to extract structured information from the target data block, specifically including:

[0014] Determining whether the comprehensive value score of the target data block is greater than the preset threshold;

[0015] When the comprehensive value score is greater than the preset threshold, a regular expression with fine extraction granularity is selected to extract structured information from the target data block; or

[0016] When the comprehensive value score is less than or equal to the preset threshold, a regular expression with a coarse extraction granularity is selected to extract structured information from the target data block.

[0017] Preferably, determining the attribute value of the text to be extracted based on the content classification label of the text to be extracted specifically includes:

[0018] Obtaining the internal importance level of the text to be extracted, and determining an importance score according to the internal importance level; wherein the importance score is used to characterize the importance of the text to be extracted within the enterprise;

[0019] Obtaining the network popularity of the content classification tag and determining a popularity score based on the network popularity; wherein the popularity score is used to represent the popularity of the content classification tag of the text to be extracted in online media;

[0020] The attribute value of the text to be extracted is determined according to the importance score and the popularity score.

[0021] Preferably, determining the attribute value of the text to be extracted based on the importance score and the popularity score specifically includes calculating the attribute value using the following formula:

[0022] ;

[0023] Among them: P attribute is the attribute value; I is the importance score; H is the popularity score; δ and γ are both adjustment coefficients less than 1, and γ is greater than δ.

[0024] Preferably, determining the user behavior value of the target data block according to the user operation record of the target data block specifically includes:

[0025] The user behavior value of the target data block is determined according to the modification frequency of the target data block, the time interval between the last modification of the target data block and the average viewing time of the target data block.

[0026] Preferably, determining the user behavior value of the target data block based on the modification frequency of the target data block, the time interval between the target data block and the last modification, and the average browsing time of the target data block specifically includes:

[0027] Determining a data activity score based on the modification frequency, determining a user importance score based on the average viewing time, and determining a data freshness of the target data block based on the time interval;

[0028] The data activity score and the user importance score are weightedly summed, and the result of the weighted sum is multiplied by the data freshness to obtain the user behavior value.

[0029] Preferably, determining the data activity score according to the modification frequency specifically includes:

[0030] Obtaining the maximum and minimum values ​​of the modification frequencies of each data block in the text to be extracted;

[0031] The modification frequency of the target data block, the maximum value, and the minimum value are normalized by a calculation formula to calculate the data activity score.

[0032] Preferably, determining the data freshness of the target data block according to the time interval specifically includes:

[0033] Determining whether the time interval is greater than the data validity period set for the target data block;

[0034] If so, the data freshness is determined to be 0; or,

[0035] If not, the data freshness is calculated using the following formula:

[0036] ;

[0037] Wherein, T is the data freshness; t is the time interval; t 有效期 The data validity period set for the target data block.

[0038] Preferably, determining the user attention score based on the average browsing time specifically includes:

[0039] Obtaining the average reading time of the text to be extracted;

[0040] By formula To determine the user importance score, where D is the user importance score; min is the operator for taking the minimum value; d is the average viewing time of the target data block; d total is the average reading time of the text to be extracted.

[0041] In another aspect, the present invention provides a device for extracting structured information from text, comprising:

[0042] A first determining unit, configured to determine the attribute value of the text to be extracted based on the content classification label of the text to be extracted;

[0043] A second determining unit is configured to determine, for a target data block in the text to be extracted, a user behavior value of the target data block based on a user operation record of the target data block; wherein the target data block is specifically any data block in the text to be extracted;

[0044] a third determining unit, configured to determine a comprehensive value score of the target data block based on the attribute value of the text to be extracted and the user behavior value of the target data block;

[0045] an extraction unit, configured to select a regular expression corresponding to an extraction granularity according to the comprehensive value score of the target data block, and extract structured information from the target data block;

[0046] The storage unit is used to store the extracted structured information in a structured storage system corresponding to the comprehensive value score.

[0047] The method for extracting structured information from a text provided by an embodiment of the present application includes determining the attribute value of the text to be extracted based on the content classification label of the text to be extracted, determining the user behavior value of the target data block based on the user operation record of the target data block for the target data block in the text to be extracted, determining the comprehensive value score of the target data block based on the attribute value of the text to be extracted and the user behavior value of the target data block, and then selecting a regular expression corresponding to the extraction granularity based on the comprehensive value score of the target data block, extracting structured information from the target data block, and then storing the extracted structured information in a structured storage system corresponding to the comprehensive value score. This method selects a regular expression corresponding to the extraction granularity by calculating the comprehensive value score of the target data block in the text to be extracted, extracts structured information from the target data block, and stores the structured information in a structured storage system corresponding to the comprehensive value score. Therefore, even if there are multiple data blocks in the text to be extracted, the method can extract structured information from each data block and store it in the corresponding structured storage system, thereby solving the problems in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 A schematic diagram of a process for extracting structured information from text provided by the present invention;

[0050] Figure 2 The present invention provides a structural block diagram of a device for extracting structured information from text;

[0051] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0052] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0053] As mentioned above, current extraction methods using neural networks and regular expressions both directly extract structured information from text. However, given that structured storage systems typically require classified storage of structured information, the structured information extracted using current extraction methods can be difficult to determine in which structured storage system the extracted information is stored. For example, in practical applications, multiple structured storage systems are typically used to store structured information separately. However, current extraction methods directly extract structured information from text using neural networks and regular expressions. However, text typically contains a large number of data blocks, and the structured information extracted from these data blocks may also need to be stored in different structured storage systems. This classified storage is clearly difficult to achieve using current extraction methods.

[0054] In view of this, the embodiments of the present application provide a method and apparatus for extracting structured information from text, which can be used to solve the problems in the prior art. For ease of understanding, the embodiments of the present application can be described as a whole here, wherein the method can be applied to software products. For example, the method provided in the embodiments of the present application can be designed as an application (i.e., a software product) and then installed on an electronic device such as a user terminal or a server terminal to apply the method provided in the embodiments of the present application.

[0055] For example, the software product can be installed on user-side electronic devices, including mobile phones, tablet computers, and other electronic devices, to execute the methods provided in the embodiments of the present application; the software product can also be installed on server-side electronic devices, including servers or server clusters, to execute the methods provided in the embodiments of the present application. In practical applications, the method can also be applied as a hardware product, for example, a dedicated hardware device can be designed to implement the method provided in the embodiments of the present application.

[0056] Figure 1 The present invention provides a method for extracting structured information from text. The method can be executed by a user-side electronic device or a server-side electronic device. Here, the method is described by taking a server (that is, a server-side electronic device) as an example. Figure 1 As shown, the method may include:

[0057] Step S11: Determine the attribute value of the text to be extracted based on the content classification label of the text to be extracted.

[0058] The text to be extracted can be any unstructured text (that is, the unstructured text mainly stores unstructured data), and structured information needs to be extracted from the unstructured text to be stored in a structured storage system. Of course, the text to be extracted includes multiple data blocks. For example, the text to be extracted can be a contract text, which includes multiple chapters, and these different chapters can be used as different data blocks; the text to be extracted can also be a news report, and different paragraphs of the news report can also be used as different data blocks; the text to be extracted can also be a text used to record business data during the daily business operations of corporate employees, and can be divided into multiple data blocks according to the time when the business data was recorded. The attribute value measures the value of the text to be extracted from the perspective of its overall attributes.

[0059] It should be noted that in actual applications, in order to facilitate the preservation of texts, it is usually necessary to classify the texts according to their content, so that the texts have corresponding content classification labels. For example, the content classification labels may include contract texts, business data texts, financial data texts, annual, monthly, and quarterly summary texts, case data, meeting minutes, etc. Therefore, in step S11, the content classification label of the text to be extracted can be first obtained, and then the attribute value of the text to be extracted can be determined. Specifically, the content classification label of the text to be extracted can be determined based on the text library in which the text to be extracted is stored or the file name, attribute value, etc. of the text to be extracted.

[0060] Of course, the specific method for determining the attribute value of the text to be extracted based on the content classification label of the text to be extracted can include first obtaining the internal importance level of the text to be extracted and determining an importance score based on the internal importance level, then obtaining the network popularity of the content classification label and determining a popularity score based on the network popularity, and then determining the attribute value of the text to be extracted based on the importance score and the popularity score. Specifically, enterprises usually classify files into importance levels to facilitate the management of file access permissions (usually to prevent leaks). Therefore, the internal importance level of a file reflects the business value, relevance, or decision-making influence of the file within the enterprise, which in turn affects the degree of impact on the enterprise after its leakage.

[0061] For example, if financial data texts and contract texts are leaked, it will have an extremely serious impact on the company's operations, so their importance level is usually extremely important; if business data texts and case data are leaked, it will also have a serious impact on the company's operations, so their importance level is usually very important; meeting minutes record important decisions and issue discussions within the company, and if leaked, it will have a certain impact on the company's operations, so its importance level is usually important.

[0062] Annual, monthly, and quarterly summary documents primarily summarize and analyze a company's work over the past year, month, or quarter, including business achievements, progress, and existing issues. While valuable for the company's internal management and decision-making, the direct impact of such documents upon disclosure is relatively small, primarily impacting the company's internal work pace and management efficiency. Therefore, annual, monthly, and quarterly summary documents are typically rated as moderately important.

[0063] Of course, there are some texts, including the company's management system record texts, brochures, etc., which usually need to be disclosed proactively. For example, companies proactively disclose brochures to promote their products. Therefore, their importance level is usually relatively unimportant.

[0064] Specifically, for the texts with content classification labels such as the company's contract texts, business data texts, financial data texts, annual, monthly and quarterly summary texts, case data, meeting minutes, etc., their importance levels can be divided as shown in Table 1.

[0065] Table 1

[0066]

[0067] Thus, in the present application, the internal importance level of the text to be extracted can be obtained first, for example, the content classification label of the text to be extracted can be obtained, and then its internal importance level can be determined according to the content classification label (for example, the internal importance level can be determined by querying Table 1 above), and then the importance score can be determined according to the internal importance level. For example, in actual applications, different internal importance levels can be assigned values ​​in advance, so that the corresponding importance score can be determined according to the internal importance level of the text to be extracted. Among them, since the importance score is determined according to the internal importance level of the text to be extracted, the importance score can be used to characterize the importance of the text to be extracted within the enterprise.

[0068] Regarding the specific method of obtaining the network popularity of the content classification tag of the text to be extracted and determining the popularity score based on the network popularity, for example, the network popularity of the content classification tag of the text to be extracted can be obtained by the following method, for example, the network search volume, social media interaction volume, and news report mention volume of the content classification tag can be obtained. Specifically, the network search volume, social media interaction volume, and news report mention volume of the content classification tag can be obtained from major search engines, major social media interaction platforms, and major online news media platforms, and then the network popularity can be calculated using Formula 1 as shown below:

[0069] P 热度=a×online search volume + b×social media interaction volume + c×news report mention volume Formula 1

[0070] In this formula, P 热度 The network popularity is calculated; a, b and c are preset weights respectively, among which the sizes of a, b and c can usually be set according to actual needs. For example, if the enterprise business pays more attention to news reports, the value of c can be set relatively high; if the enterprise business pays more attention to social media interaction, the value of b can be set relatively high; of course, if the enterprise business is more related to network search volume, the value of a can be set relatively high.

[0071] Specifically, for example, the content classification label of the text to be extracted is case data, that is, the case data of relevant cases related to the enterprise. For example, when the enterprise files a lawsuit with other enterprises or individuals due to a contract dispute, the online search volume of the relevant case in major search engines such as Baidu and 360 Search can be obtained (for example, the Baidu Index can be used as the online search volume); the social media interaction volume of major social media platforms such as QQ, WeChat, DingTalk, and Weibo, as well as the news report mentions of major online news media platforms such as Sina and Sohu, can be substituted into the above formula 1 to calculate the network popularity.

[0072] After obtaining the network popularity, the popularity score can be further determined based on the network popularity. For example, the popularity score can be determined by the following formula 2:

[0073] H=(P 热度 -P min ) / (P max -P min ) Formula 2

[0074] In Formula 2, H is the calculated heat score; P 热度 is the network popularity calculated by the above formula 1; P max The maximum value of the network popularity of the content classification label in each statistical period (for example, one week can be a statistical period) in the past year. For example, the network popularity of the content classification label in each week in the past year can be counted, and the maximum value is selected as the P max ;P min The minimum value of the network popularity of the content classification label in each statistical period in the past year. Similarly, the network popularity of the content classification label in each week in the past year can be counted, and the minimum value is selected as the P min .

[0075] Therefore, the network popularity P of the content classification label of the text to be extracted is calculated by the above formula 1. 热度Afterwards, the P 热度 Substituting this into Formula 2, the popularity score H of the content classification label of the text to be extracted is further calculated. Obviously, the popularity score H can be used to represent the popularity of the content classification label of the text to be extracted in online media.

[0076] After obtaining the importance score and popularity score H respectively in the above manner, the attribute value of the text to be extracted can be further determined based on the importance score and popularity score H. Since the importance score is used to characterize the importance of the text to be extracted within the enterprise, the popularity score H is used to characterize the content classification label of the text to be extracted and its popularity in online media, the attribute value of the text to be extracted can be measured from two perspectives: the importance within the enterprise and the popularity in online media.

[0077] In practical applications, the importance score and the heat score H can be substituted into the following formula 3 to calculate the attribute value of the text to be extracted:

[0078] Formula 3

[0079] In Formula 3, P 属性 is the calculated attribute value; I is the importance score; H is the heat score; δ and γ are adjustment coefficients less than 1, and γ is greater than δ. Among them, the attribute value P calculated by formula 3 is 属性 , combines the internal importance of the enterprise and the popularity in the network media to measure the value of the text to be extracted, and since γ is greater than δ, it reflects the internal importance of the text to be extracted to P 属性 The influence is greater, among which, through the formula three, the attribute value P under the 100-point system can be calculated 属性 .

[0080] Step S12: for the target data block in the text to be extracted, determine the user behavior value of the target data block according to the user operation record of the target data block.

[0081] Among them, the target data block is specifically any data block in the text to be extracted. For example, in actual applications, structured information can be extracted for each data block in the text to be extracted. At this time, each data block in the text to be extracted can be used as the target data block.

[0082] In actual applications, the user operation record may include the modification frequency of the target data block, the time interval between the last modification of the target data block, and the average reference time of the target data block. The modification frequency of the target data block refers to the frequency with which the user modifies the target data block. Specific modification methods may include adding new content to the target data block, deleting content in the target data block, and modifying content in the target data block. Generally speaking, the higher the modification frequency, the higher the data activity of the target data block, and thus the higher its importance in the text to be extracted. In actual applications, the number of times the user modifies the target data block can be recorded, and then the modification frequency of the target data block can be calculated by comparing it with the total time since the target data block was generated.

[0083] The time interval between the current time and the last modification of the target data block refers to the time interval between the current time and the last modification of the target data block. Specifically, the time interval can be calculated by determining the time when the target data block was last modified. In practical applications, a longer time interval indicates that the longer the time interval between the last modification of the target data block and the current time, and thus the target data block is less important in the text to be extracted. Conversely, a shorter time interval indicates that the target data block is more important in the text to be extracted.

[0084] The average viewing duration of a target data block refers to the average length of time a user spends viewing the target data block each time. In practical applications, the length of time a user spends viewing the target data block each time can be calculated by calculating the duration of each time the user swipes to the target data block on the user interface. The average viewing duration of the target data block can then be calculated by summing the total time spent viewing the target data block and dividing the sum by the number of views. Clearly, the greater the average viewing duration of a target data block, the more attention the user pays to it.

[0085] Therefore, in this application, for the target data block in the text to be extracted, the specific method of determining the user behavior value of the target data block based on the user operation record of the target data block can be to determine the user behavior value of the target data block based on the modification frequency of the target data block, the time interval between the last modification of the target data block, and the average browsing time of the target data block. Specifically, the data activity score can be first determined based on the modification frequency of the target data block, and the user importance score can be determined based on the average browsing time of the target data block, and the data freshness of the target data block can be determined based on the time interval. Then, the data activity score and the user importance score are weightedly summed, and the result of the weighted sum is multiplied by the data freshness to obtain the user behavior value, wherein the weights of the data activity score and the user importance score can generally be values ​​between 0 and 1, such as 0.3, 0.5, 0.75, etc.

[0086] The data freshness of the target data block can be calculated using the following formula:

[0087] Formula 4

[0088] In Formula 4, T is the data freshness of the target data block; t is the time interval; t 有效期 The data validity period set for the target data block can usually be set according to needs in actual applications, such as the data validity period t 有效期 It can be 1 year, half a year, etc. The data freshness T calculated by Formula 4 reflects the percentage of the remaining valid time after the target data block was last modified.

[0089] Of course, if t is greater than t 有效期 , indicating that the target data block has expired, and the value of T can be 0. Therefore, in practical applications, we can first determine whether the time interval is greater than the data validity period. If so, the data freshness value is 0. If not, the data freshness T can be calculated using Formula 4.

[0090] In addition, the method of determining the data activity score based on the modification frequency of the target data block may include obtaining the maximum and minimum values ​​of the modification frequencies of each data block in the text to be extracted, and then calculating the data activity score by normalizing the modification frequency of the target data block, the maximum and minimum values ​​through a normalized calculation formula. The normalized calculation formula may be, for example, F=100×(ff min ) / (f max -f min ). In the normalized calculation formula, F is the calculated data activity score; f is the modification frequency of the target data block; fmin is the minimum value; f max is the maximum value.

[0091] The method of determining the user's attention score based on the average reading time of the target data block may include obtaining the average reading time of the text to be extracted (that is, the average of the time the user reads the text to be extracted each time), and then determining the user's attention score based on the average reading time of the text to be extracted and the average reading time of the target data block. Specifically, the formula To determine the user's importance score, in this formula, D is the user's importance score; min is the operator for taking the minimum value; d is the average viewing time of the target data block; d total The formula means that when the average reading time of the target data block is greater than the average reading time of the text to be extracted, the user's attention score is determined to be 100 (the full score is 100); when the average reading time of the target data block is less than or equal to the average reading time of the text to be extracted, the value of D is .

[0092] In this way, the above method can be used to calculate the data activity score, user attention score, and data freshness. Since all three are normalized to a 100-point scale, the data activity score and user attention score can be further weighted and summed, and the result of the weighted sum can be multiplied by the data freshness to obtain the user behavior value. The user behavior value calculated in this way is obviously different from the attribute value, which measures the value of the text to be extracted from the perspective of the overall attributes of the text to be extracted. The user behavior value measures the value of the target data block from the perspective of various user interactive behaviors.

[0093] Step S13: Determine the comprehensive value score of the target data block based on the attribute value of the text to be extracted and the user behavior value of the target data block.

[0094] Among them, according to the attribute value obtained in the above-mentioned step S11 of the present application, it is obvious that the attribute value measures the value of the text to be extracted from the level of the overall attributes of the text to be extracted, and the user behavior value obtained through the above-mentioned step S12 measures the value of the target data block from the perspective of various interactive behaviors of the user. Therefore, in this step S13, the two can be further integrated, that is, the comprehensive value score of the target data block can be determined based on the attribute value of the text to be extracted and the user behavior value of the target data block.

[0095] Specifically, the attribute value and the user behavior value can usually be weighted and summed to calculate the comprehensive value score of the target data block. The weights of the attribute value and the user behavior value can be determined according to actual needs, such as according to the company's own business type and the content classification label of the text to be extracted. For example, if the company pays more attention to sales-related business, the weight of the user behavior value can be increased. If the content classification label is contract text, financial data text, business data text, etc., the attribute value can be set relatively high. Of course, the two can also be set to the same weight.

[0096] Step S14: According to the comprehensive value score of the target data block, a regular expression corresponding to the extraction granularity is selected to extract structured information from the target data block.

[0097] It is important to note that this application provides regular expressions for various extraction granularities. Specifically, regular expressions for coarse and fine extraction granularity, as well as preset thresholds, can be pre-set. The coarse extraction granularity regular expression can, for example, be a lookahead-lookbehind assertion regular expression. This lookahead-lookbehind assertion regular expression includes lookahead and lookbehind statements, and the keywords in the lookahead-lookbehind assertion can be one or more. This is the simplest regular expression.

[0098] The regular expression with fine extraction granularity can be multiple sets of chained regular expressions or nested regular expressions, wherein the multiple sets of chained regular expressions refer to connecting multiple expressions by using pipe characters in regular expressions, and then treating the multiple expressions connected by pipe characters as a whole as a regular expression, that is, the multiple sets of chained regular expressions. Therefore, the multiple sets of chained regular expressions can, on the one hand, achieve finer granularity and more pattern matching through each expression; the nested regular expression refers to nesting another regular expression in a regular expression, thereby also achieving more complex and refined matching functions.

[0099] Therefore, in the present application, after obtaining the comprehensive value score of the target data block through the above-mentioned step S13, for the specific implementation method of step S14, it is possible to first determine whether the comprehensive value score of the target data block is greater than the preset threshold. At this time, when the comprehensive value score is greater than the preset threshold, it means that the target data block has a relatively high value for extracting structured information. Therefore, a regular expression with fine extraction granularity can be selected to extract structured information from the target data block; or, when the comprehensive value score is less than or equal to the preset threshold, a regular expression with coarse extraction granularity can be selected to extract structured information from the target data block.

[0100] The focus of step S14 in this application is to select a regular expression of corresponding extraction granularity based on the comprehensive value score of the target data block, and then extract structured information from the target data block. However, as for how to generate a regular expression and how to use the regular expression to extract structured information from the data block, in actual applications, it can be implemented using methods in the existing technology, and this is not limited here.

[0101] Step S15: The extracted structured information is stored in a structured storage system corresponding to the comprehensive value score.

[0102] It should be further explained that the present application includes multiple structured storage systems, and these different structured storage systems are respectively used to store structured information between different comprehensive value partitions. For example, structured storage system 1 is used to store structured information with a comprehensive value partition interval of [P1, P2), structured storage system 2 is used to store structured information with a comprehensive value partition interval of [P2, P3), and structured storage system 3 is used to store structured information with a comprehensive value partition interval of [P3, P4)... In this way, after the structured information is extracted through the above-mentioned step S14, in this step S15, the structured storage system corresponding to the comprehensive value score can be determined according to the interval into which the above-mentioned comprehensive value score falls, and then the extracted structured information can be stored in the structured storage system.

[0103] The method for extracting structured information from a text provided by an embodiment of the present application includes determining the attribute value of the text to be extracted based on the content classification label of the text to be extracted, determining the user behavior value of the target data block based on the user operation record of the target data block for the target data block in the text to be extracted, determining the comprehensive value score of the target data block based on the attribute value of the text to be extracted and the user behavior value of the target data block, and then selecting a regular expression corresponding to the extraction granularity based on the comprehensive value score of the target data block, extracting structured information from the target data block, and then storing the extracted structured information in a structured storage system corresponding to the comprehensive value score. This method selects a regular expression corresponding to the extraction granularity by calculating the comprehensive value score of the target data block in the text to be extracted, extracts structured information from the target data block, and stores the structured information in a structured storage system corresponding to the comprehensive value score. Therefore, even if there are multiple data blocks in the text to be extracted, the method can be used to extract structured information from each data block and store it in a structured storage system corresponding to the value, thereby solving the problems in the prior art.

[0104] Based on the same inventive concept as the method for extracting structured information from text provided in the embodiment of the present application, the embodiment of the present application can also provide a device for extracting structured information from text. For any unclear content in the embodiment of the device, please refer to the relevant content in the above method embodiment. Figure 2 FIG. 2 is a schematic diagram of a specific structure of the apparatus 20 , which includes: a first determination unit 201 , a second determination unit 202 , a third determination unit 203 , an extraction unit 204 , and a storage unit 205 , wherein:

[0105] The first determining unit 201 is configured to determine the attribute value of the text to be extracted based on the content classification label of the text to be extracted;

[0106] A second determining unit 202 is configured to determine, for a target data block in the text to be extracted, a user behavior value of the target data block based on a user operation record of the target data block; wherein the target data block is specifically any data block in the text to be extracted;

[0107] The third determining unit 203 is configured to determine the comprehensive value score of the target data block based on the attribute value of the text to be extracted and the user behavior value of the target data block;

[0108] An extraction unit 204 is configured to select a regular expression corresponding to an extraction granularity based on the comprehensive value score of the target data block, and extract structured information from the target data block;

[0109] The storage unit 205 is used to store the extracted structured information in a structured storage system corresponding to the comprehensive value score.

[0110] The device 20 provided in the embodiment of the present application is used. Since the device 20 adopts the same inventive concept as the method provided in the embodiment of the present application, when the method can solve the problems in the prior art, the device 20 can also solve the problems in the prior art, which will not be repeated here.

[0111] The device 20 may further include a setting unit for presetting a regular expression for coarse extraction granularity, a regular expression for fine extraction granularity, and a preset threshold; and

[0112] According to the comprehensive value score of the target data block, a regular expression corresponding to the extraction granularity is selected to extract structured information from the target data block, specifically including:

[0113] Determining whether the comprehensive value score of the target data block is greater than the preset threshold;

[0114] When the comprehensive value score is greater than the preset threshold, a regular expression with fine extraction granularity is selected to extract structured information from the target data block; or

[0115] When the comprehensive value score is less than or equal to the preset threshold, a regular expression with a coarse extraction granularity is selected to extract structured information from the target data block.

[0116] Determining the attribute value of the text to be extracted based on the content classification label of the text to be extracted may specifically include:

[0117] Obtaining the internal importance level of the text to be extracted, and determining an importance score according to the internal importance level; wherein the importance score is used to characterize the importance of the text to be extracted within the enterprise;

[0118] Obtaining the network popularity of the content classification tag and determining a popularity score based on the network popularity; wherein the popularity score is used to represent the popularity of the content classification tag of the text to be extracted in online media;

[0119] The attribute value of the text to be extracted is determined according to the importance score and the popularity score.

[0120] Determining the attribute value of the text to be extracted based on the importance score and the popularity score may specifically include calculating the attribute value using the following formula:

[0121] ;

[0122] Among them: P attribute is the attribute value; I is the importance score; H is the popularity score; δ and γ are both adjustment coefficients less than 1, and γ is greater than δ.

[0123] Determining the user behavior value of the target data block according to the user operation record of the target data block may specifically include:

[0124] The user behavior value of the target data block is determined according to the modification frequency of the target data block, the time interval between the last modification of the target data block and the average viewing time of the target data block.

[0125] Determining the user behavior value of the target data block based on the modification frequency of the target data block, the time interval between the target data block and the most recent modification, and the average viewing time of the target data block may specifically include:

[0126] Determining a data activity score based on the modification frequency, determining a user importance score based on the average viewing time, and determining a data freshness of the target data block based on the time interval;

[0127] The data activity score and the user importance score are weightedly summed, and the result of the weighted sum is multiplied by the data freshness to obtain the user behavior value.

[0128] Determining the data activity score according to the modification frequency may specifically include:

[0129] Obtaining the maximum and minimum values ​​of the modification frequencies of each data block in the text to be extracted;

[0130] The modification frequency of the target data block, the maximum value, and the minimum value are normalized by a calculation formula to calculate the data activity score.

[0131] Determining the data freshness of the target data block according to the time interval may specifically include:

[0132] Determining whether the time interval is greater than the data validity period set for the target data block;

[0133] If so, the data freshness is determined to be 0; or,

[0134] If not, the data freshness can be calculated using the following formula:

[0135] ;

[0136] Wherein, T is the data freshness; t is the time interval; t 有效期 The data validity period set for the target data block.

[0137] Determining the user's importance score based on the average browsing time may specifically include:

[0138] Obtaining the average reading time of the text to be extracted;

[0139] By formula To determine the user importance score, where D is the user importance score; min is the operator for taking the minimum value; d is the average viewing time of the target data block; d total is the average reading time of the text to be extracted.

[0140] Figure 3 An example of a physical structure diagram of an electronic device is shown below. Figure 3As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other via the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the method for extracting structured information from a text provided in an embodiment of the present application, comprising determining the attribute value of the text to be extracted based on the content classification label of the text to be extracted, determining the user behavior value of the target data block in the text to be extracted based on the user operation record of the target data block, determining the comprehensive value score of the target data block based on the attribute value of the text to be extracted and the user behavior value of the target data block, and then selecting a regular expression corresponding to the extraction granularity based on the comprehensive value score of the target data block, extracting structured information from the target data block, and then storing the extracted structured information in a structured storage system corresponding to the comprehensive value score. This method calculates the comprehensive value score of the target data block in the text to be extracted, selects the regular expression corresponding to the extraction granularity, extracts structured information from the target data block, and stores the structured information in a structured storage system corresponding to the comprehensive value score. Therefore, even if there are multiple data blocks in the text to be extracted, this method can be used to extract structured information from each data block and store it in a structured storage system corresponding to the value, thereby solving the problems in the prior art.

[0141] In practical applications, the electronic device may be a user-side electronic device or a server-side electronic device.

[0142] Obviously, since the processor 310 can call the logic instructions in the memory 330 to execute the method provided in the embodiment of the present application, it can also solve the problems in the prior art.

[0143] Furthermore, the logic instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0144] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the method for extracting structured information from text provided by the embodiment of the present application, including determining the attribute value of the text to be extracted based on the content classification label of the text to be extracted, determining the user behavior value of the target data block in the text to be extracted based on the user operation record of the target data block, determining the comprehensive value score of the target data block based on the attribute value of the text to be extracted and the user behavior value of the target data block, and then selecting a regular expression corresponding to the extraction granularity based on the comprehensive value score of the target data block, extracting structured information from the target data block, and then storing the extracted structured information in a structured storage system corresponding to the comprehensive value score. This method calculates the comprehensive value score of the target data block in the text to be extracted, selects the regular expression corresponding to the extraction granularity, extracts structured information from the target data block, and stores the structured information in a structured storage system corresponding to the comprehensive value score. Therefore, even if there are multiple data blocks in the text to be extracted, this method can be used to extract structured information from each data block and store it in a structured storage system corresponding to the value, thereby solving the problems in the prior art.

[0145] Obviously, since the computer program is executed by the processor, the computer can execute the method provided by the embodiment of the present application, and thus can also solve the problems in the prior art.

[0146] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which performs the method provided in the embodiment of the present application when the computer program is executed by a processor.

[0147] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0148] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0149] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for extracting structured information from text, characterized in that: The method comprises: Determining the attribute value of the text to be extracted based on the content classification label of the text to be extracted; For a target data block in the text to be extracted, determining the user behavior value of the target data block according to the user operation record of the target data block; wherein the target data block is specifically any data block in the text to be extracted; Determining a comprehensive value score of the target data block based on the attribute value of the text to be extracted and the user behavior value of the target data block; According to the comprehensive value score of the target data block, a regular expression corresponding to the extraction granularity is selected to extract structured information from the target data block; Storing the extracted structured information in a structured storage system corresponding to the comprehensive value score; The method of determining the attribute value of the text to be extracted according to the content classification label of the text to be extracted specifically includes: Obtaining the internal importance level of the text to be extracted, and determining an importance score according to the internal importance level; wherein the importance score is used to characterize the importance of the text to be extracted within the enterprise; Obtaining the network popularity of the content classification tag and determining a popularity score based on the network popularity; wherein the popularity score is used to represent the popularity of the content classification tag of the text to be extracted in online media; Determining the attribute value of the text to be extracted according to the importance score and the popularity score; Determining the user behavior value of the target data block according to the user operation record of the target data block specifically includes: Determining the user behavior value of the target data block based on the modification frequency of the target data block, the time interval between the target data block and the last modification, and the average viewing time of the target data block; The method further includes presetting a regular expression for a coarse extraction granularity, a regular expression for a fine extraction granularity, and a preset threshold; and According to the comprehensive value score of the target data block, a regular expression corresponding to the extraction granularity is selected to extract structured information from the target data block, specifically including: Determining whether the comprehensive value score of the target data block is greater than the preset threshold; When the comprehensive value score is greater than the preset threshold, a regular expression with fine extraction granularity is selected to extract structured information from the target data block; or When the comprehensive value score is less than or equal to the preset threshold, a regular expression with a coarse extraction granularity is selected to extract structured information from the target data block.

2. The method according to claim 1, characterized in that Determining the user behavior value of the target data block based on the modification frequency of the target data block, the time interval between the target data block and the last modification, and the average access time of the target data block, specifically includes: Determining a data activity score based on the modification frequency, determining a user importance score based on the average viewing time, and determining a data freshness of the target data block based on the time interval; The data activity score and the user importance score are weightedly summed, and the result of the weighted sum is multiplied by the data freshness to obtain the user behavior value.

3. The method according to claim 2, characterized in that Determine the data activity score based on the modification frequency, specifically including: Obtaining the maximum and minimum values ​​of the modification frequencies of each data block in the text to be extracted; The modification frequency of the target data block, the maximum value, and the minimum value are normalized by a calculation formula to calculate the data activity score.

4. The method according to claim 2, characterized in that Determining the data freshness of the target data block according to the time interval specifically includes: Determining whether the time interval is greater than the data validity period set for the target data block; If so, the data freshness is determined to be 0; or, If not, the data freshness is calculated using the following formula: ； Wherein, T is the data freshness; t is the time interval; t 有效期 The data validity period set for the target data block.

5. The method according to claim 2, characterized in that Determine the user importance score based on the average browsing time, specifically including: Obtaining the average reading time of the text to be extracted; By formula To determine the user importance score, where D is the user importance score; min is the operator for taking the minimum value; d is the average viewing time of the target data block; d total is the average reading time of the text to be extracted.

6. A device for extracting structured information from text, characterized in that: include: A first determining unit, configured to determine the attribute value of the text to be extracted based on the content classification label of the text to be extracted; A second determining unit is configured to determine, for a target data block in the text to be extracted, a user behavior value of the target data block based on a user operation record of the target data block; wherein the target data block is specifically any data block in the text to be extracted; a third determining unit, configured to determine a comprehensive value score of the target data block based on the attribute value of the text to be extracted and the user behavior value of the target data block; an extraction unit, configured to select a regular expression corresponding to an extraction granularity according to the comprehensive value score of the target data block, and extract structured information from the target data block; a storage unit, configured to store the extracted structured information in a structured storage system corresponding to the comprehensive value score; The method of determining the attribute value of the text to be extracted according to the content classification label of the text to be extracted specifically includes: Obtaining the internal importance level of the text to be extracted, and determining an importance score according to the internal importance level; wherein the importance score is used to characterize the importance of the text to be extracted within the enterprise; Obtaining the network popularity of the content classification tag and determining a popularity score based on the network popularity; wherein the popularity score is used to represent the popularity of the content classification tag of the text to be extracted in online media; Determining the attribute value of the text to be extracted according to the importance score and the popularity score; Determining the user behavior value of the target data block according to the user operation record of the target data block specifically includes: Determining the user behavior value of the target data block based on the modification frequency of the target data block, the time interval between the target data block and the last modification, and the average viewing time of the target data block; Wherein, a regular expression for coarse extraction granularity, a regular expression for fine extraction granularity and a preset threshold are pre-set; and, According to the comprehensive value score of the target data block, a regular expression corresponding to the extraction granularity is selected to extract structured information from the target data block, specifically including: Determining whether the comprehensive value score of the target data block is greater than the preset threshold; When the comprehensive value score is greater than the preset threshold, a regular expression with fine extraction granularity is selected to extract structured information from the target data block; or When the comprehensive value score is less than or equal to the preset threshold, a regular expression with a coarse extraction granularity is selected to extract structured information from the target data block.

Images