Method, device, apparatus and computer-readable storage medium for extracting term data

Abstract

The present application belongs to the field of data processing and proposes a method, device, equipment and computer-readable storage medium for extracting entry data. The method includes: obtaining a news data set, determining the entry name included in the news data set; obtaining the entry text of the entry name, and determining news data related to the entry text in the news data set; using the news data related to the entry text of the entry name and the entry name as input data of an entry text extraction model, and using the entry text of the entry name as the output label of the entry text extraction model, and training the entry text extraction model. An entry text extraction operation is performed based on the trained entry text extraction model. As a result, the present application can automatically extract entry data from news data through the entry text extraction model, which is beneficial to reducing labor costs and improving the efficiency of entry extraction.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a method, apparatus, device, and computer-readable storage medium for extracting entry data. Background Art

[0002] Online encyclopedias are authoritative online knowledge service platforms. Currently, major online encyclopedias include online knowledge bases such as Wikipedia and Baidu Encyclopedia. Generally, online encyclopedias store individual entries and textual explanations of these entries. Online encyclopedias allow people to quickly access knowledge and information.

[0003] In today's rapidly evolving online information landscape, a vast amount of news data is generated daily, including numerous new terms. When organizing and extracting terms from news data, term organizers typically identify the terms within the news data and find the corresponding text descriptions, generating a collection of terms that can be used in online knowledge bases. Due to the rapid pace of news data updates, searching and organizing terms within news data is labor-intensive and inefficient. Summary of the Invention

[0004] In view of this, the embodiments of the present application provide a method, device, equipment and computer-readable storage medium for extracting entry data to solve the problem in the prior art that due to the extremely fast update speed of news data, the search and sorting process of entries in news data requires a lot of manpower and material resources, and the extraction efficiency is low.

[0005] A first aspect of an embodiment of the present application provides a method for extracting entry data, the method comprising:

[0006] Acquire a news data set, and determine the names of entries included in the news data set;

[0007] Acquire the entry text of the entry name, and determine news data related to the entry text in the news data set;

[0008] Using news data related to the entry text of the entry name and the entry name as input data of an entry text extraction model, using the entry text of the entry name as an output label of the entry text extraction model, and training the entry text extraction model;

[0009] Perform term text extraction operations based on the trained term text extraction model.

[0010] In conjunction with the first aspect, in a first possible implementation of the first aspect, determining the entry names included in the news dataset includes:

[0011] Extracting entities included in the news dataset;

[0012] The extracted entity is matched with the term name in a preset term database to determine that the entity includes the term name.

[0013] In conjunction with the first aspect, in a second possible implementation of the first aspect, determining news data related to the entry text in the news dataset includes:

[0014] Acquire keywords included in the entry text, and acquire keywords included in the news data in the news dataset;

[0015] Determining a first keyword set for the entry text according to the frequency of occurrence of keywords included in the entry text in the entry text;

[0016] determining a second keyword set for the news data based on a frequency at which keywords included in the news data appear in the news data;

[0017] News data related to the entry text is determined based on the first keyword set and the first keyword set.

[0018] In conjunction with the second possible implementation manner of the first aspect, in a third possible implementation manner of the first aspect, after determining the first keyword set and the second keyword set, the method further includes:

[0019] Determining the weight of a first keyword in the first keyword set according to the frequency of occurrence of the keyword in the first keyword set in the entry text, and determining the weight of a second keyword in the second keyword set according to the frequency of occurrence of the keyword in the second keyword set in the news data;

[0020] Determining news data related to the entry text based on the first keyword set and the first keyword set includes:

[0021] When the first keyword in the first keyword set matches the second keyword in the second keyword set, the smaller weight or the average weight between the weight of the matched first keyword and the weight of the second keyword is selected as the weight for matching the first keyword with the second keyword, and the matching degree between the entry text and the news data is determined based on the weight, and the news data related to the entry text is determined based on the matching degree.

[0022] In conjunction with the first aspect, in a fourth possible implementation of the first aspect, using news data related to the entry text of the entry name and the entry name as input data of an entry text extraction model, using the entry text of the entry name as an output label of the entry text extraction model, and training the entry text extraction model includes:

[0023] Inputting the news data and the term name into a pre-set term text extraction model, and obtaining an output term text through the term text extraction model;

[0024] Determining differences between the output entry text and the entry text of the entry name;

[0025] The parameters of the entry text extraction model are adjusted according to the difference until the difference between the entry text output by the entry text extraction model and the entry text of the entry name meets the preset requirements.

[0026] In conjunction with the first possible implementation of the first aspect, in a fifth possible implementation of the first aspect, performing an entry text extraction operation based on the trained entry text extraction model includes:

[0027] Determining a set of candidate term names in the news dataset;

[0028] Searching for news data related to the candidate entry name in the candidate entry name set;

[0029] The candidate entry name and news data related to the candidate entry name are input into the trained entry text extraction model to generate the entry text of the candidate entry name.

[0030] In conjunction with the fifth possible implementation of the first aspect, in a sixth possible implementation of the first aspect, determining the set of candidate term names in the news dataset includes:

[0031] The extracted entities are matched with term names in a preset term database, and entities that are not matched with term names are added to a candidate term name set.

[0032] A second aspect of an embodiment of the present application provides a device for extracting entry data, the device comprising:

[0033] An entry name acquisition unit, configured to acquire a news data set and determine the entry names included in the news data set;

[0034] A news data acquisition unit, configured to acquire the entry text of the entry name and determine news data related to the entry text in the news data set;

[0035] a training unit, configured to use news data related to the entry text of the entry name and the entry name as input data of an entry text extraction model, and the entry text of the entry name as an output label of the entry text extraction model, to train the entry text extraction model;

[0036] The term text extraction unit is used to perform term text extraction operations based on the trained term text extraction model.

[0037] The third aspect of an embodiment of the present application provides an entry data extraction device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in any one of the first aspects when executing the computer program.

[0038] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in any one of the first aspects are implemented.

[0039] The beneficial effects of the embodiments of the present application compared with the prior art are as follows: the present application obtains the term name in the news data, and the news data related to the term text of the term name in the news data, trains the term text extraction model through the term name, term text and news data, and performs term text extraction on the collected news data based on the trained term text extraction model, so that the present application can automatically extract term data from the news data through the term text extraction model, which is conducive to reducing labor costs and improving term extraction efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0041] Figure 1 This is a schematic diagram of an implementation scenario of a method for extracting entry data provided in an embodiment of the present application;

[0042] Figure 2 This is a schematic diagram of an implementation flow of a method for extracting entry data provided in an embodiment of the present application;

[0043] Figure 3 This is a schematic diagram of an implementation flow of a method for determining news data related to an entry provided in an embodiment of the present application;

[0044] Figure 4 This is a schematic diagram of an implementation flow of an entry text extraction operation based on a trained entry text extraction model provided in an embodiment of the present application;

[0045] Figure 5 is a schematic diagram of a device for extracting entry data provided by an embodiment of the present application;

[0046] Figure 6 It is a schematic diagram of the entry data extraction device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0048] In order to illustrate the technical solution described in this application, specific embodiments are provided below.

[0049] Currently, when organizing entries, staff or users typically determine the entry names and the entry text corresponding to the entry names, and then organize the entries one by one. Manual organization is affected by the cognitive level of different organizers, which is not conducive to improving the reliability of the organized entries. In addition, manual organization requires a large amount of manpower and is inefficient, making it difficult to adapt to the requirement of rapid entry update speed.

[0050] Based on this, the present application embodiment proposes a method for extracting entry data, such as Figure 1The diagram shows an implementation scenario of the entry data extraction method described in an embodiment of the present application. The implementation scenario includes an entry data extraction device, an entry server, and a news data server. The entry data extraction device can obtain news data from the news data server and obtain entities included in the news data. The entry data extraction device can obtain entry data from the entry server, including entry names and their corresponding entry texts. The entry data extraction device can match the entities extracted from the news data with the entry names in the entry data, and determine the relevant news data based on the entry text corresponding to the matched entry names. Based on the relevant news data and the entry name, a pre-set entry text extraction model is input, the output entry text is calculated, and it is compared with the entry text of the entry name. Based on the difference in the comparison, the parameters of the entry text extraction model are adjusted until the difference between the two meets the preset requirements. After the training of the entry text extraction model is completed, the entry data in the news data is extracted based on the entry text extraction model.

[0051] Figure 2 The following is a schematic diagram of an implementation flow of a method for extracting entry data provided in an embodiment of the present application, as detailed below:

[0052] In S201 , a news data set is acquired, and the names of entries included in the news data set are determined.

[0053] When acquiring the news dataset, news data can be collected by crawling data from multiple news websites, and a news dataset can be formed from the multiple news data. The news dataset can be limited by time. For example, the data volume of the formed news dataset includes at least one year's worth of news data. Alternatively, the data volume in the news dataset can be limited by time and source. For example, the news dataset includes at least five news websites, and the cumulative time includes news data for more than one year. In actual implementation, it is not limited to more than one year. For example, any value between 1 and 10 years can be set. Alternatively, the data volume in the news dataset can also be limited by quantity. For example, the data volume in the news dataset includes at least 10,000 pieces of data.

[0054] When determining the term names included in the news data set, the entities included in the news data in the news data set can be determined first, and then based on the included entities, the term names in the news data can be determined by matching with the term database in the term server.

[0055] The collected news dataset includes a large amount of news data. In order to extract terms (including term names and term text) from the news data, or to train a term text extraction model using the news data, it is necessary to first determine the entities included in the news data.

[0056] Among them, the entities in the embodiments of the present application refer to things that exist objectively and can be distinguished from each other. Entities can refer to specific people, things, or objects, or they can refer to concepts. The entities may include named entities, i.e., named entities are entities identified by names. When identifying entities, proper name recognition or named entity recognition (full name in English: Named Entity Recognition, abbreviated in English as NER) may also be used. Named entities generally include names of people, places, organizations, time, proper nouns, etc.

[0057] Methods for obtaining entities included in the news dataset include rule-based methods and model-based methods.

[0058] Rule-based methods can be used to extract entities from semi-structured or relatively standardized text. Common expressions can be used to identify entities within a text. For example, product prices often appear in the form of "number + yuan," and regular expressions can be used to extract these entities.

[0059] When acquiring entities using a model-based method, the selected models may include hidden Markov models, conditional random field discriminant models, recurrent neural network models, etc.

[0060] After determining the entities included in the news database through entity extraction, it is necessary to find the entities that have been extracted as entry names. The method for determining whether the extracted entity is an entry name can be based on a matching method based on an entry name database. The entry name data can include major entry organization platforms, including Baidu Encyclopedia, Wikipedia, Sogou Encyclopedia, etc.

[0061] Determine the entry name included in the extracted entity, that is, determine whether the extracted entity has been included as an entry on various entry platforms, including encyclopedia entries. Each entity can be matched with multiple entry platforms. If the entry name corresponding to the entity can be found on the entry platform, that is, if the entity is used as the entry name, it means that the entity has been included as an entry.

[0062] If an entity cannot find a corresponding entry name in the entry platform, the entity that does not match the entry name can be placed in the candidate entry name set. Entities that do match the entry name can be placed in the sample set. This allows the entry text extraction model to be trained based on the data in the sample set. When the number of entities that match the entry name reaches a predetermined number, such as more than 100,000, it can be considered that the current training sample is sufficient to train the entry text extraction model. Training samples can be generated based on the entries and news data that the entities match, and used to train the entry text extraction model.

[0063] In S202, the entry text of the entry name is obtained, and news data related to the entry text is determined in the news data set.

[0064] When training samples are generated based on terms and news data, news data related to the term can be determined, which can improve both the efficiency and accuracy of term text search.

[0065] When determining the news data related to the entry, the news data can be sorted based on the similarity between the news data and the entry text, and a predetermined number of news data with higher similarity can be selected as the news data related to the entry. Alternatively, the news data related to the entry can be determined based on keyword matching, for example, Figure 3 As shown, the implementation process of determining news data related to an entry may include:

[0066] In S301 , keywords included in the entry text are acquired, and keywords included in the news data in the news dataset are acquired.

[0067] When determining the keywords included in the term text and the news data in the news dataset, it can be done according to a supervised algorithm or an unsupervised algorithm.

[0068] Among them, supervised algorithms can train a classification model based on a document collection (including entry text or news data) with labeled keywords, and perform keyword extraction on news data or entry text that needs to determine keywords based on the trained classification model.

[0069] Unsupervised algorithms can include methods based on text statistical features, word graph models, or topic models.

[0070] In S302, a first keyword set of the entry text is determined according to the frequency of occurrence of keywords included in the entry text in the entry text.

[0071] The importance of the keyword in the entry text is determined based on the frequency of occurrence of the keyword in the entry text. A predetermined number of keywords ranked high in importance can be selected to form a first keyword set.

[0072] In order to prevent the selected first keyword set from including common words, the keywords may be screened using a pre-set common word library, and the first keyword set may be generated based on the screened keywords.

[0073] Alternatively, the common words in the keywords may be screened based on the frequency of occurrence of the keywords in combination with the inverse document frequency of the keywords.

[0074] In S303, a second keyword set of the news data is determined according to the frequency of occurrence of the keywords included in the news data in the news data.

[0075] Accordingly, before determining the second key set included in the news data, common words included in the keywords may be screened out using a pre-set common word library or in combination with the inverse document frequency of the keywords.

[0076] In S304, news data related to the entry text is determined based on the first keyword set and the first keyword set.

[0077] When determining news data related to the entry text based on the first keyword set and the second keyword set, different news data can be sorted according to the similarity between the second keyword set of different news data and the first keyword set, and a predetermined number of news data with higher similarity can be selected as news data related to the entry text.

[0078] When determining the similarity between the first keyword set and the second keyword set, the similarity between the keywords in the two keyword sets may be directly used as the similarity between the first keyword set and the second keyword set.

[0079] Alternatively, weights can be calculated for the keywords in the first and second keyword sets. After calculating the weights, identical or matching keywords in the first and second keyword sets can be searched. Based on the weights of the matching keywords in the two sets, a smaller weight or an average weight can be selected as the weight for the keyword. The weights of the matching keywords in the two sets can be summed to obtain the similarity between the two sets, namely, the first and second keyword sets. The similarity can be sorted from largest to smallest, and a predetermined amount of news data can be selected as the news data associated with the entry text.

[0080] The weights of the keywords in the first keyword set may be determined based on the frequency of occurrence of the keywords in the first keyword set in the entry text. The weights of the keywords in the second keyword set may be determined based on the frequency of occurrence of the keywords in the second keyword set in the news data.

[0081] In S203, the news data related to the term text of the term name and the term name are used as input data of the term text extraction model, and the term text of the term name is used as the output label of the term text extraction model to train the term text extraction model.

[0082] When the number of determined training samples (i.e., a term name, the term text of the term name, and the term text-related news data of the term name constitute a training sample) reaches a predetermined number, for example, more than 50,000, the training samples can be input into the term text extraction model to complete the training of the term text extraction model.

[0083] The term text extraction model can be used to input a term name and news data related to the term text of the term name, and output the term text calculated from the news data related to the term name. The output term text is compared with the term text in the training sample, that is, the term text calculated and output from the term name and the related news data is compared with the term text of the term, and the difference between the two is determined. Based on the difference, the parameters in the term text extraction model are adjusted so that the difference between the term text calculated and output by the term text extraction model and the term text in the sample meets the preset requirements, so that the output text can be used as the term text and match the input term name.

[0084] By repeatedly adjusting the parameters of the term text extraction model, the difference between the term text output by the term text extraction model and the term text in the sample is reduced. When the difference between the output term text and the term text in the sample is less than a predetermined difference, or the number of iterative calculations reaches a predetermined number, the parameters of the term text extraction model are considered to have been trained, and term text extraction operations can be performed on unknown entities based on the trained term text extraction model.

[0085] In S204, a term text extraction operation is performed based on the trained term text extraction model.

[0086] After the term text extraction model training is completed, the model can be used to extract term data from newly collected news data. Figure 4 Shown, including:

[0087] In S401, a set of candidate term names in the news dataset is determined.

[0088] When determining the candidate entry name set, the entities included in the news data set can be extracted first, the extracted entities can be matched with the entry names in the preset entry database, the matched entities can be filtered out, and the entities that are not matched to the entry names can be obtained to generate the candidate entry name set.

[0089] Alternatively, common nouns included in the entity may be further filtered out, for example, by filtering common words in the entity based on a pre-set common noun library or based on the product of the frequency of occurrence and the inverse document frequency.

[0090] In S402, news data related to the candidate entry name in the candidate entry name set is searched.

[0091] The number of times the candidate entry name appears in the news data can be determined by keyword matching, and news data with a higher number of occurrences can be selected as news data related to the candidate entry name. A predetermined number, such as K pieces of news data related to the candidate entry, can be selected. This allows the entry text required to generate an entry from the entity to be obtained based on the searched news data.

[0092] In S403, the news data related to the candidate entry name and the candidate entry name are input into the trained entry text extraction model to generate the entry text of the candidate entry name.

[0093] The term name and news data related to the term name are input into a trained term text extraction model, so that the term text extraction model outputs the term text corresponding to the term name. The required term data can be generated based on the output term text and the term name.

[0094] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0095] Figure 5 This is a schematic diagram of a device for extracting entry data provided in an embodiment of the present application. Figure 5 As shown, the device includes:

[0096] The term name acquisition unit 501 is used to acquire a news data set and determine the term names included in the news data set;

[0097] A news data acquisition unit 502 is configured to acquire the entry text of the entry name and determine news data related to the entry text in the news data set;

[0098] The training unit 503 is used to train the term text extraction model by using the news data related to the term text of the term name and the term name as input data of the term text extraction model and the term text of the term name as output label of the term text extraction model.

[0099] The term text extraction unit 504 is used to perform term text extraction operations according to the trained term text extraction model.

[0100] Figure 5 The entry data extraction device shown, and Figure 2 The entry data extraction method shown corresponds to.

[0101] Figure 6 Schematic diagram of a device for extracting entry data provided by an embodiment of the present application. Figure 6 As shown, the entry data extraction device 6 of this embodiment includes: a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and executable on the processor 60, such as an entry data extraction program. When the processor 60 executes the computer program 62, the steps of each of the above-described entry data extraction method embodiments are implemented. Alternatively, when the processor 60 executes the computer program 62, the functions of each module / unit in each of the above-described device embodiments are implemented.

[0102] For example, the computer program 62 may be divided into one or more modules / units, which are stored in the memory 61 and executed by the processor 60 to implement the present application. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, and the instruction segments are used to describe the execution process of the computer program 62 in the entry data extraction device 6.

[0103] The entry data extraction device 6 can be a computing device such as a desktop computer, a notebook, a palmtop computer, a cloud server, etc. The entry data extraction device can include, but is not limited to, a processor 60 and a memory 61. It can be understood by those skilled in the art that Figure 6 It is only an example of the entry data extraction device 6 and does not constitute a limitation of the entry data extraction device 6. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the entry data extraction device may also include input and output devices, network access devices, buses, etc.

[0104] The processor 60 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0105] The memory 61 can be an internal storage unit of the entry data extraction device 6, such as a hard disk or memory of the entry data extraction device 6. The memory 61 can also be an external storage device of the entry data extraction device 6, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the entry data extraction device 6. Furthermore, the memory 61 can also include both the internal storage unit and the external storage device of the entry data extraction device 6. The memory 61 is used to store the computer program and other programs and data required by the entry data extraction device. The memory 61 can also be used to temporarily store data that has been output or is to be output.

[0106] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0107] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0108] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0109] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0110] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0111] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0112] If the integrated module / unit is implemented as a software functional unit and sold or used as a standalone product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application can implement all or part of the process steps in the above-mentioned method embodiments through hardware related to computer program instructions. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. It should be noted that the content of the computer-readable medium can be appropriately increased or decreased based on the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media does not include electric carrier signals and telecommunication signals.

[0113] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A method for extracting entry data, characterized in that: The method comprises: Acquire a news data set, and determine the names of entries included in the news data set; Acquire the entry text of the entry name, and determine news data related to the entry text in the news data set; Using news data related to the entry text of the entry name and the entry name as input data of the entry text extraction model, and using the entry text of the entry name as the output label of the entry text extraction model, to train the entry text extraction model; Perform term text extraction operations based on the trained term text extraction model.

2. The method according to claim 1, characterized in that Determining the names of the entries included in the news dataset includes: Extracting entities included in the news dataset; The extracted entity is matched with the term name in a preset term database to determine that the entity includes the term name.

3. The method according to claim 1, characterized in that Determining news data related to the entry text in the news data set includes: Acquire keywords included in the entry text, and acquire keywords included in the news data in the news dataset; Determining a first keyword set for the entry text according to the frequency of occurrence of keywords included in the entry text in the entry text; determining a second keyword set for the news data based on a frequency at which keywords included in the news data appear in the news data; News data related to the entry text is determined based on the first keyword set and the first keyword set.

4. The method according to claim 3, characterized in that After determining the first keyword set and the second keyword set, the method further includes: Determining the weight of a first keyword in the first keyword set according to the frequency of occurrence of the keyword in the first keyword set in the entry text, and determining the weight of a second keyword in the second keyword set according to the frequency of occurrence of the keyword in the second keyword set in the news data; Determining news data related to the entry text based on the first keyword set and the first keyword set includes: When the first keyword in the first keyword set matches the second keyword in the second keyword set, the smaller weight or the average weight between the weight of the matched first keyword and the weight of the second keyword is selected as the weight for matching the first keyword with the second keyword, and the matching degree between the entry text and the news data is determined based on the weight, and the news data related to the entry text is determined based on the matching degree.

5. The method according to claim 1, wherein Using news data related to the entry text of the entry name and the entry name as input data of an entry text extraction model, using the entry text of the entry name as an output label of the entry text extraction model, and training the entry text extraction model, including: Inputting the news data and the term name into a pre-set term text extraction model, and obtaining an output term text through the term text extraction model; Determining differences between the output entry text and the entry text of the entry name; The parameters of the entry text extraction model are adjusted according to the difference until the difference between the entry text output by the entry text extraction model and the entry text of the entry name meets the preset requirements.

6. The method according to claim 2, characterized in that Perform term text extraction operations based on the trained term text extraction model, including: Determining a set of candidate term names in the news dataset; Searching for news data related to the candidate entry name in the candidate entry name set; The candidate entry name and news data related to the candidate entry name are input into the trained entry text extraction model to generate the entry text of the candidate entry name.

7. The method according to claim 6, characterized in that Determining a set of candidate term names in the news dataset includes: The extracted entities are matched with term names in a preset term database, and entities that are not matched with term names are added to a candidate term name set.

8. A device for extracting entry data, characterized in that: The device comprises: An entry name acquisition unit, configured to acquire a news data set and determine the entry names included in the news data set; A news data acquisition unit, configured to acquire the entry text of the entry name and determine news data related to the entry text in the news data set; a training unit, configured to use news data related to the entry text of the entry name and the entry name as input data of an entry text extraction model, and the entry text of the entry name as an output label of the entry text extraction model, to train the entry text extraction model; The term text extraction unit is used to perform term text extraction operations based on the trained term text extraction model.

9. A device for extracting entry data, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images