Place name identification method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a place name recognition method and device, equipment, a storage medium and a program product, and relates to the technical field of natural language processing. By adopting the method, the problems of overfitting and knowledge forgetting can be relieved. The method comprises the following steps: performing format conversion on obtained initial training data to obtain current training data consisting of a text to be extracted and an extracted place name; querying a plurality of synonyms corresponding to the extracted place names, and performing data enhancement on the current training data based on the plurality of synonyms to obtain a target training data set; constructing an auxiliary classification task according to the plurality of synonyms, the negative examples and the classification loss; based on the target training data set and the auxiliary classification task, training the initial model until model parameters of the initial model are the same as target parameters, and obtaining a preliminary training model; obtaining model fine tuning loss, and adjusting the preliminary training model according to the model fine tuning loss to obtain a pre-training model; the pre-training model is used for place name identification.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to a method, apparatus, computer equipment, computer-readable storage medium and computer program product for identifying place names. Background Art

[0002] With the development of computer technology, named entity recognition technology has emerged. This technology is an important part of the field of natural language processing. The purpose of this task is to identify entity names with specific meanings in the text, among which place name recognition is undoubtedly an important component.

[0003] In recent years, deep learning-based methods have been greatly developed, and pre-trained models have been used. Since these models are trained on a large amount of text, they can understand the semantics of the text more deeply. Therefore, fine-tuning the pre-trained models is also a very effective method. However, fine-tuning all parameters often requires too many resources. In addition, if the number of data sets is not large enough, it is easy to cause overfitting and knowledge forgetting problems. Summary of the invention

[0004] Based on this, it is necessary to provide a place name recognition method, device, computer equipment, computer readable storage medium and computer program product to address the above technical problems.

[0005] In a first aspect, the present application provides a method for identifying place names, comprising:

[0006] Acquire initial training data, and convert the format of the initial training data to obtain current training data consisting of the text to be extracted and the extracted place name;

[0007] Querying multiple synonyms corresponding to the extracted place names, and based on the multiple synonyms, performing data enhancement on the current training data to obtain a target training data set;

[0008] Acquire a plurality of negative examples having semantic differences from the extracted place names and classification losses corresponding to the negative examples, and construct an auxiliary classification task according to the plurality of synonyms, the negative examples and the classification losses;

[0009] Based on the target training data set and the auxiliary classification task, the initial model is trained until the model parameters of the initial model are the same as the target parameters, thereby obtaining a preliminary training model;

[0010] A model fine-tuning loss is obtained, and the preliminary training model is adjusted according to the model fine-tuning loss to obtain a pre-trained model; the pre-trained model is used for place name recognition.

[0011] In one embodiment, the data enhancement is performed on the current training data based on the multiple synonyms to obtain a target training data set, including:

[0012] For each of the synonyms, the corresponding vocabulary in the text to be extracted is replaced by the synonym to obtain the target text to be extracted; based on the synonyms and the target text to be extracted, target training data corresponding to the synonyms are generated; the current training data and the target training data corresponding to each of the synonyms are integrated to obtain the target training data set.

[0013] In one embodiment, before training the initial model based on the target training data set and the auxiliary classification task, the method further includes:

[0014] Construct a low-rank adapter, and obtain pre-training parameters and pre-training weights; based on the pre-training weights, perform a weighted combination of the low-rank adapter and the pre-training parameters to obtain the target parameters.

[0015] In one embodiment, constructing a low-rank adapter includes:

[0016] Obtain a first low-rank matrix, a second low-rank matrix and a bias item matrix; the bias item matrix is ​​used to adjust the bias; multiply the first low-rank matrix and the second low-rank matrix to obtain a target low-rank matrix; add the target low-rank matrix and the bias item matrix to obtain the low-rank adapter.

[0017] In one embodiment, obtaining the model fine-tuning loss includes:

[0018] The cross entropy loss and loss allocation weight of the pre-trained model are obtained; based on the loss allocation weight, the cross entropy loss and the classification loss are weightedly combined to obtain the model fine-tuning loss.

[0019] In one embodiment, constructing an auxiliary classification task based on the multiple synonyms, the negative examples and the classification loss includes:

[0020] The plurality of synonyms and the negative examples are respectively labeled to obtain first labels corresponding to the plurality of synonyms and second labels corresponding to the negative examples; training data for the auxiliary classification task is generated according to the first labels, the second labels and the classification loss; and the auxiliary classification task is constructed based on the training data.

[0021] In a second aspect, the present application also provides a place name recognition device, comprising:

[0022] A format conversion module, used for acquiring initial training data, performing format conversion on the initial training data, and obtaining current training data consisting of the text to be extracted and the extracted place name;

[0023] A data enhancement module, used for querying a plurality of synonyms corresponding to the extracted place names, and performing data enhancement on the current training data based on the plurality of synonyms to obtain a target training data set;

[0024] An auxiliary classification module, used for obtaining a plurality of negative examples having semantic differences from the extracted place names and classification losses corresponding to the negative examples, and constructing an auxiliary classification task according to the plurality of synonyms, the negative examples and the classification losses;

[0025] A model training module, used to train the initial model based on the target training data set and the auxiliary classification task until the model parameters of the initial model are the same as the target parameters, thereby obtaining a preliminary training model;

[0026] The model adjustment module is used to obtain the model fine-tuning loss, and adjust the preliminary training model according to the model fine-tuning loss to obtain a pre-trained model; the pre-trained model is used for place name recognition.

[0027] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0028] Acquire initial training data, perform format conversion on the initial training data, and obtain current training data consisting of the text to be extracted and the extracted place names; query multiple synonyms corresponding to the extracted place names, and based on the multiple synonyms, perform data enhancement on the current training data to obtain a target training data set; obtain a number of negative examples with semantic differences from the extracted place names and the classification losses corresponding to the negative examples, and construct an auxiliary classification task based on the multiple synonyms, the negative examples and the classification losses; train the initial model based on the target training data set and the auxiliary classification task until the model parameters of the initial model are the same as the target parameters, and obtain a preliminary training model; obtain the model fine-tuning loss, and adjust the preliminary training model based on the model fine-tuning loss to obtain a pre-trained model; the pre-trained model is used for place name recognition.

[0029] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0030] Acquire initial training data, perform format conversion on the initial training data, and obtain current training data consisting of the text to be extracted and the extracted place names; query multiple synonyms corresponding to the extracted place names, and based on the multiple synonyms, perform data enhancement on the current training data to obtain a target training data set; obtain a number of negative examples with semantic differences from the extracted place names and the classification losses corresponding to the negative examples, and construct an auxiliary classification task based on the multiple synonyms, the negative examples and the classification losses; train the initial model based on the target training data set and the auxiliary classification task until the model parameters of the initial model are the same as the target parameters, and obtain a preliminary training model; obtain the model fine-tuning loss, and adjust the preliminary training model based on the model fine-tuning loss to obtain a pre-trained model; the pre-trained model is used for place name recognition.

[0031] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0032] Acquire initial training data, perform format conversion on the initial training data, and obtain current training data consisting of the text to be extracted and the extracted place names; query multiple synonyms corresponding to the extracted place names, and based on the multiple synonyms, perform data enhancement on the current training data to obtain a target training data set; obtain a number of negative examples with semantic differences from the extracted place names and the classification losses corresponding to the negative examples, and construct an auxiliary classification task based on the multiple synonyms, the negative examples and the classification losses; train the initial model based on the target training data set and the auxiliary classification task until the model parameters of the initial model are the same as the target parameters, and obtain a preliminary training model; obtain the model fine-tuning loss, and adjust the preliminary training model based on the model fine-tuning loss to obtain a pre-trained model; the pre-trained model is used for place name recognition.

[0033] The above-mentioned place name recognition method, device, computer equipment, computer-readable storage medium and computer program product, by converting the format of the initial training data, efficiently obtain the current training data consisting of the text to be extracted and the extracted place names, and then extract multiple synonyms corresponding to the place names through search queries as different expressions to enrich the training data. On this basis, an auxiliary classification task is also introduced to train the model to learn the semantic relationship between different words, with the aim of enabling the model to better understand place names expressed in different ways; in addition, during fine-tuning, this scheme only selects a specific layer to add a low-rank matrix to it, while keeping the parameters of the remaining layers unchanged. During the update process, the parameters of this layer are a weighted combination of the low-rank matrix parameters and the original parameters, thereby effectively reducing resource consumption, improving the robustness of place name recognition, and alleviating the problems of overfitting and knowledge forgetting. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 A diagram showing an application environment of a method for identifying place names in an embodiment;

[0036] Figure 2 A schematic diagram of a process of a place name recognition method in one embodiment;

[0037] Figure 3 is a flowchart of a data enhancement step in an embodiment;

[0038] Figure 4 A schematic diagram of a process flow of a place name recognition method in a specific embodiment;

[0039] Figure 5 is a structural block diagram of a place name recognition device in an embodiment;

[0040] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0042] The place name recognition method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the terminal communicates with the server through the network. The data storage system can store the data that the server needs to process. The data storage system can be integrated on the server or placed on the cloud or other network servers.

[0043] Specifically, the place name recognition method provided in the embodiment of the present application can be executed by a server.

[0044] Exemplarily, the server obtains initial training data, converts the format of the initial training data, and obtains current training data consisting of the text to be extracted and the extracted place names; the server queries multiple synonyms corresponding to the extracted place names, and based on the multiple synonyms, performs data enhancement on the current training data to obtain a target training data set; the server obtains several negative examples with semantics different from the extracted place names and the classification losses corresponding to the negative examples, and constructs an auxiliary classification task based on the multiple synonyms, negative examples and classification losses; the server trains the initial model based on the target training data set and the auxiliary classification task until the model parameters of the initial model are the same as the target parameters, and obtains a preliminary training model; the server obtains the model fine-tuning loss, and adjusts the preliminary training model based on the model fine-tuning loss to obtain a pre-trained model; the pre-trained model is used for place name recognition.

[0045] In such Figure 1 In the application environment shown, the terminal can be, but is not limited to, various personal computers, notebook computers, smart phones, and tablet computers. The server can be implemented as an independent server or a server cluster consisting of multiple servers.

[0046] In one embodiment, Figure 2 As shown, a place name recognition method is provided, which is applied to Figure 1 The server in the example is used to illustrate the following steps:

[0047] Step S201, obtaining initial training data, converting the format of the initial training data, and obtaining current training data consisting of the text to be extracted and the extracted place name.

[0048] Specifically, the server collects initial training data for fine-tuning the model and converts the format of the initial training data into: ,in is the text to be extracted, For the extracted place names, get the current training data, such as: It could be "Xiao Ming is going to travel to city A next month", It is "City A".

[0049] Step S202, query multiple synonyms corresponding to the extracted place names, and based on the multiple synonyms, perform data enhancement on the current training data to obtain a target training data set.

[0050] Specifically, the server obtains all label sets of the current training data In order to perform data enhancement, we first need to obtain as many aliases of each place name as possible. This application obtains them through encyclopedia search, that is, Baidu Encyclopedia search is performed on the elements in each tag set, and the "alias" item in the search return result is retained as a synonym of the target word to obtain the target training data set. The example is as follows: If is "City A", then It is "another name for City A".

[0051] Step S203, obtaining a number of negative examples with semantic differences from the extracted place names and the classification losses corresponding to the negative examples, and constructing an auxiliary classification task based on multiple synonyms, negative examples and classification losses.

[0052] Specifically, in order to improve the model's ability to understand place name-related content, this application also designs an auxiliary classification task. For each element in the label set, k (k is a hyperparameter) words are randomly selected from the synonyms of other elements as negative examples. , the training data of the auxiliary classification task is composed as follows:

[0053]

[0054] The corresponding classification loss is , the purpose is to enable the model to have a deeper understanding of different expressions of the same place name.

[0055] Step S204, based on the target training data set and the auxiliary classification task, the initial model is trained until the model parameters of the initial model are the same as the target parameters, thereby obtaining a preliminary training model.

[0056] Specifically, the server obtains the target parameters by weighted combination, and then iteratively trains the initial model based on the target training data set and the auxiliary classification task until the model parameters of the initial model are the same as the target parameters, thereby obtaining a preliminary training model.

[0057] Step S205, obtaining the model fine-tuning loss, and adjusting the preliminary training model according to the model fine-tuning loss to obtain a pre-trained model; the pre-trained model is used for place name recognition.

[0058] It should be noted that model fine-tuning is a strategy for further training on a specific task based on a pre-trained model. Usually, fine-tuning is performed on a model pre-trained on a large-scale dataset. By continuing to train on a smaller, task-specific dataset, the model can better adapt to the requirements of the task.

[0059] Among them, the fine-tuning loss is a loss function selected based on the target task, which is usually kept consistent with the target task (such as the loss function of classification, regression, and generation tasks).

[0060] Specifically, the server allocates weights based on the losses, obtains the model fine-tuning loss by weighted combination, and then adjusts the preliminary training model according to the model fine-tuning loss to obtain a pre-trained model; the pre-trained model is used for place name recognition.

[0061] The above-mentioned place name recognition method, by converting the format of the initial training data, efficiently obtains the current training data consisting of the text to be extracted and the extracted place names, and then extracts multiple synonyms corresponding to the place names through search queries as different expressions to enrich the training data. On this basis, an auxiliary classification task is also introduced to train the model to learn the semantic relationship between different words, with the aim of enabling the model to better understand place names expressed in different ways. In addition, during fine-tuning, this scheme only selects a specific layer to add a low-rank matrix to it, while keeping the parameters of the remaining layers unchanged. During the update process, the parameters of this layer are a weighted combination of the low-rank matrix parameters and the original parameters, thereby effectively reducing resource consumption, improving the robustness of place name recognition, and alleviating the problems of overfitting and knowledge forgetting.

[0062] In one embodiment, if Figure 3 As shown, in the above step S201, based on multiple synonyms, data enhancement is performed on the current training data to obtain a target training data set, which specifically includes the following steps:

[0063] Step S301: for each synonym, the corresponding word in the text to be extracted is replaced with the synonym to obtain the target text to be extracted.

[0064] Step S302: Generate target training data corresponding to the synonyms according to the synonyms and the target text to be extracted.

[0065] Step S303 , integrating the current training data and the target training data corresponding to each synonym to obtain a target training data set.

[0066] Specifically, the server obtains the target training data set by searching the encyclopedia, that is, searching the elements in each tag set on Baidu Encyclopedia, and retaining the "alias" item in the search result as a synonym of the target word, as shown in the following example: is "City A", then is "an alias for City A" and is used Substitution of elements in The corresponding words in constitute new training data ( ), so as to quickly and accurately obtain the target training data set.

[0067] In one embodiment, before training the initial model based on the target training data set and the auxiliary classification task, the following steps are also included:

[0068] Build a low-rank adapter and obtain pre-trained parameters and pre-trained weights; based on the pre-trained weights, perform a weighted combination of the low-rank adapter and pre-trained parameters to obtain the target parameters.

[0069] Among them, the low-rank adapter is a method for fine-tuning pre-trained models, which aims to adapt to specific tasks by introducing the form of low-rank matrices, thereby improving fine-tuning efficiency and reducing computational and storage overhead during training.

[0070] Specifically, the pre-training parameters of the model often contain a large amount of semantic information obtained in the pre-training stage. Directly using the constructed fine-tuning dataset for parameter optimization can easily lead to overfitting. Therefore, in order to ensure the previous learning content, the target parameters of the final model are obtained by weighted combination. The specific calculation formula is as follows:

[0071]

[0072] In the above formula, is the target parameter, are pre-trained parameters, is a low-rank adapter.

[0073] In one embodiment, in the above embodiment, constructing a low-rank adapter specifically includes the following steps:

[0074] A first low-rank matrix, a second low-rank matrix and a bias matrix are obtained; the bias matrix is ​​used to adjust the bias; the first low-rank matrix and the second low-rank matrix are multiplied to obtain a target low-rank matrix; the target low-rank matrix and the bias matrix are added to obtain a low-rank adapter.

[0075] Specifically, the low-rank adapter is mainly composed of two parts, low-rank matrices A and B, and a bias term C. The product of A and B constitutes a low-rank matrix, which is used to adjust the weight of the model. Matrix C is used to adjust the bias. The specific calculation formula of the low-rank adapter is as follows:

[0076]

[0077] Before training, A is initialized with a Gaussian distribution, and B and C are initialized to 0. During fine-tuning, the adapter parameters are updated with the gradient of the loss function. Low-rank adapters allow only the parameters of specific layers to be updated during model fine-tuning, which reduces the amount of parameters updated and reduces the amount of computation and memory required compared to traditional full parameter updates.

[0078] In one embodiment, in the above step S205, obtaining the model fine-tuning loss specifically includes the following steps:

[0079] Get the cross entropy loss and loss allocation weight of the pre-trained model; based on the loss allocation weight, perform a weighted combination of the cross entropy loss and the classification loss to obtain the model fine-tuning loss.

[0080] Specifically, the model fine-tuning loss is calculated as follows:

[0081]

[0082] In the above formula, is the cross entropy loss of the model, is the classification loss for the auxiliary task.

[0083] In this embodiment, by obtaining the cross entropy loss and loss allocation weight of the pre-trained model, based on the loss allocation weight, the cross entropy loss and classification loss are weightedly combined in the calculation formula of the input model fine-tuning loss, so as to quickly and accurately obtain the model fine-tuning loss.

[0084] In one embodiment, in the above step S203, constructing an auxiliary classification task according to multiple synonyms, negative examples and classification losses specifically includes the following steps:

[0085] Multiple synonyms and negative examples are labeled respectively to obtain first labels corresponding to the multiple synonyms and second labels corresponding to the negative examples; training data for an auxiliary classification task is generated according to the first labels, the second labels and the classification loss; and an auxiliary classification task is constructed based on the training data.

[0086] Specifically, the server can extract the place name Synonyms with the same semantics are marked as "1" and will be compared with the extracted place names. Negative examples with opposite semantics are marked as "2", and their corresponding classification loss is , according to the "1" label, "2" label and classification loss, generate training data for auxiliary classification tasks; finally, based on the training data, construct auxiliary classification tasks, which is conducive to enabling the model to have a deeper understanding of different expressions of the same place name.

[0087] In one embodiment, Figure 4 As shown, a method for identifying place names in a specific embodiment is provided, which specifically includes the following steps:

[0088] Step S401, obtaining initial training data, converting the format of the initial training data, and obtaining current training data consisting of the text to be extracted and the extracted place name.

[0089] Step S402, query multiple synonyms corresponding to the extracted place names, and for each synonym, use the synonym to replace the corresponding vocabulary in the text to be extracted to obtain the target text to be extracted; generate target training data corresponding to the synonym based on the synonym and the target text to be extracted; integrate the current training data and the target training data corresponding to each synonym to obtain the target training data set.

[0090] Step S403, obtain a number of negative examples with semantic differences from the extracted place names and the classification loss corresponding to each negative example, mark multiple synonyms and negative examples respectively, obtain first labels corresponding to multiple synonyms and second labels corresponding to negative examples; generate training data for the auxiliary classification task according to the first label, the second label and the classification loss; and construct the auxiliary classification task based on the training data.

[0091] Step S404, obtaining a first low-rank matrix, a second low-rank matrix and a bias item matrix; the bias item matrix is ​​used to adjust the bias; multiplying the first low-rank matrix and the second low-rank matrix to obtain a target low-rank matrix; adding the target low-rank matrix and the bias item matrix to obtain a low-rank adapter.

[0092] Step S405, obtaining pre-training parameters and pre-training weights; based on the pre-training weights, performing weighted combination on the low-rank adapter and the pre-training parameters to obtain target parameters.

[0093] Step S406, obtaining the cross entropy loss and loss allocation weight of the pre-trained model; based on the loss allocation weight, performing a weighted combination of the cross entropy loss and the classification loss to obtain the model fine-tuning loss.

[0094] Step S407, adjusting the preliminary training model according to the model fine-tuning loss to obtain a pre-trained model; the pre-trained model is used for place name recognition.

[0095] The beneficial effects brought by the above embodiments are as follows:

[0096] 1) This application obtains different expressions of the target place name through search to enrich the training data.

[0097] 2) This application introduces an auxiliary classification task to enable the model to better understand place names expressed in different ways.

[0098] 3) This application only selects specific layers and uses low-rank matrices during fine-tuning, while also retaining some original parameters, effectively reducing resource consumption, improving the robustness of place name recognition, and alleviating the problems of overfitting and knowledge forgetting.

[0099] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0100] Based on the same inventive concept, the embodiment of the present application also provides a place name recognition device for implementing the place name recognition method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more place name recognition device embodiments provided below can refer to the limitations of the place name recognition method above, and will not be repeated here.

[0101] In an exemplary embodiment, Figure 5 As shown, a place name recognition device is provided, comprising:

[0102] The format conversion module 501 is used to obtain initial training data, perform format conversion on the initial training data, and obtain current training data consisting of the text to be extracted and the extracted place name;

[0103] A data enhancement module 502 is used to query multiple synonyms corresponding to the extracted place names, and based on the multiple synonyms, perform data enhancement on the current training data to obtain a target training data set;

[0104] An auxiliary classification module 503 is used to obtain a number of negative examples with semantic differences from the extracted place names and classification losses corresponding to the negative examples, and construct an auxiliary classification task according to a plurality of synonyms, negative examples and classification losses;

[0105] A model training module 504 is used to train the initial model based on the target training data set and the auxiliary classification task until the model parameters of the initial model are the same as the target parameters, thereby obtaining a preliminary training model;

[0106] The model adjustment module 505 is used to obtain the model fine-tuning loss, and adjust the preliminary training model according to the model fine-tuning loss to obtain a pre-trained model; the pre-trained model is used for place name recognition.

[0107] In one embodiment, the data enhancement module 502 is also used to replace the corresponding vocabulary in the text to be extracted with the synonym for each synonym to obtain the target text to be extracted; generate target training data corresponding to the synonym based on the synonym and the target text to be extracted; integrate the current training data and the target training data corresponding to each synonym to obtain the target training data set.

[0108] In one embodiment, the place name recognition device also includes a parameter acquisition module for constructing a low-rank adapter and acquiring pre-trained parameters and pre-trained weights; based on the pre-trained weights, the low-rank adapter and pre-trained parameters are weighted combined to obtain target parameters.

[0109] In one embodiment, the parameter acquisition module is also used to obtain a first low-rank matrix, a second low-rank matrix and a bias item matrix; the bias item matrix is ​​used to adjust the bias; the first low-rank matrix and the second low-rank matrix are multiplied to obtain a target low-rank matrix; the target low-rank matrix and the bias item matrix are added to obtain a low-rank adapter.

[0110] In one embodiment, the model adjustment module 505 is further used to obtain the cross entropy loss and loss allocation weight of the pre-trained model; based on the loss allocation weight, the cross entropy loss and the classification loss are weightedly combined to obtain the model fine-tuning loss.

[0111] In one embodiment, the auxiliary classification module 503 is further used to label multiple synonyms and negative examples respectively to obtain first labels corresponding to the multiple synonyms and second labels corresponding to the negative examples; generate training data for the auxiliary classification task according to the first label, the second label and the classification loss; and construct an auxiliary classification task based on the training data.

[0112] Each module in the above-mentioned place name recognition device can be implemented in whole or in part by software, hardware and a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0113] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 6As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store initial training data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for recognizing place names is implemented.

[0114] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0115] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0116] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0117] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0118] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0119] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0120] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0121] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for identifying place names, characterized in that: The method comprises: Acquire initial training data, and convert the format of the initial training data to obtain current training data consisting of the text to be extracted and the extracted place name; Querying multiple synonyms corresponding to the extracted place names, and based on the multiple synonyms, performing data enhancement on the current training data to obtain a target training data set; Acquire a plurality of negative examples having semantic differences from the extracted place names and classification losses corresponding to the negative examples, and construct an auxiliary classification task according to the plurality of synonyms, the negative examples and the classification losses; Based on the target training data set and the auxiliary classification task, the initial model is trained until the model parameters of the initial model are the same as the target parameters, thereby obtaining a preliminary training model; A model fine-tuning loss is obtained, and the preliminary training model is adjusted according to the model fine-tuning loss to obtain a pre-trained model; the pre-trained model is used for place name recognition.

2. The method according to claim 1, characterized in that The step of performing data enhancement on the current training data based on the plurality of synonyms to obtain a target training data set includes: For each of the synonyms, the corresponding words in the text to be extracted are replaced with the synonyms to obtain the target text to be extracted; Generating target training data corresponding to the synonyms according to the synonyms and the target text to be extracted; The current training data and the target training data corresponding to each of the synonyms are integrated to obtain the target training data set.

3. The method according to claim 1, characterized in that Before training the initial model based on the target training data set and the auxiliary classification task, the method further includes: Build low-rank adapters and obtain pre-trained parameters and pre-trained weights; Based on the pre-trained weights, the low-rank adapter and the pre-trained parameters are weighted combined to obtain the target parameters.

4. The method according to claim 3, characterized in that The low-rank adapter is constructed, comprising: Obtaining a first low-rank matrix, a second low-rank matrix, and a bias item matrix; the bias item matrix is ​​used to adjust the bias; Multiplying the first low-rank matrix and the second low-rank matrix to obtain a target low-rank matrix; The target low-rank matrix and the bias matrix are added to obtain the low-rank adapter.

5. The method according to claim 1, characterized in that The obtaining of the model fine-tuning loss includes: Obtaining the cross entropy loss and loss distribution weight of the pre-trained model; Based on the loss allocation weight, the cross entropy loss and the classification loss are weightedly combined to obtain the model fine-tuning loss.

6. The method according to any one of claims 1 to 5, characterized in that: The constructing an auxiliary classification task according to the plurality of synonyms, the negative examples and the classification loss comprises: Marking the multiple synonyms and the negative examples respectively to obtain first labels corresponding to the multiple synonyms and second labels corresponding to the negative examples; Generate training data for the auxiliary classification task according to the first label, the second label, and the classification loss; Based on the training data, the auxiliary classification task is constructed.

7. A place name recognition device, characterized in that: The device comprises: A format conversion module, used for acquiring initial training data, performing format conversion on the initial training data, and obtaining current training data consisting of the text to be extracted and the extracted place name; A data enhancement module, used for querying a plurality of synonyms corresponding to the extracted place names, and performing data enhancement on the current training data based on the plurality of synonyms to obtain a target training data set; An auxiliary classification module, used to obtain a plurality of negative examples having semantic differences from the extracted place names and classification losses corresponding to the negative examples, and construct an auxiliary classification task according to the plurality of synonyms, the negative examples and the classification losses; A model training module, used to train the initial model based on the target training data set and the auxiliary classification task until the model parameters of the initial model are the same as the target parameters, thereby obtaining a preliminary training model; The model adjustment module is used to obtain the model fine-tuning loss, and adjust the preliminary training model according to the model fine-tuning loss to obtain a pre-trained model; the pre-trained model is used for place name recognition.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images