Place name recognition method and device, computer device, readable storage medium and program product

Abstract

The application relates to a place name recognition method and device, equipment, a storage medium and a program product, and relates to the technical field of natural language processing. The method can alleviate the problems of overfitting and knowledge forgetting. The method comprises the following steps: performing format conversion on obtained initial training data to obtain current training data composed of to-be-extracted text and extracted place names; a plurality of synonyms corresponding to the extracted place names are queried, and the current training data is subjected to data enhancement based on the plurality of synonyms to obtain a target training data set; an auxiliary classification task is constructed according to the plurality of synonyms, negative examples and classification loss; the initial model is trained based on the target training data set and the auxiliary classification task until the model parameters of the initial model are the same as target parameters, and a preliminary training model is obtained; a model fine-tuning loss is obtained, the preliminary training model is adjusted according to the model fine-tuning loss, and a pre-training model is obtained; and the pre-training model is used for place name recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, in particular to a place name recognition method and device, computer equipment, computer readable storage medium and computer program product. BACKGROUND

[0002] With the development of computer technology, named entity recognition technology appears, which is an important part of natural language processing. The purpose of this task is to identify entity names with specific meanings in text, and place name recognition is undoubtedly an important part.

[0003] In recent years, deep learning-based methods have developed greatly, and pre-trained models have appeared. Since these models have been trained on a large amount of text, they can better understand the semantics of the text, so fine-tuning the pre-trained model is also an effective method. However, fine-tuning all parameters often requires excessive resources, and if the number of data sets is not large enough, it is easy to cause overfitting and knowledge forgetting problems. SUMMARY

[0004] Therefore, it is necessary to provide a place name recognition method, device, computer equipment, computer readable storage medium and computer program product to solve the above technical problems.

[0005] In a first aspect, the present application provides a place name recognition method, comprising:

[0006] Obtaining initial training data, performing format conversion on the initial training data to obtain current training data composed of to-be-extracted text and extracted place names;

[0007] Querying a plurality of synonyms corresponding to the extracted place names, and performing data augmentation on the current training data based on the plurality of synonyms to obtain a target training data set;

[0008] Obtaining a plurality of negative examples with different semantics from the extracted place names and classification losses corresponding to the negative examples, and constructing an auxiliary classification task according to the plurality of synonyms, the negative examples and the classification losses;

[0009] Training an initial model based on the target training data set and the auxiliary classification task until the model parameters of the initial model are the same as the target parameters to obtain a preliminary training model;

[0010] Obtaining a model fine-tuning loss, adjusting the preliminary training model according to the model fine-tuning loss to obtain a pre-trained model; the pre-trained model is used for place name recognition.

[0011] In one of the embodiments, the data augmentation on the current training data based on the plurality of synonyms comprises:

[0012] For each of the synonyms, the corresponding words in the text to be extracted are replaced by the synonym to obtain target text to be extracted; the target training data corresponding to the synonym is generated according to the synonym and the target text to be extracted; and the current training data and the target training data corresponding to each of the synonyms are integrated to obtain the target training data set.

[0013] In one of the embodiments, before the training of the initial model based on the target training data set and the auxiliary classification task, the method further comprises:

[0014] a low-rank adapter is constructed, pre-training parameters and pre-training weights are obtained, and the low-rank adapter and the pre-training parameters are combined based on the pre-training weights to obtain the target parameters.

[0015] In one of the embodiments, the construction of the low-rank adapter comprises:

[0016] a first low-rank matrix, a second low-rank matrix and a bias term matrix are obtained, the bias term matrix is used to adjust the bias, the first low-rank matrix and the second low-rank matrix are multiplied to obtain a target low-rank matrix, and the target low-rank matrix and the bias term matrix are added to obtain the low-rank adapter.

[0017] In one of the embodiments, the obtaining of the model fine-tuning loss comprises:

[0018] The cross-entropy loss of the pre-training model and a loss distribution weight are obtained, and the cross-entropy loss and the classification loss are combined based on the loss distribution weight to obtain the model fine-tuning loss.

[0019] In one of the embodiments, the construction of the auxiliary classification task according to the plurality of synonyms, the negative examples and the classification loss comprises:

[0020] The plurality of synonyms and the negative examples are labeled respectively to obtain first labels corresponding to the plurality of synonyms and second labels corresponding to the negative examples; training data of the auxiliary classification task is generated according to the first labels, the second labels and the classification loss; and the auxiliary classification task is constructed based on the training data.

[0021] In a second aspect, the application further provides a place name recognition device, comprising:

[0022] The format conversion module is configured to obtain initial training data, perform format conversion on the initial training data, and obtain current training data composed of to-be-extracted text and extracted place names.

[0023] The data enhancement module is configured to query a plurality of synonyms corresponding to the extracted place names, perform data enhancement on the current training data based on the plurality of synonyms, and obtain a target training data set.

[0024] The auxiliary classification module is configured to obtain a plurality of negative examples different in semantics from the extracted place names and classification losses corresponding to the negative examples, construct an auxiliary classification task according to the plurality of synonyms, the negative examples, and the classification losses.

[0025] The model training module is configured to train an initial model based on the target training data set and the auxiliary classification task until model parameters of the initial model are the same as target parameters, and obtain a preliminary training model.

[0026] The model adjustment module is configured to obtain a model fine-tuning loss, adjust the preliminary training model according to the model fine-tuning loss, and obtain a pre-training model; the pre-training model is used for place name recognition.

[0027] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, the memory stores a computer program, and the processor realizes the following steps when executing the computer program:

[0028] obtain initial training data, perform format conversion on the initial training data, and obtain current training data composed of to-be-extracted text and extracted place names; query a plurality of synonyms corresponding to the extracted place names, perform data enhancement on the current training data based on the plurality of synonyms, and obtain a target training data set; obtain a plurality of negative examples different in semantics from the extracted place names and classification losses corresponding to the negative examples, construct an auxiliary classification task according to the plurality of synonyms, the negative examples, and the classification losses; train an initial model based on the target training data set and the auxiliary classification task until model parameters of the initial model are the same as target parameters, and obtain a preliminary training model; obtain a model fine-tuning loss, adjust the preliminary training model according to the model fine-tuning loss, and obtain a pre-training model; the pre-training model is used for place name recognition.

[0029] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to realize the following steps:

[0030] The initial training data is obtained, the initial training data is format-converted to obtain current training data composed of to-be-extracted text and extracted place names, a plurality of synonyms corresponding to the extracted place names are queried, the current training data is data-augmented based on the plurality of synonyms to obtain a target training data set, a plurality of negative examples different in semantics from the extracted place names and classification losses corresponding to the negative examples are obtained, an auxiliary classification task is constructed according to the plurality of synonyms, the negative examples and the classification losses, an initial model is trained based on the target training data set and the auxiliary classification task until model parameters of the initial model are the same as target parameters, an initial training model is obtained, a model fine-tuning loss is obtained, and the initial training model is adjusted according to the model fine-tuning loss to obtain a pre-training model, and the pre-training model is used for place name recognition.

[0031] In a fifth aspect, the present application also provides a computer program product comprising a computer program which, when executed by a processor, implements the following steps:

[0032] The initial training data is obtained, the initial training data is format-converted to obtain current training data composed of to-be-extracted text and extracted place names, a plurality of synonyms corresponding to the extracted place names are queried, the current training data is data-augmented based on the plurality of synonyms to obtain a target training data set, a plurality of negative examples different in semantics from the extracted place names and classification losses corresponding to the negative examples are obtained, an auxiliary classification task is constructed according to the plurality of synonyms, the negative examples and the classification losses, an initial model is trained based on the target training data set and the auxiliary classification task until model parameters of the initial model are the same as target parameters, an initial training model is obtained, a model fine-tuning loss is obtained, and the initial training model is adjusted according to the model fine-tuning loss to obtain a pre-training model, and the pre-training model is used for place name recognition.

[0033] The place name recognition method, device, computer device, computer readable storage medium and computer program product obtain current training data composed of to-be-extracted text and extracted place names by format-converting initial training data, search and query a plurality of synonyms corresponding to the extracted place names as different expression modes to enrich training data, and introduce an auxiliary classification task to train a model to learn semantic relationships between different words, so that the model can better understand place names expressed in different ways. In addition, only a specific layer is selected to add a low-rank matrix in the fine-tuning process, while parameters of the remaining layers remain unchanged. In the updating process, the parameters of the layer are a weighted combination of the low-rank matrix parameters and the original parameters, so that the resource consumption is effectively reduced, the robustness of the place name recognition is improved, and the problems of overfitting and knowledge forgetting are alleviated. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a diagram illustrating the application environment of a place name recognition method in one embodiment.

[0036] Figure 2 This is a flowchart illustrating a place name recognition method in one embodiment;

[0037] Figure 3 This is a flowchart illustrating the data augmentation steps in one embodiment;

[0038] Figure 4 This is a flowchart illustrating a place name recognition method in a specific embodiment;

[0039] Figure 5 This is a structural block diagram of a place name recognition device in one embodiment;

[0040] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0042] The place name recognition method provided in this application can be applied to, for example... Figure 1 The application environment shown depicts a scenario where the terminal communicates with the server via a network. The data storage system stores the data that the server needs to process. This data storage system can be integrated onto the server, or it can be hosted in the cloud or on other network servers.

[0043] Specifically, the place name recognition method provided in this application embodiment can be executed by a server.

[0044] Exemplarily, the server obtains initial training data, converts the initial training data in format, and obtains current training data composed of to-be-extracted text and extracted place names; the server queries a plurality of synonyms corresponding to the extracted place names, performs data enhancement on the current training data based on the plurality of synonyms, and obtains a target training data set; the server obtains a plurality of negative examples different in semantics from the extracted place names and classification losses corresponding to the negative examples, constructs an auxiliary classification task according to the plurality of synonyms, the negative examples and the classification losses, trains the initial model based on the target training data set and the auxiliary classification task, until model parameters of the initial model are the same as target parameters, and obtains a preliminary training model; the server obtains a model fine-tuning loss, adjusts the preliminary training model according to the model fine-tuning loss, and obtains a pre-training model; and the pre-training model is used for place name recognition.

[0045] In the application environment as shown in Figure 1 , the terminal can be, but is not limited to, various personal computers, notebook computers, smart phones and tablet computers. The server can be implemented by an independent server or a server cluster composed of multiple servers.

[0046] In one embodiment, as shown in Figure 2 , a place name recognition method is provided. Taking the server in Figure 1 as an example, the method comprises the following steps:

[0047] Step S201, obtaining initial training data, converting the initial training data in format, and obtaining current training data composed of to-be-extracted text and extracted place names.

[0048] Specifically, the server collects initial training data for model fine-tuning, and converts the format of the initial training data to: , wherein is the to-be-extracted text, is the extracted place name, and the current training data is obtained, such as: may be "Next month Xiaoming will go to City A for fun", and "City A" is.

[0049] Step S202, querying a plurality of synonyms corresponding to the extracted place names, and performing data enhancement on the current training data based on the plurality of synonyms to obtain a target training data set.

[0050] Specifically, the server obtains all label sets In order to perform data enhancement, firstly, as many aliases of each place name as possible are acquired. The application acquires the aliases by means of encyclopedia search, that is, each element in the label set is searched in Baidu Encyclopedia, and the "alias" item in the search return result is retained as a synonym of the target word, to obtain a target training data set. For example, if is "A city", then is an alias of "A city".

[0051] In step S203, a plurality of negative examples different from the semantic of the extracted place name and a classification loss corresponding to the negative examples are acquired, and an auxiliary classification task is constructed according to the plurality of synonyms, the negative examples and the classification loss.

[0052] Specifically, in order to improve the understanding ability of the model for the place name related content, the application also designs an auxiliary classification task. For each element in the label set, k (k is a hyperparameter) words are randomly selected from the synonyms of other elements as negative examples The training data of the auxiliary classification task is as follows:

[0053]

[0054] The corresponding classification loss is , so that the model can more deeply understand different expressions of the same place name.

[0055] In step S204, the initial model is trained based on the target training data set and the auxiliary classification task, until the model parameters of the initial model are the same as the target parameters, to obtain a preliminary training model.

[0056] Specifically, the server acquires the target parameters by weighted combination, and then iteratively trains the initial model based on the target training data set and the auxiliary classification task, until the model parameters of the initial model are the same as the target parameters, to obtain a preliminary training model.

[0057] In step S205, a model fine-tuning loss is acquired, and the preliminary training model is adjusted according to the model fine-tuning loss, to obtain a pre-training model; the pre-training model is used for place name recognition.

[0058] It should be noted that model fine-tuning is a strategy of further training on the basis of a pre-training model for a specific task. Usually, fine-tuning is performed on a model pre-trained on a large-scale data set, and the model is further trained on a smaller, specific task data set, so that the model can better adapt to the requirements of the task.

[0059] The fine-tuning loss is a loss function selected based on the target task, and is usually consistent with the target task (such as the loss functions of classification, regression and generation tasks).

[0060] Specifically, the server obtains a model fine-tuning loss by a weighted combination manner based on the loss assignment weight, and adjusts the preliminary training model according to the model fine-tuning loss to obtain a pre-training model, which is used for place name recognition.

[0061] The above place name recognition method efficiently obtains current training data composed of the text to be extracted and the extracted place name by performing format conversion on the initial training data, and extracts a plurality of synonyms corresponding to the extracted place name through search query as different expression manners for enriching the training data. On this basis, an auxiliary classification task is introduced for the model to learn the semantic relationship between different vocabularies, so as to enable the model to better understand the place names expressed in different manners. In addition, only a specific layer is selected to add a low-rank matrix during fine-tuning, while the parameters of the remaining layers remain unchanged. The parameters of the layer in the updating process are a weighted combination of the low-rank matrix parameters and the original parameters, thereby effectively reducing resource consumption, improving the robustness of place name recognition, and alleviating the problems of overfitting and knowledge forgetting.

[0062] In one of the embodiments, as shown in Figure 3 The step S201 includes the following steps:

[0063] In step S301, for each synonym, the corresponding vocabulary in the text to be extracted is replaced by the synonym to obtain the target text to be extracted.

[0064] In step S302, the target training data corresponding to the synonym is generated according to the synonym and the target text to be extracted.

[0065] In step S303, the current training data and the target training data corresponding to each synonym are integrated to obtain the target training data set.

[0066] Specifically, the server obtains the target training data set by searching the encyclopedia, i.e., searching the Baidu Encyclopedia for each element in the label set, and retaining the "alias" item in the search return result as the synonym of the target word to obtain the target training data set. For example, if is "A city", then is an alias of "A city", and the elements in are used to replace the corresponding vocabularies in to form new training data , so as to quickly and accurately obtain the target training data set.

[0067] In one of the embodiments, before training the initial model based on the target training data set and the auxiliary classification task, the following steps are further included:

[0068] constructing a low-rank adapter, and obtaining pre-training parameters and pre-training weights; and performing weighted combination on the low-rank adapter and the pre-training parameters based on the pre-training weights to obtain target parameters.

[0069] The low-rank adapter is a method for fine-tuning a pre-trained model, which aims to adapt to specific tasks by introducing a low-rank matrix form, thereby improving fine-tuning efficiency and reducing calculation and storage overheads during training.

[0070] Specifically, the pre-training parameters of the model often contain a large amount of semantic information obtained during the pre-training phase, and directly using the constructed fine-tuning data set for parameter optimization can easily lead to overfitting. Therefore, in order to ensure the previous learning content, the target parameters of the final model are obtained by using a weighted combination method, and the specific calculation formula is as follows:

[0071]

[0072] In the above formula, is the target parameter, is the pre-training parameter, is the low-rank adapter.

[0073] In one of the embodiments, the low-rank adapter is constructed, and the construction specifically includes the following steps:

[0074] obtaining a first low-rank matrix, a second low-rank matrix, and a bias term matrix; the bias term matrix is used to adjust the bias; multiplying the first low-rank matrix and the second low-rank matrix to obtain a target low-rank matrix; and adding the target low-rank matrix and the bias term matrix to obtain the low-rank adapter.

[0075] Specifically, the low-rank adapter mainly consists of two parts, low-rank matrices A and B, and a bias term C. The product of A and B constitutes a low-rank matrix, which is used to adjust the weight of the model, and the matrix C is used to adjust the bias. The specific calculation formula of the low-rank adapter is as follows:

[0076]

[0077] Before training, A is initialized with a Gaussian distribution, B and C are initialized to 0, and during the fine-tuning process, the parameters of the adapter are updated with the gradient of the loss function. The low-rank adapter allows only specific layer parameters to be updated during model fine-tuning, reducing the amount of updated parameters. Compared with traditional full parameter update, the low-rank adapter reduces the amount of calculation and the demand for memory.

[0078] In one of the embodiments, in the step S205, the model fine-tuning loss is obtained, and the obtaining specifically includes the following steps:

[0079] The cross-entropy loss of the pre-trained model and a loss distribution weight are obtained; and the cross-entropy loss and the classification loss are combined by weighting based on the loss distribution weight to obtain the model fine-tuning loss.

[0080] Specifically, the calculation formula of the model fine-tuning loss is as follows:

[0081]

[0082] In the above formula, is the cross-entropy loss of the model, is the classification loss of the auxiliary task.

[0083] In the embodiment, the cross-entropy loss of the pre-trained model and the loss distribution weight are obtained, the cross-entropy loss and the classification loss are combined by weighting based on the loss distribution weight in the calculation formula of the model fine-tuning loss, so that the model fine-tuning loss is quickly and accurately obtained.

[0084] In one of the embodiments, in the step S203, the auxiliary classification task is constructed according to the multiple synonyms, the negative examples and the classification loss, and specifically includes the following steps:

[0085] The multiple synonyms and the negative examples are respectively marked to obtain the first label corresponding to the multiple synonyms and the second label corresponding to the negative examples; the training data of the auxiliary classification task is generated according to the first label, the second label and the classification loss; and the auxiliary classification task is constructed based on the training data.

[0086] Specifically, the server can mark the synonyms with the same semantics as the extracted place names as "1", and mark the negative examples with opposite semantics as the extracted place names as "2", and the corresponding classification loss is The training data of the auxiliary classification task is generated according to the "1" label, the "2" label and the classification loss; and finally the auxiliary classification task is constructed based on the training data, which is beneficial to enable the model to more deeply understand different expression forms of the same place name.

[0087] In one of the embodiments, as shown in Figure 4 , a place name recognition method in one specific embodiment is provided, and specifically includes the following steps:

[0088] In step S401, the initial training data is obtained, and the initial training data is format-converted to obtain the current training data composed of the to-be-extracted text and the extracted place names.

[0089] In step S402, a plurality of synonyms corresponding to the extracted place name are queried, for each synonym, a corresponding word in the to-be-extracted text is replaced by using the synonym to obtain a target to-be-extracted text; target training data corresponding to the synonym is generated according to the synonym and the target to-be-extracted text; the current training data and the target training data corresponding to each synonym are integrated to obtain a target training data set.

[0090] In step S403, a plurality of negative examples with different semantics from the extracted place name and classification losses corresponding to the negative examples are obtained, the plurality of synonyms and the negative examples are labeled respectively to obtain first labels corresponding to the plurality of synonyms and second labels corresponding to the negative examples; training data of an auxiliary classification task is generated according to the first labels, the second labels and the classification losses; and the auxiliary classification task is constructed based on the training data.

[0091] In step S404, a first low-rank matrix, a second low-rank matrix and a bias term matrix are obtained; the bias term matrix is used to adjust a bias; the first low-rank matrix and the second low-rank matrix are multiplied to obtain a target low-rank matrix; and the target low-rank matrix and the bias term matrix are added to obtain a low-rank adapter.

[0092] In step S405, pre-training parameters and pre-training weights are obtained; the low-rank adapter and the pre-training parameters are combined by weighting based on the pre-training weights to obtain target parameters.

[0093] In step S406, a cross-entropy loss of a pre-training model and a loss distribution weight are obtained; the cross-entropy loss and the classification loss are combined by weighting based on the loss distribution weight to obtain a model fine-tuning loss.

[0094] In step S407, the preliminary training model is adjusted according to the model fine-tuning loss to obtain a pre-training model; the pre-training model is used for place name recognition.

[0095] The beneficial effects brought by the above embodiments are as follows:

[0096] 1) The present application searches to obtain different expression modes of the target place name, which is used to enrich the training data.

[0097] 2) The present application introduces an auxiliary classification task, which aims to enable the model to better understand the place names with different expression modes.

[0098] 3) The present application selects only specific layers and uses a low-rank matrix when fine-tuning, while also retaining part of the original parameters, which effectively reduces resource consumption, improves the robustness of place name recognition, and alleviates the problems of overfitting and knowledge forgetting.

[0099] It should be understood that although the steps in the flowcharts involved in the embodiments described above are shown in sequence according to the arrows, the steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, the execution of the steps is not strictly limited in sequence, and the steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts involved in the embodiments described above can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of the steps or stages is not necessarily sequential, but can be alternately or alternately executed with at least part of other steps or steps or stages in other steps.

[0100] Based on the same inventive concept, the embodiments of the present application also provide a place name recognition device for implementing the above-mentioned place name recognition method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more place name recognition device embodiments provided below can refer to the limitations of the place name recognition method described above, which will not be repeated here.

[0101] In one exemplary embodiment, as shown in Figure 5 A place name recognition device is provided, comprising:

[0102] The format conversion module 501 is configured to obtain initial training data, perform format conversion on the initial training data, and obtain current training data composed of to-be-extracted text and extracted place names.

[0103] The data enhancement module 502 is configured to query a plurality of synonyms corresponding to the extracted place names, perform data enhancement on the current training data based on the plurality of synonyms, and obtain a target training data set.

[0104] The auxiliary classification module 503 is configured to obtain a plurality of negative examples with different semantics from the extracted place names and classification losses corresponding to the negative examples, and construct an auxiliary classification task according to the plurality of synonyms, the negative examples, and the classification losses.

[0105] The model training module 504 is configured to train the initial model based on the target training data set and the auxiliary classification task until the model parameters of the initial model are the same as the target parameters, and obtain a preliminary training model.

[0106] The model adjustment module 505 is configured to obtain a model fine-tuning loss, adjust the preliminary training model according to the model fine-tuning loss, and obtain a pre-training model. The pre-training model is used for place name recognition.

[0107] In an embodiment, the data enhancement module 502 is further configured to replace, for each synonym, a corresponding word in the text to be extracted with the synonym to obtain target text to be extracted; generate target training data corresponding to the synonym according to the synonym and the target text to be extracted; and integrate the current training data and the target training data corresponding to each synonym to obtain a target training data set.

[0108] In an embodiment, the place name recognition apparatus further includes a parameter acquisition module configured to construct a low-rank adapter and acquire pre-training parameters and pre-training weights; and combine the low-rank adapter and the pre-training parameters based on the pre-training weights to obtain target parameters.

[0109] In an embodiment, the parameter acquisition module is further configured to acquire a first low-rank matrix, a second low-rank matrix, and a bias term matrix; adjust a bias using the bias term matrix; multiply the first low-rank matrix and the second low-rank matrix to obtain a target low-rank matrix; and add the target low-rank matrix and the bias term matrix to obtain the low-rank adapter.

[0110] In an embodiment, the model adjustment module 505 is further configured to acquire a cross-entropy loss of the pre-training model and a loss distribution weight; and combine the cross-entropy loss and the classification loss based on the loss distribution weight to obtain a model fine-tuning loss.

[0111] In an embodiment, the auxiliary classification module 503 is further configured to mark a plurality of synonyms and negative examples respectively to obtain first labels corresponding to the plurality of synonyms and second labels corresponding to the negative examples; generate training data for an auxiliary classification task according to the first labels, the second labels, and a classification loss; and construct the auxiliary classification task based on the training data.

[0112] The above-described modules of the place name recognition apparatus can be implemented in whole or in part by software, hardware, or a combination thereof. The above-described modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in the computer device in software form, so as to be called and executed by a processor to perform operations corresponding to the above-described modules.

[0113] In an exemplary embodiment, a computer device, which can be a server, is provided, and an internal structure diagram of the computer device can be as shown in FIG. 8. Figure 6As shown in the figure. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through the system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store initial training data. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through network connection. The computer program is executed by the processor to implement a place name recognition method.

[0114] Those skilled in the art can understand that, Figure 6 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0115] In one embodiment, a computer device is also provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in each of the above method embodiments.

[0116] In one embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the steps in each of the above method embodiments.

[0117] In one embodiment, a computer program product is provided, including a computer program, and the computer program is executed by a processor to implement the steps in each of the above method embodiments.

[0118] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0119] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0120] The technical features of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present application.

[0121] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A method of place name recognition, characterized by, The method comprises: obtaining initial training data, and performing format conversion on the initial training data to obtain current training data composed of to-be-extracted text and extracted place names; querying a plurality of synonyms corresponding to the extracted place names, and performing data augmentation on the current training data based on the plurality of synonyms to obtain a target training data set; obtaining a plurality of negative examples different in semantics from the extracted place names and classification losses corresponding to the negative examples, and constructing an auxiliary classification task according to the plurality of synonyms, the negative examples and the classification losses; training an initial model based on the target training data set and the auxiliary classification task until model parameters of the initial model are the same as target parameters, to obtain a preliminary training model; obtaining a model fine-tuning loss, and adjusting the preliminary training model according to the model fine-tuning loss to obtain a pre-training model; the pre-training model is used for place name recognition.

2. The method of claim 1, wherein, The method comprises: for each synonym, replacing a corresponding word in the to-be-extracted text with the synonym to obtain target to-be-extracted text; generating target training data corresponding to the synonym according to the synonym and the target to-be-extracted text; integrating the current training data and the target training data corresponding to each synonym to obtain the target training data set.

3. The method of claim 1, wherein, Before training the initial model based on the target training data set and the auxiliary classification task, the method further comprises: constructing a low-rank adapter, and obtaining pre-training parameters and pre-training weights; weighting and combining the low-rank adapter and the pre-training parameters based on the pre-training weights to obtain the target parameters.

4. The method of claim 3, wherein, The method comprises: obtaining a first low-rank matrix, a second low-rank matrix and a bias term matrix; the bias term matrix is used to adjust bias; multiplying the first low-rank matrix and the second low-rank matrix to obtain a target low-rank matrix; adding the target low-rank matrix and the bias term matrix to obtain the low-rank adapter.

5. The method of claim 1, wherein, The method comprises: obtaining a cross-entropy loss of the pre-training model and a loss distribution weight; weighting and combining the cross-entropy loss and the classification loss based on the loss distribution weight to obtain the model fine-tuning loss.

6. The method according to any one of claims 1 to 5, characterized in that, The method comprises: respectively labeling the plurality of synonyms and the negative examples to obtain first labels corresponding to the plurality of synonyms and second labels corresponding to the negative examples; generating training data of the auxiliary classification task according to the first labels, the second labels and the classification losses; constructing the auxiliary classification task based on the training data.

7. A place name recognition apparatus characterized by comprising: The device comprises: a format conversion module configured to obtain initial training data, and perform format conversion on the initial training data to obtain current training data composed of to-be-extracted text and extracted place names; The data enhancement module is configured to query a plurality of synonyms corresponding to the extracted place name, perform data enhancement on the current training data based on the plurality of synonyms, and obtain a target training data set; The auxiliary classification module is configured to obtain a plurality of negative examples different from the extracted place name in semantics and classification losses corresponding to the negative examples, construct an auxiliary classification task according to the plurality of synonyms, the negative examples, and the classification losses. The model training module is configured to train an initial model based on the target training data set and the auxiliary classification task until model parameters of the initial model are the same as target parameters, and obtain a preliminary training model. The model adjustment module is configured to obtain a model fine-tuning loss, adjust the preliminary training model according to the model fine-tuning loss, and obtain a pre-training model; the pre-training model is used for place name recognition. 8.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-7. The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 6.

Images