Training method and device of natural language processing model, equipment and storage medium

Abstract

Embodiments of the present disclosure relate to a natural language processing model training method and device, equipment and a storage medium. The natural language processing model training method comprises: obtaining training data, wherein the training data comprises extensible markup language (XML) data of at least one page; extracting information of a control on the at least one page from the XML data of the at least one page; and training a preset natural language processing model based on the information of the control on the at least one page to obtain a natural language processing model that can be used to extract page features. The natural language processing model trained by the training method provided by the embodiments of the present disclosure has the ability to extract page features, thereby providing a basis for page analysis.

Description

TECHNICAL FIELD

[0001] The embodiments of the present disclosure relate to the technical field of artificial intelligence, and particularly, to a natural language processing model training method and device, equipment and a storage medium. BACKGROUND

[0002] In the related art, a natural language processing model (such as a pre-training language model, also referred to as a Bert model) can be used to extract semantic and grammatical features in natural language, thereby providing analysis basis for downstream tasks such as sentiment analysis, question and answer systems, etc. However, the natural language processing model cannot extract features in a page and cannot provide basis for page analysis. SUMMARY

[0003] To solve the above technical problems or at least partially solve the above technical problems, the embodiments of the present disclosure provide a natural language processing model training method, device, equipment and storage medium.

[0004] In a first aspect, the embodiments of the present disclosure provide a natural language processing model training method, comprising:

[0005] obtaining training data, wherein the training data includes extensible markup language (XML) data of at least one page;

[0006] extracting information of controls on the at least one page from the XML data of the at least one page;

[0007] training a preset natural language processing model based on the information of the controls on the at least one page to obtain a natural language processing model that can be used to extract page features.

[0008] Optionally, the extracting information of controls on the at least one page from the XML data of the at least one page comprises:

[0009] extracting at least one of the following information of the controls:

[0010] style information of the controls, types of content in the controls, and text used to describe the controls that has been included in the XML data.

[0011] Optionally, the training a preset natural language processing model based on the information of the controls on the at least one page comprises:

[0012] for each page, concatenating information of multiple controls included in the page to generate a sentence used to describe the page;

[0013] training the preset natural language processing model based on the sentences corresponding to the pages.

[0014] Optionally, the information of the plurality of controls included on the page is concatenated together to generate a sentence for describing the page, including:

[0015] The information of the plurality of controls included on the page is concatenated together in sequence based on the arrangement order of the plurality of controls on the page to generate a sentence for describing the page.

[0016] Optionally, the natural language processing model is trained based on the sentence corresponding to each page, including:

[0017] The sentences corresponding to each page are concatenated together to generate a target sentence;

[0018] The natural language processing model is trained based on the target sentence.

[0019] Optionally, the natural language processing model is trained based on the target sentence, including:

[0020] The target sentence is encoded based on a vocabulary to obtain a vector corresponding to the target sentence;

[0021] The natural language processing model is trained based on the vector;

[0022] The vocabulary includes a corresponding relationship between the information of the control and the encoding mode.

[0023] Optionally, the natural language processing model includes a pre-trained language model Bert;

[0024] The natural language processing model is trained based on the vector, including:

[0025] The Bert model is trained by masked language model (MLM) and / or next sentence prediction (NSP) based on the vector.

[0026] In a second aspect, the embodiments of the present disclosure provide a natural language processing model training device, including:

[0027] A data acquisition unit is configured to acquire training data, wherein the training data includes extensible markup language (XML) data of at least one page;

[0028] A control information extraction unit is configured to extract information of a control on the at least one page from the XML data of the at least one page;

[0029] A model training unit is configured to train a preset natural language processing model based on the information of the control on the at least one page to obtain a natural language processing model that can be used to extract page features.

[0030] Optionally, the control information extraction unit extracts information of the control from the XML data of the at least one page, including:

[0031] at least one of style information of the control, a type of content in the control, and text included in the XML data and used to describe the control.

[0032] Optionally, the model training unit includes:

[0033] a control information concatenator unit, configured to concatenate information of a plurality of controls included on each page together to generate a sentence used to describe the page;

[0034] a training sub-unit, configured to train a preset natural language processing model based on the sentence corresponding to each page.

[0035] Optionally, the control information concatenator unit concatenates the information of the plurality of controls included on the page together in sequence based on an arrangement order of the plurality of controls on the page to generate the sentence used to describe the page.

[0036] Optionally, the training sub-unit includes:

[0037] a target sentence concatenator module, configured to concatenate the sentences corresponding to the pages together to generate a target sentence;

[0038] a model training module, configured to train a preset natural language processing model based on the target sentence.

[0039] Optionally, the training module includes:

[0040] a vector generation sub-module, configured to perform encoding processing on the target sentence based on a vocabulary table to obtain a vector corresponding to the target sentence;

[0041] a model training sub-module, configured to train the natural language processing model based on the vector;

[0042] The vocabulary table includes a corresponding relationship between information of a control and an encoding manner.

[0043] Optionally, the natural language processing model includes a pre-trained language model Bert.

[0044] The model training sub-module performs masked language model MLM training and / or next sentence prediction NSP training on the Bert model based on the vector.

[0045] In a third aspect, an embodiment of the present disclosure provides a computing device, including:

[0046] A memory and a processor, wherein the memory has stored therein a computer program, and when the computer program is executed by the processor, the method of the first aspect can be implemented.

[0047] A fourth aspect of the embodiments of the present disclosure provides a computer readable storage medium, the storage medium has stored therein a computer program, and when the computer program is executed by a processor, the method of the first aspect can be implemented.

[0048] The technical solutions provided by the embodiments of the present disclosure have the following advantages compared with the prior art.

[0049] The technical solutions provided by the embodiments of the present disclosure have the following advantages compared with the prior art. BRIEF DESCRIPTION OF DRAWINGS

[0050] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and serve to explain the principles of the present disclosure together with the specification.

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced as follows, and obviously, other drawings can also be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0052] Figure 1 is a flow chart of a method for training a natural language processing model provided by the embodiments of the present disclosure;

[0053] Figure 2 is a flow chart of a method for training a natural language processing model provided by the embodiments of the present disclosure;

[0054] Figure 3 is a structural schematic diagram of a training device for a natural language processing model provided by the embodiments of the present disclosure;

[0055] Figure 4 is a structural schematic diagram of a computing device in the embodiments of the present disclosure. DETAILED DESCRIPTION

[0056] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced as follows, and obviously, other drawings can also be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0057] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0058] Figure 1 This is a flowchart illustrating a training method for a natural language processing model according to an embodiment of this disclosure. This method can be executed by a computing device. Exemplarily, the computing device can be an electronic device such as a desktop computer or an all-in-one computer, or it can be a server. Figure 1 As shown, the method provided in this embodiment includes steps S101-S103.

[0059] Step S101: Obtain training data, which includes at least one page of Extensible Markup Language (XML) data.

[0060] In this embodiment of the disclosure, the training data is raw sample data used to train a natural language processing model for extracting features from sample pages. The training data includes Extensible Markup Language (XML) data for at least one page, and each page's XML data contains sub-data representing controls on the page.

[0061] In this embodiment of the disclosure, the computing device can obtain training data using a preset method. In some implementations, the preset method may include reading pages or loading from local storage. For example, in one embodiment of the disclosure, the computing device can obtain the XML data of at least one page, i.e., the training data, by reading the page tree data structure stored in local storage. It should be noted that in other embodiments of the disclosure, the acquisition of training data may not be limited to reading pages or loading from local storage.

[0062] Step S102: Extract information about controls on at least one page from the XML data of at least one page.

[0063] Controls on a webpage are user interface objects used to interact with the user, enabling data input or manipulation. These controls can be buttons, list boxes, edit boxes, progress bars, or similar elements on the page.

[0064] The information about a control is the information in the XML data used to describe the characteristics of the control. Specifically, the information about a control can be static information, that is, data that only needs to be extracted from the XML and does not need to be loaded from the server.

[0065] In some embodiments of the present disclosure, the information of the control can include at least one of the following: style information of the control, type of content in the control, and text included in the XML data for describing the control.

[0066] The style information of the control represents information of a certain control style. For example, in the case that the control includes text, the style information of the control can include font of the text, font size of the text, and line spacing of the text. For another example, in the case that the control includes both image and text, the style information of the control can include relative position relationship between the image region and the text region, display size of the image, display region size of the text, font and font size of the text in the text region, etc.

[0067] The type of content in the control is used to represent a specific type of content contained in a certain control. For example, in the case that a certain control only includes text, the type of content in the control is text. For another example, in the case that a certain control includes both text and image, the type of content in the control can include text and image.

[0068] The text included in the XML data for describing the control is a text added by a control designer in the XML to represent the design concept when designing the control, or a text representing a specific feature of the control.

[0069] Of course, in other embodiments of the present disclosure, the information of the control can also include other information in addition to the aforementioned information.

[0070] Corresponding to the aforementioned various possible information of the control, the step S102 in the embodiments of the present disclosure extracts the information of the control on the at least one page from the XML data of the at least one page, which can specifically include: extracting at least one of the style information of the control, the type of content in the control, and the text included in the XML data for describing the control.

[0071] In the embodiments of the present disclosure, the information of the control on the at least one page can be extracted from the XML data of the at least one page in the following method: first, the data representing a certain specific control in the page in the XML data is extracted, and then the XML data representing the specific control is extracted to obtain the information of the specific control.

[0072] It should be noted that after the various types of information of a certain control are extracted, the various types of information need to be combined according to a pre-set combination rule, and then the information of the control is obtained, so that all the information of the control conforms to the pre-set combination rule.

[0073] Step S103: training the pre-set natural language processing model based on the information of the control on the at least one page to obtain a natural language processing model that can be used to extract page features.

[0074] In the embodiments of the present disclosure, after obtaining the information of the controls in each page in the training data, the information of the controls in each page can be used to train the pre-set natural language processing model to obtain a natural language processing model for extracting page features.

[0075] Figure 2 is a flowchart of a method for training a natural language processing model provided by the embodiments of the present disclosure. As shown in Figure 2 In some embodiments of the present disclosure, step S103 of training the pre-set natural language processing model based on the information of the controls on at least one page to obtain a natural language processing model that can be used to extract page features can include steps S1031-S1032.

[0076] Step S1031: For each page, concatenate the information of the multiple controls included in the page to generate a sentence for describing the page.

[0077] In the embodiments of the present disclosure, the information of each control is taken as a word for describing the control. The information of the multiple controls on the page is concatenated to obtain a sentence for describing the page. The sentence for describing the page includes not only the information of the controls in the page, but also the relative position information of the controls in the page.

[0078] In some embodiments of the present disclosure, the information of the multiple controls included in the page can be concatenated based on the arrangement order of the controls in the page to generate a sentence for describing the page. For example, if the controls in the page are arranged and displayed in the order from left to right and from top to bottom, the information of the multiple controls can be concatenated in the order from left to right and from top to bottom to generate a sentence for describing the page.

[0079] In some other embodiments of the present disclosure, the controls in the page can be classified based on the information of the controls, the information of the controls in each classification is concatenated in the arrangement order of the controls in the page to obtain a sub-sentence in each classification. Subsequently, the sub-sentences of the classifications are concatenated in a pre-set classification order to generate a sentence for describing the page.

[0080] In some other embodiments of the present disclosure, the information of the multiple controls included in the page can also be concatenated randomly to generate a sentence for describing the page.

[0081] Step S1032: Train the pre-set natural language processing model based on the sentences corresponding to the pages.

[0082] In some embodiments of the present disclosure, after obtaining the sentences corresponding to the respective pages, the sentences corresponding to the respective pages can be directly used as training samples to train the preset natural language processing model, so as to obtain a natural language processing model that can be used to extract page features.

[0083] For example, in the case of a natural language processing model being a Bert model, training the preset natural language model based on the sentences corresponding to the respective pages can be to perform a cloze MLM (Masked LM) training on the Bert model based on the sentences corresponding to the respective pages. Specifically, for each sentence corresponding to a page, the information of some controls in the sentence can be randomly hidden, and the information of the hidden controls can be predicted by using the information of other controls in the sentence, so as to realize a self-supervised training of the Bert model by using the cloze MLM.

[0084] In some embodiments of the present disclosure, the number of controls included in each page can be different, so that the lengths of the sentences corresponding to the respective pages are not the same. In order to ensure that the lengths of the sentences corresponding to the respective pages match the input interface dimensions of the preset natural language processing model when the sentences corresponding to the respective pages are input into the preset natural language processing model, after obtaining the sentences corresponding to the respective pages, the sentences corresponding to the respective pages can be processed into sentences with the same length, and then the preset natural language processing model can be trained based on the sentences with the same length. In specific embodiments, some shorter sentences can be processed by padding to make the lengths of the sentences corresponding to the respective pages the same.

[0085] In some embodiments of the present disclosure, when training the preset natural language processing model based on the sentences corresponding to the respective pages, the sentences corresponding to the respective pages can be first encoded based on a vocabulary table to obtain vectors corresponding to the respective pages, and then the natural language processing model can be trained based on the vectors corresponding to the respective pages. The vocabulary table includes a correspondence between the information of the controls and the encoding mode, and the vocabulary table is generated by performing a deduplication processing on the extracted information of the plurality of controls and based on the remaining information after the deduplication processing.

[0086] In some other embodiments of the present disclosure, the step S1032 of training the preset natural language processing model based on the sentences corresponding to the respective pages can include steps S1032A-S1032B.

[0087] The step S1032A includes concatenating the sentences corresponding to the respective pages to generate a target sentence.

[0088] The step S1032B includes training the preset natural language processing model based on the target sentence.

[0089] In some embodiments of the present disclosure, after obtaining the sentences corresponding to the respective pages, the sentences corresponding to the respective pages can be concatenated to generate a target sentence, and then the target sentence can be used to train the preset natural language processing model to obtain a natural language processing model that can be used to extract page features.

[0090] For example, in the case of the natural language processing model being a Bert model, training the preset natural language model based on the target sentence can be Next Sentence Prediction (NSP) training of the Bert model based on the target sentence.

[0091] In a specific implementation, when concatenating the sentences corresponding to the respective pages in step S1032A, the corresponding sentences can be concatenated based on the connection relationship between the pages, or the corresponding sentences can be concatenated based on a randomly determined relationship, which is not particularly limited in the embodiments of the present disclosure. Subsequently, in step S1032B, the natural language model can be trained based on the target sentence and the incorrect sentence constructed based on the target sentence.

[0092] In some embodiments of the present disclosure, when training the preset natural language processing model based on the target sentence, the target sentence can be encoded based on a vocabulary table to obtain a vector corresponding to the target sentence, and then the natural language processing model can be trained based on the vector corresponding to the target sentence. The vocabulary table includes a correspondence between the information of the control and the encoding mode, and the vocabulary table is generated by performing deduplication processing on the extracted information of the plurality of controls and based on the remaining information after the deduplication processing.

[0093] It should be noted that, in the case of the natural language processing model being a Bert model, step S1032 can perform Next Sentence Prediction (NSP) training of the Bert model based on the target sentence, and can also perform Masked Language Model (MLM) training.

[0094] Through the foregoing analysis of steps S101-S103, by using the training method of the natural language processing model provided in the embodiments of the present disclosure, after obtaining the extensible markup language (XML) data including at least one page, the information of the control in the XML data is extracted, and then the preset natural language processing model is trained based on the information of the control in the page, so that the trained natural language processing model has the ability to extract page features, thereby providing a basis for page analysis.

[0095] The preset natural language processing model mentioned in the foregoing is a Bert model, and in other embodiments of the present disclosure, the preset natural language processing model can also be a LUMFiT model or an OpenAI GPT-2 model.

[0096] Figure 3 is a structural schematic diagram of a training apparatus of a natural language processing model provided by an embodiment of the present disclosure. The training apparatus can be understood as the above-mentioned computing device or part of the functional modules in the above-mentioned computing device. As shown in the figure, the training apparatus of the natural language processing model 300 includes a data acquisition unit 301, a control information extraction unit 302, and a model training unit 303. Figure 3

[0097] The data acquisition unit 301 is configured to acquire training data, wherein the training data includes extensible markup language (XML) data of at least one page.

[0098] The control information extraction unit 302 is configured to extract information of controls on the at least one page from the XML data of the at least one page.

[0099] The model training unit 303 is configured to train a preset natural language processing model based on the information of the controls on the at least one page, to obtain a natural language processing model that can be used to extract features of a page.

[0100] In some embodiments of the present disclosure, the control information extraction unit 302 extracts information of the controls from the XML data of the at least one page, including at least one of style information of the controls, types of content in the controls, and text included in the XML data for describing the controls.

[0101] In some embodiments of the present disclosure, the model training unit 303 includes a control information concatenation subunit and a training subunit. The control information concatenation subunit is configured to concatenate information of multiple controls included in a page together for each page to generate a sentence for describing the page. The training subunit is configured to train the preset natural language processing model based on the sentences corresponding to the pages.

[0102] In some embodiments of the present disclosure, the control information concatenation subunit concatenates the information of the multiple controls together in sequence based on an arrangement order of the multiple controls on the page to generate the sentence for describing the page.

[0103] In some embodiments of the present disclosure, the training subunit includes a target sentence concatenation module and a model training module. The target sentence concatenation module is configured to concatenate the sentences corresponding to the pages together to generate a target sentence. The model training module is configured to train the preset natural language processing model based on the target sentence.

[0104] ​In some embodiments of this disclosure, the training module includes a vector generation submodule and a model training submodule. The vector generation submodule encodes the target sentence based on a vocabulary to obtain the vector corresponding to the target sentence. The model training submodule trains a natural language processing model based on the vectors. The vocabulary includes the correspondence between control information and encoding methods.

[0105] In some embodiments of this disclosure, the natural language processing model includes a pre-trained language model, BERT. A model training submodule performs cloze test (MLM) training and / or sentence prediction (NSP) training on the BERT model based on vectors.

[0106] The apparatus provided in this embodiment is capable of performing the above-described... Figures 1-2 The methods in any of the embodiments are similar in execution and beneficial effects, and will not be described again here.

[0107] This disclosure also provides a computing device, which includes a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it can perform the above-described functions. Figures 1-2 The method of any of the embodiments.

[0108] Example, Figure 4 This is a schematic diagram of the structure of a computing device according to an embodiment of this disclosure. See below for details. Figure 4 The diagram illustrates a structural schematic suitable for implementing the computing device 400 in the embodiments of this disclosure. The computing device 400 in the embodiments of this disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 4 The computing device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0109] like Figure 4 As shown, the computing device 400 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage device 408 into a random access memory (RAM) 403. The RAM 403 also stores various programs and data required for the operation of the computing device 400. The processing unit 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0110] Generally, the following devices can be connected to the I / O interface 405: input devices 406 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, and so on; output devices 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, and so on; storage devices 408 including, for example, a magnetic tape, a hard disk, and so on; and communication devices 409. The communication devices 409 can allow the computing device 400 to communicate with other devices wirelessly or through wires to exchange data. Although Figure 4 The computing device 400 is shown with various devices, but it is understood that not all of the shown devices are required to be implemented or present. More or fewer devices can alternatively be implemented or present.

[0111] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for executing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through the communication devices 409, or installed from the storage devices 408, or installed from the ROM 402. When the computer program is executed by the processing devices 401, the above-described functions defined in the methods of embodiments of the present disclosure are performed.

[0112] It is noted that the aforementioned computer-readable medium of the present disclosure can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the foregoing. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus or device. In the present disclosure, the computer-readable signal medium can include a data signal propagated in baseband or propagated as a carrier wave in a propagated data signal, in which the computer-readable program code is contained. Such a propagated data signal can take any of a variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium that is not a storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wire, cable, RF, infrared, or any suitable combination thereof.

[0113] In some embodiments, the client, server, or both can communicate using any current known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any current known or future developed networks.

[0114] The aforementioned computer-readable medium can be included within the aforementioned computing device; or can exist entirely or be separate from the aforementioned computing device.

[0115] The computer readable medium described above carries one or more programs, when the one or more programs are executed by the computing device, cause the computing device to: obtain training data, the training data comprising extensible markup language, XML, data of at least one page; extract information of a control on the at least one page from the XML data of the at least one page; and train a preset natural language processing model based on the information of the control on the at least one page to obtain a natural language processing model that can be used to extract page features.

[0116] Computer program code for carrying out operations of the present disclosure can be written in any one or more programming languages, including object oriented programming languages such as Java, Smalltalk, C++ or conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0117] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a procedure, or a part of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in a different order than that noted in the figures. For example, two blocks noted in succession can in fact be executed substantially concurrently or in the opposite order, depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flow diagrams, and combinations of blocks in the block diagrams and / or flow diagrams, can be implemented by dedicated hardware-based systems that perform the specified functions or operations, or can be implemented by a combination of dedicated hardware-based systems and computer instructions.

[0118] The units described in the embodiments of the present disclosure can be implemented by software, or by hardware. In some cases, the names of the units do not constitute a limitation on the units themselves.

[0119] The functionality described herein above can be performed, at least in part, by one or more hardware logic components. For example, and without limitation, an example type of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Application-specific Integrated Circuits (ASICs), Application-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0120] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0121] The present disclosure also provides a computer readable storage medium, wherein the storage medium stores a computer program. When the computer program is executed by a processor, the method of any of the above embodiments can be implemented. The implementation manner and beneficial effects are similar, and will not be described here. Figures 1-2 The present disclosure also provides a computer readable storage medium, wherein the storage medium stores a computer program. When the computer program is executed by a processor, the method of any of the above embodiments can be implemented. The implementation manner and beneficial effects are similar, and will not be described here.

[0122] It should be noted that, in this document, relational terms such as "first" and "second", and the like, are used solely to distinguish one entity or action from another entity or action, without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without limitation, an element preceded by "comprises... a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0123] The foregoing is merely illustrative of the various implementations of the present disclosure and the general principles thereof. Numerous modifications can be made to these illustrations, and equivalents can be substituted therefor, without departing from the scope of the present disclosure. The specific embodiments commensurate with the specific application are intended to be illustrative only and not limiting of the scope of the application as set forth in the following claims.

Claims

1. A method for training a natural language processing model, the method comprising: The method comprises: acquiring training data, wherein the training data comprises extensible markup language (XML) data of at least one page; extracting information of controls on the at least one page from the XML data of the at least one page; training a preset natural language processing model based on the information of the controls on the at least one page to obtain a natural language processing model that can be used to extract page features; the training of the preset natural language processing model based on the information of the controls on the at least one page comprises: for each page, concatenating information of multiple controls included on the page to generate a sentence for describing the page; training the preset natural language processing model based on the sentences corresponding to the pages; the concatenation of the information of the multiple controls included on the page to generate the sentence for describing the page comprises: classifying each of the controls based on the information of the control, and concatenating the information of the controls in each classification according to an arrangement order of the controls in the page to obtain a sub-sentence in each classification; concatenating the sub-sentences of the classifications according to a preset classification order to generate the sentence for describing the page.

2. The method of claim 1, wherein, the extraction of the information of the controls on the at least one page from the XML data of the at least one page comprises: extracting at least one of the following information of the controls: style information of the controls, types of contents in the controls, and texts for describing the controls that have been included in the XML data.

3. The method of claim 1, wherein, the concatenation of the information of the multiple controls included on the page to generate the sentence for describing the page comprises: concatenating the information of the multiple controls included on the page in an arrangement order of the controls on the page to generate the sentence for describing the page.

4. The method of claim 1, wherein, the training of the preset natural language processing model based on the sentences corresponding to the pages comprises: concatenating the sentences corresponding to the pages to generate a target sentence; training the preset natural language processing model based on the target sentence.

5. The method of claim 4, wherein, the training of the preset natural language processing model based on the target sentence comprises: performing encoding processing on the target sentence based on a vocabulary table to obtain a vector corresponding to the target sentence; training the natural language processing model based on the vector; the vocabulary table comprises a corresponding relationship between information of controls and encoding manners.

6. The method of claim 5, wherein, the natural language processing model comprises a pre-trained language model Bert. the training of the natural language processing model based on the vector comprises: performing masked language model (MLM) training and / or next sentence prediction (NSP) training on the Bert model based on the vector.

7. A training device for a natural language processing model, characterized in that, The method comprises: a data acquisition unit configured to acquire training data, wherein the training data comprises extensible markup language (XML) data of at least one page; a control information extraction unit configured to extract information of controls on the at least one page from the XML data of the at least one page. The model training unit is configured to train a preset natural language processing model based on the information of the controls on the at least one page to obtain a natural language processing model that can be used to extract page features. The model training unit includes: The control information connector unit is configured to concatenate the information of the controls included on each page to generate a sentence for describing the page. The training subunit is configured to train a preset natural language processing model based on the sentences corresponding to the pages. The control information connector unit is configured to classify the controls based on the information of the controls in the page, concatenate the information of the controls in each classification according to the arrangement order of the controls in the page, obtain a sub-sentence in each classification, and concatenate the sub-sentences of each classification according to a preset classification order to generate a sentence for describing the page.

8. The apparatus of claim 7, wherein, The control information extraction unit extracts the information of the controls from the XML data of the at least one page, including: The style information of the controls, the type of the content in the controls, and at least one of the texts included in the XML data for describing the controls.

9. The apparatus of claim 7, wherein: The control information connector unit concatenates the information of the controls included on the page in the arrangement order of the controls on the page to generate a sentence for describing the page.

10. The apparatus of claim 7, wherein, The training subunit includes: A target sentence connector module configured to concatenate the sentences corresponding to the pages to generate a target sentence. A model training module configured to train a preset natural language processing model based on the target sentence.

11. The apparatus of claim 10, wherein, The training module includes: A vector generation sub-module configured to encode the target sentence based on a vocabulary table to obtain a vector corresponding to the target sentence. A model training sub-module configured to train the natural language processing model based on the vector. The vocabulary table includes a corresponding relationship between the information of the controls and the encoding method.

12. The apparatus of claim 11, wherein, The natural language processing model includes a pre-trained language model Bert. The model training sub-module performs masked language model MLM training and / or next sentence prediction NSP training on the Bert model based on the vector.

13. A computing device, comprising: A memory and a processor, wherein the memory stores a computer program that, when executed by the processor, implements the method of any one of claims 1-6. The storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1-6.

14. A computer-readable storage medium, characterized in that, ​

Images