Corpus processing method and device, electronic equipment and storage medium

Abstract

The application provides a corpus processing method and device, electronic equipment and a storage medium. The method comprises the following steps: obtaining a corpus processing request, wherein the corpus processing request comprises a corpus table file, attribute information of a model corresponding to the corpus table file, and a processing type; analyzing the corpus table file according to the attribute information of the model, and obtaining corpus content in the corpus table file; and processing the corpus content according to the processing type, and determining a corpus record of the model. Thus, the screening, editing and batch processing functions of the table file can be utilized, the user can conveniently adjust and process the corpus table file, and the corpus processing efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a corpus processing method and device, electronic equipment and storage medium. BACKGROUND

[0002] At present, natural language processing technology involves training and using a large number of models. When training a model, a file in txt format is used as a processing medium of corpus. When training a model, a file in txt format is read, the file is parsed to obtain corpus, and the corpus is used as input of the model to train the model.

[0003] In the above scheme, when a user adjusts corpus in a file in txt format, for example, deletes, modifies or adds, due to the functional limitation of the file in txt format, it is difficult to effectively adjust the corpus in the file in txt format, thereby reducing the corpus processing efficiency. SUMMARY

[0004] The present application aims to at least solve one of the technical problems in the related art to some extent.

[0005] The present application provides a corpus processing method and device, electronic equipment and storage medium.

[0006] The first aspect of the present application provides a corpus processing method, comprising: obtaining a corpus processing request, wherein the corpus processing request comprises a corpus table file, attribute information of a model corresponding to the corpus table file, and a processing type; parsing the corpus table file according to the attribute information of the model to obtain corpus content in the corpus table file; processing the corpus content according to the processing type to determine corpus records of the model.

[0007] The corpus processing method of the present application, by obtaining a corpus processing request, wherein the corpus processing request comprises a corpus table file, attribute information of a model corresponding to the corpus table file, and a processing type; parsing the corpus table file according to the attribute information of the model to obtain corpus content in the corpus table file; processing the corpus content according to the processing type to determine corpus records of the model, can utilize the filtering, editing and batch processing functions of the table file to facilitate the user to adjust the corpus table file and other processing, thereby improving the corpus processing efficiency.

[0008] Optionally, the attribute information comprises a type of the model; and the parsing the corpus table file according to the attribute information of the model to obtain the corpus content in the corpus table file comprises: determining a parsing strategy for the corpus table file according to the type in the attribute information; and parsing the corpus table file according to the parsing strategy to obtain the corpus content.

[0009] Optionally, the processing type includes at least one of the following types: a new type, an added type, and a replacement type.

[0010] Optionally, the attribute information includes an identifier of the model; and the processing of the corpus content according to the processing type to determine the corpus record of the model includes: when the processing type is the new type, generating corpus record information including the identifier of the model; and generating the corpus record of the model according to the corpus record information and the corpus content.

[0011] Optionally, the processing of the corpus content according to the processing type to determine the corpus record of the model includes: when the processing type is the added type, querying existing corpus record information matching the attribute information; adding the corpus content to existing corpus content corresponding to the existing corpus record information to obtain processed corpus content; and determining the existing corpus record information and the processed corpus content as the corpus record of the model.

[0012] Optionally, the processing of the corpus content according to the processing type to determine the corpus record of the model includes: when the processing type is the replacement type, querying existing corpus record information matching the attribute information; replacing existing corpus content corresponding to the existing corpus record information with the corpus content; and determining the existing corpus record information and the corpus content as the corpus record of the model.

[0013] Optionally, the method further includes: obtaining a corpus download request, wherein the corpus download request includes corpus record information to be downloaded; querying existing corpus records according to the corpus record information to obtain corpus content to be downloaded corresponding to the corpus record information; and performing format conversion on the corpus content to be downloaded to obtain a corpus table file to be downloaded and provide the corpus table file to a requester of the corpus download request.

[0014] An embodiment of the second aspect of the present application provides a corpus processing apparatus, including: an obtaining module configured to obtain a corpus processing request, wherein the corpus processing request includes a corpus table file, attribute information of a model corresponding to the corpus table file, and a processing type; an analyzing module configured to analyze the corpus table file according to the attribute information of the model to obtain corpus content in the corpus table file; and a processing module configured to process the corpus content according to the processing type to determine a corpus record of the model.

[0015] The corpus processing device of the embodiment of the application obtains a corpus processing request, wherein the corpus processing request comprises a corpus table file, attribute information of a model corresponding to the corpus table file, and a processing type; the corpus table file is parsed according to the attribute information of the model to obtain corpus content in the corpus table file; and the corpus content is processed according to the processing type to determine a corpus record of the model, so that the screening, editing and batch processing functions of the table file can be utilized to facilitate the user to adjust and process the corpus table file, and the corpus processing efficiency is improved.

[0016] Optionally, the attribute information comprises a type of the model.

[0017] The parsing module is specifically configured to,

[0018] determine a parsing strategy for the corpus table file according to the type in the attribute information;

[0019] parse the corpus table file according to the parsing strategy to obtain the corpus content.

[0020] Optionally, the processing type comprises at least one of the following types: a new creation type, an addition type and a replacement type.

[0021] Optionally, the attribute information comprises an identifier of the model.

[0022] The processing module is specifically configured to,

[0023] when the processing type is the new creation type, generate corpus record information comprising the identifier of the model;

[0024] generate the corpus record of the model according to the corpus record information and the corpus content.

[0025] Optionally, the processing module is specifically configured to,

[0026] when the processing type is the addition type, query existing corpus record information matched with the attribute information;

[0027] add the corpus content to existing corpus content corresponding to the existing corpus record information to obtain processed corpus content;

[0028] determine the existing corpus record information and the processed corpus content as the corpus record of the model.

[0029] Optionally, the processing module is specifically configured to,

[0030] when the processing type is the replacement type, query existing corpus record information matched with the attribute information;

[0031] replacing the existing corpus content corresponding to the existing corpus record information with the corpus content;

[0032] determining the existing corpus record information and the corpus content as the corpus record of the model.

[0033] Optionally, the apparatus further comprises a query module and a format conversion module.

[0034] The acquisition module is further configured to acquire a corpus download request, wherein the corpus download request comprises corpus record information to be downloaded.

[0035] The query module is configured to query an existing corpus record according to the corpus record information, and acquire corpus content to be downloaded corresponding to the corpus record information.

[0036] The format conversion module is configured to perform format conversion on the corpus content to be downloaded, to obtain a corpus table file to be downloaded and provide the corpus table file to a requester of the corpus download request.

[0037] The third aspect of the present application provides an electronic device, comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the corpus processing method provided in the first aspect of the present application.

[0038] The fourth aspect of the present application provides a non-transitory computer readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the corpus processing method provided in the first aspect of the present application.

[0039] The fifth aspect of the present application provides a computer program product, when an instruction processor in the computer program product executes, the corpus processing method provided in the first aspect of the present application is executed.

[0040] Additional aspects and advantages of the present application will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0041] The above and / or additional aspects and advantages of the present application will become apparent and be readily appreciated from the following description, including the accompanying drawings, wherein:

[0042] Figure 1 A flowchart of the corpus processing method provided in the first aspect of the present application;

[0043] Figure 2A schematic diagram of an acquisition interface for a corpus processing request;

[0044] Figure 3 A schematic diagram of an acquisition interface for a corpus processing request when the processing type is an addition type;

[0045] Figure 4 A schematic diagram of a process for corpus processing by a user through an AI algorithm platform;

[0046] Figure 5 A schematic diagram of a process for the corpus processing method provided in Embodiment Two of the present application;

[0047] Figure 6 A schematic diagram of the structure of the corpus processing device provided in Embodiment Three of the present application;

[0048] Figure 7 A schematic diagram of the structure of another corpus processing device;

[0049] Figure 8 A block diagram of an electronic device for a corpus processing method according to an exemplary embodiment. DETAILED DESCRIPTION

[0050] The embodiments of the present application are described in detail below, examples of which are shown in the accompanying drawings, in which the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the accompanying drawings are exemplary and are intended to explain the present application, and cannot be understood as limiting the present application.

[0051] Currently, natural language processing technology involves the training and use of a large number of models. When training a model, a file in txt format is used as a processing medium for corpus. When training a model, a file in txt format is read, the file is parsed to obtain corpus, and the corpus is used as input for the model to train the model.

[0052] In the above scheme, when the user adjusts the corpus in the file in txt format, such as deleting, modifying, or adding, due to the functional limitations of the file in txt format, it is difficult to effectively adjust the corpus in the file in txt format, thereby reducing the corpus processing efficiency.

[0053] To solve the above problems, the present application provides a corpus processing method, device, electronic equipment and storage medium.

[0054] Figure 1 A schematic diagram of the corpus processing method provided in Embodiment One of the present application, it should be noted that the corpus processing method can be applied to a corpus processing device. The corpus processing device can be applied to an AI algorithm platform.

[0055] AsFigure 1 As shown in the figure, the corpus processing method comprises the following steps:

[0056] In step 101, a corpus processing request is obtained, wherein the corpus processing request comprises a corpus table file, attribute information of a model corresponding to the corpus table file, and a processing type.

[0057] In the embodiment of the present application, the attribute information of the model can comprise at least one of the following parameters: an identifier of the model, a type of the model. The type of the model can comprise, for example, a classification type, a clustering type, and a context type, etc., which can be set according to actual needs. When the type of the model is the classification type, the model can correspond to a classification model; when the type of the model is the clustering type, the model can correspond to a clustering model; and when the type of the model is the context type, the model can correspond to a context model.

[0058] The corpus contents required by different models during training are different. For example, the corpus contents required by a classification model during training comprise the corpus itself and corpus classification. The corpus contents required by a context model during training comprise the corpus itself, corpus classification, and the association relationship between corpora.

[0059] The processing type can comprise at least one of the following types: a new type, an added type, and a replacement type. The new type indicates that corpus contents required during training need to be newly created for the model; the added type indicates that corpus contents required during training need to be added to the model; and the replacement type indicates that corpus contents required during training need to be replaced.

[0060] In the embodiment of the present application, the process in which the corpus processing apparatus obtains the corpus processing request can be, for example, for a specific model, when a user's batch import operation is detected, it is determined that the corpus processing request is obtained, the table file imported in batches is taken as the corpus table file, and it is determined that the processing type of the corpus processing request is the added type. The process in which the corpus processing apparatus obtains the corpus processing request can also be, for example, for a specific model, when a user's batch replacement operation is detected, it is determined that the corpus processing request is obtained, the new table file to be replaced in batches is taken as the corpus table file, and it is determined that the processing type of the corpus processing request is the replacement type. The acquisition interface of the corpus processing request when the processing type is the new type or the replacement type is shown in FIG. 3. Figure 2

[0061] In the embodiment of the present application, the acquisition interface of the corpus processing request when the processing type is the new type is shown in FIG. 4. Figure 3 ​As shown, the process of the corpus processing apparatus obtaining the corpus processing request may be, for example, for a specific model, when detecting that the user selects the corpus pool operation of adding, determining that the corpus processing request is obtained, and the processing type of the corpus processing request is the new type. One corpus pool corresponds to one model, and one model is one corpus record.

[0062] In the embodiment of the present application, generally, there is a certain amount of repeated corpus in model training. The flowchart of the process of the user processing the corpus through the AI algorithm platform is as shown in Figure 4 As shown, the user processes the corpus through the front-end engineering of the AI algorithm platform. For a specific model, when the processing type of the corpus processing request is the new type, the corpus content can be added or edited one by one. The corpus content is directly parsed, and the parsed corpus content is stored in the database. When the processing type of the corpus processing request is the new type or the replacement type, the corpus table file can be uploaded in batches. The corpus table file is stored in the database directory corresponding to the oss platform, and the corresponding storage link is transmitted to the back-end engineering. After receiving the storage link, the back-end engineering downloads the corresponding corpus table file from the oss platform according to the storage link, parses the corpus table file, and stores the parsed corpus content in the corresponding corpus record in the database.

[0063] In the embodiment of the present application, the user clicks the corpus download button as shown in Figure 2 The corpus download button generates a corpus download request, and the corpus download request includes corpus record information to be downloaded. The front-end engineering of the AI algorithm platform queries the existing corpus record according to the corpus record information, and transmits the corpus record information of the corpus record to the back-end engineering. The back-end engineering generates a corpus download task according to the corpus record information, downloads the corresponding to-be-downloaded corpus content from the database according to the corpus download task, and temporarily stores the to-be-downloaded corpus content in the memory. After the to-be-downloaded corpus content is obtained, the back-end engineering writes the to-be-downloaded corpus content temporarily stored in the memory into a corpus table file according to the format requirement of the table file, wherein the corpus table file is an Excel file. The corpus table file is uploaded to the oss platform, and a corresponding corpus download link is generated and provided to the front-end engineering of the AI algorithm. After receiving the corpus download link, the front-end engineering sends an HTTP request to the oss platform according to the corpus download link to obtain the corresponding corpus table file.

[0064] Step 102, according to the attribute information of the model, the corpus table file is parsed to obtain the corpus content in the corpus table file.

[0065] In an embodiment of the present application, in an example, the attribute information can include a type of the model; correspondingly, the process of step 102 performed by the corpus processing apparatus can be, for example, determining a parsing strategy for the corpus table file according to the type in the attribute information, parsing the corpus table file according to the parsing strategy, and obtaining the corpus content.

[0066] In an embodiment of the present application, the type of the model can include, for example, a classification type, a clustering type, a context type, and the like, which can be set according to actual needs. For example, when the type of the model is the classification type, the corpus content required during training of the model includes corpus itself and corpus classification. Correspondingly, the parsing strategy for the corpus table file is to determine a column in which the corpus itself is located and a column in which the corpus classification is located in the corpus table file, read the content on the columns from the corpus table file, and obtain the corpus content of the model of the classification type. For another example, when the type of the model is the context type, the corpus content required during training of the model includes corpus itself, corpus classification, and an association relationship between corpora. Correspondingly, the parsing strategy for the corpus table file is to determine a column in which the corpus itself is located, a column in which the corpus classification is located, and a column in which the association relationship between corpora is located in the corpus table file, read the content on the columns from the corpus table file, and obtain the corpus content of the model of the context type.

[0067] Step 103: processing the corpus content according to the processing type to determine the corpus record of the model.

[0068] In an embodiment of the present application, the processing type can include at least one of the following types: a new creation type, an addition type, and a replacement type. The corpus record can include corpus record information and corpus content; the corpus record information includes at least one of the following parameters: a corpus record number, an associated robot identifier, a type of the model, an identifier of the model, and information such as an update or creation time, as shown in Figure 3 .

[0069] In an embodiment of the present application, in an example, when the processing type is the new creation type, the corpus record information including the identifier of the model is generated; the corpus record of the model is generated according to the corpus record information and the corpus content.

[0070] In another example, when the processing type is the addition type, the existing corpus record information matched with the attribute information is queried; the corpus content is added to the existing corpus content corresponding to the existing corpus record information to obtain processed corpus content; the existing corpus record information and the processed corpus content are determined as the corpus record of the model.

[0071] In another example, in the case of the processing type being the replacement type, the query matches the existing corpus record information with the attribute information; the existing corpus content corresponding to the existing corpus record information is replaced with the corpus content; the existing corpus record information and the corpus content are determined as the corpus record of the model.

[0072] In the embodiment of the present application, in the corpus processing process of the new type and the replacement type, the user can directly input the corpus content through the front-end engineering of the AI algorithmist platform, or upload an Excel file including the corpus content, or use both, wherein the Excel file can be a corpus table file. If the corpus table file is uploaded through the Excel file, the front-end engineering uploads the corpus table file to the database directory of the oss platform, and the front-end engineering of the AI algorithmist platform obtains the storage link and transmits it to the back-end engineering. After receiving the storage link, the back-end engineering downloads the corresponding corpus table file from the oss platform according to the storage link, parses the corpus table file, and stores the parsed corpus content in the corresponding corpus record of the database.

[0073] In summary, the corpus processing request is obtained, wherein the corpus processing request includes: a corpus table file, attribute information of a model corresponding to the corpus table file, and a processing type; the corpus table file is parsed according to the attribute information of the model to obtain the corpus content in the corpus table file; the corpus content is processed according to the processing type to determine the corpus record of the model, so that the filtering, editing, and batch processing functions of the table file can be used to facilitate the user to adjust the corpus table file and other processing, and improve the corpus processing efficiency.

[0074] Figure 5 The flowchart of the corpus processing method provided in Embodiment Two of the present application is shown in FIG. 8. Figure 5 As shown in Embodiment Two of the present application, the method further includes: Figure 1

[0075] Step 201, obtaining a corpus download request, wherein the corpus download request includes: corpus record information to be downloaded.

[0076] In the embodiment of the present application, the user clicks the corpus download button as shown in FIG. 9 through the front-end engineering of the AI algorithmist platform to generate a corpus download request, wherein the corpus download request includes the corpus record information to be downloaded. Figure 2

[0077] Step 202, querying the existing corpus record according to the corpus record information to obtain the corpus content to be downloaded corresponding to the corpus record information.

[0078] ​​In the embodiment of the present application, the front-end engineering of the AI consultant platform queries the existing corpus record according to the corpus record information, and transmits the corpus record information of the corpus record to the back-end engineering. The back-end engineering generates a corpus download task according to the corpus record information, downloads the corresponding to-be-downloaded corpus content from the database according to the corpus download task, and temporarily stores in the memory.

[0079] In step 203, the to-be-downloaded corpus content is format-converted to obtain a to-be-downloaded corpus table file and provide the to-be-downloaded corpus table file to the requester of the corpus download request.

[0080] In the embodiment of the present application, after the to-be-downloaded corpus content is acquired, the back-end engineering writes the to-be-downloaded corpus content temporarily stored in the memory into a corpus table file according to the format requirement of the table file, wherein the corpus table file is an Excel file. The corpus table file is uploaded to an oss platform, and a corresponding corpus download link is generated and provided to the front-end engineering of the AI consultant. After receiving the corpus download link, the front-end engineering sends an HTTP request to the oss platform according to the corpus download link to acquire the corresponding corpus table file.

[0081] In summary, the corpus processing request is acquired, wherein the corpus processing request includes: a corpus table file, attribute information of a model corresponding to the corpus table file, and a processing type; the corpus table file is parsed according to the attribute information of the model to acquire corpus content in the corpus table file; and the corpus content is processed according to the processing type to determine corpus records of the model, so that the screening, editing and batch processing functions of the table file can be utilized to facilitate the user to adjust and process the corpus table file, thereby improving the corpus processing efficiency.

[0082] Figure 6 The structure diagram of the corpus processing device provided in Embodiment Three of the present application.

[0083] As shown in Figure 6 the corpus processing device 600 includes an acquisition module 601, an analysis module 602 and a processing module 603.

[0084] The acquisition module 601 is configured to acquire a corpus processing request, wherein the corpus processing request includes: a corpus table file, attribute information of a model corresponding to the corpus table file, and a processing type.

[0085] The analysis module 602 is configured to parse the corpus table file according to the attribute information of the model to acquire corpus content in the corpus table file.

[0086] The processing module 603 is configured to process the corpus content according to the processing type to determine corpus records of the model.

[0087] As a possible implementation manner of the embodiment of the present application, the parsing module 602 is specifically configured to determine a parsing strategy for the corpus table file according to the type in the attribute information; and parse the corpus table file according to the parsing strategy to obtain the corpus content.

[0088] As a possible implementation manner of the embodiment of the present application, the processing module 603 is specifically configured to, when the processing type is a new type, generate corpus record information including an identifier of the model;

[0089] generate the corpus record of the model according to the corpus record information and the corpus content.

[0090] As a possible implementation manner of the embodiment of the present application, the processing module 603 is specifically configured to, when the processing type is an adding type, query existing corpus record information matched with the attribute information;

[0091] add the corpus content to existing corpus content corresponding to the existing corpus record information to obtain processed corpus content;

[0092] determine the existing corpus record information and the processed corpus content as the corpus record of the model.

[0093] As a possible implementation manner of the embodiment of the present application, the processing module 603 is specifically configured to, when the processing type is a replacing type, query existing corpus record information matched with the attribute information;

[0094] replace existing corpus content corresponding to the existing corpus record information with the corpus content;

[0095] determine the existing corpus record information and the corpus content as the corpus record of the model.

[0096] As a possible implementation manner of the embodiment of the present application, Figure 7 for another structure diagram of a corpus processing apparatus, in Figure 6 the embodiment shown, the corpus processing apparatus 600 further includes a query module 604 and a format conversion module 605;

[0097] The acquisition module 601 is further configured to acquire a corpus download request, wherein the corpus download request includes corpus record information to be downloaded.

[0098] The query module 604 is configured to query existing corpus records according to the corpus record information to acquire corpus content to be downloaded corresponding to the corpus record information.

[0099] The format conversion module 605 is configured to convert the format of the corpus content to be downloaded, to obtain a corpus table file to be downloaded and provide the corpus table file to the requestor of the corpus download request.

[0100] The corpus processing apparatus of the embodiment of the present application acquires a corpus processing request, wherein the corpus processing request comprises a corpus table file, attribute information of a model corresponding to the corpus table file, and a processing type; the corpus table file is parsed according to the attribute information of the model to acquire corpus content in the corpus table file; and the corpus content is processed according to the processing type to determine a corpus record of the model, so that the screening, editing and batch processing functions of the table file can be utilized to facilitate the user to adjust and process the corpus table file, and the corpus processing efficiency is improved.

[0101] To achieve the above-mentioned embodiments, the present application further provides an electronic device, such as Figure 8 As shown in the figure, Figure 8 is a block diagram of an electronic device for a corpus processing method according to an exemplary embodiment.

[0102] As shown in the figure, Figure 8 The electronic device 1100 comprises:

[0103] The memory 1110 and the processor 1120, the bus 1130 connecting different components (including the memory 1110 and the processor 1120), the memory 1110 stores a computer program, and the processor 1120 executes the program to realize the coupon processing method of the embodiment of the present application.

[0104] The bus 1130 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to industry standard architecture (ISA) bus, micro channel architecture (MAC) bus, enhanced ISA bus, video electronics standards association (VESA) local bus, and peripheral component interconnect (PCI) bus.

[0105] The electronic device 1100 typically includes a variety of electronic device readable media. These media can be any available media that can be accessed by the electronic device 1100, including volatile and nonvolatile media, removable and non-removable media.

[0106] The storage 1160, which can be implemented as a non- volatile storage device such as a magnetic disk drive and / or optical disk drive, is coupled to the bus 1130 by a storage interface 1162. The storage interface 1162 facilitates access to information such as computer- readable instructions, data structures, program modules, and the like. Figure 8 not shown, a magnetic hard disk drive, read-only memory (ROM) 1144, a flash memory Figure 8 drive, a floppy disk drive, including the media associated therewith, and other like computer program products. The storage 1160 can be used to store various software components that facilitate the operation of the electronic device 1100. The storage 1160 can also be used to store various data such as electronic messages, documents, images, and the like.

[0107] A program / utility 1180 having a set (at least one) of program modules 1170, including

[0108] The program modules 1170 typically carry out the functions and / or methodologies of embodiments of the present disclosure as described herein. The electronic device 1100 can also communicate with one or more external devices 1190 such as a keyboard or pointing device, a display 1091, etc.; one or more devices that enable a user to interact with the electronic device 1100; and / or one or more devices that enable the electronic device 1100 to communicate with one or more other computing devices. Such communication can be via an input / output (I / O) interface 1192. Still yet, the electronic device 1100 can communicate with one or more networks such as a local area network (LAN), a general wide area network (WAN), and / or a public network such as the Internet, via a network adapter 1193. As Figure 8 illustrated, the network adapter 1193 communicates with the other components of the electronic device 1100 via the bus 1130. It should be appreciated that the network adapter 1193 and / or the other components of the electronic device 1100 can be communicatively coupled in other manners, consistent with the claimed subject matter. Figure 8 It should be appreciated that the claimed subject matter can be implemented with other computer system configurations, including, but not limited to, minicomputers, mainframe computers, as well as personal computers, hand-held computing devices, microprocessor-based or programmable consumer electronics, network PCs, and the like. As should be clear, the claimed subject matter can be also be implemented in a distributed computing environment, where tasks are performed by local and remote processing devices that can be in communication through a

[0109] The processor 1120 performs various function applications and data processing by running programs stored in the memory 1110.

[0110] It should be noted that the implementation process and technical principles of the electronic device of the present embodiment are described in the foregoing description of the monitoring processing method of the present disclosure, which will not be repeated here.

[0111] In order to implement the above-mentioned embodiments, the present application further provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the corpus processing method described in the above-mentioned embodiments.

[0112] In order to implement the above-mentioned embodiments, the present disclosure further provides a computer program product, when the instruction processor in the computer program product executes, the corpus processing method described in the above-mentioned embodiments is executed.

[0113] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples without contradiction.

[0114] In addition, the terms "first", "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "multiple" is at least two, for example, two, three, etc., unless otherwise specifically limited.

[0115] Any process or method descriptions in flow charts or otherwise described herein can be understood as representing code modules, segments, or portions of code which include one or more executable instructions for implementing specific logic functions (or steps) in the process, and the preferred embodiments of the present application include additional implementations in which the order of steps can differ from those shown or discussed, including a step can occur at other times, including as recited in the description, and the present application includes the possibility that some steps can be performed in parallel or with reverse order, as appropriate, with the appropriate functionality provided.

[0116] The logic and / or steps represented in the flowcharts and / or described herein, for example, can be considered as a sequence of instructions to implement logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, apparatus, or device, such as a computer-based system, processor- based system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable medium can be a computer- readable storage medium or a computer-readable signal medium. The computer- readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include the following: an electrical connection having one or more wires (electrical connections), a portable computer diskette (a magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium can even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example, via optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.

[0117] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the above embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. As such, in some embodiments, specifically configured hardware can be used to implement at least some of the functionality described herein. For example, if implemented in hardware, the hardware can include any or a combination of the following: a discrete logic circuit having logic gates for implementing logic functions upon data signals, an application specific integrated circuit having appropriate combinational logic gates, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0118] Those of skill in the art would understand that information and signals can be represented using any of a variety of technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that can be referenced throughout the above description can be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0119] In addition, each of the functional units in the various embodiments of the present application can be integrated in one processing module, or each of the units can be physically present separately, or two or more units can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software functional module. When the integrated module is realized in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

[0120] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. A corpus processing method, characterized by, The method comprises the following steps: acquiring a corpus processing request, wherein the corpus processing request comprises a corpus table file, attribute information of a model corresponding to the corpus table file, and a processing type; parsing the corpus table file according to the attribute information of the model to obtain corpus content in the corpus table file; processing the corpus content according to the processing type to determine corpus records of the model; wherein the attribute information comprises a type of the model; the processing type comprises at least one of the following types: a new type, an added type, and a replacement type; the processing of the corpus content according to the processing type to determine the corpus records of the model comprises: when the processing type is the added type, querying existing corpus record information matching the attribute information; adding the corpus content to existing corpus content corresponding to the existing corpus record information to obtain processed corpus content; determining the existing corpus record information and the processed corpus content as the corpus records of the model; the parsing of the corpus table file according to the attribute information of the model to obtain the corpus content in the corpus table file comprises: determining a parsing strategy for the corpus table file according to the type in the attribute information; parsing the corpus table file according to the parsing strategy to obtain the corpus content.

2. The method of claim 1, wherein, the attribute information comprises an identifier of the model; the processing of the corpus content according to the processing type to determine the corpus records of the model comprises: when the processing type is the new type, generating corpus record information comprising the identifier of the model; generating the corpus records of the model according to the corpus record information and the corpus content.

3. The method of claim 1, wherein, the processing of the corpus content according to the processing type to determine the corpus records of the model comprises: when the processing type is the replacement type, querying existing corpus record information matching the attribute information; replacing existing corpus content corresponding to the existing corpus record information with the corpus content; determining the existing corpus record information and the corpus content as the corpus records of the model.

4. The method of claim 1, wherein, The method further comprises the following steps: acquiring a corpus download request, wherein the corpus download request comprises corpus record information to be downloaded; querying existing corpus records according to the corpus record information to obtain corpus content to be downloaded corresponding to the corpus record information; performing format conversion on the corpus content to be downloaded to obtain a corpus table file to be downloaded and provide the corpus table file to a requester of the corpus download request.

5. A corpus processing device, characterized by, The method comprises the following steps: an acquiring module is configured to acquire a corpus processing request, wherein the corpus processing request comprises a corpus table file, attribute information of a model corresponding to the corpus table file, and a processing type; a parsing module is configured to parse the corpus table file according to the attribute information of the model to obtain corpus content in the corpus table file; a processing module is configured to process the corpus content according to the processing type to determine corpus records of the model. The attribute information includes a type of the model. The processing type includes at least one of a new type, an added type, and a replacement type. The processing module is specifically configured to, when the processing type is the added type, query existing corpus record information matched with the attribute information; add the corpus content to existing corpus content corresponding to the existing corpus record information to obtain processed corpus content; determine the existing corpus record information and the processed corpus content as the corpus record of the model. The analysis module is specifically configured to, determine an analysis strategy for the corpus table file according to the type in the attribute information; analyze the corpus table file according to the analysis strategy to obtain the corpus content.

6. The apparatus of claim 5, wherein, The attribute information includes an identifier of the model. The processing module is specifically configured to, when the processing type is the new type, generate corpus record information including the identifier of the model; generate the corpus record of the model according to the corpus record information and the corpus content.

7. The apparatus of claim 5, wherein, The processing module is specifically configured to, when the processing type is the replacement type, query existing corpus record information matched with the attribute information; replace existing corpus content corresponding to the existing corpus record information with the corpus content; determine the existing corpus record information and the corpus content as the corpus record of the model.

8. The apparatus of claim 5, wherein, Further comprising: a query module and a format conversion module; The acquisition module is further configured to acquire a corpus download request, wherein the corpus download request includes corpus record information to be downloaded; The query module is configured to query existing corpus records according to the corpus record information to acquire corpus content to be downloaded corresponding to the corpus record information; The format conversion module is configured to perform format conversion on the corpus content to be downloaded to obtain a corpus table file to be downloaded and provide the corpus table file to a requester of the corpus download request.

9. An electronic device, comprising: comprise: at least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method of any one of claims 1-4.

10. A non-transitory computer-readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to execute the method of any one of claims 1-4.

Images