Text analysis method and device, computer readable medium and electronic device

Abstract

The application provides a text analysis method and device, a computer readable medium and an electronic device. The method comprises: splitting a text to be analyzed into a plurality of first text blocks; for each field model in at least one field model, determining a first text block matched with the field model, the field model comprising an attribute field of a category; for each field model, constructing a first prompt text according to the field model and the first text block matched with the field model, the first prompt text being used to instruct a model to generate a field value corresponding to the attribute field in the field model; for the first prompt text corresponding to each field model, inputting the first prompt text into a predetermined model, and obtaining an analysis result corresponding to the field model according to an output result of the predetermined model; and aggregating the analysis results corresponding to the field models to obtain an analysis result of the text to be analyzed. The application can improve the accuracy of complex task analysis of long text.

Description

Technical Field

[0001] This application relates to the field of text processing technology, and more specifically, to a text parsing method, apparatus, computer-readable medium, and electronic device. Background Technology

[0002] With the development of information technology, text parsing technology has also made great progress.

[0003] Currently, technologies that use language models to parse text have emerged.

[0004] However, current text parsing techniques have low accuracy when parsing long texts for complex tasks. Summary of the Invention

[0005] The embodiments of this application provide a text parsing method, apparatus, computer-readable medium, and electronic device, which can at least to some extent improve the accuracy of parsing long texts for complex tasks.

[0006] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0007] According to one aspect of the embodiments of this application, a text parsing method is provided, the method comprising: splitting the text to be parsed into a plurality of first text blocks; for each field model in at least one field model, determining at least one first text block that matches the field model, the field model including a category of attribute fields; for each field model in the at least one field model, constructing a first prompt text based on the field model and the at least one first text block that matches the field model, the first prompt text being used to instruct the model to generate a field value corresponding to the attribute field in the field model; for each first prompt text corresponding to a field model, inputting the first prompt text into a predetermined model, and obtaining a parsing result corresponding to the field model based on the output result of the predetermined model; and aggregating the parsing results corresponding to each of the field models to obtain a parsing result for the text to be parsed.

[0008] According to one aspect of the embodiments of this application, a text parsing apparatus is provided, the apparatus comprising: a text splitting unit for splitting a text to be parsed into a plurality of first text blocks; a text block matching unit for determining at least one first text block matching a field model for each field model in at least one field model, the field model including a category of attribute fields; a prompt text construction unit for constructing a first prompt text for each field model in at least one field model based on the field model and the at least one first text block matching the field model, the first prompt text being used to instruct the model to generate a field value corresponding to an attribute field in the field model; a prompt text input unit for inputting the first prompt text corresponding to each field model into a predetermined model, and obtaining a parsing result corresponding to the field model based on the output result of the predetermined model; and an aggregation unit for aggregating the parsing results corresponding to each of the field models to obtain a parsing result for the text to be parsed.

[0009] In some embodiments of this application, based on the foregoing scheme, the field model further includes definition information corresponding to each attribute field in the attribute fields of the category, and the device further includes a splitting and mapping unit; before determining at least one first text block matching the field model, the splitting and mapping unit is configured to: for each first text block, split the first text block into multiple second text blocks with a size smaller than the first text block; map each second text block to a corresponding second text vector; the text block matching unit is configured to: map the first text information in the definition information to a first text vector; determine at least one second text vector most similar to the first text vector among each second text vector, and take the second text blocks corresponding to the at least one second text vector as second text blocks matching the field model; take the first text blocks to which each second text block matching the field model belongs as first text blocks matching the field model.

[0010] In some embodiments of this application, based on the foregoing scheme, the text block matching unit is further configured to: determine, among the plurality of first text blocks, a first text block containing the second text information in the definition information, as the first text block that matches the field model.

[0011] In some embodiments of this application, based on the foregoing scheme, the splitting and mapping unit is configured to: split the first text block multiple times for each first text block, and when splitting the first text block each time, split the first text block into multiple second text blocks with smaller sizes according to the size corresponding to the current splitting operation, wherein the sizes corresponding to different splitting operations of the first text block are different.

[0012] In some embodiments of this application, based on the foregoing scheme, the prompt text input unit is configured to: verify the output result of the predetermined model; if the output result passes the verification, then the output result is used as the parsing result corresponding to the field model.

[0013] In some embodiments of this application, based on the foregoing scheme, the first prompt text is used to instruct the model to generate structured data in the target format. The prompt text input unit is configured to: determine whether the output result of the predetermined model is structured data in the target format; after verifying the output result of the predetermined model, the prompt text construction unit is further configured to: if the output result of the predetermined model is not structured data in the target format, construct a new first prompt text based on the field model corresponding to the output result and at least one first text block matching the field model, and input the new first prompt text into the predetermined model to instruct the predetermined model to correct the output result to structured data in the target format.

[0014] In some embodiments of this application, based on the foregoing scheme, the device further includes a field value correction unit, and the prompt text input unit is configured to: for each attribute field, verify whether the field value corresponding to the attribute field in the output result conforms to the definition information corresponding to the attribute field in the field model corresponding to the output result; after verifying the output result of the predetermined model, the field value correction unit is used to: if the field value corresponding to the target attribute field in the output result does not conform to the definition information corresponding to the target attribute field in the field model corresponding to the output result, construct a second prompt text according to the definition information corresponding to the target attribute field and at least one first text block matching the field model, and input the second prompt text into the predetermined model to instruct the predetermined model to correct the field value corresponding to the target attribute field to a new field value that conforms to the definition information corresponding to the target attribute field; if the new field value corresponding to the target attribute field output by the predetermined model conforms to the definition information corresponding to the target attribute field, replace the field value corresponding to the target attribute field in the output result with the new field value corresponding to the target attribute field.

[0015] In some embodiments of this application, based on the foregoing scheme, the text to be parsed includes relevant information corresponding to each of at least one object, and the device further includes an object name acquisition unit and an information return unit; after splitting the text to be parsed into multiple first text blocks, the object name acquisition unit is used to: sequentially traverse the multiple first text blocks, and during the traversal of each first text block, whenever a first text block is traversed, construct a third prompt text based on the first text block to instruct the model to generate an object name, and input the third prompt text into a specified model to obtain the output result of the specified model, and obtain the object name in the first text block based on the output result; after obtaining the parsing result of the text to be parsed, the information return unit is used to: return the parsing result of the text to be parsed and the object names in each of the first text blocks to the front end.

[0016] In some embodiments of this application, based on the foregoing scheme, the object name acquisition unit is configured as follows: if the output result exists in the text to be parsed, then the output result is used as the object name in the first text block; if the output result does not exist in the text to be parsed, then a new third prompt text is constructed according to the corresponding first text block to instruct the model to generate the object name, and the new third prompt text is input into the specified model to obtain a new output result output by the specified model; if the new output result exists in the text to be parsed, then the new output result is used as the object name in the first text block.

[0017] According to one aspect of the embodiments of this application, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the text parsing method as described in the above embodiments.

[0018] According to one aspect of the embodiments of this application, an electronic device is provided, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the text parsing method as described in the above embodiments.

[0019] According to one aspect of the embodiments of this application, a computer program product is provided, the computer program product including computer instructions stored in a computer-readable storage medium, a processor of a computer device reading the computer instructions from the computer-readable storage medium, and the processor executing the computer instructions, causing the computer device to perform the text parsing method as described in the above embodiments.

[0020] In some embodiments of this application, the technical solutions involve first splitting the text to be parsed into multiple first text blocks, each containing a portion of the text to be parsed. Then, for each field model used to parse the text, a matching first text block is identified. A corresponding first prompt text is constructed based on the field model and the matching first text block. This first prompt text is then input into a predetermined model, and the parsing result corresponding to the field model is obtained based on the output of the predetermined model. Finally, the parsing results corresponding to each field model are aggregated to obtain the parsing result of the text to be parsed. Since each field model includes a category of attribute fields, this application breaks down the complex task of parsing the text into simpler sub-tasks at the granularity of tuples composed of a small number of fields. The input to the model is the construction of the first prompt text based on the field model and at least one matching first text block. This makes each sub-task more explicit and clear, thus improving the accuracy of parsing complex tasks involving long texts. Furthermore, since the entire text parsing process requires no manual operation, it saves labor costs and improves text parsing efficiency.

[0021] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0023] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of this application can be applied is shown;

[0024] Figure 2 A flowchart of a text parsing method according to an embodiment of this application is shown;

[0025] Figure 3 A block diagram of a system architecture according to an embodiment of this application is shown;

[0026] Figure 4 A timing diagram of the overall scheme according to one embodiment of this application is shown;

[0027] Figure 5 A schematic diagram illustrating the contents of a field model according to an embodiment of this application is shown;

[0028] Figure 6 An embodiment according to this application is shown. Figure 2 A flowchart of the steps preceding step 250 and details of step 250 in the embodiment;

[0029] Figure 7 An embodiment according to this application is shown. Figure 2 Flowcharts of steps 210 and 280 in the embodiment;

[0030] Figure 8 A flowchart illustrating the process of obtaining the object name in a first text block based on the output result according to an embodiment of this application is shown;

[0031] Figure 9 A schematic diagram of the first part of a text segmentation and recall process according to an embodiment of this application is shown;

[0032] Figure 10 A schematic diagram of the second part of a text segmentation and recall process according to an embodiment of this application is shown;

[0033] Figure 11 A schematic diagram of the third part of a text segmentation and recall process according to an embodiment of this application is shown;

[0034] Figure 12 A schematic diagram of the fourth part of a text segmentation and recall process according to an embodiment of this application is shown;

[0035] Figure 13 A schematic diagram of the fifth part of a text segmentation and recall process according to an embodiment of this application is shown;

[0036] Figure 14 A schematic diagram of the first part of a prompt word generation process according to an embodiment of this application is shown;

[0037] Figure 15 A schematic diagram of the second part of a prompt word generation process according to an embodiment of this application is shown;

[0038] Figure 16 A schematic diagram of the third part of a prompt word generation process according to an embodiment of this application is shown;

[0039] Figure 17 A schematic diagram of the fourth part of a prompt word generation process according to an embodiment of this application is shown;

[0040] Figure 18 A block diagram of a text parsing apparatus according to an embodiment of this application is shown;

[0041] Figure 19A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0042] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0043] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0044] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0045] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0046] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0047] Therefore, this application first provides a text parsing method. The text parsing method provided by the embodiments of this application can overcome the above-mentioned defects in related technologies and can at least achieve the following technical effects:

[0048] 1. The solution proposed in this application can parse contract text based on a general large language model. It understands the content of the article as a whole through the field model and the parser and then outputs the results. There is no need to train a separate contract text processing model, which can save computing resources. Moreover, there is no need to extract character data from the original contract text. It can not only ensure that the parsing results conform to the definition of the field model, but also has stronger reasoning ability.

[0049] 2. This application uses a custom field model and retrieval enhancement to generate data, which does not depend on the structure of the original contract text or have specific format requirements for the contract text. This improves the robustness of the system and makes it more applicable to the format of the contract text. Moreover, relevant paragraphs can be extracted based on the field model, and the output structured data corresponds to the field model rather than the hierarchical structure of the original text, making it more standardized and easier for downstream users to access.

[0050] 3. This application automatically generates prompt word chains through field models, eliminating the need for manual maintenance of training sets and separate training of large models for contract parsing tasks, thus saving on training set maintenance and update costs. Furthermore, this application automatically extracts text fragments that hit keywords through retrieval enhancement that includes mixed retrieval, thereby narrowing the scope of questions and improving the accuracy of results, without requiring additional differentiation of different types of information in the contract text when configuring information extraction.

[0051] In other words, the text parsing method provided in this application is actually a contract text parsing method based on a field model and a large language model. It realizes the automatic parsing of contracts into structured data and can achieve the following technical effects:

[0052] 1. A field model is proposed as a way to decompose large language model tasks. It uses tuples composed of a small number of fields as the granularity and automatically generates prompt word chains each time, thereby splitting complex contract parsing tasks into simple small tasks. The prompt word engineering improves logical reasoning, association and prediction capabilities.

[0053] 2. Define a result parser for the large language model based on the field model to ensure that the parsing results conform to the metadata definition, which facilitates data display and downstream access.

[0054] 3. Based on the characteristics of standardized terminology in contract texts, a hybrid retrieval and parent document retrieval approach is adopted to enhance retrieval generation, providing high-precision retrieval enhancement capabilities and thus improving the accuracy of results.

[0055] In the copyright operation process of video platforms, the operations team regularly reads the latest contract texts, interprets the rights and interests related to the subject matter (usually film and television works) in the contract texts, and after verifying with the legal department, extracts structured data to support the standardized operation process downstream.

[0056] The solution in this application embodiment can be applied to the decomposition of contract documents. After the operation provides the field model, the system can automatically interpret and extract various attribute values ​​of the subject matter in the contract, and output the rights in the form of structured data. After aggregating various models, a binary pair of "subject matter -> rights" is finally formed, which is handed over to the operation for confirmation as a draft, thereby improving work efficiency.

[0057] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of this application can be applied is shown. For example... Figure 1 As shown, the system architecture 100 may include an operation terminal 110, a backend server 120, a file server 130, an inference server 140, and a database server 150. The operation terminal 110, file server 130, inference server 140, and database server 150 are all connected to the backend server 120. The operation terminal 110 is a terminal used by operators and has a client deployed on it capable of accessing the text parsing service. The backend server 120 has a text parsing service deployed on it. The file server 130 stores the contract documents to be parsed. The inference server 140 has a large model deployed on it, and the database server 150 has a vector database deployed on it. The backend server 120 can serve as the execution entity of the scheme in this application embodiment. When the text parsing method provided in this application embodiment is applied... Figure 1In the system architecture shown, a process can be as follows: First, the operator configures the field model to the text parsing service on the backend server 120 through the client on the operation terminal 110. The backend server 120 stores the field model. Then, the operator accesses the text parsing service on the backend server 120 through the client on the operation terminal 110 to submit the field model identifier and the corresponding contract identifier to the text parsing service. Next, the text parsing service retrieves the corresponding contract file from the file server 130 according to the contract identifier, extracts the text to be parsed from the contract file, and splits the text to be parsed into multiple first text blocks. Then, the text parsing service splits each first text block into multiple second text blocks with a size smaller than the first text block, maps the second text blocks to corresponding second text vectors, and stores each second text block and its corresponding second text vector in the vector database on the database server 150. Finally, the text parsing service extracts the field model information from each field model... The text parsing service extracts first text information from the corresponding field model, maps this first text information to a first text vector, and retrieves the second text block corresponding to the second text vector that matches the first text vector from the vector database. Subsequently, the text parsing service determines the first text block to which these retrieved second text blocks belong, as the first text block that matches the corresponding field model. Then, for each field model, the text parsing service constructs a first prompt text containing the field model and the first text block that matches the field model, and obtains the output result returned by the large model by inputting the first prompt text to the large model on the inference server 140. The text parsing service obtains the parsing result corresponding to the corresponding field model based on the output result. After obtaining the parsing results corresponding to each field model, the text parsing service aggregates the parsing results corresponding to each field model to obtain the parsing result of the text to be parsed, and returns the parsing result of the text to be parsed to the operation terminal 110.

[0058] In one embodiment of this application, after the parsing result of the text to be parsed is returned to the operation terminal 110, the operator will submit a score for the parsing result corresponding to each field model through the page displaying the parsing result of the text to be parsed. The text parsing service will divide the parsing result corresponding to each field model into positive and negative samples according to the score of the parsing result corresponding to each field model, and store the positive and negative samples.

[0059] In one embodiment of this application, the first prompt text constructed by the text parsing service also includes positive and negative samples corresponding to the corresponding field model.

[0060] It should be understood that Figure 1The number of operating terminals, backend servers, file servers, inference servers, and database servers shown is merely illustrative. Depending on implementation needs, there can be any number of operating terminals, and the number of backend servers, file servers, inference servers, and database servers can also be arbitrary. That is, there can be multiple operating terminals, and the backend servers, file servers, inference servers, and database servers can all be server clusters composed of multiple servers.

[0061] It should be noted that, Figure 1 The illustration shown is merely one embodiment of this application. Although in Figure 1 In the embodiment of the scheme, the operating terminal is a desktop computer, but in other embodiments of this application, the operating terminal can also be various types of terminal devices such as smartphones, laptops, tablets, vehicle terminals, portable wearable devices, and workstations; although Figure 1 The solution in this embodiment is used for parsing contract documents, but in other embodiments of this application, it can also be used for parsing various types of documents, including legal documents; although in Figure 1 In the embodiment, both the positive and negative samples and the field model are stored on the backend server. However, in other embodiments of this application, the positive and negative samples and the field model can also be stored in other locations besides the backend server, and the storage locations of the positive and negative samples and the field model can be different. Figure 1 In the embodiments described, text information is mapped to text vectors through a text parsing service. However, in other embodiments of this application, the text parsing service may also map text information to text vectors by calling interfaces of other services. This application does not limit the scope of protection of this application in any way.

[0062] It is easy to understand that the text parsing method provided in the embodiments of this application is generally executed by a server, and correspondingly, the text parsing device is generally located in the server. However, in other embodiments of this application, the terminal device may also have similar functions to the server, thereby executing the text parsing scheme provided in the embodiments of this application.

[0063] Therefore, the embodiments of this application can be applied to terminals or servers. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, and this application does not impose any restrictions.

[0064] The implementation details of the technical solutions in the embodiments of this application are described in detail below:

[0065] Figure 2 A flowchart of a text parsing method according to an embodiment of this application is shown. This text parsing method can be executed by various devices with processing and computing capabilities. Specifically, it can be executed by a target device, such as a user terminal or a cloud server. User terminals include, but are not limited to, mobile phones, computers, smart voice interaction devices, smart home appliances, vehicle terminals, aircraft, smartwatches, etc. Please refer to... Figure 2 As shown, this text parsing method includes at least the following steps:

[0066] In step 210, the text to be parsed is split into multiple first text blocks.

[0067] The text to be parsed can be text from a contract document, and each first text block contains a portion of the text to be parsed. Below, we will describe the solution of this application embodiment using text from a contract document as an example. Before detailing step 210, we will first introduce other content.

[0068] Contract documents refer to legally binding documents that embody the will of the contracting parties and are formed throughout the entire process from signing to termination of the contract, as well as other related documents and materials that have archival value.

[0069] A contract document may include one or more subject matters. The subject matter refers to the specific goods, services, rights, or other matters stipulated in the contract. In a contract, the subject matter describes the main content and object of the contract. For example, in a film and television copyright contract, the film or television series itself is usually the subject matter, because the contract involves the transfer, licensing, or other related matters of the film or television series' copyright.

[0070] Figure 3 A block diagram of a system architecture according to one embodiment of this application is shown. Please refer to... Figure 3As shown, the system architecture of this application embodiment mainly includes four parts: text parsing service, vector database, AI agent service, and access layer service. Among them, the text parsing service can be a contract parsing service; the vector database can be privately deployed and can provide the ability to store and retrieve text vector data; the AI ​​agent service provides a text vectorization interface and an LLM dialogue interface, which is the large language model calling interface. Figure 3 The Elasticsearch service in the database can store information related to the target.

[0071] Before splitting the text to be parsed into multiple first text blocks, it is necessary to obtain the text to be parsed first.

[0072] Figure 3 The text parsing service can retrieve files to be parsed from external file systems and process them into string format data, which serves as the parsed text. The format of the files retrieved by the text parsing service can be .docx, .doc, .pdf, .txt, .xls, etc.

[0073] Before splitting the text to be parsed into multiple first text blocks, it is usually necessary to obtain the field model used to parse the text.

[0074] Figure 4 A timing diagram of the overall scheme according to one embodiment of this application is shown.

[0075] Please see Figure 4 As shown in the sequence diagram, the overall solution mainly involves five objects: users, access layer, contract parsing, vector database, and AI agent service. Contract parsing refers to the aforementioned text parsing service, and users are the operations personnel who can interact with the access layer through the front-end page. Figure 4 As shown, it may include the following steps:

[0076] 1. Operational configuration field model.

[0077] Operations personnel submit configured field models to the access layer via the front-end page. The access layer can store the configured field models. Operations personnel can configure at least one field model.

[0078] When configuring field models, operators can also associate field models with contract types or contract numbers (contract identifiers). This way, when contract documents need to be parsed, the system can automatically use the field model associated with the contract document; similarly, when text parsing using field models is required, the system can automatically parse the contract document associated with the field model.

[0079] The field model in this application refers to a group of fields aggregated from fields strongly related to business logic. This includes textual descriptions of the business logic and the fields themselves, representing a type of attribute of the subject matter. In the process, it represents the smallest granularity of information output by the large language model in a single analysis. The combination of all field models yields the rights and interests of the subject matter confirmed in the contract. Figure 3 In the process, operators can configure field models, and these field model configurations can be stored in the access layer.

[0080] Figure 5 A schematic diagram illustrating the contents of a field model according to an embodiment of this application is shown. Please refer to... Figure 5 As shown, the content of the field model may include field name, field alias, field format (e.g., whether it is an array, text rules), enumeration value, enumeration value alias, enumeration value description, etc. Therefore, the field model in this application includes a category of attribute fields, such as... Figure 5 The field model includes attribute fields for the category of editing rights. Different field models can include attribute fields for different categories.

[0081] Operators can base on Figure 3 The information configuration field model is provided by the metadata service. In other words, the content of the field model can be a part of the metadata information in the metadata service, and the field model can be generated based on the metadata information in the metadata service.

[0082] 2. Select a contract and model.

[0083] Operations personnel select the contract file to be parsed and the field model used for parsing the contract file through the front-end page. The access layer obtains the identifier of the field model and the contract number. When the association between the field model and the contract type or contract number (contract identifier) ​​is configured, once the user selects a field model, the front-end page can automatically display the associated contract information for the user to choose from. Similarly, once the user selects contract information, the front-end page can also automatically display the information of the associated field model for the user to choose from. The contract file to be parsed selected by the operations personnel can be the most recently generated contract file in the file system.

[0084] 3. Input the model and contract number.

[0085] The access layer sends the field model and contract number to the contract parsing service. The contract parsing service can obtain the corresponding contract file based on the contract number and parse the obtained contract file based on the field model.

[0086] Step 210 will be described in detail below.

[0087] The size of the first text blocks into which the text to be parsed is divided can be the same; for example, the text to be parsed can be divided into first text blocks of 1024 characters each. Of course, the sizes of different first text blocks can also be different. Multiple first text blocks are obtained sequentially from the text to be parsed, and the content in different first text blocks can be completely non-overlapping. However, the content in two adjacent first text blocks can partially overlap; for example, the last part of the content in the first first text block can be used as the starting part of the content in the second first text block. This reduces sentence breaks and ensures the integrity of the content in each first text block. Of course, for text with a relatively clear structure, it can also be divided into blocks according to the document's table of contents or other structural elements.

[0088] Please continue reading Figure 3 As shown, the text processing module in the text parsing service has the function of text segmentation.

[0089] Figure 6 An embodiment according to this application is shown. Figure 2 A flowchart detailing the steps preceding step 250 and step 250 itself is provided in this embodiment. Please refer to [link / reference needed]. Figure 6 As shown, before performing step 250, which involves determining at least one first text block that matches the field model, the method may further include the following steps:

[0090] In step 220, for each first text block, the first text block is split into multiple second text blocks with a size smaller than the first text block.

[0091] Specifically, the different second text blocks into which the same first text block is split can be the same size or different sizes. For example, a first text block of 1024 characters can be split into a second text block of 64 characters or 128 characters. The different second text blocks into which the same first text block is split can be similar in size but not exactly the same. For example, one second text block into which the same first text block is split can be 63 characters in size, and another second text block into size can be 66 characters in size.

[0092] Of course, the first text block can also be split multiple times.

[0093] In one embodiment of this application, for each first text block, splitting the first text block into multiple second text blocks with a size smaller than the first text block includes: splitting the first text block multiple times for each first text block, and splitting the first text block into multiple second text blocks with a size smaller than the first text block according to the size corresponding to the current splitting operation when splitting the first text block each time, wherein the size corresponding to different splitting operations of the first text block is different.

[0094] For example, the first text block can be split multiple times into blocks of 128 characters, 64 characters, and 32 characters. The first split yields a second text block of 128 characters; the second split yields a second text block of 64 characters; and the third split yields a second text block of 32 characters. As can be seen, the sizes corresponding to the different splitting operations on the first text block can have a multiple relationship.

[0095] Of course, the different splitting operations mentioned above can be performed in parallel, meaning that there is no specific order between the different splitting operations.

[0096] Of course, the size of the second text block obtained in each split can also be different. For example, multiple different size ranges can be set, with each split operation corresponding to one size range; each split operation splits according to the corresponding size range, thus obtaining a second text block whose size belongs to the size range corresponding to that split operation. For example, a size range can be 60-68, then the size of the second text block obtained by splitting according to that size range will be between 60 and 68.

[0097] In step 230, each second text block is mapped to its corresponding second text vector.

[0098] It can be called Figure 3 The AI ​​agent service provides a text vectorization interface to map each second text block to its corresponding second text vector.

[0099] The text vectorization interface can be provided by a privately deployed open-source component. After each second text block is input into the interface, the interface returns a second text vector represented as a floating-point array. The second text vector can be stored in a vector database for subsequent text retrieval. Figure 3 The Chinese text processing module can use its vectorization storage capability to store the obtained second text vector into the vector database.

[0100] Please continue reading Figure 4 As shown, after step 3, the following may also be included:

[0101] 4. Parse the contract document into text and divide it into blocks.

[0102] The contract parsing service extracts text from the contract document and divides the text into a first text block and a second text block.

[0103] 5. Vectorize the text blocks and store them in the database.

[0104] The contract parsing service inputs the second text block into the text vectorization interface provided by the AI ​​agent service, and obtains the vectorized text (i.e. the second text vector) returned by the AI ​​agent service. The contract parsing service then stores the obtained second text vector into the vector database.

[0105] In one embodiment of this application, the text to be parsed includes relevant information corresponding to each of at least one of the objects.

[0106] The object here can be the subject matter of the contract (such as a film or television series). A contract document can include relevant information corresponding to one or more films or television series. For example, a contract document can include information related to the editing rights of film or television series A, the editing rights of film or television series B, and the editing rights of film or television series C, etc.

[0107] Figure 7 An embodiment according to this application is shown. Figure 2 The flowcharts for steps 210 and 280 in this embodiment are shown below. Please refer to [link / reference]. Figure 7 As shown, after splitting the text to be parsed into multiple first text blocks, the method may include the following steps:

[0108] In step 240, multiple first text blocks are traversed sequentially. During the traversal of each first text block, whenever a first text block is traversed, a third prompt text is constructed based on the first text block to instruct the model to generate the object name. The third prompt text is then input into the specified model to obtain the output result of the specified model. The object name in the first text block is obtained based on the output result.

[0109] The specified model can be a Large Language Model (LLM), also known as a large model, which can be multimodal.

[0110] The specified model can have hundreds of billions of parameters, meaning it can have over 100 billion parameters, or the number of parameters for a single model can be in the billions. The specified model can be the same large model as the predetermined model described below, or it can be a different large model. In this embodiment, the first text block is traversed and interpreted by calling a large language model.

[0111] The third prompt text may include the first text block and related instructions.

[0112] Prompt text, also known as cue words, is a keyword or phrase used to help retrieve and understand relevant information in a contract document. It can be used to activate a large language model to generate specific outputs or complete specific tasks.

[0113] The specified model can output object names based on third-party prompt text; that is, the output of the specified model can be object names. The object name can be the name of the target object.

[0114] When traversing each first text block, if the object name obtained from analyzing a certain first text block is the same as the object name obtained from analyzing a previous first text block, the latest obtained object name can be removed.

[0115] Figure 8 A flowchart illustrating the process of obtaining the object name in a first text block based on the output result, according to one embodiment of this application, is shown. See also... Figure 8 As shown, obtaining the object name in the first text block based on the output result can specifically include the following steps:

[0116] In step 810, if the output result exists in the text to be parsed, the output result is used as the object name in the first text block.

[0117] The name of the subject matter in the output must be in the original contract text. Therefore, if the output exists in the text to be parsed, it can be used directly as the name of the subject matter.

[0118] In step 820, if the output result does not exist in the text to be parsed, a new third prompt text is constructed based on the corresponding first text block to indicate the name of the object generated by the model, and the new third prompt text is input into the specified model to obtain a new output result from the specified model.

[0119] The new third prompt text may include the corresponding first text block.

[0120] If the output does not exist in the text to be parsed, the prompt words are adjusted and the large language model is called again to reinterpret the corresponding text block. The original output can be discarded. The original output can be stored separately. When parsing other contract documents later, the constructed third prompt text can include information indicating that the model's output should not contain the original output.

[0121] In step 830, if the new output exists in the text to be parsed, the new output is used as the object name in the first text block.

[0122] Please continue reading Figure 3As shown, the prompt word engineering module mainly includes two prompt word chains. The first prompt word chain first calls LLM to interpret the text and extract the contract-related subject matter names. Then, it matches the subject matter names with the original text (the original contract text) to determine whether the subject matter names exist in the original text, thus verifying the matching results to ensure the accuracy of the subject matter names. Finally, it retrieves other information related to the subject matter names from the Elasticsearch service. After obtaining the parsing results of the text to be parsed, the parsing results, the subject matter names, and other information related to the subject matter names can be returned to the front end. Alternatively, the subject matter names can be verified by checking whether they exist in the metadata service.

[0123] Please continue reading Figure 4 As shown, after step 5, the following may also be included:

[0124] 6. The LLM parsing contract includes the subject matter.

[0125] The contract parsing service calls the interface provided by the AI ​​agent service to parse the name of the subject matter from the contract text, and the AI ​​agent service returns the name of the subject matter to the contract parsing service.

[0126] 7. Verify the name of the subject matter.

[0127] The contract parsing service verifies whether the obtained subject matter name is correct.

[0128] In step 250, for each field model in at least one field model, at least one first text block matching the field model is determined, the field model including a category of attribute fields.

[0129] At least one field model is used to parse the text and the file to be parsed. The attribute fields in the field model are... Figure 5 Fields such as field name and authorization rights.

[0130] In one embodiment of this application, the field model further includes definition information corresponding to each attribute field in a category of attribute fields.

[0131] The definition information corresponding to the attribute field can be Figure 5 Each field, such as the field name and the authorized rights, corresponds to a single row of information.

[0132] In one embodiment of this application, determining at least one first text block that matches a field model includes: identifying a first text block containing second text information from the definition information among a plurality of first text blocks as the first text block that matches the field model.

[0133] The steps in this embodiment are actually precise matching operations. The second text information may include... Figure 5 At least one of the following: field name, field alias, enumeration value, enumeration value alias.

[0134] Therefore, this application embodiment uses keywords for precise matching to determine the first text block that matches the field model.

[0135] Please continue reading Figure 6 As shown, determining at least one first text block that matches the field model may include the following steps:

[0136] In step 251, the first text information in the definition information is mapped to a first text vector.

[0137] The first text information in the definition can be any information different from the second text information. The first text information can be descriptive information corresponding to the attribute field. For example, the first text information can be... Figure 5 The enumeration values ​​are explained below. The first text information can be mapped to a first text vector by calling the text vectorization interface of the AI ​​agent service.

[0138] In step 252, at least one second text vector that is most similar to the first text vector is determined from each second text vector, and the second text block corresponding to each of the at least one second text vector is taken as the second text block that matches the field model.

[0139] The similarity between the first text vector and each of the second text vectors can be determined by cosine similarity, and the at least one second text vector with the highest similarity is taken as the at least one second text vector most similar to the first text vector.

[0140] It can also determine the Euclidean distance between the first text vector and each of the second text vectors, and map the Euclidean distance to similarity according to a predetermined rule, wherein the similarity is negatively correlated with the Euclidean distance.

[0141] In step 253, the first text block to which each second text block that matches the field model belongs is taken as the first text block that matches the field model.

[0142] If different second text blocks belong to the same first text block, then only one first text block can be retained.

[0143] The steps in this application embodiment are actually a semantic recall process.

[0144] This application implements vector matching, which, combined with the aforementioned precise matching, enables hybrid retrieval and effectively improves the accuracy of matching. At the same time, since the lengths of the enumeration values ​​corresponding to different attribute fields are often different, and the second text block in this application can be obtained by splitting the first text block multiple times according to different sizes, this application can more accurately match the second text block and improve the accuracy of file parsing.

[0145] This application Figure 3 The text processing module in this embodiment provides both vector matching and exact matching capabilities.

[0146] Specifically, in this embodiment, the first text block matching the field model can be determined firstly through the second text information, and then the first text block matching the field model can be determined again through the first text information. The number of first text blocks matching the field model determined by the first text information can be determined based on the total number of characters of all first text blocks matching the field model determined by the second text information and a predetermined context length. Specifically, the predetermined context length can be a value that is less than the context length of the predetermined model, and it can be close to the context length of the predetermined model. The second text blocks corresponding to each second text vector can be sorted from high to low similarity with the first text vector to obtain a sorted list. After determining the first text block that matches the field model through the second text information, the first text block that matches the field model is taken as the first current set. Then, each time, the first second text block is taken from the sorted list, and the calculation is performed to determine if the first text block to which the second text block belongs is added to the current set to obtain a new current candidate set. After deduplicating the first text blocks in the new current candidate set, the total number of characters of the first text blocks in the new current candidate set is calculated. If the total number of characters is less than the predetermined context length, the first text block to which the second text block belongs is added to the current set to obtain a new current set. Then, the next second text block is taken from the sorted list until the total number of characters of the first text blocks in the new current candidate set reaches the predetermined context length. The final new current set is taken as all the first text blocks that match the field model.

[0147] In this embodiment of the application, by determining which second text blocks belong to the first text blocks that can be used as the first text blocks that match the field model with a predetermined context length, the model can process more first text blocks at once, thereby improving the accuracy of text parsing.

[0148] Figures 9-13 This describes the text segmentation and recall process in an embodiment of this application. Figures 9-13In this context, identical nodes in different attached figures represent connections between the two figures. In other words, the same node represents a corresponding position in different attached figures, which corresponds to a position in the text segmentation and recall process. For example, Figure 9 Node 1 in the middle will be with Figure 10 The nodes in the text are connected to node 1, which means they represent the same location in the text segmentation and recall process.

[0149] Figure 9 A schematic diagram of the first part of a text segmentation and recall process according to an embodiment of this application is shown. Please see below. Figure 9 As shown, for the original contract text, 1024 characters can be divided into a first text block each time. There can be a small amount of overlap between different first text blocks, which also reduces the number of sentence breaks.

[0150] Figure 10 A schematic diagram of the second part of a text segmentation and recall process according to an embodiment of this application is shown. See also... Figure 10 As shown, the dashed boxes represent the entire contract text, and each solid box represents a first text block obtained from the split. It is the base file block and can be directly used to fill the prompt. After obtaining the first text block, it is also necessary to split each first text block into smaller blocks according to different sizes.

[0151] Figure 11 A schematic diagram of the third part of a text segmentation and recall process according to an embodiment of this application is shown. Please see below. Figure 11 As shown, the dashed box represents the first text block; therefore, the dashed box in this figure is equivalent to... Figure 10 The solid-line box in each dashed box represents a second text block obtained by splitting the first text block according to a certain size. Therefore, this application splits the first text block according to sizes of 32 characters, 64 characters, and 128 characters respectively, and obtains second text blocks corresponding to different sizes.

[0152] Figure 12 A schematic diagram of the fourth part of a text segmentation and recall process according to an embodiment of this application is shown. Please see below. Figure 12 As shown, the system will determine the similarity score between each second text block and the first text information, and sort the second text blocks according to each similarity score.

[0153] Figure 13 A schematic diagram of the fifth part of a text segmentation and recall process according to an embodiment of this application is shown. See also... Figure 13 As shown, the system will obtain the first matching text block based on the sorting result of the second text block.

[0154] In step 260, for each field model in at least one field model, a first prompt text is constructed based on the field model and at least one first text block that matches the field model. The first prompt text is used to instruct the model to generate field values ​​corresponding to attribute fields in the field model.

[0155] This application constructs a first prompt text by retrieving at least one first text block that matches the field model, which can narrow the range of text that the model needs to analyze, thereby improving the accuracy of the parsing results.

[0156] The initial prompt text may contain a field model and at least one first text block that matches the field model. Alternatively, the field model may be supplemented by extracting field information from metadata information in a metadata service, and the field model in the initial prompt text may be this supplemented field model.

[0157] In one embodiment of this application, the first prompt text may further include positive and / or negative samples, which are determined based on the user's rating of the historical parsing results corresponding to the field model.

[0158] Specifically, the first prompt text may contain content indicating that a positive sample should be output when the provided first text block contains a positive sample, or it may contain content indicating that a positive sample should not be output when the provided first text block contains a negative sample.

[0159] In this embodiment of the application, by setting content related to positive and / or negative samples in the first prompt text, the large language model can achieve text parsing more accurately.

[0160] Please continue reading Figure 4 As shown, after step 7, the following may also be included:

[0161] The contract parsing service will iterate through all field models and execute steps 8-11.

[0162] 8. Obtain relevant sample examples for the model.

[0163] The contract parsing service obtains positive and negative samples related to the field model from the access layer, and the access layer can return the positive and negative samples it requested to the contract parsing service.

[0164] 9. Recall the relevant texts.

[0165] The contract parsing service retrieves vectors that match the field model from the vector database, and then determines the first text block that matches the field model.

[0166] In step 270, for each field model, the first prompt text is input into the predetermined model, and the parsing result corresponding to the field model is obtained based on the output of the predetermined model.

[0167] In this embodiment of the application, a field may be assigned to multiple models for multiple parsings. Each time, a certain enumeration value of the field is parsed. For example, "authorization rights" includes the enumeration value "editing rights". The fields related to "editing rights" include "editing rights definition conditions" and "editing rights other definition conditions". When parsing the field model related to "editing rights", the enumeration value of "authorization rights" provided to the large model is only "editing rights". The final aggregated value of "authorization rights" is the union of the "authorization rights" values ​​in each field model.

[0168] In one embodiment of this application, obtaining a parsing result corresponding to a field model based on the output of a predetermined model includes: verifying the output of the predetermined model; if the output passes the verification, then the output is used as the parsing result corresponding to the field model.

[0169] In one embodiment of this application, the first prompt text is used to instruct the model to generate structured data in the target format.

[0170] The target format for structured data can be JSON. Alternatively, solutions such as GBNF syntax in llama.cpp or guidance libraries can be introduced to restrict the output of large language models to JSON-formatted structured data.

[0171] In one embodiment of this application, verifying the output of a predetermined model includes: determining whether the output of the predetermined model is structured data in a target format; after verifying the output of the predetermined model, the method further includes: if the output of the predetermined model is not structured data in a target format, constructing a new first prompt text based on the field model corresponding to the output and at least one first text block matching the field model, and inputting the new first prompt text into the predetermined model to instruct the predetermined model to correct the output of the predetermined model to structured data in the target format.

[0172] The new first prompt text may contain the field model corresponding to the output result and at least one first text block that matches the field model.

[0173] If the new first prompt text is input into the predefined model and the output result in the correct target format is still not obtained, a regular expression can be generated based on the corresponding field model. The field value can be extracted from the current or previous output result of the predefined model using the regular expression as the corresponding parsing result.

[0174] In one embodiment of this application, the output result of the predetermined model is validated, including: for each attribute field, validating whether the field value corresponding to the attribute field in the output result conforms to the definition information of the attribute field in the field model corresponding to the output result; after validating the output result of the predetermined model, the method further includes: if the field value corresponding to the target attribute field in the output result does not conform to the definition information of the target attribute field in the field model corresponding to the output result, then constructing a second prompt text based on the definition information of the target attribute field and at least one first text block matching the field model, and inputting the second prompt text into the predetermined model to instruct the predetermined model to correct the field value corresponding to the target attribute field to a new field value that conforms to the definition information of the target attribute field; if the new field value corresponding to the target attribute field output by the predetermined model conforms to the definition information of the target attribute field, then replacing the field value corresponding to the target attribute field in the output result with the new field value corresponding to the target attribute field.

[0175] The system can iterate through and validate each attribute field in the output, determining whether the value of each attribute field conforms to the definition information of that attribute field in the corresponding field model. For example, the definition information of an attribute field may include an enumerated value or a date format. If it does not conform, a correction prompt is generated for the attribute field according to the field model configuration. The large language model is then called to obtain the correction result, and validation is performed again. If the validation is successful after correction, the field value is changed to the corrected value and populated back into the context for subsequent operations.

[0176] In step 280, the parsing results corresponding to each field model are aggregated to obtain the parsing results of the text to be parsed.

[0177] The parsing results of the aggregated text can be returned to the user for confirmation.

[0178] Once the parsing results of the text to be parsed are obtained, they can be returned to the user. The user can provide a score for the parsing results corresponding to the field model. The system can then classify the parsing results corresponding to the field model into positive and negative samples based on this score. For example, the user can score the parsing results corresponding to the field model between 1 and 10. Parsing results with a score of 5 or higher can be considered positive samples, indicating high-quality parsing results; parsing results with a score less than 5 can be considered negative samples, indicating low-quality parsing results.

[0179] Please continue reading Figure 7 As shown, after obtaining the parsing result of the text to be parsed, the method further includes:

[0180] In step 290, the parsing results of the text to be parsed and the object names in each first text block are returned to the front end.

[0181] The object names in each first text block are the same as the object names in the text to be parsed.

[0182] Please continue reading Figure 3 As shown, in the prompt word engineering module, the second prompt word chain selects the corresponding first text block based on the matched vector and selects the sample corresponding to the field model. Then, prompt words are automatically generated based on the field model, the first text block, and the sample. After task decomposition, the large language model is called through the LLM dialogue interface to interpret the text for text parsing. Finally, the parsing results are aggregated by performing data entity aggregation operations. In the second prompt word chain, a custom parser can also be called to parse the output of the large language model based on the field model. For unqualified fields, LLM tuning is iteratively called to ensure the accuracy of the output of the large language model.

[0183] Subsequently, the access layer can display the aggregated parsing results of the text to be parsed to the user. The user can provide feedback to the access layer on the parsing results corresponding to each field model. The access layer can determine the positive and negative samples based on the scores and store the positive and negative samples.

[0184] Figures 14-17 A prompt word generation process according to one embodiment of this application is illustrated. Please refer to [link to relevant documentation]. Figures 14-17 The system can identify samples that match the field model from the sample set, and it can also identify text blocks that match the field model from the original contract text. After supplementing the field model with field information, the supplemented field model, samples, and text blocks are used to construct prompt words. The constructed prompt words can be used for multi-turn dialogues with the large language model. That is, each dialogue only inputs a part of the prompt words into the large language model. The advantage of doing this is that it makes the task performed by the large language model more explicit each time, thereby improving the accuracy of the output results of the large language model.

[0185] Please continue reading Figure 4 As shown, after step 9, the following may also be included:

[0186] 10. Model, sample, text concatenation prompts, call LLM.

[0187] The contract parsing service concatenates the field model, sample, and text into prompt words, and inputs the prompt words into the large model; the large model can then return JSON format data to the contract parsing service.

[0188] 11. Format and enumeration value parsing and validation; if unqualified, request another LLM iteration with context.

[0189] The contract parsing service verifies the output of the large model in terms of format, enumeration values, etc. If the verification fails, it constructs corresponding prompt words based on the context and requests the large model to parse it again.

[0190] 12. Results of the combined analysis.

[0191] The contract parsing service aggregates the parsing results corresponding to each field model, and obtains the parsing results in the format of "contract -> subject matter -> rights information", and returns the parsing results to the access layer.

[0192] 13. Results Display.

[0193] The access layer displays the parsed results to the user.

[0194] 14. Users rate the results.

[0195] The access layer obtains the user's rating of the parsing results corresponding to each field model, and marks and stores the parsing results corresponding to each field model based on the rating.

[0196] In summary, the text parsing method provided in the embodiments of this application can achieve at least the following technical effects:

[0197] 1. A method for automatically breaking down film and television contracts is provided, which is efficient and fast.

[0198] 2. Based on the output of the large model, the parsing results are made more logical, and the retrieval-enhanced generation reduces the illusion of the large model output.

[0199] 3. By extracting and validating structured data using business-related field groups as the basic granularity, complex tasks are automatically broken down, ensuring the accuracy of the output results.

[0200] The following describes an embodiment of the apparatus described in this application, which can be used to execute the text parsing method described in the above embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the text parsing method described above.

[0201] Figure 18 A block diagram of a text parsing apparatus according to an embodiment of this application is shown. Please refer to... Figure 18As shown, a text parsing apparatus 1800 according to an embodiment of this application includes: a text splitting unit 1810, a text block matching unit 1820, a prompt text construction unit 1830, a prompt text input unit 1840, and an aggregation unit 1850. The text splitting unit 1810 is used to split the text to be parsed into multiple first text blocks; the text block matching unit 1820 is used to determine at least one first text block that matches each field model in at least one field model, wherein the field model includes a category of attribute fields; the prompt text construction unit 1830 is used to construct a first prompt text for each field model in at least one field model, based on the field model and the at least one first text block that matches the field model, wherein the first prompt text is used to instruct the model to generate field values ​​corresponding to the attribute fields in the field model; the prompt text input unit 1840 is used to input the first prompt text corresponding to each field model into a predetermined model, and obtain the parsing result corresponding to the field model based on the output result of the predetermined model; and the aggregation unit 1850 is used to aggregate the parsing results corresponding to each field model to obtain the parsing result of the text to be parsed.

[0202] In some embodiments of this application, based on the foregoing scheme, the field model further includes definition information corresponding to each attribute field in the attribute fields of the category, and the device further includes a splitting and mapping unit; before determining at least one first text block matching the field model, the splitting and mapping unit is configured to: for each first text block, split the first text block into multiple second text blocks with a size smaller than the first text block; map each second text block to a corresponding second text vector; the text block matching unit 1820 is configured to: map the first text information in the definition information to a first text vector; determine at least one second text vector most similar to the first text vector among each second text vector, and take the second text blocks corresponding to the at least one second text vector as second text blocks matching the field model; take the first text blocks to which each second text block matching the field model belongs as first text blocks matching the field model.

[0203] In some embodiments of this application, based on the foregoing scheme, the text block matching unit 1820 is further configured to: determine, among the plurality of first text blocks, a first text block containing the second text information in the definition information, as the first text block matching the field model.

[0204] In some embodiments of this application, based on the foregoing scheme, the splitting and mapping unit is configured to: split the first text block multiple times for each first text block, and when splitting the first text block each time, split the first text block into multiple second text blocks with smaller sizes according to the size corresponding to the current splitting operation, wherein the sizes corresponding to different splitting operations of the first text block are different.

[0205] In some embodiments of this application, based on the foregoing scheme, the prompt text input unit 1840 is configured to: verify the output result of the predetermined model; if the output result passes the verification, then the output result is used as the parsing result corresponding to the field model.

[0206] In some embodiments of this application, based on the foregoing scheme, the first prompt text is used to instruct the model to generate structured data in the target format. The prompt text input unit 1840 is configured to: determine whether the output result of the predetermined model is structured data in the target format; after verifying the output result of the predetermined model, the prompt text construction unit 1830 is further configured to: if the output result of the predetermined model is not structured data in the target format, construct a new first prompt text based on the field model corresponding to the output result and at least one first text block matching the field model, and input the new first prompt text into the predetermined model to instruct the predetermined model to correct the output result to structured data in the target format.

[0207] In some embodiments of this application, based on the foregoing scheme, the device further includes a field value correction unit, and the prompt text input unit 1840 is configured to: for each attribute field, verify whether the field value corresponding to the attribute field in the output result conforms to the definition information corresponding to the attribute field in the field model corresponding to the output result; after verifying the output result of the predetermined model, the field value correction unit is used to: if the field value corresponding to the target attribute field in the output result does not conform to the definition information corresponding to the target attribute field in the field model corresponding to the output result, construct a second prompt text according to the definition information corresponding to the target attribute field and at least one first text block matching the field model, and input the second prompt text into the predetermined model to instruct the predetermined model to correct the field value corresponding to the target attribute field to a new field value that conforms to the definition information corresponding to the target attribute field; if the new field value corresponding to the target attribute field output by the predetermined model conforms to the definition information corresponding to the target attribute field, replace the field value corresponding to the target attribute field in the output result with the new field value corresponding to the target attribute field.

[0208] In some embodiments of this application, based on the foregoing scheme, the text to be parsed includes relevant information corresponding to each of at least one object, and the device further includes an object name acquisition unit and an information return unit; after splitting the text to be parsed into multiple first text blocks, the object name acquisition unit is used to: sequentially traverse the multiple first text blocks, and during the traversal of each first text block, whenever a first text block is traversed, construct a third prompt text based on the first text block to instruct the model to generate an object name, and input the third prompt text into a specified model to obtain the output result of the specified model, and obtain the object name in the first text block based on the output result; after obtaining the parsing result of the text to be parsed, the information return unit is used to: return the parsing result of the text to be parsed and the object names in each of the first text blocks to the front end.

[0209] In some embodiments of this application, based on the foregoing scheme, the object name acquisition unit is configured as follows: if the output result exists in the text to be parsed, then the output result is used as the object name in the first text block; if the output result does not exist in the text to be parsed, then a new third prompt text is constructed according to the corresponding first text block to instruct the model to generate the object name, and the new third prompt text is input into the specified model to obtain a new output result output by the specified model; if the new output result exists in the text to be parsed, then the new output result is used as the object name in the first text block.

[0210] Figure 19 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown.

[0211] It should be noted that, Figure 19 The computer system 1900 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0212] like Figure 19As shown, the computer system 1900 includes a Central Processing Unit (CPU) 1901, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 1902 or programs loaded from storage portion 1908 into Random Access Memory (RAM) 1903, such as performing the methods described in the above embodiments. Various programs and data required for system operation are also stored in RAM 1903. The CPU 1901, ROM 1902, and RAM 1903 are interconnected via bus 1904. An Input / Output (I / O) interface 1905 is also connected to bus 1904.

[0213] The following components are connected to I / O interface 1905: input section 1906 including keyboard, mouse, etc.; output section 1907 including cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; storage section 1908 including hard disk, etc.; and communication section 1909 including network interface card, modem, etc. Communication section 1909 performs communication processing via a network such as the Internet. Drive 1910 is also connected to I / O interface 1905 as needed. Removable media 1911, such as disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1910 as needed so that computer programs read from them can be installed into storage section 1908 as needed.

[0214] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1909, and / or installed from removable medium 1911. When the computer program is executed by central processing unit (CPU) 1901, it performs various functions defined in the system of this application.

[0215] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0216] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0217] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0218] In one aspect, this application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods described in the above embodiments.

[0219] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0220] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of this application.

[0221] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0222] The data collection and processing plan outlined in this application must be implemented in strict accordance with the requirements of relevant national laws and regulations, obtaining the informed consent or separate consent of the data subject (or having a legal basis as stipulated by the relevant national laws and regulations), and conducting subsequent data use and processing within the scope authorized by laws and regulations and the data subject.

[0223] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A text parsing method, characterized in that, The method includes: The text to be parsed is split into multiple first text blocks; For each field model in at least one field model, determine at least one first text block that matches the field model, the field model including a category of attribute fields; For each field model in the at least one field model, a first prompt text is constructed based on the field model and at least one first text block that matches the field model. The first prompt text is used to instruct the model to generate a field value corresponding to an attribute field in the field model. For each field model, the first prompt text is input into a predetermined model, and the parsing result corresponding to the field model is obtained based on the output of the predetermined model. The parsing results corresponding to each of the field models are aggregated to obtain the parsing results for the text to be parsed.

2. The text parsing method according to claim 1, characterized in that, The field model also includes definition information corresponding to each attribute field in the attribute fields of the category. Before determining at least one first text block that matches the field model, the method further includes: For each first text block, the first text block is split into multiple second text blocks with a size smaller than the first text block; Each of the second text blocks is mapped to a corresponding second text vector; The determination of at least one first text block matching the field model includes: Map the first text information in the defined information to a first text vector; Among the various second text vectors, at least one second text vector that is most similar to the first text vector is determined, and the second text block corresponding to the at least one second text vector is taken as the second text block that matches the field model. The first text block to which each second text block that matches the field model belongs is taken as the first text block that matches the field model.

3. The text parsing method according to claim 2, characterized in that, The determination of at least one first text block matching the field model further includes: Among the plurality of first text blocks, the first text block containing the second text information in the definition information is determined as the first text block that matches the field model.

4. The text parsing method according to claim 2, characterized in that, The step of splitting each first text block into multiple second text blocks of smaller size includes: For each first text block, the first text block is split multiple times. When splitting the first text block each time, the first text block is split into multiple second text blocks with smaller sizes according to the size corresponding to the current splitting operation. The sizes corresponding to different splitting operations of the first text block are different.

5. The text parsing method according to claim 2, characterized in that, The step of obtaining the parsing result corresponding to the field model based on the output of the predetermined model includes: The output of the predetermined model is validated; If the output result passes the validation, then the output result will be used as the parsing result corresponding to the field model.

6. The text parsing method according to claim 5, characterized in that, The first prompt text is used to instruct the model to generate structured data in the target format. The verification of the output of the predetermined model includes: Determine whether the output of the predetermined model is structured data in the target format; After verifying the output of the predetermined model, the method further includes: If the output of the predetermined model is not structured data in the target format, a new first prompt text is constructed based on the field model corresponding to the output and at least one first text block matching the field model, and the new first prompt text is input into the predetermined model to instruct the predetermined model to correct the output of the predetermined model to structured data in the target format.

7. The text parsing method according to claim 5, characterized in that, The verification of the output results of the predetermined model includes: For each attribute field, verify whether the field value corresponding to the attribute field in the output result conforms to the definition information of the attribute field in the field model corresponding to the output result; After verifying the output of the predetermined model, the method further includes: If the field value corresponding to the target attribute field in the output result does not conform to the definition information of the target attribute field in the field model corresponding to the output result, then a second prompt text is constructed based on the definition information of the target attribute field and at least one first text block that matches the field model, and the second prompt text is input into the predetermined model to instruct the predetermined model to correct the field value corresponding to the target attribute field to a new field value that conforms to the definition information of the target attribute field. If the new field value output by the predetermined model that corresponds to the target attribute field matches the definition information of the target attribute field, then the field value corresponding to the target attribute field in the output result will be replaced with the new field value corresponding to the target attribute field.

8. The text parsing method according to claim 1, characterized in that, The text to be parsed includes relevant information corresponding to each of at least one object. After splitting the text to be parsed into multiple first text blocks, the method further includes: The plurality of first text blocks are traversed sequentially. During the traversal of each first text block, whenever a first text block is traversed, a third prompt text for instructing the model to generate an object name is constructed based on the first text block, and the third prompt text is input into the specified model to obtain the output result of the specified model. The object name in the first text block is obtained based on the output result. After obtaining the parsing result of the text to be parsed, the method further includes: The parsing results of the text to be parsed and the object names in each of the first text blocks are returned to the front end.

9. The text parsing method according to claim 8, characterized in that, The step of obtaining the object name in the first text block based on the output result includes: If the output result exists in the text to be parsed, then the output result is used as the object name in the first text block; If the output result does not exist in the text to be parsed, a new third prompt text is constructed based on the corresponding first text block to indicate the name of the object generated by the model, and the new third prompt text is input into the specified model to obtain a new output result output by the specified model; If the new output exists in the text to be parsed, then the new output is used as the object name in the first text block.

10. A text parsing device, characterized in that, The device includes: A text splitting unit is used to split the text to be parsed into multiple first text blocks; A text block matching unit is configured to determine at least one first text block that matches each of the at least one field model, the field model including a category of attribute fields; A prompt text construction unit is configured to construct a first prompt text for each field model in the at least one field model, based on the field model and at least one first text block matching the field model, wherein the first prompt text is used to instruct the model to generate a field value corresponding to an attribute field in the field model; The prompt text input unit is used to input the first prompt text corresponding to each field model into a predetermined model, and obtain the parsing result corresponding to the field model based on the output result of the predetermined model; The aggregation unit is used to aggregate the parsing results corresponding to each of the field models to obtain the parsing result of the text to be parsed.

11. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the text parsing method as described in any one of claims 1 to 9.

12. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the text parsing method as described in any one of claims 1 to 9.

13. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium, a processor of a computer device reading the computer instructions from the computer-readable storage medium, and the processor executing the computer instructions to cause the computer device to perform the text parsing method as described in any one of claims 1 to 9.

Images