Composite information entity extraction method and device

Abstract

The invention provides a composite information entity extraction method and device. The method comprises the following steps: acquiring a to-be-processed document; wherein the to-be-processed document comprises a composite information entity composed of a plurality of information entities. The number of characters contained in the composite information entity is greater than a threshold value. And extracting a first type of information entity representing the beginning part of the composite information entity and a second type of information entity representing the ending part of the composite information entity from the to-be-processed document. And determining the composite information entity contained in the to-be-processed document based on the first type of information entity and the second type of information entity, and extracting the determined composite information entity.

Description

Technical Field

[0001] The implementation methods of this specification relate to the field of artificial intelligence. More specifically, the implementation methods of this specification relate to a method and device for extracting composite information entities. Background Art

[0002] With the development of artificial intelligence technology, the amount of data that people need to process has increased dramatically. How to quickly and efficiently extract effective information from open-domain texts has become an important issue facing people. In the process of natural language processing, entity extraction or named entity recognition is the core task of text mining and information extraction.

[0003] The task of entity extraction is mainly to automatically extract entities from text information by modeling text information and classify them into predefined categories. However, for overly complex entity objects, it is often impossible to effectively model or classify them, making it difficult to achieve simple and effective extraction. Summary of the invention

[0004] To overcome the problems existing in the related art, this specification provides the following methods and devices.

[0005] In a first aspect of the embodiments of the present specification, a method for extracting a composite information entity is provided; the method comprises:

[0006] Acquire a document to be processed; wherein the document to be processed contains a composite information entity composed of a plurality of information entities; the number of characters contained in the composite information entity is greater than a threshold;

[0007] Extract a first type of information entity representing the beginning part of the composite information entity and a second type of information entity representing the ending part of the composite information entity from the document to be processed; determine the composite information entity contained in the document to be processed based on the first type of information entity and the second type of information entity, and extract the determined composite information entity.

[0008] In a second aspect of the embodiments of this specification, an information retrieval method is provided; the method comprises:

[0009] Acquire a document to be processed; the document to be processed includes a plurality of composite information entities composed of a plurality of information entities;

[0010] Extracting a first type of information entity representing the beginning of the composite information entity and a second type of information entity representing the ending of the composite information entity from the document to be processed;

[0011] Based on the first type of information entities and the second type of information entities, determining a plurality of composite information entities contained in the document to be processed, and extracting the determined plurality of composite information entities;

[0012] The extracted multiple composite information entities are used as search objects, and based on the acquired search conditions, a target composite information entity that meets the search conditions is searched from the multiple composite information entities.

[0013] In a third aspect of the embodiments of the present specification, a composite information entity extraction device is provided, the device comprising:

[0014] A first acquisition unit is used to acquire a document to be processed; wherein the document to be processed includes a composite information entity composed of a plurality of information entities; and the number of characters included in the composite information entity is greater than a threshold;

[0015] The first extraction unit is used to extract the first type of information entity representing the beginning part of the composite information entity and the second type of information entity representing the ending part of the composite information entity from the document to be processed; based on the first type of information entity and the second type of information entity, determine the composite information entity contained in the document to be processed, and extract the determined composite information entity.

[0016] In a fourth aspect of the embodiments of the present specification, an information retrieval device is provided; the device comprises:

[0017] A second acquisition unit is used to acquire a document to be processed; the document to be processed includes a plurality of composite information entities composed of a plurality of information entities;

[0018] A second extraction unit is used to extract a first type of information entity representing the beginning part of the composite information entity and a second type of information entity representing the ending part of the composite information entity from the document to be processed; based on the first type of information entity and the second type of information entity, determine a plurality of composite information entities contained in the document to be processed, and extract the determined plurality of composite information entities;

[0019] The retrieval unit is used to use the extracted multiple composite information entities as retrieval objects, and based on the acquired retrieval conditions, retrieve a target composite information entity that meets the retrieval conditions from the multiple composite information entities.

[0020] In a fifth aspect of the embodiments of the present specification, a storage medium is provided; a computer program is stored on the storage medium, and the computer program implements the steps of the method described above when executed.

[0021] In a sixth aspect of the embodiments of the present specification, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the above method is implemented when the processor executes the program.

[0022] The above embodiments of this specification have at least the following beneficial effects:

[0023] In the above technical scheme, by identifying the beginning and ending parts of a composite information entity composed of several information entities, the recognition and extraction of composite information entities that are difficult to identify directly are realized, thereby solving the problem that it is difficult to extract extremely long composite information entities efficiently and accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification, and for ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0025] Figure 1 The schematic diagram schematically shows the architecture of a composite information entity extraction method according to an embodiment of the present specification;

[0026] Figure 2 A flowchart of a composite information entity extraction method according to an embodiment of the present specification is schematically shown;

[0027] Figure 3 A schematic diagram schematically shows a composite information entity extraction method according to an embodiment of the present specification;

[0028] Figure 4 A schematic diagram schematically shows a method for extracting information entities according to an embodiment of the present specification;

[0029] Figure 5 A flowchart of an information retrieval method according to an embodiment of the present specification is schematically shown;

[0030] Figure 6 A block diagram schematically shows a composite information entity extraction device according to an embodiment of the present specification;

[0031] Figure 7 A block diagram schematically shows an information retrieval device according to an embodiment of the present specification;

[0032] Figure 8 A hardware structure diagram of a computer device in which an information retrieval method according to an embodiment of the present specification is located is schematically shown.

[0033] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0034] The principles and spirit of this specification will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement this specification, and are not intended to limit the scope of this specification in any way. On the contrary, these embodiments are provided to make this specification more thorough and complete, and to fully convey the scope of this specification to those skilled in the art.

[0035] Those skilled in the art know that the embodiments of this specification can be implemented as a system, device, equipment, method or computer program product. Therefore, this specification can be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0036] With the development of artificial intelligence technology, the amount of data that people need to process has increased dramatically. How to quickly and efficiently extract effective information from open-domain texts has become an important issue facing people. Entity extraction or named entity recognition (NER) is the core task of text mining and information extraction.

[0037] The entity extraction task is mainly to automatically extract entities from text information by modeling text information and classify them into predefined categories. When performing entity extraction tasks in specific fields, the technical solutions of related technologies are usually implemented in a way based on deep learning models. Deep learning models are models trained using training samples.

[0038] However, for some special entities, their structure is too complex to be effectively classified or effectively modeled using conventional methods, and it is difficult to achieve simple and effective extraction using conventional deep learning models.

[0039] For example, a composite information entity consisting of several information entities often has a complex structure and contains a large number of characters, which can reach hundreds or even thousands of words. Conventional entity extraction models can usually only process shorter entities with a length ranging from a few characters to more than a dozen characters, and cannot extract such composite information entities.

[0040] In a related technology, a deep learning model can be trained for the above-mentioned composite information entity to obtain a special model for this type of composite information entity. However, due to the complex structure and long length of this type of composite information entity, the trained deep learning model often has poor performance and cannot accurately and effectively identify it, which cannot meet the needs.

[0041] To overcome the problems existing in the related art, this specification provides the following embodiments.

[0042] The scheme of this specification is described in detail below with reference to the accompanying drawings.

[0043] See also Figure 1 , Figure 1 FIG. 1 is a schematic diagram of the architecture of a composite information entity extraction system provided by an exemplary embodiment. Figure 1 As shown, the system may include a network 10, a server 11, and several electronic devices, such as a mobile phone 12, a mobile phone 13, and a mobile phone 14.

[0044] The server 11 may be a physical server including an independent host, or the server 11 may be a virtual server, a cloud server, etc. hosted by a host cluster. The mobile phones 12-14 are just one type of electronic device that the user may use. In fact, the user may also use electronic devices such as tablet devices, laptop computers, PDAs (Personal Digital Assistants), wearable devices (such as smart glasses, smart watches, etc.), etc., and one or at least one embodiment of this specification does not limit this. The network 10 may include various types of wired or wireless networks.

[0045] In one embodiment, the server 11 can cooperate with the mobile phones 12-14; wherein the mobile phones 12-14 can accept user operations and upload the received commands and files to the server 11 through the network 10, and then the server 11 processes the files based on the solution of this specification. In another embodiment, the mobile phones 12-14 can independently implement the solution of this specification; wherein the mobile phones 12-14 accept user operations and process the received commands and files based on the solution of this specification to realize the extraction of composite information entities.

[0046] See also Figure 2 , Figure 2 is a flowchart of a composite information entity extraction method provided by an exemplary embodiment, the method is applied to a processing device, and the processing device can be Figure 1 The server 11 or mobile phones 12-14 etc. are shown.

[0047] The method may include the following steps:

[0048] Step 202: Acquire a document to be processed; wherein the document to be processed contains a composite information entity composed of a plurality of information entities; and the number of characters contained in the composite information entity is greater than a threshold.

[0049] In the composite information entity extraction method, the document to be processed is a document containing the composite information entity to be extracted.

[0050] The document to be processed should be in a document format, such as an unformatted text format (txt format), a rich text format (rtf format), a portable document format (pdf format), etc. If the original carrier of the document to be processed is not in a document format, it can be pre-processed to convert it into a document to be processed in a document format. This manual does not specifically describe how to convert the original document into a document format.

[0051] The above-mentioned document to be processed may contain one or more composite information entities to be extracted. A composite information entity may usually be composed of several information entities. It is not a simple word or phrase, but a complex text fragment composed of multiple sentences or even paragraphs. In some composite information entities, rich text information such as figures and tables may also be included.

[0052] For the convenience of processing, the number of characters contained can usually be used to determine whether the entity to be extracted is a composite information entity. An entity whose number of characters contained is greater than a threshold is determined as the above-mentioned composite information entity. The above-mentioned threshold can be preset. For example, the threshold can be set to 64 or 128, that is, an entity whose number of characters contained is greater than 64 or 128 is determined as a composite information entity.

[0053] Step 204: Extract a first type of information entity representing the beginning part of the composite information entity and a second type of information entity representing the ending part of the composite information entity from the document to be processed; based on the first type of information entity and the second type of information entity, determine the composite information entity contained in the document to be processed, and extract the determined composite information entity.

[0054] In actual application scenarios, the extraction of composite information entities in the document to be processed is usually to solve certain specific problems. The composite information entities to be extracted are usually complex entities with specific domain semantics.

[0055] In addition to having certain domain semantics, these composite information entities also usually have certain rules in their structure. For example, their beginning or ending parts usually have certain specific common features.

[0056] Therefore, for those composite information entities that are difficult to extract directly from the document to be processed, their structural features can be utilized to identify their beginning and ending parts to achieve the recognition of composite information entities.

[0057] The beginning and ending parts of a compound information entity usually meet the extraction requirements of traditional entity extraction and can be extracted using traditional entity extraction methods, such as using a customized deep learning model.

[0058] Specifically, the beginning part of the composite information entity may be defined as the first type of information entity; and the ending part of the composite information entity may be defined as the second type of information entity.

[0059] The beginning or end of a compound information entity may be defined by the number of characters, for example, the first five characters of the compound information entity may be defined as the beginning of the compound information entity, or the last seven characters of the compound information entity may be defined as the end of the compound information entity, etc. The beginning or end of a compound information entity may also be defined by the number of words or sentences, for example, the first three words of the compound information entity may be defined as the beginning of the compound information entity, or the last sentence of the compound information entity may be defined as the end of the compound information entity. This specification does not limit the specific method for determining the beginning or end of a compound information entity.

[0060] Firstly, the first type of information entity and the second type of information entity are extracted from the document to be processed, that is, the beginning part and the end part of the composite information entity are extracted.

[0061] Once the beginning and ending parts of the composite information entity are determined, the composite information entity can be determined and completely extracted from the document to be processed.

[0062] The above embodiment realizes the recognition and extraction of composite information entities that are difficult to directly recognize by identifying the beginning and ending parts of a composite information entity composed of several information entities, thereby solving the problem that it is difficult to efficiently and accurately extract super-long composite information entities.

[0063] See also Figure 3 , Figure 3 A schematic diagram of a composite information entity extraction method according to an embodiment of the present specification is schematically shown.

[0064] The following combination Figure 3 The overall process of the composite information entity extraction method is explained.

[0065] After obtaining the document to be processed, the beginning and the end of the composite information entity are defined respectively, and the beginning and the end of the composite information entity are extracted. The extraction of the beginning and the end of the composite information entity can be performed simultaneously or sequentially.

[0066] After the beginning and the end of the composite information entity are extracted, the beginning and the end can be paired, and the beginning, the end, and the middle part of the beginning and the end can be merged to obtain the composite information entity to be extracted.

[0067] In an exemplary embodiment shown in this specification, information entity extraction models may be trained in advance for the first category of information entities and the second category of information entities, respectively, to achieve extraction of the first category of information entities and the second category of information entities.

[0068] Documents containing compound information entities can be used as samples, and corresponding sample labels can be added to the beginning and end parts of the compound information entities as inputs of the above-mentioned information entity extraction model for supervised training.

[0069] The above-mentioned information entity extraction model can be a classification model, which classifies and judges the text fragments in the input document that meet the first category information entity or the second category information entity to determine whether they are the first category information entity or the second category information entity.

[0070] Whether the information entity extraction model is sufficiently trained can be judged by its recognition performance indicators.

[0071] Taking the recognition of the first type of information entities as an example, based on the recognition results of the first type of information entities in the sample by the above-mentioned information entity extraction model and the first type of information entities actually corresponding to the sample, the performance indicators of these prediction algorithms for the sample data can be calculated.

[0072] The above performance indicators may include performance indicators used to describe the accuracy of the information entity extraction model in identifying samples. This specification does not limit the specific types of the above performance indicators used to describe the accuracy of the prediction algorithm in predicting data for sample data. For example, performance indicators such as mean square error, root mean square error, accuracy, precision, recall rate, F1 value, etc. may be used, and this specification does not specifically limit this.

[0073] When the performance index reaches the predetermined threshold, it can be considered that the recognition ability of the information entity extraction model has reached the expectation and can be used to identify the documents to be processed.

[0074] In an exemplary embodiment shown in this specification, the above-mentioned information entity extraction model can be extracted using a deep learning model based on the Universal Information Extraction (UIE) framework.

[0075] Since the beginning or ending part of a compound information entity, namely the first type of information entity and the second type of information entity mentioned above, can also be a complex entity composed of multiple words, conventional named entity recognition methods still have difficulty in identifying them.

[0076] Therefore, the UIE-based model can be used to extract the first type of information entities and the second type of information entities.

[0077] UIE can automatically extract specific types of information from text. This process involves converting unstructured or semi-structured text into structured data for further analysis and utilization. UIE can identify and extract important information such as entities, relationships, and events in text. These entities can be names of people, places, organizations, etc., relationships can be connections between people, relationships between companies and products, etc., and events can be meetings, transactions, natural disasters, etc.

[0078] Through word segmentation, named entity recognition, relationship extraction, event extraction, attribute extraction, etc., UIE can extract complex entities with specific structures from documents, such as the first and second types of information entities mentioned above.

[0079] After the first type of information entities and the second type of information entities in the document to be processed are extracted, the composite information entity to be extracted can be obtained based on the extracted first type of information entities and the second type of information entities.

[0080] In an illustrative embodiment shown in this specification, the document to be processed contains multiple composite information entities to be extracted, and the multiple composite information entities may be two or more.

[0081] When the document to be processed contains multiple composite information entities to be extracted, the extracted first-category information entities and second-category information entities will also be correspondingly multiple.

[0082] Therefore, it is necessary to pair the extracted first-category information entities with the second-category information entities in pairs to determine the composite information entity.

[0083] The pairing can be completed based on the positional relationship between the extracted first-category information entities and the second-category information entities. Since multiple information entities are usually independent of each other and do not overlap, in the document to be processed, the first first-category information entity extracted is used as the target first-category information entity in order from front to back, and the second-category information entity corresponding to the target first-category information entity is the first second-category information entity extracted, that is, the first second-category information entity located after the first first-category information entity. Based on the target first-category information entity and the target second-category information entity, the first composite information entity can be determined, and the composite information entity is composed of the target first-category information entity, the target second-category information entity, and the text between the first-category information entity and the second-category information entity.

[0084] Similarly, the remaining first-category information entities are taken as target first-category information entities in turn, and the above steps are executed to determine the remaining composite information entities in the document to be processed.

[0085] In an exemplary embodiment shown in this specification, the result of the information extraction model judging whether the target text segment is a first-category information entity or a second-category information entity is usually not a binary yes or no result, but first obtains a probability, or confidence, representing the probability that the target text segment is a first-category information entity or a second-category information entity. When the confidence exceeds a preset confidence threshold, it is judged to be true, that is, it is determined that the target text is a first-category information entity or a second-category information entity.

[0086] For example, the confidence threshold of the information extraction model is 90%; the information extraction model calculates that the confidence of text segment A is the first type of information entity is 98%, and the confidence of text segment B is the first type of information entity is 60%; then the information extraction model will determine text segment A as the first type of information entity, and will not determine text segment B as the first type of information entity. In addition, the information extraction model believes that the probability that the text segment A it determines is the beginning of the composite information entity is 90%.

[0087] The output of the information extraction model may also include the confidence levels corresponding to the first category information entities and the second category information entities determined by the model.

[0088] Since the extraction of the information extraction model may not be 100% accurate, the confidence level can be used as additional reference information for error correction when problems occur in the extraction.

[0089] For example, the information extraction model may fail to identify the beginning or end of a compound information entity, or may mistakenly identify a text fragment that is not the beginning or end of a compound information entity as the beginning or end of a compound information entity.

[0090] For the case where the information extraction model has no extraction errors, in the document to be extracted, the extracted first-category information entities and second-category information entities should appear alternately.

[0091] Therefore, whether there is an extraction error can be determined by checking whether the first type of information entities and the second type of information entities appear alternately.

[0092] Specifically, whether the second-category information entity is incorrectly identified may be determined by determining whether there is more than one second-category information entity between any two adjacent first-category information entities.

[0093] If it is determined that there are more than one second-category information entities between two adjacent first-category information entities, the confidence levels corresponding to the more than one second-category information entities are compared, and the second-category information entity with the highest confidence level is determined to be the real second-category information entity, and the other second-category information entities are determined to be identification errors and not second-category information entities.

[0094] For the situation where there is more than one first-category information entity between any two adjacent second-category information entities, a similar method may be used for processing.

[0095] For example, see Figure 4 , Figure 4 A schematic diagram of an information entity extraction method according to an implementation of this specification is schematically shown.

[0096] In the document to be processed 40, the first type of information entities extracted by the information extraction model include information entity 411 and information entity 412, and the second type of information entities extracted include information entity 421 and information entity 422, etc.

[0097] There are two second-category information entities, information entity 421 and information entity 422 , between two adjacent first-category information entities, information entity 411 and information entity 412 .

[0098] This does not conform to the rule that the first type of information entity and the second type of information entity appear alternately. Therefore, it is determined that the information extraction model has an error in extraction, and only one of information entity 421 and information entity 422 is a true second type of information entity.

[0099] At this time, the confidences corresponding to information entity 421 and information entity 422 can be compared. For example, when the confidence corresponding to information entity 421 is 94% and the confidence corresponding to information entity 422 is 82%, the confidence corresponding to information entity 421 is higher. Information entity 421 can be determined as a second-category information entity, while information entity 422 can be determined as a recognition error and no longer identified as a second-category information entity.

[0100] After determining that the information entity 421 is the second type of information entity corresponding to the first type of information entity 411, the first information entity 411 can be used as the beginning part of the composite information entity to be extracted, and the second information entity 421 can be used as the ending part of the composite information entity to determine the composite information entity.

[0101] In an exemplary embodiment shown in this specification, the document to be processed is a reply letter from the responding party to the inquiry of the inquiring party; the composite information entity includes the question text that the inquiring party asks the responding party of the reply letter.

[0102] The following is an application scenario of this solution. When the stock exchange finds that the standards are not met (generally manifested as inaccurate information disclosure or incomplete content) during the review of the company's relevant announcements, it will issue an inquiry letter for financial reports, mergers and acquisitions, related transactions, abnormal stock fluctuations, and social hot spots reported by the media, listing the specific terms of the relevant rules in detail, and requiring the company to provide explanations, supplementary disclosures and error corrections within the specified time based on the listed terms and its own behavior, or requiring the company's legal advisors and financial advisors to express targeted opinions.

[0103] The reply letter to the inquiry letter is a document in which the company responds to the questions in the inquiry letter. The reply letter usually contains the various inquiry questions in the inquiry letter and the corresponding reply texts. In addition to directly responding to the inquirer, the company usually publishes the inquiry letter for reference by investors and market participants.

[0104] When an enterprise receives an inquiry letter and needs to respond, it may need to refer to other enterprises' responses to similar questions in the inquiry letter. At this time, it can search for other enterprises' responses to the inquiry letter and search for similar questions.

[0105] To improve search efficiency, the question part of the reply letter to the inquiry letter can be extracted first, and then searched in the question part.

[0106] At this time, the reply letter can be treated as a document to be processed, and the question text that the inquirer asks the replyer of the reply letter can be treated as a composite information entity. The methods in the various embodiments of this specification can be used to extract the composite information entity, that is, the question text that the inquirer asks the replyer of the reply letter, from the reply letter.

[0107] In an exemplary embodiment shown in this specification, after the compound information entity is extracted from the text to be processed, the extracted compound information entity can also be used as a search object to retrieve the required information based on the search conditions specified by the user.

[0108] Taking the above scenario of extracting question text from a reply letter as an example, after extracting the question text from the reply letter, the enterprise can set corresponding search conditions from the extracted question text according to the question text in the inquiry letter received by the enterprise, search for similar questions from the reply letter, and use its reply as reference material for replying to the inquiry letter.

[0109] The method in the above embodiment is described below with reference to a specific example.

[0110] In the above application scenario, the text to be processed may be a reply letter to an inquiry letter.

[0111] For example, part of the response letter could look like this:

[0112] "The company's board of directors has conducted verification and implementation in accordance with the requirements of the inquiry letter, and now responds to the issues raised in the inquiry letter as follows:

[0114] 1. Please respond to the following inquiries:

[0115] (1) The main recipients of the issuer’s equity incentive are employees who have worked in the company for two years or more, and their investment amount is composed of their own funds / self-raised funds and bonuses.

[0116] (2) The issuer will amortize the share-based payment amount over a 24-month period starting from the month of grant.

[0117] (3) The resigned employees involved in the issuer’s equity incentive and share holding schemes have transferred their equity interests to others.

[0118] (4) The issuer determines fair value based on the price-to-earnings ratio corresponding to the highest price in the history of internal transfers and uses this ratio as the basis for recognizing the amount of share-based payments generated from previous grants.

[0119] Please issue:

[0120] (1) Explain whether the method and results of determining the fair value of shares during the implementation of employee equity incentives and the transfer of shares on the employee stock ownership platform are reasonable, whether share-based payments should be recognized, and whether the relevant accounting treatment (especially the phased recognition of share-based payment expenses and the treatment of corresponding shares after employees leave) complies with the requirements of enterprise accounting standards and common practices. A table should also be provided to explain the incentive targets, the number of shares held and the corresponding number of equity incentives, and the calculation process of share-based payment expenses.

[0121] (2) Quantitatively analyze the impact of a high proportion of corrections of prior accounting errors on the exercise price and share-based payment expenses, and disclose risks and provide reminders of major events when necessary.

[0122] (3) Explain whether the capital increase has been paid in full, explain the rationality and compliance of the termination of the proxy holding of equity through the capital increase, and explain whether the proxy holding has been completely cleared in combination with the cash flow, and whether there are other interest arrangements, etc. (4) Explain in detail whether the specific provisions of the "Employee Stock Ownership Management Measures" and other regulations on the service period are clear, explain whether the amortization of the share-based payment amount over 24 months starting from the month of grant is in compliance with the relevant provisions of the "Enterprise Accounting Standards", and simulate and calculate the impact on the main financial indicators.

[0123] Please ask the sponsor institution and the reporting accountant to verify the above matters and express clear opinions.

[0124] From the above example, we can see that in the text fragment of the above reply letter, there is an extremely long question text, which also quotes the question text that the inquirer asked to the responder, because the question text is a composite information entity.

[0125] However, according to conventional entity recognition methods, it is usually impossible to properly identify the composite information entities contained in the above question text.

[0126] This type of composite information entity, such as the question text asked by the inquirer to the respondent quoted in the reply letter in the above example, usually has relatively fixed structure or context information at the beginning and end, and has relatively obvious characteristics. Therefore, in practical applications, training samples can be constructed based on these obvious characteristics to train an information entity extraction model, and the information entity extraction model can be used to extract the question text asked by the inquirer to the respondent quoted in the above question text.

[0127] As in the above example, at least one first-class information entity representing the beginning of the composite information entity can be extracted from the text to be processed; for example, based on the above example, the string corresponding to the first-class information entity can be: "Please reply according to the following inquiry:". In addition, a second-class information entity representing the ending of the composite information entity can also be extracted; for example, based on the above example, the string corresponding to the second-class information entity can be: "Express a clear opinion". Then, based on the extracted first-class information entity and second-class information entity, the complete composite information entity contained in the above text to be processed can be determined, that is, the inquiry question text.

[0128] For example, in practical applications, the information entity consisting of the first type of information entity, the second type of information entity, and the text between the first type of information entity and the second type of information entity can be determined as the composite information entity contained in the document to be processed. Accordingly, the reply text to the inquiry question text usually corresponds to the position of the inquiry question text, such as usually being located after it.

[0129] In actual applications, companies that need to reply to inquiry letters can use the above method to extract the question texts contained in the reply letters of other companies as composite information entities, and search for questions of interest in the extracted question texts. Then, based on the relative position relationship between the question text and the reply text, further search for the reply text corresponding to the question of interest as a reference when replying to the reply letter.

[0130] In an exemplary embodiment of the present specification, an information retrieval method is also provided.

[0131] like Figure 5 As shown, Figure 5 is a flowchart of an information retrieval method provided by an exemplary embodiment, the method is applied to a processing device, and the processing device can be Figure 1 The server 11 or mobile phones 12-14 etc. are shown.

[0132] The method may include the following steps:

[0133] Step 502: Acquire a document to be processed; wherein the document to be processed contains a composite information entity composed of a plurality of information entities; and the number of characters contained in the composite information entity is greater than a threshold.

[0134] In the composite information entity extraction method, the document to be processed is a document containing the composite information entity to be extracted.

[0135] The document to be processed should be in a document format, such as an unformatted text format (txt format), a rich text format (rtf format), a portable document format (pdf format), etc. If the original carrier of the document to be processed is not in a document format, it can be pre-processed to convert it into a document to be processed in a document format. This manual does not specifically describe how to convert the original document into a document format.

[0136] The above-mentioned document to be processed may contain one or more composite information entities to be extracted. A composite information entity may usually be composed of several information entities. It is not a simple word or phrase, but a complex text fragment composed of multiple sentences or even paragraphs. In some composite information entities, rich text information such as figures and tables may also be included.

[0137] For the convenience of processing, the number of characters contained can usually be used to determine whether the entity to be extracted is a composite information entity. An entity whose number of characters contained is greater than a threshold is determined as the above-mentioned composite information entity. The above-mentioned threshold can be preset. For example, the threshold can be set to 64 or 128, that is, an entity whose number of characters contained is greater than 64 or 128 is determined as a composite information entity.

[0138] Step 504: extracting a first type of information entity representing the beginning of the composite information entity and a second type of information entity representing the ending of the composite information entity from the document to be processed; based on the first type of information entity and the second type of information entity, determining a plurality of composite information entities contained in the document to be processed, and extracting the determined plurality of composite information entities;

[0139] In actual application scenarios, the extraction of composite information entities in the document to be processed is usually to solve certain specific problems. The composite information entities to be extracted are usually complex entities with specific domain semantics.

[0140] In addition to having certain domain semantics, these composite information entities also usually have certain rules in their structure. For example, their beginning or ending parts usually have certain specific common features.

[0141] Therefore, for those composite information entities that are difficult to extract directly from the document to be processed, their structural features can be utilized to identify their beginning and ending parts to achieve the recognition of composite information entities.

[0142] The beginning and ending parts of a compound information entity usually meet the extraction requirements of traditional entity extraction and can be extracted using traditional entity extraction methods, for example, using a customized deep learning model.

[0143] Specifically, the beginning part of the composite information entity may be defined as the first type of information entity; and the ending part of the composite information entity may be defined as the second type of information entity.

[0144] The beginning or end of a compound information entity may be defined by the number of characters, for example, the first five characters of the compound information entity may be defined as the beginning of the compound information entity, or the last seven characters of the compound information entity may be defined as the end of the compound information entity, etc. The beginning or end of a compound information entity may also be defined by the number of words or sentences, for example, the first three words of the compound information entity may be defined as the beginning of the compound information entity, or the last sentence of the compound information entity may be defined as the end of the compound information entity. This specification does not limit the specific method for determining the beginning or end of a compound information entity.

[0145] Firstly, the first type of information entity and the second type of information entity are extracted from the document to be processed, that is, the beginning part and the end part of the composite information entity are extracted.

[0146] Once the beginning and ending parts of the composite information entity are determined, the composite information entity can be determined and completely extracted from the document to be processed.

[0147] Step 506: taking the extracted multiple composite information entities as search objects, and based on the acquired search conditions, searching for target composite information entities that meet the search conditions from the multiple composite information entities.

[0148] After the composite information entities in the document to be processed are extracted, the extracted composite information entities can be used as search objects, and based on the search conditions specified by the user, the required target composite information entities can be searched from the extracted composite information entities.

[0149] Through the above method, specific semantic parts in the target document can be extracted and screened as retrieval objects in the target document. Compared with searching directly in the target document, a large amount of text that is not the retrieval object is excluded, thereby improving the search accuracy and efficiency.

[0150] Still taking the application scenario of a company replying to an inquiry letter from a stock exchange as an example, when a company receives an inquiry letter and needs to respond, it may need to refer to other companies' responses to similar questions in the inquiry letter. At this time, you can search for other companies' reply letters to the inquiry letter and search for similar questions.

[0151] Since the reply in the reply letter is a reply to the question text, the keywords in the question text often appear multiple times in the reply text, and these keywords may be mentioned in the replies to other questions. If you search the reply letter directly, you may get a large number of search results in the reply content, which requires further screening, which is inefficient. Therefore, it is necessary to extract the question text part in the reply letter in advance for efficient search.

[0152] In an exemplary embodiment shown in this specification, the document to be processed may also include document fragments corresponding to the positions of each composite information entity;

[0153] Retrieving the target compound information entity can also be a means of retrieving the document fragment corresponding to the target compound information entity. The target compound information entity contains keywords suitable for retrieval, and the corresponding document fragment contains more information required for retrieval.

[0154] The document fragment corresponding to the compound information entity may correspond to the position of the compound information entity, for example, may be connected before or after the compound information entity.

[0155] Still taking the application scenario of an enterprise replying to the inquiry letter of the stock exchange as an example, the document to be processed is the reply letter in which the responding party responds to the inquiry of the inquirer, the composite information entity is the question text that the inquirer asks the responding party of the reply letter, and the document fragment corresponding to the position of the composite information entity is the reply content of the responding party in response to the question text.

[0156] As mentioned above, companies that receive inquiry letters need to refer to the reply texts of other companies to similar questions in the inquiry letters. In order to retrieve the corresponding reply texts, a search method for the question text is adopted.

[0157] When similar questions in reply letters from other companies are retrieved, the reply texts corresponding to the similar questions will be obtained.

[0158] In the exemplary embodiment of this specification, a composite information entity extraction device is also provided. Figure 6 , Figure 6 It is a block diagram of a complex information entity extraction device according to an embodiment of this specification.

[0159] The device may include the following components:

[0160] The first acquisition unit 610 is used to acquire a document to be processed; wherein the document to be processed includes a composite information entity composed of a plurality of information entities; and the number of characters included in the composite information entity is greater than a threshold;

[0161] The first extraction unit 620 is used to extract the first type of information entity representing the beginning part of the composite information entity and the second type of information entity representing the ending part of the composite information entity from the document to be processed; based on the first type of information entity and the second type of information entity, determine the composite information entity contained in the document to be processed, and extract the determined composite information entity.

[0162] Optionally, the first extraction unit 620 is specifically configured to:

[0163] Using an information entity extraction model, a first type of information entity representing the beginning part of the composite information entity and a second type of information entity representing the ending part of the composite information entity are extracted from the document to be processed; wherein the information entity extraction model is an information entity extraction model obtained by supervised training based on several document samples containing composite information entities; sample labels are added to the beginning and ending parts of the composite information entity contained in the document samples, respectively.

[0164] Optionally, the information entity extraction model is an information entity extraction model using a UIE algorithm.

[0165] Optionally, the document to be processed contains multiple composite information entities;

[0166] The first extraction unit 620 is specifically configured to:

[0167] Take each extracted first-category information entity as the target first-category information entity in turn and perform the following steps:

[0168] Taking the second-category information entity following the target first-category information entity as the target second-category information entity corresponding to the target first-category information entity;

[0169] An information entity consisting of the target first-category information entity, the target second-category information entity, and the text between the first-category information entity and the second-category information entity is determined as the composite information entity contained in the document to be processed.

[0170] Optionally, the output result of the information entity extraction model also includes confidences corresponding to the first type of information entity and the second type of information entity; wherein the confidence corresponding to the first type of information entity indicates the probability that the first type of information entity is the beginning part; and the confidence corresponding to the second type of information entity indicates the probability that the second type of information entity is the ending part;

[0171] The device also includes:

[0172] A first correction unit 630, configured to determine whether there is more than one second-category information entity between any two adjacent first-category information entities;

[0173] If there is more than one second-category information entity, retain the second-category information entity with the highest confidence among the more than one second-category information entities;

[0174] determining whether there is more than one first-category information entity between any two adjacent second-category information entities;

[0175] If there are more than one first-category information entities, the first-category information entity with the highest confidence among the more than one first-category information entities is retained.

[0176] Optionally, the document to be processed is a reply letter from the responding party to the inquiry of the inquiring party;

[0177] The composite information entity includes the text of the question that the inquirer asks the responder of the reply letter.

[0178] Optionally, the device further comprises:

[0179] The first retrieval unit 640 is configured to use the extracted multiple composite information entities as retrieval objects, and based on the acquired retrieval conditions, retrieve a target composite information entity that meets the retrieval conditions from the multiple composite information entities.

[0180] In the exemplary embodiment of this specification, an information retrieval device is also provided. Figure 7 , Figure 7 This is a block diagram of an information search device according to an embodiment of this specification.

[0181] The device may include the following components:

[0182] The second acquisition unit 710 is used to acquire a document to be processed; the document to be processed includes a plurality of composite information entities composed of a plurality of information entities;

[0183] The second extraction unit 720 is used to extract the first type of information entity representing the beginning part of the composite information entity and the second type of information entity representing the ending part of the composite information entity from the document to be processed; based on the first type of information entity and the second type of information entity, determine multiple composite information entities contained in the document to be processed, and extract the determined multiple composite information entities;

[0184] The second retrieval unit 730 is used to use the extracted multiple composite information entities as retrieval objects, and based on the acquired retrieval conditions, retrieve target composite information entities that meet the retrieval conditions from the multiple composite information entities. Optionally, the cache space corresponding to the cache area includes a first cache space for caching the global context information, and a second cache space for caching the local context information corresponding to the user input text;

[0185] Optionally, the second extraction unit 720 is specifically configured to:

[0186] Using an information entity extraction model, a first type of information entity representing the beginning part of the composite information entity and a second type of information entity representing the ending part of the composite information entity are extracted from the document to be processed; wherein the information entity extraction model is an information entity extraction model obtained by supervised training based on several document samples containing composite information entities; sample labels are added to the beginning and ending parts of the composite information entity contained in the document samples, respectively.

[0187] Optionally, the information entity extraction model is an information entity extraction model using a UIE algorithm.

[0188] Optionally, the document to be processed contains multiple composite information entities;

[0189] The second extraction unit 720 is specifically configured to:

[0190] Take each extracted first-category information entity as the target first-category information entity in turn and perform the following steps:

[0191] Taking the second-category information entity following the target first-category information entity as the target second-category information entity corresponding to the target first-category information entity;

[0192] An information entity consisting of the target first-category information entity, the target second-category information entity, and the text between the first-category information entity and the second-category information entity is determined as the composite information entity contained in the document to be processed.

[0193] Optionally, the output result of the information entity extraction model also includes confidences corresponding to the first type of information entity and the second type of information entity; wherein the confidence corresponding to the first type of information entity indicates the probability that the first type of information entity is the beginning part; and the confidence corresponding to the second type of information entity indicates the probability that the second type of information entity is the ending part;

[0194] The device also includes:

[0195] A second correction unit 740, configured to determine whether there is more than one second-category information entity between any two adjacent first-category information entities;

[0196] If there is more than one second-category information entity, retain the second-category information entity with the highest confidence among the more than one second-category information entities;

[0197] determining whether there is more than one first-category information entity between any two adjacent second-category information entities;

[0198] If there are more than one first-category information entities, the first-category information entity with the highest confidence among the more than one first-category information entities is retained.

[0199] Optionally, the device further comprises:

[0200] The searching unit 750 is configured to search the document segment corresponding to the position of the target composite information entity in the document to be processed.

[0201] Optionally, the to-be-processed document includes a reply letter from the responding party in response to the inquiry of the inquiring party;

[0202] The composite information entity includes the text of the question that the inquiring party asks the responding party of the reply letter;

[0203] The document segment corresponding to the position of the composite information entity includes the reply content of the replying party in response to the question text.

[0204] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0205] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. The device embodiments described above are only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on at least one network unit. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this specification. Ordinary technicians in this field can understand and implement it without paying creative work.

[0206] In an exemplary embodiment of the present specification, an embodiment of a device and a terminal to which the device is applied is also provided.

[0207] The embodiments of the device of this specification can be applied to computer devices, such as servers or terminal devices. The device embodiments can be implemented by software, hardware, or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor in which it is located reading the corresponding computer program instructions in the non-volatile memory into the memory and running them. From the hardware level, if Figure 8 As shown, Figure 8 A hardware structure diagram of a computer device 80 in which a device of an embodiment of this specification is located, except Figure 8 In addition to the processor 810, memory 830, network interface 820, and non-volatile memory 840 shown, the server or electronic device where the device is located in the embodiment may also include other hardware according to the actual function of the computer device, which will not be described in detail.

[0208] In the exemplary embodiments of this specification, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of this specification is stored. In some possible embodiments, various aspects of this specification can also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of this specification described in the above "Exemplary Method" section of this specification.

[0209] According to the embodiment of the present specification, the program product for implementing the above method can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present specification is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0210] The program product may adopt any combination of one or at least one readable medium. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or at least one wire, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0211] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0212] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0213] Program code for performing the operations of the present specification may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0214] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0215] Although this specification contains many specific implementation details, these should not be interpreted as limiting the scope of any invention or the scope of the claimed protection, but are mainly used to describe the features of the specific embodiments of specific inventions. Certain features described in at least one embodiment in this specification may also be implemented in combination in a single embodiment. On the other hand, the various features described in a single embodiment may also be implemented separately in at least one embodiment or in any suitable sub-combination. In addition, although the features may work in certain combinations as described above and even initially claim protection, one or at least one feature from the claimed combination may be removed from the combination in some cases, and the claimed combination may point to a sub-combination or a variation of a sub-combination.

[0216] Similarly, although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring these operations to be performed in the particular order shown or to be performed sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system units and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged into at least one software product.

[0217] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the particular order or sequential order shown to achieve the desired results. In some implementations, multitasking and parallel processing may be advantageous.

[0218] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

Claims

1. A composite information entity extraction method, comprising: Acquire a document to be processed; wherein the document to be processed contains a composite information entity composed of a plurality of information entities; the number of characters contained in the composite information entity is greater than a threshold; Extracting a first type of information entity representing the beginning of the composite information entity and a second type of information entity representing the ending of the composite information entity from the document to be processed; Based on the first-category information entity and the second-category information entity, the composite information entity included in the to-be-processed document is determined, and the determined composite information entity is extracted.

2. The method according to claim 1, wherein: The step of extracting the first type of information entity representing the beginning of the composite information entity and the second type of information entity representing the ending of the composite information entity from the document to be processed comprises: Using an information entity extraction model, a first type of information entity representing the beginning part of the composite information entity and a second type of information entity representing the ending part of the composite information entity are extracted from the document to be processed; wherein the information entity extraction model is an information entity extraction model obtained by supervised training based on several document samples containing composite information entities; sample labels are added to the beginning and ending parts of the composite information entity contained in the document samples, respectively.

3. The method according to claim 2, wherein: The information entity extraction model is an information entity extraction model that adopts the UIE algorithm.

4. The method according to claim 1, wherein: The document to be processed includes multiple composite information entities; Determining the composite information entity contained in the to-be-processed document based on the first-category information entity and the second-category information entity includes: Take each extracted first-category information entity as the target first-category information entity in turn and perform the following steps: Taking the second-category information entity following the target first-category information entity as the target second-category information entity corresponding to the target first-category information entity; An information entity consisting of the target first-category information entity, the target second-category information entity, and the text between the first-category information entity and the second-category information entity is determined as the composite information entity contained in the document to be processed.

5. The method according to claim 3, wherein: The output result of the information entity extraction model also includes confidences corresponding to the first type of information entity and the second type of information entity; wherein the confidence corresponding to the first type of information entity indicates the probability that the first type of information entity is the beginning part; and the confidence corresponding to the second type of information entity indicates the probability that the second type of information entity is the ending part; Before determining the composite information entity included in the to-be-processed document based on the first-category information entity and the second-category information entity, the method further includes: Determine whether there is more than one second-category information entity between any two adjacent first-category information entities; If there is more than one second-category information entity, retain the second-category information entity with the highest confidence among the more than one second-category information entities; determining whether there is more than one first-category information entity between any two adjacent second-category information entities; If there are more than one first-category information entities, the first-category information entity with the highest confidence among the more than one first-category information entities is retained.

6. The method according to claim 1, wherein: The document to be processed is a reply letter from the responding party to the inquiry of the inquiring party; The composite information entity includes the text of the question that the inquirer asks the responder of the reply letter.

7. The method according to claim 1, wherein: The document to be processed includes multiple composite information entities; The method further comprises: The extracted multiple composite information entities are taken as search objects, and based on the acquired search conditions, a target composite information entity that meets the search conditions is searched from the multiple composite information entities.

8. An information retrieval method comprising: Get the documents to be processed; The document to be processed contains a plurality of composite information entities composed of a plurality of information entities; the number of characters contained in the composite information entity is greater than a threshold; Extracting a first type of information entity representing the beginning of the composite information entity and a second type of information entity representing the ending of the composite information entity from the document to be processed; determining multiple composite information entities contained in the document to be processed based on the first type of information entity and the second type of information entity, and extracting the determined multiple composite information entities; The extracted multiple composite information entities are used as search objects, and based on the acquired search conditions, a target composite information entity that meets the search conditions is searched from the multiple composite information entities.

9. The method according to claim 8, wherein: The document to be processed also includes document fragments corresponding to the positions of the composite information entities respectively; The method further comprises: The document segment corresponding to the position of the target composite information entity is searched in the document to be processed.

10. The method according to claim 9, wherein: The documents to be processed include a reply letter from the responding party in response to the inquiry from the inquiring party; The composite information entity includes the text of the question that the inquiring party asks the responding party of the reply letter; The document fragment corresponding to the position of the composite information entity includes the reply content of the replying party in response to the question text; The step of searching the document to be processed for a document segment corresponding to the position of the target composite information entity includes: The reply content corresponding to the target composite information entity position is searched in the document to be processed.

11. A composite information entity extraction device, the device comprising: A first acquisition unit is used to acquire a document to be processed; wherein the document to be processed includes a composite information entity composed of a plurality of information entities; and the number of characters included in the composite information entity is greater than a threshold; The first extraction unit is used to extract the first type of information entity representing the beginning part of the composite information entity and the second type of information entity representing the ending part of the composite information entity from the document to be processed; based on the first type of information entity and the second type of information entity, determine the composite information entity contained in the document to be processed, and extract the determined composite information entity.

12. An information retrieval device, comprising: A second acquisition unit, used to acquire a document to be processed; The document to be processed contains a plurality of composite information entities composed of several information entities; A second extraction unit is used to extract a first type of information entity representing the beginning part of the composite information entity and a second type of information entity representing the ending part of the composite information entity from the document to be processed; based on the first type of information entity and the second type of information entity, determine a plurality of composite information entities contained in the document to be processed, and extract the determined plurality of composite information entities; The retrieval unit is used to use the extracted multiple composite information entities as retrieval objects, and based on the acquired retrieval conditions, retrieve a target composite information entity that meets the retrieval conditions from the multiple composite information entities.

13. A storage medium having a computer program stored thereon, which, when executed, implements the steps of the method according to any one of claims 1 to 10.

14. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the following method is implemented: wherein, when the processor executes the program, the method as claimed in any one of claims 1 to 10 is implemented.

Images