Document description information generation method, electronic equipment, chip system and storage medium

Abstract

The invention discloses a document description information generation method, electronic equipment, a chip system and a storage medium, and the method comprises the steps: inputting a to-be-processed document into a text extraction model for processing, and obtaining document description information of the to-be-processed document. Performing text duplication detection on the generated document description information, and if the document description information has duplication texts and the number of times of re-acquiring the document description information of the to-be-processed document by using the text extraction model is not greater than a set number-of-times threshold value, adjusting model parameters of the text extraction model according to the number of duplication times of the duplication texts, and processing the to-be-processed document input parameter adjusted text extraction model to obtain new document description information. Through the method, text repetition detection can be automatically performed on the document description information, and the document description information is regenerated when repetition exists, so that repeated texts in the repeated document description information are reduced, and the generation effect of the document description information is improved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a method for generating document description information, an electronic device, a chip system and a storage medium. Background Art

[0002] Summary generation is of great significance in the information age. It can help readers quickly understand the core content of a document and can also provide support for search engine optimization. At present, summary generation methods are prone to repeated text problems during the generation process, that is, the model is prone to repeated phrases or sentences when generating summaries, which will make the generated text content appear unnatural or lengthy, and ultimately lead to poor semantic retrieval and preview effects. Therefore, reducing repeated text in document summaries and effectively improving the effect of summary generation is a very important task. Summary of the invention

[0003] The embodiments of the present application provide a document description information generation method, an electronic device, a chip system and a storage medium, which can perform text duplication detection on the document description information of a document and regenerate the document description information when there is duplication, so as to reduce or avoid the existence of repeated text in the document description information, thereby improving the generation effect of the document description information.

[0004] In the first aspect, the embodiment of the present application provides a method for generating document description information, which can be applied to electronic devices, the method comprising: inputting a document to be processed into a text extraction model for processing, obtaining first document description information of the document to be processed; performing text duplication detection on the first document description information; in response to detecting duplicate text in the first document description information, and the number of retries is less than or equal to the number threshold, adjusting the model parameters of the text extraction model; wherein the number of retries refers to the number of times the document description information of the document to be processed is re-acquired using the text extraction model, and the adjusted model parameters of the text extraction model are associated with the generation probability of duplicate text; inputting the document to be processed into the text extraction model after the model parameters are adjusted for processing, obtaining second document description information of the document to be processed; determining the target document description information of the document to be processed according to the second document description information. It can be seen that in this technical solution, the electronic device can perform text duplication detection on the document description information generated each time, and adjust the parameters of the text generation model in the case of duplication, so as to reduce or avoid the existence of duplicate text in the document description information, thereby improving the generation effect of the document description information.

[0005] In combination with the first aspect, in a possible manner, the method further includes: displaying the document content of the initial document on the document display interface, the initial document being included in the document database; in response to the detected editing operation, editing the document content to obtain the document to be processed; in response to the triggering of the generation of the document description information of the document to be processed, executing the step of inputting the document to be processed into the text extraction model for processing to obtain the first document description information of the document to be processed. It can be seen that for a document that already exists in the document database, but is edited by the user in a subsequent process, resulting in a change in the document content, the present application can regenerate the document description information of the changed document content to improve the generation effect of the document description information.

[0006] In combination with the first aspect, in a possible manner, the method further includes: in response to a new document being added to the document database, executing a step of inputting the document to be processed into a text extraction model for processing to obtain first document description information of the document to be processed; wherein the document to be processed is a new document added to the document database. It can be seen that for the new document added to the document database, the present application can detect and generate corresponding document description information, avoid omission of documents, make the document data in the document database more complete, and improve the user search experience.

[0007] In conjunction with the first aspect, in one possible manner, the above-mentioned performing text duplication detection on the first document description information includes: using each detection window of a plurality of detection windows to perform text duplication detection on the first document description information respectively, and the plurality of detection windows have different window sizes. It can be seen that in the embodiment of the present application, using detection windows of different window sizes to perform text duplication detection on the document description information has a higher accuracy rate.

[0008] In combination with the first aspect, in a possible manner, the above-mentioned use of each detection window in the multiple detection windows to perform text duplication detection on the first document description information separately includes: in response to the detected text not being included in the text set, adding the detected text to the text set; the detected text is any text under the target detection window during the process of sliding detection of the first document description information using the target detection window; in response to the detected text being included in the text set, determining that there is duplicate text in the first document description information, and the duplicate text includes the detected text. It can be seen that when performing text duplication detection, the texts are detected one by one, which can improve the accuracy of duplication detection.

[0009] In conjunction with the first aspect, in a possible manner, the above-mentioned adjustment of the model parameters of the text extraction model includes: adjusting the model parameters of the text extraction model according to the number of repetitions of the repeated text, where the number of repetitions of the repeated text is obtained in the process of performing text repetition detection on the first document description information. It can be seen that determining the number of repetitions of the repeated text is very important for adjusting the model parameters.

[0010] In combination with the first aspect, in a possible manner, the above-mentioned adjustment of the model parameters of the text extraction model according to the number of repetitions of the repeated text includes: determining the adjustment data of the repeated text according to the number of repetitions of the repeated text and the constraint function; adjusting the original parameters of the repeated text in the text extraction model according to the repeated text adjustment data, and the adjustment data of the repeated text is used to reduce the generation probability of the repeated text. It can be seen that the model parameters can be determined by the constraint function and the number of repetitions of the repeated text. For repeated text with a large number of repetitions, the generation probability can be reduced by the constraint function, so as to reduce the repeated text in the document description information.

[0011] In conjunction with the first aspect, in a possible manner, the method further includes: in response to detecting duplicate text in the first document description information and the number of retries being greater than a threshold number, generating a prompt message; and displaying the prompt message on the document display interface, the prompt message being used to indicate that the first document description information generated by the text extraction model does not meet the document description information condition. It can be seen that when the generated document description information does not meet the document description information condition, a prompt message will be displayed on the interface to facilitate the user to use other means to generate the document description information of the document to avoid document omission.

[0012] In combination with the first aspect, in a possible manner, the method further includes: performing compliance detection on the target document description information of the to-be-processed document; in response to the target document description information passing the compliance detection, generating a vector representation of the target document description information; and storing the target document description information and the vector representation of the target document description information in a document database. It can be seen that storing the document description information and the corresponding vector representation of the document in the document database can facilitate users to quickly obtain matching content when searching.

[0013] In conjunction with the first aspect, in a possible manner, the method further includes: displaying a search interface, the search interface displaying search information; in response to detecting a search trigger operation, determining a vector representation that matches the vector representation of the search information from a document database; searching for document description information corresponding to the vector representation that matches the vector representation of the search information, and displaying the corresponding document description information on the search interface. In this technical solution, the document description information that matches the search information can be quickly determined from the document database based on the semantic information of the search information, thereby improving the user experience.

[0014] In a second aspect, the present application provides an electronic device, which includes: one or more processors, a display screen and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute: inputting a document to be processed into a text extraction model for processing to obtain first document description information of the document to be processed; performing text duplication detection on the first document description information; in response to detecting repeated text in the first document description information, and the number of retries is less than or equal to a number threshold, adjusting the model parameters of the text extraction model; wherein the number of retries refers to the number of times the document description information of the document to be processed is re-acquired using the text extraction model, and the adjusted model parameters of the text extraction model are associated with the generation probability of repeated text; inputting the document to be processed into the text extraction model after the model parameters are adjusted for processing to obtain second document description information of the document to be processed; and determining the target document description information of the document to be processed based on the second document description information.

[0015] In combination with the second aspect, in one possible manner, the one or more processors call the computer instructions to cause the electronic device to execute: displaying the document content of an initial document on a document display interface, the initial document being included in a document database; editing the document content in response to a detected editing operation to obtain a document to be processed; and in response to a trigger for generating document description information of the document to be processed, inputting the document to be processed into a text extraction model for processing to obtain the first document description information of the document to be processed.

[0016] In combination with the second aspect, in one possible manner, the one or more processors call the computer instructions to cause the electronic device to execute: in response to a new document being added to a document database, execute the step of inputting the document to be processed into a text extraction model for processing to obtain first document description information of the document to be processed; wherein the document to be processed is a new document added to the document database.

[0017] In combination with the second aspect, in one possible manner, the one or more processors call the computer instructions to cause the electronic device to perform text duplication detection on the first document description information, including: using each detection window of a plurality of detection windows to perform text duplication detection on the first document description information respectively, and the window sizes of the plurality of detection windows are different.

[0018] In combination with the second aspect, in one possible manner, the one or more processors call the computer instructions to cause the electronic device to perform text duplication detection on the first document description information using each detection window in multiple detection windows, including: in response to the detected text not being included in the text collection, adding the detected text to the text collection; the detected text is any text under the target window during the process of sliding detection of the first document description information using the target detection window; in response to the detected text being included in the text collection, determining that duplicate text exists in the first document description information, and the duplicate text includes the detected text.

[0019] In combination with the second aspect, in one possible manner, the one or more processors call the computer instructions to cause the electronic device to execute adjustment of the model parameters of the text extraction model, including: adjusting the model parameters of the text extraction model according to the number of repetitions of the repeated text, and the number of repetitions of the repeated text is obtained during the process of performing text repetition detection on the first document description information.

[0020] In combination with the second aspect, in one possible manner, the one or more processors call the computer instructions to enable the electronic device to adjust the model parameters of the text extraction model according to the number of repetitions of the repeated text, including: determining the adjustment data of the repeated text according to the number of repetitions of the repeated text and the constraint function; adjusting the original parameters of the repeated text in the text extraction model according to the repeated text adjustment data, and the repeated text adjustment data is used to lower the generation probability of the repeated text.

[0021] In combination with the second aspect, in one possible manner, the one or more processors call the computer instructions to cause the electronic device to execute: in response to detecting repeated text in the first document description information and the number of retries is greater than a threshold number, generating a prompt message; and displaying the prompt message on the document display interface, the prompt message being used to indicate that the first document description information generated by the text extraction model does not meet the document description information conditions.

[0022] In combination with the second aspect, in one possible manner, the one or more processors call the computer instructions to cause the electronic device to execute: performing a compliance check on the target document description information of the document to be processed; in response to the target document description information passing the compliance check, generating a vector representation of the target document description information; and storing the target document description information and the vector representation of the target document description information in a document database.

[0023] In combination with the second aspect, in one possible manner, the one or more processors call the computer instructions to cause the electronic device to execute: displaying a search interface, the search interface displaying search information; in response to detecting a search trigger operation, determining from a document database a vector representation that matches the vector representation of the search information; searching for document description information corresponding to the vector representation that matches the vector representation of the search information, and displaying the corresponding document description information on the search interface.

[0024] In a third aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the method described in the first aspect above.

[0025] In a fourth aspect, the present application provides a chip system, which includes one or more processors, a memory and an interface circuit. The memory, the interface circuit and one or more processors are interconnected through lines. Program instructions are stored in the memory. When the program instructions are executed by the processor, the chip system implements the method described in the first aspect above.

[0026] In a fifth aspect, the present application provides a computer program product comprising instructions, which, when executed on an electronic device, enables the electronic device to execute the method described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1A-Figure 1D A schematic diagram of a document editing interface provided in an embodiment of the present application;

[0028] Figure 2A and Figure 2B A schematic diagram of a new interface for a direct document provided in an embodiment of the present application;

[0029] Figure 3A and Figure 3B A schematic diagram of a document search interface provided in an embodiment of the present application;

[0030] Figure 4 A flowchart of a method for generating document description information provided in an embodiment of the present application;

[0031] Figure 5A and Figure 5B A schematic diagram of a display interface for prompt information provided in an embodiment of the present application;

[0032] Figure 6 A flowchart of another method for generating document description information provided in an embodiment of the present application;

[0033] Figure 7 A flowchart of a method for generating document description information based on a software module provided in an embodiment of the present application;

[0034] Figure 8 A schematic diagram of interaction between software modules provided in an embodiment of the present application;

[0035] Fig. 9 A schematic diagram of a software layered architecture provided for an embodiment of the present application;

[0036] Fig.10 A schematic diagram of the software structure of an electronic device provided in an embodiment of the present application;

[0037] Fig.11 A schematic diagram of the hardware results of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0039] The terms "first", "second", "third", etc. in the embodiments of the present application are distinguished from different objects, rather than being used to describe a specific order. In addition, the terms "include" and "have" and any variation thereof are intended to cover non-exclusive inclusions. For example, a series of steps or units are included, or alternatively, steps or units not listed are also included, or other steps or units inherent to these processes, methods, products or devices are optionally included. The terms "one embodiment" or "some embodiments", etc. mean that the specific features, structures or characteristics described in conjunction with the embodiment are included in one or more embodiments of the present application. Thus, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in the differences in the embodiments of the present application are not necessarily all with reference to the same embodiment, but mean "one or more but not all embodiments", unless otherwise particularly emphasized.

[0040] In the embodiments of the present application, words such as "exemplary", "for example", or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary", "for example", or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example", or "for example" is intended to present related concepts in a specific way.

[0041] In addition, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can mean: a, or b, or c, or a and b, or a and c, or b and c, or a, b and c, where a, b, c can be single or multiple.

[0042] Before introducing the document description information generation method provided by the present application, the relevant terms involved in this embodiment are introduced:

[0043] Large Language Model (LLM): It is an artificial intelligence model based on deep learning technology, which is designed to understand and generate human language. These models are trained on large amounts of text data and can perform a variety of natural language processing tasks, such as text classification, question answering, dialogue system construction, machine translation, etc. Large language models usually adopt the Transformer architecture and are trained through pre-training and fine-tuning. In the pre-training stage, the model learns the grammar, semantics and context information of the language on massive unlabeled text data; in the fine-tuning stage, it optimizes for specific tasks to improve the performance of the model in downstream tasks. The parameter scale of these models usually reaches billions or even hundreds of billions, such as GPT-3 and GPT-4. Large language models are widely used, including but not limited to natural language generation, dialogue systems, machine translation, sentiment analysis, knowledge extraction and reasoning, etc. In addition, they are also used in some emerging fields, such as speech recognition, computer vision, and multimodal learning. Large language models are one of the important ways to artificial intelligence, showing powerful natural language understanding and generation capabilities, and providing initial possibilities for the realization of general artificial intelligence.

[0044] Artificial Intelligence (AI) is a field of science and technology that studies how to enable computers to simulate and perform human intelligence tasks. It is committed to developing systems that can perceive, understand, learn, reason, and other intelligent behaviors, and tries to understand the essence of intelligence and produce intelligent machines that can respond in a similar way to human intelligence.

[0045] Vector representation (Embedding): For example, in deep learning and natural language processing, "embedding" refers to the process of converting data (such as text, images, etc.) into a low-dimensional dense vector that can capture the semantic information of the data. In these fields, embedding is often understood as a "vectorization" or "vector representation" technique.

[0046] Vector representation model (Embedding model): is a technology widely used in machine learning and natural language processing (NLP), which aims to map high-dimensional data (such as text, pictures, audio, etc.) to a low-dimensional continuous vector space. This technology can capture the potential relationships and semantic information of the data, thereby improving model performance and efficiency. The core idea of ​​Embedding is to convert discrete, non-continuous data into continuous vector representations, which enables computers to process and analyze data more efficiently. In the field of NLP, Embedding technology is often used to convert words or sentences into vectors to facilitate model training and reasoning. For example, in text processing, Embedding can map each word to a point in a high-dimensional space, and the distance between these points reflects the semantic similarity between words. In this embodiment, after obtaining the document description information of the document to be processed, the vector representation model can be used to determine the vector representation of the document description information, so that semantic search can be achieved in the search process.

[0047] In the information age, establishing a mapping relationship between search information and a document can help users quickly find the required articles from a huge database and improve the user experience. In order to determine the document description information of a document in a preparatory manner, thereby establishing a mapping relationship between the document and the search information, the present application can provide a method for generating document description information and an electronic device. After a text extraction model of document description information is deployed in a text product of a terminal (usually a fine-tuned large language model), when it is detected that there is a document to be processed locally or in the cloud of the terminal, the document to be processed is input into the text extraction model for processing to obtain the document description information of the document to be processed, and the document description information includes summary information and keywords of the document to be processed. The generated document description information is subjected to text repetition detection. If there is repeated text (specifically repeated words) in the document description information, and the number of times the document description information of the document to be processed is re-obtained using the text extraction model is not greater than the set number threshold, the model parameters of the text extraction model are adjusted according to the number of repetitions of the repeated text in the document description information, and the document to be processed is input into the text extraction model after the adjusted parameters are processed to obtain new document description information. Repeatedly perform text duplication detection, and when the generated document description information does not contain duplicate text, or when the generated document description information contains duplicate text but the number of times the document description information of the document to be processed is re-acquired using the text extraction model is greater than a threshold number, output the document description information generation result of the document to be processed, which result is either accurate document description information generated, or the document description information generated by the text extraction model does not meet the requirements. Through such a method, electronic devices (such as mobile phones, tablets, computers, etc.) can automatically detect whether there are new documents or modified documents, and perform text duplication detection in the process of generating the document description information of the document, and regenerate the document description information when there is duplication, so as to reduce duplicate text in the duplicate document description information and improve the effect of the generated document description information.

[0048] The document description information generation method provided in the present application can be applied to an electronic device, which can be an electronic device with display screen hardware and corresponding software support.

[0049] For example, the electronic device may be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a household appliance, or the like. The present application does not impose any restrictions on the specific type of the electronic device.

[0050] The document description information generation method of the present application is applicable to documents in the form of electronic documents. The document format may be a portable document format (PDF), a presentation (PowerPoint, PPT), an editable text file (word), or a long picture. The document corresponding to the document may have the same or different format as the document.

[0051] For the sake of convenience, the present application takes the electronic document displayed on a display device as an example for illustration. The display device may be an electronic device with display hardware and corresponding software support. For example, the display device may be a projector, a television, a mobile phone, a tablet computer, a laptop computer, a netbook, and other devices.

[0052] The method for generating document description information according to the embodiment of the present application is applicable to the following scenarios, for example, after performing an editing operation on a document contained in document data, document description information of the document after the editing operation is generated.

[0053] like Figure 1A As shown, the electronic device can display a main screen interface, which includes a document display interface. The document content 101 of the initial document is displayed on the document display interface. The initial document is included in a document database. The user can perform editing operations on the document content in 101. The editing operations include adding to the document content of the initial document, modifying the document content of the initial document, and deleting the document content of the initial document.

[0054] like Figure 1B As shown, new content 102 is added to the document content 101 of the initial document. If the user clicks Save and stores it in the document database, the document content of the initial document stored in the document database is changed. Then, the electronic device responds to the detected editing operation (new addition operation) and performs new addition processing on the document content to obtain a document to be processed. If the user clicks on a control for generating document description information, such as a "summary generation" control, the electronic device can input the document to be processed into a text extraction model to regenerate the document description information, and use the new document description information to replace the original document description information of the initial document.

[0055] like Figure 1CAs shown, the content 103 is modified in the document content 101 of the initial document. If the user clicks Save and stores it in the document database, the document content of the initial document stored in the document database is changed. Then, in response to the detected editing operation (modification operation), the electronic device modifies the document content to obtain the document to be processed. If the user clicks on a control for generating document description information, such as a "summary generation" control, the electronic device can input the document to be processed into the text extraction model to regenerate the document description information, and use the new document description information to replace the original document description information of the initial document.

[0056] like Figure 1D As shown, the content 104 is modified in the document content 101 of the initial document. If the user clicks Save and stores it in the document database, the document content of the initial document stored in the document database is changed. Then, in response to the detected editing operation (deletion operation), the electronic device deletes the document content to obtain the document to be processed. If the user clicks on the control for generating document description information, such as the "summary generation" control, the electronic device can input the document to be processed into the text extraction model to regenerate the document description information, and use the new document description information to replace the original document description information of the initial document. In addition, deletion can also be deletion of the entire document. In this case, it is necessary to find the document description information that has been generated for the document and delete it from the document database to reduce the memory occupied.

[0057] Among them, it should be noted that Figure 1A-Figure 1D It is an exemplary document display interface. Optionally, in an embodiment of the present application, it may include multiple electronic devices. The initial document may be edited on one electronic device, and the document description information may be generated on another electronic device.

[0058] The method for generating document description information according to the embodiment of the present application is also applicable to the following scenarios. For example, the document to be processed is a newly added document that does not exist in the document database of the electronic device. After receiving the newly added document, the electronic device will generate document description information for the newly added document.

[0059] like Figure 2A As shown, what is displayed is the document display interface of the electronic device opened by the user. At this time, the document display interface has no document content, as shown in 201, and is blank.

[0060] like Figure 2BAs shown, when a user imports a new document, the electronic device obtains the new document and displays the new document content 202 on the document display interface. For the document content in 202, the electronic device can input the document content into a text extraction model for processing to obtain document description information, and store the document description information and the document in a document database for subsequent searches.

[0061] After the document description information of each document is generated and stored in a database, it can be applied to a document search scenario. The following takes the document search scenario as an example to exemplarily introduce the application scenario of the embodiment of the present application.

[0062] like Figure 3A As shown, the electronic device can display the interface of the main screen, which includes a search page, and the search page includes a search box and a result display box. If the user enters the search keyword "artificial intelligence" in the search box 301, the computer searches the database for the vector representation corresponding to "artificial intelligence" through "artificial intelligence" and determines the document description information of the document matching it. Figure 3A The result display box 302 displays the document names of documents matching "artificial intelligence", and the corresponding area 303 includes the document names of multiple documents. Optionally, these document names can be sorted according to relevance, with the most relevant ones being placed at the front.

[0063] like Figure 3B As shown, the electronic device can also display the interface of the main screen, which includes a search page, the search page includes a search box and a result display box, the result display box can also include a directory display box and a summary information display box, wherein the directory display box 304 displays Figure 3A When the user selects any document in the directory display box 304, such as selecting "Document 1", the summary information corresponding to "Document 1" can be displayed in the summary information display box 305 to achieve a preview effect.

[0064] Based on the scenarios described above, the following describes how document description information and vector representation are generated. Figure 4 , Figure 4 A flowchart of a method for generating document description information provided in an embodiment of the present application, the method may include but is not limited to the following steps:

[0065] S401, input the document to be processed into the text extraction model for processing, and obtain the first document description information of the document to be processed. In some embodiments, before the electronic device inputs the document to be processed into the text extraction model for generating the document description information, it needs to first determine the document to be processed. Among them, the document to be processed includes two types, one is the document already in the document database; the other is not in the document database and is newly added by the user.

[0066] For the first case: the document display interface displays the document content of the initial document, and the initial document is included in the document database. This behavior can be actively triggered by the user, that is, actively opening the initial document in the document database in the document display interface of the electronic device, and then editing the document content of the initial document. After the editing process, the user triggers the control on the interface for generating document description information, such as the "summary generation" control, and the electronic device will respond to the detected editing operation, edit the document content, and obtain the document to be processed, and then input the document to be processed into the text extraction model for processing to obtain the first document description information of the document to be processed. The editing process here includes document addition processing, document deletion processing, and document modification processing, as shown below. Figure 1A-Figure 1D As shown, no further repetition will be given here.

[0067] For the second case: a new document is added to the document database, that is, the electronic device detects that a new document is added locally or in the cloud, and at this time, the new document can be used as a document to be processed. In response to the addition of a new document in the document database, the electronic device treats the new document as a document to be processed, and inputs the document to be processed into a text extraction model for processing to obtain the first document description information of the document to be processed.

[0068] For the document to be processed in the first case, the electronic device inputs the document to be processed into the text extraction model for processing to obtain the first document description information of the document to be processed. The first document description information here is used to update the original document description information of the document to be processed, because in case 1, the document content has changed, so the corresponding document description information will also change. For the document to be processed in the second case, since the document is newly added, the first document description information is also generated for the first time. It should be noted that before the document to be processed is input into the text extraction model, the compliance check can also be performed on the document to be processed to check whether there are illegal words in the document to be processed. If so, it will not be processed and can jump directly to step S406.

[0069] In an embodiment of the present application, the first document description information includes summary information of the document to be processed and keywords of the document to be processed. The keywords are central words that summarize the document to be processed. For example, for a document about artificial intelligence, the keywords may include artificial intelligence, model, etc. The summary information is the central idea of ​​the document to be processed, summarizing the document in brief content so that readers can clearly understand the general content of a document.

[0070] In some embodiments, the text extraction model is a large language model that has been trained and fine-tuned. For any document, the text extraction model can extract document description information, and the output format of these document description information is consistent after training.

[0071] S402: Perform text duplication detection on the first document description information.

[0072] In some embodiments, the electronic device can perform text duplication detection on the first document description information by sliding window string matching. In order to achieve accurate duplication detection, each detection window in multiple detection windows can be used to perform text duplication detection on the first document description information, and the window sizes of the multiple detection windows are different. The number of detection windows is related to the window size of the first document description information. Min_window_size is defined as the minimum window of the first document description information, and max_window_size is defined as the maximum window of the first document description information. The size of the detection window is min_window_size to max_window_size.

[0073] Wherein, the first document description information includes N words, N is a positive integer, and the target detection window in multiple detection windows is taken as an example to explain the detection process of N words using the target detection window. In this process, a text set is first defined, and the text set is used to record the text of the window detected during the detection process. In response to the detected text not being included in the text set, the detected text is added to the text set; the detected text is any text under the target detection window during the process of sliding detection of the N words included in the first document description information using the target detection window. In response to the detected text being included in the text set, it is determined that there is a repeated text in the first document description information, and the repeated text includes the detected text. In layman's terms, it is to first determine a detection window, and then use the detection window to perform sliding detection on the first description document, that is, to divide the first description document into multiple detected windows, and compare the text in each detected window with the text in the text set in turn, and then determine whether there is a repetition.

[0074] Based on the text duplication detection result, if there is duplicate text in the first document description information, step S403 is executed; if there is no duplicate text in the first document description information, step S404 is executed.

[0075] The following example illustrates the process of text duplication detection. Assume that the document description information is T, and W is the word sequence in the text, W = {w 1 ,w 2 ,…,w n}, where n is the total number of words. Define min_window_size as the minimum window of W, and max_window_size as the maximum window of W. Define a detection window s, where s∈[min_window_size,max_window_size], and define a text set S to store the detected words. For each starting index i, where i∈[0,ns], define a window W i =(w i ,w i+1 ,…w i+s-1 ), if W i ∈S, then it is considered that there is a duplicate text, otherwise W i Add the words in to the text set S and loop until the traversal ends.

[0076] Take an example, let the document description information T be "a kind of artificial intelligence method", then W = {a kind of human, a kind of artificial intelligence method}, let the size of the detection window s be 2, then start from the first "a kind", slide detection 11 times: the first time "a kind" does not exist in the text set, then "a kind" is added to the text set; the second time "a kind of person" does not exist in the text set, then "a kind of person" is added to the text set; according to this sliding window, detect in sequence, until the last time, the detection "method" does not exist in the text set, then "method" is added to the text set. In this process, it can be concluded that "a kind" is repeated, "a kind of person" is repeated, and "artificial" is repeated. Then according to the detection logic, the window is expanded in sequence. When the size of the detection window s is 3, "a kind of person" is repeated, and "a kind of artificial intelligence" is repeated. Based on this method, it can be determined whether there is repeated text in the document description information. At the same time, the number of repetitions of repeated text can also be determined during the detection process, and the number of repetitions of repeated text can usually be recorded by a counter.

[0077] S403: Determine whether the number of times the document description information of the document to be processed is re-acquired using the text extraction model is less than or equal to a number threshold.

[0078] In some embodiments, when there is repeated text in the first document description information, it is necessary to determine the number of times the text extraction model reacquires the document description information of the document to be processed and set it as the number of retries. In the embodiment of the present application, the number of times the text extraction model is used to reacquire the document description information of the document to be processed is set as the number of retries. The number of retries is determined during the continuous processing of the text extraction model. The text extraction model records each time it is processed. Therefore, the number of retries can be determined directly. First, determine the number of times the text extraction model processes the document description information. The number of retries is the number of processing times minus 1. If the number of times the text extraction model processes the document description information is 1, the number of retries is 0.

[0079] In this process, it is determined whether the number of retries is less than or equal to the number threshold. If the number of retries is less than or equal to the number threshold, step S405 is executed. If the number of retries is greater than the number threshold, step S406 is executed. The number threshold can be customized. The set value should not be too large, which will affect the efficiency, nor too small, which may result in unsatisfactory generated document description information. Therefore, a more moderate value can be set based on past experience.

[0080] S404: Determine the first document description information as target document description information of the document to be processed, and perform compliance detection on the target document description information of the document to be processed.

[0081] In some embodiments, after determining the target document description information of the document to be processed, the target document description information needs to be stored so that subsequent users can find the corresponding document according to the search keyword. Before storage, the target document description information of the document to be processed needs to be checked for compliance, the purpose of which is to detect whether there are abnormal words or abnormal formats in the target document description information.

[0082] In response to detecting compliance with the target document description information, a vector representation of the target document description information is generated, and the target document description information and the vector representation of the target document description information are stored in a database. The vector representation of the target document description information is obtained by inputting the target document description information into a vector representation model for processing. In response to detecting non-compliance with the target document description information, the target document description information of the document to be processed is deleted, and the process jumps to step S406.

[0083] S405: Adjust model parameters of the text extraction model.

[0084] Among them, the model parameters are associated with the generation probability of repeated text. In the embodiment of the present application, the main purpose is to reduce the generation probability of repeated text by using the model parameters, so as to reduce the repeated text in the document description information.

[0085] In some embodiments, the model parameters of the text extraction model are determined based on the number of repeated texts. Repeated texts specifically refer to repeated words in the document description information. The number of repetitions of each word can be determined when performing text repetition detection on the document description information. When a repetition is detected, the number of repetitions of the corresponding word is increased by 1. When the document description information detection is completed, the number of repetitions of each repeated word can be determined.

[0086] Furthermore, when adjusting the model parameters of the text extraction model according to the number of repetitions of the repeated text, the adjustment data of the repeated text is first determined according to the number of repetitions of the repeated text and the constraint function, and the constraint function includes one of a logarithmic form or a constant form. Then, according to the repeated text adjustment data, the original parameters of the repeated text in the text extraction model are adjusted, and the repeated text adjustment data is used to lower the generation probability of the repeated text. In the actual application process, the original parameters of the repeated text in the text extraction model can be directly replaced with the adjustment data. Among them, the constraint function is also called a penalty function. As the name suggests, this value is to penalize the repeated text to reduce the generation probability of the repeated text.

[0087] The following are two ways to determine the adjustment data for repeated text:

[0088] Method 1: The constraint function is in logarithmic form, and the adjustment data of repeated text is determined in logarithmic form. Set the constraint function to r(w i ,k),r(w i ,k) is the adjustment data of the repeated text. The adjustment data of the repeated text can be specifically shown as formula (1):

[0089] r(w i ,k)=1+αlog (k+1) (1)

[0090] Among them, w i is a repeated word, k is a repeated word w i The number of repetitions, α is a parameter that controls the growth rate. When k = 0, r(w i ,k)=1, that is, no penalty is imposed. When k>0, r(w i ,k) increases with the increase of k, that is, the penalty for words with more repeated occurrences is increased to reduce the probability of the word appearing.

[0091] Method 2: The constraint function is in constant form, that is, the adjustment data of the repeated text is determined in constant form. Set the constraint function to r(w i ,k),r(w i ,k) is the adjustment data of the repeated text. The adjustment data of the repeated text can be specifically shown as formula (2):

[0092]

[0093] Among them, m is the number of repetitions of the repeated word. The more repetitions, the greater the number of repetitions. i , the larger the value of k).

[0094] Based on the adjusted data of repeated text obtained above, the generation probability of the adjusted repeated text can be determined. Since the model parameters are increased, the generation probability of the repeated analysis is reduced.

[0095] In some feasible embodiments, the predicted generation probability of the repeated text can be determined according to the adjustment data of the repeated text and the current generation probability of the repeated text. The corresponding general process is as follows:

[0096] ①For each generated word, determine the original generation probability P of each word t After the document to be processed is input into the text extraction model for processing, the document description information and the generation probability of each word in the document description information can be obtained. Since the text extraction model has defined the output during the training process, the document description information and the generation probability of each word in the document description information can be directly output.

[0097] ②For each generated word w i , if the word is repeated, we will generate a probability P for it t (w i ) to make adjustments. If the word w i If the word is not repeated, no adjustment is required. Specifically, the adjustment is made based on the adjustment data of the repeated text (i.e., repeated words). i The probability calculation process can be found in formula (3):

[0098] P′ t (w i )=P t (w i ) / r(w i ,k) (3)

[0099] ③Finally, P′t is renormalized to ensure that the sum of the probabilities of all words is 1. The distribution of the renormalized probability distribution P″t can be calculated by the following formula (4):

[0100]

[0101] S406: Generate prompt information and display the prompt information on the user interface, where the prompt information is used to indicate that the first document description information generated by the text extraction model does not meet the document description information condition.

[0102] In some feasible embodiments, based on the above description, it can be known that when there are repeated texts in the first document description information and the number of retries is greater than the number threshold, the electronic device can generate a prompt message and display the prompt message on the document display interface. Usually, the prompt message is displayed on the administrator's document display interface, in order to let the administrator know that the first document description information generated by the text extraction model does not meet the document description information conditions (there may be too much repeated text in the document description letter, or there are sensitive words in the document description letter, etc.), so that the administrator can use other text extraction models to generate the document description information of the document to be processed. If other means are not used for intervention, the document description information of the document to be processed cannot be successfully stored in the library, and the user cannot search for the document, thereby affecting the user experience.

[0103] In addition, in some possible embodiments, when performing compliance detection, if it is detected that the target document description information is not compliant, the electronic device may also generate a prompt message to prompt the administrator that the document description information of the document to be processed is not compliant, or, when non-compliance is detected, determine the non-compliant words and replace or delete the non-compliant words. In addition, after the electronic device performs compliance detection on the target document description information, it also detects the entire document to be processed. If the document to be processed is also non-compliant, then the document to be processed can be prohibited from being stored.

[0104] like Figure 5A As shown, a document display interface is displayed in the electronic device. If the first document description information generated by the text extraction model does not meet the document description information condition, a prompt box 501 is displayed in the document display interface, and the words "document description information of document 2 does not meet the document description information condition" can be displayed in 501. If the administrator clicks the "Confirm" control, it can jump to the display interface of the document description information, so that the administrator can manually modify the document description information; if the administrator clicks the "Cancel" control, it means that the administrator does not process the document description information, and does not need to store the document description information for storage management temporarily.

[0105] like Figure 5B As shown, a document display interface is displayed in the electronic device. If it is detected that the generated document description information is not compliant, a prompt box 502 is displayed in the document display interface, and the words "Document description information of document 3 is not compliant, please confirm" can be displayed in 502. If the administrator clicks the "Confirm" control, it can jump to the display interface of the document description information, so that the administrator can manually modify the non-compliant parts of the document description information; if the administrator clicks the "Cancel" control, it means that the administrator does not process the document description information, and does not need to store the document description information for storage management for the time being.

[0106] S407: Input the document to be processed into the text extraction model with adjusted model parameters for processing to obtain second document description information of the document to be processed.

[0107] In some feasible embodiments, after the text extraction model is parameter-adjusted, the document to be processed is input into the text extraction model after the model parameters are adjusted for processing, and the second document description information of the document to be processed is obtained. There are differences between the second document description information and the first document description information. Based on the above description, it can be determined that the second document description information has less repeated text than the first document description information.

[0108] Furthermore, in order to generate more accurate document description information, text duplication detection may be performed on the second document description information, specifically by repeatedly executing steps S402 - S406 .

[0109] S408: Determine target document description information of the document to be processed according to the second document description information.

[0110] In some feasible embodiments, if there is no duplicate text in the second document description information, the second document description information is directly used as the target document description information. If there is still duplicate text in the second document description information, the model parameters are adjusted again, and the third document description information is generated using the text extraction model, and then the text duplication detection and the number of retries are continued. Based on such logic, the document description information can be repeatedly generated multiple times until there is no duplicate text in the document description information, or the number of retries is greater than the number threshold. Based on this approach, the duplicate text in the document description information can be reduced, and the generation effect of the document description information can be improved.

[0111] Based on the above description, it can be known that a large amount of document description information and vector representations are stored in the document database, so users can search for documents. The specific process may include: displaying a search interface, the search interface displays search information; Figure 3A As shown, the search information may be "artificial intelligence". In response to detecting a search trigger operation, the electronic device determines a vector representation that matches the vector representation of the search information from a document database; searches the document database for document description information corresponding to the vector representation that matches the vector representation of the search information, and finally displays the corresponding document description information on the search interface. For example Figure 3A As shown, the name of the document can be displayed in the search interface; for example Figure 3B As shown, summary information of the document can be displayed in the search interface.

[0112] In an embodiment of the present application, document description information is generated by repeated detection, multiple rounds of retries, and continuous adjustment of model parameters. This ensures the automated handling of abnormal situations of repeated generation while ensuring the diversity of the generated description information, and makes the generated document description information more effective.

[0113] In order to more clearly explain the method for generating document description information in the embodiment of the present application, see Figure 6 , taking a document as an example, the entire document description process is outlined, including the following steps:

[0114] (1) Perform document preprocessing.

[0115] (2) Use the text extraction model to process the document and obtain document description information.

[0116] (3) Perform duplication detection on document description information.

[0117] (4) Determine whether there is duplicate text in the document description information.

[0118] (5) If there is duplicate text in the document description information, determine whether the number of retries is less than or equal to the number threshold.

[0119] (6) If the number of retries is less than or equal to the threshold, the parameters of the text extraction model are adjusted. At the same time, the document is reprocessed using the model with adjusted parameters to obtain document description information, and (3)-(6) are repeated. If the number of retries is greater than the threshold, the process ends.

[0120] (7) If there is no duplicate text in the document description information, the document description information is input into the risk control module.

[0121] (8) Determine whether the document description information is safe.

[0122] (9) If the document description information is safe, the document description information is stored in the document database.

[0123] (10) Generate a vector representation of document description information.

[0124] (11) If the document description information is not safe, the process ends.

[0125] In step (11), if it is detected that the document description information is unsafe, you can choose not to display the prompt information directly, not to store it directly, or you can replace the unsafe areas in the document description information, such as sensitive words, with appropriate words.

[0126] pass Figure 6 The entire process of generating document description information can be clearly seen.

[0127] The following describes the process of generating document description information by an electronic device in combination with the specific scenario corresponding to the document status of the document to be processed and the interaction of the internal modules of the electronic device. Figure 7 As shown, the following steps may be included but are not limited to:

[0128] S701, the system AI search service module detects documents to be processed.

[0129] The document status of the document to be processed may indicate that the document to be processed may be a newly added document or may indicate that the original document is modified. In this step, the document path and document status of the document to be processed may be determined, wherein the document path indicates where the document to be processed comes from, and the document status indicates whether the document is newly added or modified in the original document.

[0130] S702, the system AI search service module sends the document path and document status of the document to be processed to the summary generation plug-in module.

[0131] S703, the summary generation plug-in module forwards the document path and document status of the document to be processed to the summary generation service process.

[0132] The summary generation plug-in module is used to connect the system AI search service module and the summary generation service process. Its purpose is to promote the summary generation service process and mainly provide a communication module. Among them, the summary generation plug-in module and the summary generation service process are both related to the generation of document description information. The "summary" is used as an example and does not limit the document description information.

[0133] S704, the summary generation service process obtains the content of the document to be processed according to the document path, and detects the content of the document to be processed.

[0134] Among them, the detection of the content of the document to be processed includes analysis, cleaning and risk control management.

[0135] S705, the summary generation service process sends the document to be processed to the model reasoning engine module, requesting the generation of document description information and corresponding vector representation.

[0136] Among them, a text extraction model and a vector representation model are deployed in the model reasoning engine module. Therefore, when the summary generation service process initiates a request, the model reasoning engine module can realize the generation of document description information, and when the document description information meets the generation conditions, the vector representation model is used to generate the corresponding vector representation.

[0137] In the process of the text extraction model generating the document description information of the document to be processed, it is necessary to perform the above-mentioned duplicate text detection, determination of the number of retries and compliance detection to ensure the reliability of the generated document description information.

[0138] S706: The model inference engine module returns the document description information and the corresponding vector representation to the summary generation service process.

[0139] S707, the summary generation service process saves the document description information and the corresponding vector representation.

[0140] The document description information and the corresponding vector representation may be stored in a document database to ensure that subsequent users can quickly search for corresponding content during searches.

[0141] S708: The summary generation service process returns the document description information generation result of the document to be processed to the summary generation plug-in module.

[0142] S709, the summary generation plug-in module returns the document description information generation result of the document to be processed to the system AI search service module.

[0143] Among them, the document description information generation results of the document to be processed include generation success and generation failure. Optionally, in the case of generation failure, prompt information can be input to the user through the system AI search service module to prompt the user the reason for the generation failure, so that the user can take measures to handle it.

[0144] Furthermore, Figure 7 The specific processing steps between the modules shown can be found in Figure 8 ,from Figure 8 It can be seen that the specific process executed by each module, among which the system AI search service module also includes a communication module and a document detection module. The document detection module mainly detects the document status, which includes document addition, document modification and document deletion, etc. The document to be processed is sent to the communication module in the summary generation service process through the communication module.

[0145] The summary generation service process obtains the documents to be processed through the communication module, puts each document into a queue, parses the content of each document, performs risk control and data cleansing, and then requests the document description information of each document from the model reasoning engine module; after obtaining the document description information of each document, it performs risk control management on the document description information, and requests the model reasoning engine module for the vector representation of each document description information, and finally stores the document description information and vector representation in the database for subsequent user searches.

[0146] The model inference engine module mainly deploys text extraction models and vector representation models through deployment tools. After receiving a document description information generation request initiated by the summary generation service process, the text extraction model is used to determine the document description information and return it; after receiving a vector representation generation request for the document description information initiated by the summary generation service process, the vector representation model is used to determine the vector representation of the document description information and return it.

[0147] Combine the following Fig. 9 The software system architecture provided summarizes the generation process and application process of the above document description information. Fig. 9 As shown, the layered architecture of the electronic device can be divided into several layers by software, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In this embodiment, the Android system is divided into application layer, service layer, model layer and reasoning deployment layer from top to bottom.

[0148] The application programs provided by the application layer may include search applications and document applications. The search application mainly provides a search interface for users to facilitate their searches; the document application is mainly used for document display. When the user completes the search, the search content is displayed on the document interface.

[0149] The service layer mainly provides services for various applications in the application layer. Intelligent middle-office services are deployed in the service layer, and intelligent middle-office services can be used to intelligently provide services for various applications in the application layer. For the search and documents in this application layer, the intelligent middle-office services in the server layer provide search services for search applications and document services for document applications.

[0150] The model layer is mainly used to deploy models to support the services of the service layer. In the embodiment of the present application, a large language model that can extract document description information is deployed in the model layer, which specifically refers to a text extraction model; at the same time, a vector representation model is also deployed in the model layer, which mainly performs vector representation processing on the document description information to obtain the vector representation corresponding to each document description information.

[0151] The inference deployment layer mainly deploys large model deployment accelerators to accelerate the processing speed of the models deployed in the model layer.

[0152] Among them, it should be noted that Fig. 9 The implementation logic of each layer is from bottom to top, and the lower layer provides technical support for the upper layer so that users can search for documents and display the searched documents in the document display interface.

[0153] like Fig.10As shown, the layered architecture of the electronic device can be divided into several layers by software, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into an application layer, an application framework layer, and a kernel layer from top to bottom.

[0154] The application layer may include a series of application packages, which may include camera, gallery, video playback, live broadcast, map, calendar, music, call and other applications. In the embodiment of the present application, the application layer may also include a display module. The display module may be used to control the user interface to output prompt information about the document description information, such as prompting the user that the document description information generated does not meet the document description information conditions, or prompting the user that the document description information is not safe.

[0155] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0156] The application framework layer may include a window manager. The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0157] The application framework layer can also include a view system, a notification manager, etc.

[0158] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0159] The notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages and can automatically disappear after a short stay without user interaction. In an embodiment of the present application, the notification manager can output prompt information to the user, for example, prompting the user that the document description information generated by the user does not meet the document description information conditions, or prompting the user that the document description information is unsafe.

[0160] In an embodiment of the present application, the application framework layer may also include a system AI search service module, a summary generation plug-in module, a summary generation service process, and a model reasoning engine module. Among them, the system AI search service module can be used to detect the documents to be processed and determine whether it is necessary to trigger the generation of document description information. The summary generation plug-in module is mainly used for communication and is responsible for sending relevant information of the document to be processed to the summary generation service process. The summary generation service process is mainly used to detect the documents to be processed and request the model reasoning engine module to generate document description information and vector representation. The model reasoning engine module is mainly used to deploy text extraction models and vector representation models to generate document description information and vector representation.

[0161] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, sensor driver, processor driver, etc.

[0162] It should be noted that the various functional modules included in the above software structure are only exemplary and do not constitute a specific limitation on the software architecture of the mobile phone of the present application. In other embodiments, the various functional modules included in the above software structure may be more or less, and the present application does not limit this. Although the embodiments of the present application are described using the Android system as an example, the basic principles are also applicable to electronic devices based on operating systems such as iOS or Windows.

[0163] It should be understood that in the embodiments of the present application, the electronic device may be a mobile phone, a tablet computer, a laptop computer, a PDA, a mobile internet device (MID), a wearable device, a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal in industrial control, a wireless terminal in self driving, a wireless terminal in remote medical surgery, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, a wireless terminal in a smart home, a personal digital assistant (PDA), etc., and the embodiments of the present application are not limited to this.

[0164] Exemplarily, the hardware structure of the electronic device in the embodiment of the present application is introduced below.

[0165] like Fig.11 As shown, the electronic device may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0166] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0167] NPU is a neural network computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. NPU can realize applications such as intelligent cognition of electronic devices, such as image recognition, face recognition, speech recognition, text understanding, etc. In the embodiment of the present application, NPU can be used to extract text from the document to be processed to determine the document description information of the document to be processed.

[0168] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0169] The electronic device implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0170] The wireless communication function of the electronic device can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor.

[0171] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0172] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to electronic devices. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0173] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as WiFi networks), Bluetooth (BT), BLE broadcast, global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to electronic devices. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the electromagnetic wave signal and filters it, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0174] The electronic device implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0175] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light emitting diode (OLED), an active matrix organic light emitting diode or an active matrix organic light emitting diode (AMOLED), a flexible light emitting diode (FLED), Miniled, MicroLed, MicrooLed, a quantum dot light emitting diode (QLED), etc. In some embodiments, the electronic device may include 1 or N display screens 194, where N is a positive integer greater than 1. In the embodiment of the present application, the display screen 194 can be used to display Figure 1A-Figure 1D The document display interface shown, Figure 2A and Figure 2B The document display interface shown, Figure 3A and Figure 3BThe search interface shown, Figure 5A and Figure 5B The document display interface shown.

[0176] The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor 180K can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device, which is different from the position of the display screen 194.

[0177] It is to be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0178] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in this application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk), etc.

[0179] It should be noted that, for the above-mentioned various method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0180] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0181] The modules in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0182] A person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing the relevant hardware through a program, and the program may be stored in a computer-readable storage medium, which may include a flash drive, ROM, RAM, a magnetic disk or an optical disk, etc.

[0183] The above disclosure is only a preferred embodiment of the present application, which is only a part of the embodiments of the present application and cannot be used to limit the scope of rights of the present application.

Claims

1. A method for generating document description information, characterized in that: The method comprises: Inputting the document to be processed into a text extraction model for processing to obtain first document description information of the document to be processed; Performing text duplication detection on the first document description information; In response to detecting duplicate text in the first document description information, and the number of retries is less than or equal to a number threshold, adjusting the model parameters of the text extraction model; wherein the number of retries refers to the number of times the document description information of the to-be-processed document is re-acquired using the text extraction model, and the adjusted model parameters of the text extraction model are associated with the generation probability of the duplicate text; Inputting the document to be processed into the text extraction model after the model parameters are adjusted for processing, so as to obtain second document description information of the document to be processed; The target document description information of the document to be processed is determined according to the second document description information.

2. The method according to claim 1, characterized in that The method further comprises: Displaying the document content of an initial document on a document display interface, wherein the initial document is included in a document database; In response to the detected editing operation, editing the document content to obtain the document to be processed; In response to a trigger for generating document description information of the document to be processed, the step of inputting the document to be processed into a text extraction model for processing to obtain first document description information of the document to be processed is performed.

3. The method according to claim 1, characterized in that The method further comprises: In response to a new document being added to the document database, executing the step of inputting the document to be processed into the text extraction model for processing to obtain first document description information of the document to be processed; The document to be processed is a document newly added to the document database.

4. The method according to any one of claims 1 to 3, characterized in that: The performing text duplication detection on the first document description information includes: Each detection window in a plurality of detection windows is used to perform text duplication detection on the first document description information; the plurality of detection windows have different window sizes.

5. The method according to claim 4, characterized in that The performing text duplication detection on the first document description information using each detection window in the plurality of detection windows respectively includes: In response to the detected text not being included in the text set, adding the detected text to the text set; the detected text is any text under the target detection window during the process of sliding detection of the first document description information using the target detection window; In response to the detected text being included in a text collection, it is determined that duplicate text exists in the first document description information, and the duplicate text includes the detected text.

6. The method according to any one of claims 1 to 3, characterized in that: The adjusting of the model parameters of the text extraction model includes: The model parameters of the text extraction model are adjusted according to the number of repetitions of the repeated text, and the number of repetitions of the repeated text is obtained in the process of performing text repetition detection on the first document description information.

7. The method according to claim 6, characterized in that The adjusting of the model parameters of the text extraction model according to the number of repetitions of the repeated text includes: Determining adjustment data of the repeated text according to the number of repetitions of the repeated text and a constraint function; According to the repeated text adjustment data, original parameters of the repeated text in the text extraction model are adjusted, and the repeated text adjustment data is used to lower the generation probability of the repeated text.

8. The method according to claim 1, characterized in that The method further comprises: In response to detecting duplicate text in the first document description information, and the number of retries being greater than the number threshold, generating prompt information; The prompt information is displayed on a document display interface, where the prompt information is used to indicate that the first document description information generated by the text extraction model does not meet the document description information condition.

9. The method according to claim 1, characterized in that The method further comprises: Performing compliance detection on target document description information of the document to be processed; In response to the target document description information passing the compliance detection, generating a vector representation of the target document description information; The target document description information and the vector representation of the target document description information are stored in a document database.

10. The method according to claim 9, characterized in that The method further comprises: Displaying a search interface, wherein the search interface displays search information; In response to detecting a search trigger operation, determining a vector representation matching the vector representation of the search information from the document database; Search for document description information corresponding to the vector representation that matches the vector representation of the search information, and display the corresponding document description information on the search interface.

11. An electronic device, characterized in that: The electronic device comprises: one or more processors, a memory and a touch screen; the memory is used to store program code; the processor is used to run the program code, so that the electronic device implements the method according to any one of claims 1-10.

12. A chip system, characterized in that: The chip system includes one or more processors, a memory and an interface circuit, the memory, the interface circuit and the one or more processors are interconnected via lines, and program instructions are stored in the memory; when the program instructions are executed by the processor, the chip system implements the method as described in any one of claims 1-10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images