Document generation method, device and equipment

Abstract

The present application provides a method, apparatus, and device for document generation, relating to the field of artificial intelligence technology. The method comprises: obtaining an initial document; if the initial document includes an image, inputting the image into a multimodal model to obtain image description text, and combining the text in the initial document with the image description text to obtain a first text, wherein the image description text is text used to describe the content of the image; inputting the first text into a large language model to preprocess the first text to obtain a preprocessed text; and generating a target document based on the preprocessed text. The present application can improve the efficiency of document generation.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, and device for generating a document. Background Art

[0002] In businesses and schools, document writing is a complex and time-consuming process, typically requiring multiple stages. First, the writer develops a preliminary concept and creates an initial document. Then, the initial document undergoes review by multiple stakeholders. Following the review, the writer needs to refine the document based on the reviewer's feedback.

[0003] However, due to the busy work of the document writer, there is often not enough time to write detailed document content, resulting in a long document writing cycle and low efficiency. Therefore, a method that can improve the efficiency of document writing is needed to assist the writer in completing the document writing. Summary of the Invention

[0004] This application provides a method, apparatus, and device for document generation, which can automatically generate a final target document from a draft document to improve the efficiency of document writing. The technical solution is as follows:

[0005] In a first aspect, a method for generating a document is provided, the method comprising:

[0006] An initial document is obtained. If the initial document includes an image, the image is input into a multimodal model to obtain image description text. The text in the initial document and the image description text are combined to obtain a first text, where the image description text is text describing the image. The first text is input into a large language model to preprocess the first text, obtaining a first preprocessed text. A target document is generated based on the first preprocessed text.

[0007] The technical solution provided by this application can automatically generate a final target document based on a user-provided initial document and a large language model. This allows the author to complete a rough initial document without having to spend a significant amount of time refining it, saving the user time and improving the author's efficiency. Furthermore, if the initial document contains images, the technical solution provided by this application can retain the image descriptions when generating the target document, rather than discarding the image content, thereby ensuring the integrity of the target document.

[0008] In one possible implementation, the method further includes:

[0009] Target audio is obtained and converted into audio description text, where the audio description text is text used to describe the content of the target audio. The audio description text is input into the large language model to preprocess the audio description text to obtain a second preprocessed text. Furthermore, a target document is generated based on the first preprocessed text and the second preprocessed text.

[0010] In the technical solution provided by the present application, after the writer has written the initial document, the initial document can be reviewed by relevant personnel. In the review meeting, the participants make speeches and propose amendments and additional comments on the initial document. The speeches of the participants can be recorded to obtain the target audio. Then, the target audio is converted into an audio description text, and the audio description text is preprocessed to obtain a second preprocessed text. The preprocessing may include correcting grammatical and logical errors, supplementing and expanding the content, etc. Then, the first preprocessed text and the second preprocessed text obtained above can be combined to generate the target document. In this way, it can be avoided that when the writer manually writes the target document, he forgets the review opinions in the review meeting, resulting in defects in the target document.

[0011] In a possible implementation, obtaining target audio includes:

[0012] Receive the target audio sent by the endpoint; or receive the target audio sent by the conference system.

[0013] In the technical solution provided in this application, the target audio can be the speeches of the participants in the review meeting for the initial document. Specifically, the review meeting can be an online meeting held on a conference system, which records the review meeting and uses the real-time recorded audio stream as the target audio. Alternatively, at the end of the review meeting, the author can download the complete recording of the review meeting through a terminal.

[0014] In a possible implementation, generating a target document according to the first preprocessed text includes:

[0015] The target technical field to which the content of the first preprocessed text belongs is identified, and a prompt word corresponding to the target technical field is obtained. The first preprocessed text and the prompt word are input into the large language model, so that the large language model processes the first preprocessed text according to the prompt word to obtain an optimized text. A target document is generated based on the optimized text in a specified format.

[0016] In the technical solution provided in the present application, in order to improve the accuracy and completeness of the target document generation, corresponding prompt words can be matched according to the technical field to which the content of the preprocessed document belongs, and when generating the target document, the matched prompt words and the preprocessed text are input into the large language model together, so that the prompt words can be used to instruct the large language model to generate a target document that meets the prompt word requirements for the preprocessed text.

[0017] In a possible implementation, generating a target document according to the first preprocessed text and the second preprocessed text includes:

[0018] The content of the first preprocessed text and the target technical field to which the second preprocessed text belongs are identified, and a prompt word corresponding to the target technical field is obtained. The first preprocessed text, the second preprocessed text, and the prompt word are input into the large language model, so that the large language model processes the first preprocessed text and the second preprocessed text according to the prompt word to obtain an optimized text. A target document is generated based on the optimized text in a specified format.

[0019] In the technical solution provided in the present application, in order to improve the accuracy and completeness of the target document generation, corresponding prompt words can be matched according to the technical field to which the content of the preprocessed document belongs, and when generating the target document, the matched prompt words and the preprocessed text are input into the large language model together, so that the prompt words can be used to instruct the large language model to generate a target document that meets the prompt word requirements for the preprocessed text.

[0020] In a second aspect, a device for generating a document is provided, the device comprising:

[0021] Data entry module, used to obtain initial documents;

[0022] an image-to-text conversion module, configured to, when the initial document includes an image, input the image into a multimodal model to obtain image description text, and combine the text in the initial document with the image description text to obtain a first text, wherein the image description text is text used to describe the image;

[0023] A generation module is used to input the first text into a large language model to preprocess the first text to obtain a first preprocessed text; and generate a target document based on the first preprocessed text.

[0024] In a possible implementation, the data input module is further configured to:

[0025] Get the target audio;

[0026] The device also includes a speech-to-text conversion module, which is used to:

[0027] Converting the target audio into an audio description text, wherein the audio description text is a text used to describe the target audio content;

[0028] The generation module is used to input the audio description text into the large language model to preprocess the audio description text to obtain a second preprocessed text; and generate a target document based on the first preprocessed text and the second preprocessed text.

[0029] In a possible implementation, the data input module is used to:

[0030] Receive the target audio sent by the terminal; or,

[0031] Receive the target audio sent by the conference system.

[0032] In a possible implementation, the generating module is configured to:

[0033] Identify the target technical field to which the content of the first preprocessed text belongs;

[0034] Obtain prompt words corresponding to the target technical field;

[0035] Inputting the first preprocessed text and the prompt word into the large language model, so that the large language model processes the first preprocessed text according to the prompt word to obtain an optimized text;

[0036] Generate a target document according to the optimized text in a specified format.

[0037] In a possible implementation, the generating module is configured to:

[0038] Identify the content of the first preprocessed text and the target technical field to which the second preprocessed text belongs;

[0039] Obtain prompt words corresponding to the target technical field;

[0040] Inputting the first preprocessed text, the second preprocessed text, and the prompt word into the large language model, so that the large language model processes the first preprocessed text and the second preprocessed text according to the prompt word to obtain an optimized text;

[0041] Generate a target document according to the optimized text in a specified format.

[0042] In a third aspect, a computing device cluster is provided, comprising at least one computing device, each computing device comprising a processor and a memory;

[0043] The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the document generation method as described in the first aspect and its possible implementations.

[0044] In a fourth aspect, a computer-readable storage medium is provided, comprising computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster performs the document generation method as described in the first aspect and its possible implementations.

[0045] In a fifth aspect, a computer program product comprising instructions is provided. When the instructions are executed by a computing device cluster, the computing device cluster executes the document generation method as described in the first aspect and its possible implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a flowchart of a method for generating a document provided in an embodiment of the present application;

[0047] Figure 2 This is a schematic diagram of a system framework for document generation provided by an embodiment of the present application;

[0048] Figure 3 This is a flowchart of a method for generating a document provided in an embodiment of the present application;

[0049] Figure 4 This is a schematic diagram of a system framework for document generation provided by an embodiment of the present application;

[0050] Figure 5 This is a schematic diagram of the structure of a document generation device provided in an embodiment of the present application;

[0051] Figure 6 is a schematic diagram of a computing device provided in an embodiment of the present application;

[0052] Figure 7 is a schematic diagram of a computing device cluster provided in an embodiment of the present application;

[0053] Figure 8 This is a schematic diagram of a computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0055] To facilitate understanding of the embodiments of the present application, the following explains the terms involved in the embodiments of the present application:

[0056] Multimodal Large Model (MLM):

[0057] A large-scale machine learning model that can process multiple data types, such as text, images, and audio. For example, an image can be input into a multimodal model, and the multimodal model will output image description text, which is text that describes the image content.

[0058] Large Language Model (LLM):

[0059] A deep learning model with a large number of parameters that is specifically designed for generating and understanding natural language. For example, it can understand the input text content and process it according to requirements to generate text that meets the requirements.

[0060] Automatic Speech Recognition (ASR):

[0061] A technology that converts audio signals into text can be achieved through the ASR model, which is a machine learning model.

[0062] document:

[0063] In the embodiments of the present application, documents include but are not limited to various technical documents, solution designs, white papers, etc. The document format can be a slide presentation document (PowerPoint, PPT), a word document, etc. The application scenarios include but are not limited to design solution writing, user manual writing, report material writing, etc.

[0064] Prompt:

[0065] Input text used to guide the large language model to generate specific content.

[0066] In businesses and schools, document writing is a complex and time-consuming process, typically requiring multiple stages. First, the writer develops a preliminary concept and creates a first draft. Then, the review phase begins, where the draft is reviewed and expanded upon from various perspectives. After the review, the writer needs to supplement and revise the draft based on the review comments to create the final document. Some technical documents typically include a large amount of content, including technical background, existing technologies, the technical problems to be solved by the solution, key ideas, solution architecture diagrams, specific scenarios, and detailed implementation procedures. However, due to their busy schedules, business personnel often lack the time to supplement and revise the draft, resulting in a lengthy document writing cycle. Furthermore, because writers often supplement and revise the draft long after the review, comments raised in the review meeting may be missed, resulting in poor quality in the final document. Therefore, a more efficient method is needed to assist writers in document generation.

[0067] An embodiment of the present application provides a method for document generation, which can be implemented by a computing device. In this method, the computing device obtains an initial document. When the initial document includes an image, the image is input into a multimodal model to obtain image description text, and the text in the initial document and the image description text are combined to obtain a first text. Furthermore, the first text is input into a large language model to preprocess the first text through the large language model to obtain a preprocessed text. The preprocessing may include the revision of erroneous content and may also include the expansion of the solution content in the first text. Finally, a target document is generated based on the preprocessed text. The entire process is automated and does not require human intervention, which saves the writer's time and improves the efficiency of document writing.

[0068] The document generation method provided by the embodiment of the present application is described below with reference to the accompanying drawings. Figure 1 , the method may include the following steps:

[0069] Step 101: Obtain an initial document.

[0070] During implementation, when it is necessary to write a technical document, the writer can first simply write an initial document after having an idea. The initial document can be a Word document, PPT document, etc. The initial document can include a simple description of the technical background, existing technology, technical problems to be solved by the solution, main ideas, solution architecture diagram, specific scenarios and detailed implementation process.

[0071] The writer can then upload the initial document to a computing device.

[0072] Step 102: When the initial document includes an image, the image is input into a multimodal model to obtain image description text, and the text in the initial document and the image description text are combined to obtain a first text.

[0073] The image description text is text used to describe the image content.

[0074] In practice, when writing an initial document, the author may insert an image, such as a picture or video, into the document to save time or to describe the image. In this case, the computing device can read the initial document, extract the image from the initial document if it contains an image, and input the image into the multimodal model to obtain image description text. The computing device then extracts text from the initial document and replaces the image position with the corresponding image description text to obtain the first text.

[0075] Alternatively, when the initial document contains an image, the initial document is input into the multimodal model, which automatically recognizes the image in the initial document and generates image description text, and then outputs a first text, wherein the first text includes the text in the initial document and the image description text.

[0076] Step 103: Input the first text into the large language model to preprocess the first text to obtain a first preprocessed text.

[0077] During implementation, the computing device inputs the first text into the large language model, preprocesses the first text using the large language model, and outputs a first preprocessed text. Preprocessing may include correcting errors or expanding solution content. Errors may include errors or defects in the first text, such as technical description errors, grammatical errors, and logical errors.

[0078] In addition, when inputting the first text into the large language model, a prompt word may also be input at the same time. For example, the prompt word may be "Please correct the incorrect content of the input text and reasonably expand the solution described in the text."

[0079] Step 104: Generate a target document based on the first preprocessed text.

[0080] In practice, in order to improve the quality of the target document finally generated, some prompt words can be designed and stored in advance for different technical fields. For example, the technical fields may include cloud computing technology, database technology, optical communication technology, wireless communication technology, new energy technology, automotive technology, artificial intelligence technology, etc. The embodiment of this application does not limit the specific technical field division. As shown in Table 1 below:

[0081] Table 1

[0082]

[0083] In step 104, the target technical field to which the content of the first preprocessed text belongs can be identified, and the prompt word corresponding to the target technical field can be obtained from the stored correspondence between technical fields and prompt words. Here, the target technical field can be identified by matching keywords in the first preprocessed text with keywords in the technical field to identify the target technical field.

[0084] Then, the first preprocessed text and the prompt words corresponding to the target technical field are input into the large language model, so that the large language model processes the first preprocessed text according to the prompt words to obtain an optimized text.

[0085] Then, the optimized text is converted into a target document according to the specified format. For example, a document template may be pre-stored, and the document template includes multiple titles, such as technical background, existing technology, technical problems to be solved by the solution, main ideas, solution architecture diagram, specific scenarios and detailed implementation process, etc. Correspondingly, the optimized text may also include the above titles, and the content of the optimized text may be filled into the document template according to the corresponding titles to obtain the target document. In order to further improve the accuracy and completeness of the target document, after obtaining the target document, the target document may be input into a large language model so that the large language model optimizes the target document to ensure the rigor of the logic and the correctness and integrity of the content. When inputting the target document into the large language model, prompt words may be input at the same time, such as "Please check the logic and content in the input document, and correct the erroneous content and inaccurate logic."

[0086] Furthermore, for the aforementioned solution architecture diagram, prior art, and detailed implementation process, the optimized text output by the large language model may contain drawing code. In this case, if the presence of drawing code is detected in the optimized text, the corresponding drawing application is called to generate the corresponding architecture diagram, flow chart, etc. based on the drawing code. The drawing code in the optimized text is then replaced with the architecture diagram, flow chart, etc. generated based on the drawing code. The drawing code may be mermaid code, PlantUML code, etc., and this embodiment of the application does not limit this.

[0087] In one possible implementation, a correspondence between technical fields and technical knowledge may be pre-stored. Accordingly, in step 104, after identifying the target technical field to which the content of the first pre-processed text belongs, the technical knowledge corresponding to the target technical field may be obtained from the correspondence between the technical field and technical knowledge. Furthermore, the first pre-processed text, the prompt words corresponding to the target technical field, and the technical knowledge corresponding to the target technical field may be input into a large language model, so that the large language model processes the first pre-processed text based on the prompt words and technical knowledge to obtain an optimized text. The technical knowledge mentioned here may be existing technical knowledge, i.e., technical knowledge that has been disclosed in the form of papers, patents, conferences, etc.

[0088] The following describes the target technical field to which the content of the first preprocessed text belongs:

[0089] The computing device can be deployed with a pre-trained technical field recognition model. After obtaining the first preprocessed text, the computing device inputs the first preprocessed text into the technical field recognition model, and the technical field recognition model outputs the target technical field to which the content of the first preprocessed text belongs.

[0090] Alternatively, after obtaining the first preprocessed text, the computing device inputs the first preprocessed text into the large language model, which then outputs the target technical field to which the content of the first preprocessed text belongs. Furthermore, when inputting the first preprocessed text into the large language model, a prompt word is also input simultaneously. For example, the prompt word could be "Please analyze the technical field to which the input text belongs." Alternatively, possible technical fields can be added to the prompt word for the large language model to select, such as "Please analyze whether the technical field to which the input text belongs is cloud computing technology, database technology, optical communication technology, wireless communication technology, new energy technology, automotive technology, or artificial intelligence technology."

[0091] In another possible implementation, after the author completes the initial document, it can be reviewed by relevant personnel. During the review meeting, participants can speak and propose revisions and supplementary comments to the initial document. The participant's speech can be recorded to obtain the target audio. For example, the participant can provide a detailed description of the content that needs to be added under a certain heading, or the participant can simply briefly mention the addition of a brief introduction to a specific technology under a certain heading, and the specified technology is already existing technology. Or, the participant can propose the addition of specific deliverables for a specific team under a certain heading.

[0092] There are many ways to obtain the target audio. The following are some examples to illustrate:

[0093] Method 1:

[0094] After the review meeting, the author can upload the complete audio of the review meeting, or a processed version of the complete audio, as the target audio to a computing device. Processing of the complete audio may include removing irrelevant content and optimizing audio quality. Irrelevant content can include content unrelated to the technical discussion, such as greetings between participants.

[0095] Method 2:

[0096] like Figure 2 As shown, a document system and a conference system can be deployed in a computing device, and the embodiment of the present application can be implemented by a document system. The document system and the conference system are interconnected, and the above-mentioned review meeting can be conducted online through the conference system. During the review meeting, the conference system records the speeches of the participants and sends the real-time audio stream as the target audio to the document system. Alternatively, after the review meeting, the conference system sends the complete audio of the review meeting, or the audio after processing the complete audio, as the target audio to the document system.

[0097] After acquiring the target audio, the computing device inputs the target audio into the ASR model, which converts the target audio into audio description text. The audio description text is then input into the large language model to preprocess the audio description text, generating a second preprocessed text. This preprocessing may include correcting errors or expanding the solution content.

[0098] In addition, when inputting audio description text into the large language model, you can also input prompt words at the same time. For example, the prompt word can be "Please correct the incorrect content of the input text and reasonably expand the solution described in the text."

[0099] In combination with the above several possible implementations, the process of generating the optimized text in step 104 can be as follows:

[0100] The computing device identifies the target technical field to which the contents of the first and second preprocessed texts belong, and obtains the prompt word corresponding to the target technical field from the stored correspondence between technical fields and prompt words. Furthermore, the computing device obtains the technical knowledge corresponding to the target technical field from the stored correspondence between technical fields and technical knowledge. Furthermore, the computing device inputs the first and second preprocessed texts, the prompt word corresponding to the target technical field, and the technical knowledge corresponding to the target technical field into the large language model, so that the large language model processes the first and second preprocessed texts based on the prompt word and technical knowledge to produce optimized texts.

[0101] Of course, in the processing here, only the first preprocessed text and the prompt word can be input, or the first preprocessed text and technical knowledge, or the first preprocessed text, the second preprocessed text and the prompt word, or the first preprocessed text, the second preprocessed text and technical knowledge. The above content can be input in combination, and the embodiment of the present application does not limit this.

[0102] The following describes the target technical field to which the contents of the first preprocessed text and the second preprocessed text belong:

[0103] The computing device can be deployed with a pre-trained technical field identification model. After obtaining the first preprocessed text and the second preprocessed text, the computing device inputs the first preprocessed text and the second preprocessed text into the technical field identification model, and the technical field identification model outputs the target technical field to which the contents of the first preprocessed text and the second preprocessed text belong.

[0104] Alternatively, after obtaining the first preprocessed text and the second preprocessed text, the computing device inputs the first preprocessed text and the second preprocessed text into the large language model, and the large language model outputs the target technical field to which the contents of the first preprocessed text and the second preprocessed text belong. In addition, when inputting the large language model into the first preprocessed text and the second preprocessed text, a prompt word is also input at the same time, for example, the prompt word is "Please analyze which technical field the next two input texts belong to in common." Alternatively, possible technical fields can be added to the prompt word for the large language model to select, for example, "Please analyze which technical field the next two input texts belong to in common: cloud computing technology, database technology, optical communication technology, wireless communication technology, new energy technology, automotive technology, or artificial intelligence technology."

[0105] The following example uses cloud computing as an example to illustrate a corresponding prompt word for cloud computing technology:

[0106] "Hello, I have a task that needs your help. I have three texts in the field of cloud computing technology, namely the first draft I wrote, the text corresponding to the review recording, and some technical knowledge related to cloud computing. My goal is to sort out the technical background of the solution, existing technologies, technical problems to be solved by the solution, main ideas, solution architecture diagram, specific scenarios and detailed implementation process from these texts. If it involves system architecture, describe the system architecture in detail in a few paragraphs, and generate an architecture diagram represented by mermaid code. If it involves a method flow chart, describe the method flow in detail in a few paragraphs, and generate a method flow chart represented by mermaid code. In addition, you can expand the text content I provided through Internet search. It is important to note that the output content should not involve various sensitive words."

[0107] The following examples illustrate the processing performed by several large language models based on the second pre-processed text:

[0108] Processing 1:

[0109] The second pre-processed text includes the words of a participant, "Add the first content under the main idea of the plan." When the large language model generates the optimized content, it can add the above first content under the title "Main idea of the plan."

[0110] Processing 2:

[0111] The second preprocessed text includes the words of a participant, "Add a brief introduction to the specified technology under the existing technology." When generating the optimized content, the large language model obtains the relevant introduction to the specified technology through online search, and can add the obtained relevant introduction to the specified technology under the title "Existing Technology."

[0112] Process three:

[0113] The second preprocessed text includes the words of a participant, "Add the specified deliverables of the specified team under the detailed implementation process." When generating the optimized content, the large language model accesses the internal knowledge base to obtain the specified deliverables of the specified team, and can add the obtained specified deliverables under the title "Detailed Implementation Process."

[0114] In one possible implementation, in combination with the above-mentioned target audio, the above-mentioned initial document may only include titles such as technical background, existing technology, technical problems to be solved by the solution, main ideas, solution architecture diagram, specific scenarios and detailed implementation process.

[0115] The technical solutions provided by the embodiments of the present application are described below with reference to the accompanying drawings:

[0116] like Figure 3As shown, in the technical solution provided by the embodiments of the present application, for an initial document, a multimodal model is used to convert the images therein into image description text. This text is combined with the original text of the initial document to form a first text, which is then input into a large language model to correct errors and expand the solution content, resulting in a first preprocessed text. For the target audio of the review meeting of the initial document, an ASR model is used to convert the target audio into audio description text. This audio description text is then input into a large language model to correct errors and expand the solution content, resulting in a second preprocessed text. Furthermore, prompt words and technical knowledge corresponding to the target technical fields to which the first and second preprocessed texts belong are obtained. The first and second preprocessed texts, as well as the prompt words and technical knowledge corresponding to the target technical fields, are input into the large language model. The large language model processes the first preprocessed text based on the prompt words and technical knowledge, resulting in an optimized text. Then, for the drawing code in the optimized text, a corresponding drawing application can be called to generate corresponding flowcharts, framework diagrams, etc. The generated flowcharts and framework diagrams are added to the optimized text, and the target document is generated based on the optimized text in a specified format. Finally, the target document can be input into the large language model so that the large language model can check and correct the logic and content of the target document.

[0117] The present application also provides a method for document generation. In this method, there may be no initial document or the initial document is blank. In this case, a target document can be generated based on target audio and a large language model. The target audio can be a real-time audio stream of a review meeting sent by a cloud conference system. Figure 4 , the method may include the following steps:

[0118] Step 201: Acquire target audio.

[0119] In practice, relevant personnel can hold review meetings through the cloud conferencing system, during which they can share initial documents, which can be blank or just have a title. The cloud conferencing system records the speeches of the participants and generates an audio stream that is sent to the computing device.

[0120] In addition, the cloud conference system can also identify which page of the initial document is currently shared, and send page indication information and the audio stream generated at the same time to the computing device, wherein the page indication information is used to indicate which page of the initial document is currently shared.

[0121] Step 202: Convert the target audio into audio description text.

[0122] The audio description text is text used to describe the target audio content.

[0123] In implementation, after receiving the audio stream, the computing device may input the audio stream into an ASR model, and the ASR model converts the audio stream into audio description text.

[0124] Step 203: Input the audio description text into the large language model, preprocess the audio description text, and obtain a preprocessed text.

[0125] During implementation, the computing device inputs the page indication information and the audio description text corresponding to the audio stream received simultaneously with the page indication information into the large prediction model, so as to preprocess the audio description text through the large language model to obtain the preprocessed text.

[0126] Here are some examples of how to generate preprocessed text:

[0127] Method 1:

[0128] The attendees have explained in detail the content that needs to be added to the page indicated by the page indication information. This part of the audio description text can be directly used as the pre-processed text within the page indicated by the page indication information.

[0129] Method 2:

[0130] The attendees only roughly explained that a brief introduction to the specified technology can be added to the page indicated by the page indication information, and the specified technology is an existing technology. The large language model can then obtain a brief introduction to the specified technology through an online search as the processed text within the page indicated by the page indication information.

[0131] Method 3:

[0132] If a participant suggests that the specified deliverable content of a specified team should be added to the page indicated by the page indication information, the company's internal knowledge base may be searched for relevant information of the specified deliverable content based on the team name and department name of the specified team, and the content will be used as processed text within the page indicated by the page indication information.

[0133] Method 4:

[0134] The large language model performs AI-assisted content expansion based on the audio description text, and uses the expanded content as preprocessed text.

[0135] Step 204: Generate a target document based on the preprocessed text.

[0136] In practice, to improve the quality of the final generated target document, prompt words can be pre-designed and stored for different technical fields. For example, technical fields may include cloud computing technology, database technology, optical communication technology, wireless communication technology, new energy technology, automotive technology, artificial intelligence technology, etc. The present application embodiment does not limit the specific technical field division.

[0137] In step 204, the target technical field to which the content of the first preprocessed text belongs can be identified, and the prompt word corresponding to the target technical field can be obtained from the stored correspondence between technical fields and prompt words. Here, identifying the target technical field can be performed by matching keywords in the first preprocessed text with keywords in the technical field to identify the target technical field.

[0138] Then, the first preprocessed text and the prompt words corresponding to the target technical field are input into the large language model, so that the large language model processes the first preprocessed text according to the prompt words to obtain an optimized text.

[0139] The optimized text is then formatted to generate a target document. For example, a document template can be pre-selected and stored, which includes multiple headings, such as technical background, existing technology, technical problems to be solved by the solution, main ideas, solution architecture diagram, specific scenarios, and detailed implementation process. Correspondingly, the optimized text can also include these headings. The content of the optimized text can then be filled into the document template according to the corresponding headings to obtain the target document.

[0140] In addition, for the above-mentioned solution architecture diagram, existing technology, detailed implementation process, etc., the optimized document output by the large language model may contain drawing codes. In this case, the drawing program corresponding to these drawing codes can be called to generate the corresponding architecture diagram, flow chart, etc., and the above-mentioned drawing code can be replaced with the generated architecture diagram, flow chart, etc.

[0141] Based on the same technical concept, the embodiment of the present application also provides a device for generating a document, which can be applied to a computing device, see Figure 5 , the apparatus may include:

[0142] Data input module 510, used to obtain initial documents;

[0143] An image-to-text conversion module 520 is configured to, when the initial document includes an image, input the image into a multimodal model to obtain image description text, and combine the text in the initial document with the image description text to obtain a first text, wherein the image description text is text used to describe the image;

[0144] The generation module 530 is configured to input the first text into a large language model to preprocess the first text to obtain a first preprocessed text; and generate a target document based on the first preprocessed text.

[0145] In a possible implementation, the data input module 510 is further configured to:

[0146] Get the target audio;

[0147] The device also includes a speech-to-text conversion module, which is used to:

[0148] Converting the target audio into an audio description text, wherein the audio description text is a text used to describe the target audio content;

[0149] The generation module 530 is used to input the audio description text into the large language model to preprocess the audio description text to obtain a second preprocessed text; and generate a target document based on the first preprocessed text and the second preprocessed text.

[0150] In a possible implementation, the data input module 510 is configured to:

[0151] Receive the target audio sent by the terminal; or,

[0152] Receive the target audio sent by the conference system.

[0153] In a possible implementation, the generating module 530 is configured to:

[0154] Identify the target technical field to which the content of the first preprocessed text belongs;

[0155] Obtain prompt words corresponding to the target technical field;

[0156] Inputting the first preprocessed text and the prompt word into the large language model, so that the large language model processes the first preprocessed text according to the prompt word to obtain an optimized text;

[0157] Generate a target document according to the optimized text in a specified format.

[0158] In a possible implementation, the generating module 530 is configured to:

[0159] Identify the content of the first preprocessed text and the target technical field to which the second preprocessed text belongs;

[0160] Obtain prompt words corresponding to the target technical field;

[0161] Inputting the first preprocessed text, the second preprocessed text, and the prompt word into the large language model, so that the large language model processes the first preprocessed text and the second preprocessed text according to the prompt word to obtain an optimized text;

[0162] Generate a target document according to the optimized text in a specified format.

[0163] The technical solution provided by this application can automatically generate a final target document based on a user-provided initial document and a large language model. This allows the author to complete a rough initial document without having to spend a significant amount of time refining it, saving the user time and improving the author's efficiency. Furthermore, if the initial document contains images, the technical solution provided by this application can retain the image descriptions when generating the target document, rather than discarding the image content, thereby ensuring the integrity of the target document.

[0164] It should be noted that the document generation apparatus provided in the above embodiments only uses the division of the above-mentioned functional modules as an example for document generation. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the computing device can be divided into different functional modules to complete all or part of the functions described above. In addition, the document generation apparatus provided in the above embodiments and the document generation method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0165] The data input module 510, the image-to-text conversion module 520, and the generation module 530 can all be implemented via software or hardware. For example, the implementation of the feature content generation module 340 will be described below using the generation module 530 as an example. Similarly, the implementation of the data input module 510 and the image-to-text conversion module 520 can refer to the implementation of the generation module 530.

[0166] As an example of a software functional unit, the generation module 530 may include code running on a computing instance. A computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, the generation module 530 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, with each AZ including one data center or multiple geographically close data centers. Typically, a region may include multiple AZs.

[0167] Similarly, the multiple hosts / virtual machines / containers used to run the code can be distributed within the same virtual private cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Cross-region communication between two VPCs within the same region, or between VPCs in different regions, requires a communication gateway within each VPC to interconnect the VPCs.

[0168] As an example of a hardware functional unit, generation module 530 may include at least one computing device, such as a server. Alternatively, generation module 530 may be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0169] The multiple computing devices included in the generation module 530 can be distributed in the same region or in different regions. The multiple computing devices included in the generation module 530 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the generation module 530 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, GALs, and other computing devices.

[0170] The embodiment of the present application also provides a computing device 700, which can be used as the above-mentioned server or terminal. Figure 6 As shown, computing device 700 includes a bus 702, a processor 704, a memory 706, and a communication interface 708. Processor 704, memory 706, and communication interface 708 communicate with each other via bus 702. Computing device 700 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in computing device 700.

[0171] The bus 702 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 The fact that only one line is used in the figure does not mean that there is only one bus or only one type of bus. Bus 702 may include a path for transmitting information between various components of computing device 700 (eg, memory 706, processor 704, communication interface 708).

[0172] The processor 704 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0173] The memory 706 may include volatile memory, such as random access memory (RAM). The memory 706 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0174] Memory 706 stores executable program code. Processor 704 executes this executable program code to implement the aforementioned functions of data input module 510, image-to-text conversion module 520, and generation module 530, thereby implementing the storage resource allocation method. Specifically, memory 706 stores instructions for executing the document generation method.

[0175] Alternatively, the memory 706 stores executable codes, and the processor 704 executes the executable codes to respectively implement the functions of the aforementioned document generation apparatus, thereby implementing the document generation method. In other words, the memory 706 stores instructions for executing the document generation method.

[0176] The communication interface 708 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 700 and other devices or a communication network.

[0177] Embodiments of the present application also provide a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0178] like Figure 7 As shown, the computing device cluster includes at least one computing device 700. The memory 706 in one or more computing devices 700 in the computing device cluster may store the same instructions for executing the document generation method.

[0179] In some possible implementations, the memory 706 of one or more computing devices 700 in the computing device cluster may also store partial instructions for executing the document generation method. In other words, the combination of one or more computing devices 700 can jointly execute the instructions for the document generation method.

[0180] It should be noted that the memory 706 in different computing devices 700 in the computing device cluster can store different instructions, each used to implement the storage resource allocation portion of the function. In other words, the instructions stored in the memory 706 in different computing devices 700 can implement the functions of one or more modules among the data input module 510, the image-to-text conversion module 520, and the generation module 530.

[0181] In some possible implementations, one or more computing devices in a computing device cluster may be connected via a network, which may be a wide area network or a local area network. Figure 8 A possible implementation is shown. Figure 8 As shown, two computing devices 700A and 700B are connected via a network. Specifically, the connection to the network is achieved through communication interfaces within each computing device. In this possible implementation, memory 706 within computing device 700A stores instructions for executing the functions of data input module 510 and image-to-text conversion module 520. Simultaneously, memory 706 within computing device 700B stores instructions for executing the functions of generation module 530.

[0182] Figure 8 The connection mode between the computing device clusters shown may be based on the consideration that document generation requires a large amount of computing resources, and therefore the function of the generation module 530 is considered to be executed by the computing device 700B.

[0183] It should be understood that Figure 8 The functionality of the computing device 700A shown in FIG. 7 may also be implemented by multiple computing devices 700. Similarly, the functionality of the computing device 700B may also be implemented by multiple computing devices 700.

[0184] In this application, the terms "first," "second," and the like are used to distinguish between identical or similar items having substantially the same function or effect. It should be understood that "first," "second," and "nth" do not have a logical or temporal dependency, nor do they limit the quantity or order of execution. It should also be understood that although the following description uses the terms "first," "second," and the like to describe various elements, these elements should not be limited by these terms. These terms are simply used to distinguish one element from another.

[0185] It should also be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0186] In this application, the term "at least one" means one or more, and the term "plurality" means two or more. For example, "plurality of second devices" means two or more second devices. The terms "system" and "network" are often used interchangeably herein.

[0187] It should be understood that the terminology used in the description of the various examples herein is for the purpose of describing particular examples only and is not intended to be limiting. As used in the description of the various examples and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0188] It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the listed items. The term "and / or" describes an association between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this application generally indicates that the associated objects are in an "or" relationship.

[0189] It should also be understood that the terms “if” and “if” may be interpreted to mean “when” or “upon” or “in response to determining” or “in response to detecting.” Similarly, the phrases “if it is determined that…” or “if [stated condition or event] is detected” may be interpreted to mean “upon determining…” or “in response to determining…” or “upon detecting [stated condition or event]” or “in response to detecting [stated condition or event],” depending on the context.

[0190] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the embodiments of the present application.

[0191] All information, data and signals involved in this application are authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0192] Embodiments of the present application also provide a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored on any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes a method for allocating storage resources.

[0193] Embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium can be any available medium capable of being stored by a computing device, or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, or a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute a method for allocating storage resources.

[0194] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the embodiments of the present application.

[0195] All information, data and signals involved in this application are authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

Claims

1. A method for generating a document, characterized in that: The method comprises: Get initial documentation; In a case where the initial document includes an image, inputting the image into a multimodal model to obtain image description text, and combining the text in the initial document and the image description text to obtain a first text, wherein the image description text is text for describing the image; Inputting the first text into a large language model to preprocess the first text to obtain a first preprocessed text; Acquire target audio, wherein the content of the target audio includes modification suggestions for the initial document; Converting the target audio into an audio description text, wherein the audio description text is a text used to describe the target audio content; Inputting the audio description text into the large language model to preprocess the audio description text to obtain a second preprocessed text; Identify the target technical field to which the content of the first preprocessed text and the content of the second preprocessed text belong; In the correspondence between technical fields and prompt words, determining the prompt word corresponding to the target technical field; inputting the first preprocessed text, the second preprocessed text, and a prompt word corresponding to the target technical field into the large language model, so that the large language model processes the first preprocessed text and the second preprocessed text according to the prompt word to obtain an optimized text; If drawing code is detected in the optimized text, calling a drawing application corresponding to the drawing code, generating a corresponding architecture diagram and / or flowchart based on the drawing code, and replacing the drawing code in the optimized text with the architecture diagram and / or the flowchart; Generate the target document according to the specified format for the replaced optimized text.

2. The method according to claim 1, characterized in that The acquiring of the target audio includes: Receive the target audio sent by the terminal; or, Receive the target audio sent by the conference system.

3. A document generation device, characterized in that: The device comprises: A data input module is used to obtain an initial document; obtain a target audio, wherein the content of the target audio includes modification suggestions for the initial document; an image-to-text conversion module, configured to, when the initial document includes an image, input the image into a multimodal model to obtain image description text, and combine the text in the initial document with the image description text to obtain a first text, wherein the image description text is text used to describe the image; A speech-to-text conversion module, configured to convert the target audio into an audio description text, wherein the audio description text is a text used to describe the target audio content; The generation module is configured to input the first text into a large language model to preprocess the first text to obtain a first preprocessed text; input the audio description text into the large language model to preprocess the audio description text to obtain a second preprocessed text; identify the target technical field to which the content of the first preprocessed text and the content of the second preprocessed text belong; determine the prompt word corresponding to the target technical field in the correspondence between technical fields and prompt words; input the first preprocessed text, the second preprocessed text, and the prompt word corresponding to the target technical field into the large language model, so that the large language model processes the first and second preprocessed texts according to the prompt word to obtain an optimized text; if drawing code is detected in the optimized text, call a drawing application corresponding to the drawing code, generate a corresponding architecture diagram and / or flow chart based on the drawing code, and replace the drawing code in the optimized text with the architecture diagram and / or flow chart; and generate a target document using the replaced optimized text in a specified format.

4. The device according to claim 3, characterized in that The data input module is used to: Receive the target audio sent by the terminal; or, Receive the target audio sent by the conference system.

5. A computing device cluster, characterized in that: comprising at least one computing device, each computing device including a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in a memory of the at least one computing device, so that the computing device cluster executes the document generation method according to any one of claims 1 to 2.

6. A computer-readable storage medium, characterized in that The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster performs the document generation method according to any one of claims 1 to 2.

7. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster is enabled to perform the document generation method according to any one of claims 1 to 2.

Images