Information processing method and electronic device

Abstract

This application discloses an information processing method and an electronic device. The method includes: obtaining input information; obtaining first data based on the input information; the first data including multiple text blocks; determining a first prompt word, the first prompt word being used to reflect the order of the multiple text blocks, and guiding the multiple steps generated by the model to maintain consistency with the order of the text blocks used for the generation steps in the order of sorting; inputting the input information, the first data and the first prompt word into the model to obtain output information; the output information including at least multiple steps.

Description

Technical Field

[0001] This application relates to the field of information processing technology, and in particular to an information processing method and an electronic device. Background Technology

[0002] To improve the accuracy of model inference, before the model generates an answer, a retrieval-enhanced generative model can be used to retrieve information related to the question from the database, and then the retrieved information can be used to guide the model in generating the answer.

[0003] However, in the current process of generating answer results based on multiple retrieved pieces of information, the model often fails to effectively utilize these multiple retrieved pieces of information, resulting in low accuracy of the inferred answer results. Summary of the Invention

[0004] On the one hand, this application provides an information processing method, including:

[0005] Obtain input information;

[0006] First data is obtained based on the input information; the first data includes multiple text blocks.

[0007] A first prompt word is determined, which is used to reflect the order of the multiple text blocks, and the order of the multiple steps generated by the model is consistent with the order of the text blocks used to generate the steps in the order of the sorting.

[0008] The input information, the first data, and the first prompt word are input into the model to obtain output information; the output information includes at least the aforementioned steps.

[0009] In one possible implementation, the information processing method further includes:

[0010] Obtain the graph identifier bound to the text block;

[0011] A second prompt word is determined, which is used to indicate the image identifier bound to the text block, and the image identifier is used to identify an image;

[0012] The step of inputting the input information, the first data, and the first prompt word into the model includes:

[0013] Input the input information, the first data, the first prompt word, and the second prompt word into the model;

[0014] The output information further includes: the graph identifier bound to the step, which is used to characterize the image to be inserted in the step.

[0015] In yet another possible implementation, obtaining the graph identifier bound to the text block includes:

[0016] Obtain second data, which includes: candidate image identifiers associated with each of the plurality of text blocks;

[0017] If the total number of candidate graph identifiers associated with the multiple text blocks exceeds a set number, the graph identifiers to be bound to the text blocks are selected from the candidate graph identifiers associated with the text blocks based on the set number.

[0018] If the total number of candidate graph identifiers associated with the multiple text blocks does not exceed the set number, the candidate graph identifier associated with the text block is determined as the graph identifier bound to the text block.

[0019] In another possible implementation, the step of filtering the graph identifiers to be bound to the text block from at least one candidate graph identifiers associated with the text block based on the set number includes:

[0020] The function type of the text block associated with the candidate image identifier, the image function category corresponding to the image represented by the candidate image identifier, and the degree of correlation between the candidate image identifier and the input information are determined, wherein the function type is used to characterize whether the text block belongs to the step content;

[0021] Based on the function type, image function category, and relevance of the candidate image identifier, an importance score for the candidate image identifier is determined.

[0022] Based on the importance score of the candidate image identifiers, the set number of candidate image identifiers are selected.

[0023] The selected candidate graph identifiers are determined as the graph identifiers bound to the text blocks associated with the candidate graph identifiers.

[0024] In yet another possible implementation, determining the first prompt word includes:

[0025] Obtain the function type and source location information of the text block, wherein the function type is used to characterize whether the text block belongs to the step content, and the source location information is used to characterize the position of the text block in its source document;

[0026] Based on the source location information of the text blocks and the function type of the text blocks, determine the order of at least one target text block belonging to the step content among the plurality of text blocks;

[0027] Generate a first prompt word, which is used to characterize the order of the at least one target text block.

[0028] In another possible implementation, the candidate graph identifier associated with the text block is obtained as follows:

[0029] Obtain the text metadata of the text block, the candidate image identifiers of multiple candidate images, and the image metadata of the candidate images. The text metadata includes at least one of the following: the page position of the text block in the source document, the page coordinates of the text block in the page of the source document, the image number referenced by the text block, the function type of the text block, and the semantic features of the text block. The image metadata includes at least one of the following: the page position of the candidate image in the source document, the page coordinates of the candidate image in the page of the source document, the image number corresponding to the candidate image, and the semantic features of the candidate image.

[0030] Determine the degree of association between the candidate image and the text block;

[0031] The candidate image identifiers of candidate images that meet the correlation requirements are determined as the candidate image identifiers associated with the text block;

[0032] The degree of correlation is determined based on at least one of the following:

[0033] The positional difference between the page position of the candidate image and the page position of the text block;

[0034] The coordinate difference between the page coordinates of the candidate image and the page coordinates of the text block;

[0035] Whether the image number of the candidate image belongs to the image number referenced in the text block;

[0036] Does the function type of the text block indicate whether the text block belongs to the step content?

[0037] The similarity between the semantic features of the candidate image and the semantic features of the text block.

[0038] In another possible implementation, the semantic features of the candidate image are generated based on at least one of the following: the vector encoding of the candidate image, the title content associated with the candidate image, and the text identified from the candidate image.

[0039] In yet another possible implementation, the information processing method further includes:

[0040] Based on the graph identifier bound in the above steps, the target image to be inserted in the above steps is obtained;

[0041] The target image is inserted into the step of adding the output information.

[0042] In another aspect, this application also provides an electronic device, including: an input unit, a processor, and a display unit;

[0043] The input unit is used to obtain input information;

[0044] The processor is configured to obtain first data based on the input information; the first data includes multiple text blocks; determine a first prompt word, the first prompt word being used to reflect the order of the multiple text blocks, guiding the multiple steps generated by the model to maintain consistency with the order of the text blocks used to generate the steps in the order; input the input information, the first data, and the first prompt word into the model to obtain output information; the output information includes at least the multiple steps.

[0045] The display unit is used to display the output information.

[0046] In one possible implementation, the processor is further configured to: obtain the graph identifier bound to the text block; determine a second prompt word, the second prompt word being used to indicate the graph identifier bound to the text block, the graph identifier being used to identify an image;

[0047] When the processor inputs the input information, the first data, and the first prompt word into the model, it is specifically used to: input the input information, the first data, the first prompt word, and the second prompt word into the model;

[0048] The output information further includes: the graph identifier bound to the step, which is used to characterize the image to be inserted in the step. Attached Figure Description

[0049] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0050] Figure 1 A flowchart illustrating the information processing method provided in this application;

[0051] Figure 2 Another flowchart illustrating the information processing method provided in this application;

[0052] Figure 3 Another flowchart illustrating the information processing method provided in this application;

[0053] Figure 4 Another flowchart illustrating the information processing method provided in this application;

[0054] Figure 5 This is a schematic diagram illustrating one implementation process for determining candidate graph identifiers associated with text blocks in this application;

[0055] Figure 6 A schematic diagram of the composition structure of an electronic device provided in this application. Detailed Implementation

[0056] The solution proposed in this application is applicable to any scenario where the model is invoked to infer the answer information corresponding to the input information. The solution proposed in this application can improve the accuracy of the model in generating answer information based on the retrieved search information.

[0057] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is only for explaining specific embodiments and is not intended to limit the application. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0058] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0059] like Figure 1 This illustration shows a flowchart of an information processing method provided in this application. The method in this embodiment can be applied to electronic devices, such as mobile phones and desktop computers, as well as host devices, servers, or device nodes in cloud platforms; no specific limitations are imposed. In this application, the electronic device can run or invoke a model to process the obtained input information.

[0060] The method in this embodiment may include:

[0061] S101, obtain input information.

[0062] The input information can be information that the electronic device needs to process using a model, such as user-submitted questions, chat conversations, or images to be analyzed. Alternatively, the input information can also be derived from user-submitted questions or chat conversations, with no specific restrictions.

[0063] In this application, the type of input information is not limited, such as text, voice, or images.

[0064] S102, based on the input information, first data is obtained, the first data including multiple text blocks.

[0065] The first data refers to data retrieved based on the input information. Correspondingly, the text blocks within the first data are the retrieved text blocks that match the input information.

[0066] For example, based on input information, multiple text blocks are retrieved from the database using retrieval enhancement techniques, and the similarity between the vector features of each of these multiple text blocks and the vector features of the input information exceeds a set threshold.

[0067] In this application, a text block is data composed of text content. For example, a text block can be a text statement or a text paragraph. Furthermore, a text block can consist of continuous text content extracted or split from a document.

[0068] S103, determine the first prompt word.

[0069] The first prompt word is used to reflect the sorting of the multiple text blocks in the first data, and the order of the multiple steps used to guide the model generation is consistent with the order of the text blocks used in the generation steps in the sorting.

[0070] In this application, the order of multiple text blocks is the inherent sequential relationship of the multiple text blocks themselves. The order of the multiple text blocks is related to the attributes of the text blocks themselves (such as the attribute features recorded in the metadata). Therefore, the order of the multiple text blocks is the inherent sorting order of the multiple text blocks and is not related to the matching degree between each text block and the input information.

[0071] For example, in this application, text blocks can be extracted or split from the source document to which the text block belongs. Therefore, different text blocks have different page positions, paragraph positions, and page positions in the source document, resulting in different text blocks having different order. This gives different text blocks an inherent order that is not affected by the input information.

[0072] This application's research found that multiple text blocks obtained based on input information usually have an inherent sorting order. However, multiple text blocks are generally spliced ​​or organized according to their matching degree (or similarity) with the input information. Therefore, when the model processes these multiple text blocks, it is easy to combine the matching degree of multiple text blocks with the input information to organize the steps in the output information. As a result, the order of the contents of multiple text blocks in the generation steps is disordered, which affects the accuracy of the results inferred by the model.

[0073] For example: The first data block consists of three text blocks. These three text blocks describe the operation content of the three steps in the source document. For example, text block 1 describes the operation content of the first step in the device installation process, text block 2 describes the operation content of the second step in the device installation process, and text block 3 describes the operation content of the third step in the device installation process. Therefore, the order of these three text blocks should be text block 1, text block 2, and text block 3.

[0074] The order of text blocks based on their matching degree with the input information is: text block 2, text block 1, and text block 3. Therefore, after inputting text blocks 2, 1, and 3 into the model in sequence, the order of the text blocks used in the multiple steps generated by the model based on these three text blocks will be inconsistent. For example, if the model generates three steps required to install equipment, with the order of steps 1, 2, and 3, where step 1 is generated based on text block 2, step 2 is generated based on text block 1, and step 3 is generated based on text block 3, then the order of the equipment installation operations described by these three generated steps will inevitably be incorrect, leading to low accuracy in the model's generated results.

[0075] Therefore, in order to avoid the order of multiple steps generated by the model being disordered with the order of the text blocks on which the steps depend, this application uses a first prompt word to indicate the order of the multiple text blocks. This guides the multiple steps generated by the model to maintain the order of the text blocks used to generate the step in the same order, thereby reducing the possibility of the model organizing multiple text blocks in disorder.

[0076] The phrase "the order of the multiple steps in the model generation guided by the first prompt word is consistent with the order of the text blocks used in the generation steps in the sorting" means that the order of any two steps in the model generation is consistent with the order of the text blocks used in those two steps in the sorting of multiple text blocks.

[0077] For example, step a of the model generation is based on text block A, while step b is based on text block B. In the sorting of multiple text blocks, text block A comes first, followed by text block B. Text blocks A and B can be either adjacent in order or non-adjacent. For instance, text block A might be the 3rd in the sort, while text block B could be the 4th or 5th.

[0078] Based on this, in the order of the multiple steps generated by the model, step a should be placed before step b. Of course, step a and step b can be two adjacent steps or two non-adjacent steps.

[0079] It should be noted that the number of subsequent steps generated by the model can be the same as or different from the number of text blocks. For example, each step can be generated based on one or more text blocks. However, when the text blocks are ordered, the model uses at least one text block that is adjacent in order among the ordered text blocks to generate a step. Based on this, when guiding the model to generate multiple steps based on the order of multiple text blocks, it is still possible to ensure that the order of any two steps in the multiple steps is consistent with the order of the at least one text block used in each of those two steps in the text block order.

[0080] Using the example above, suppose step a is generated based on text block A and text block C, and text block A and text block C are two adjacent text blocks that precede text block B. Then step b, which is generated based on text block B, needs to be a step that follows step a.

[0081] S104, input the input information, the first data and the first prompt word into the model to obtain the output information.

[0082] The output information includes at least multiple steps. These multiple steps are generated based on multiple text blocks in the first data. The number of steps may not be the same as the number of text blocks; for example, a certain step may be generated based on multiple text blocks.

[0083] In this application, the multiple steps in the output information are solution steps given in response to the input information. For example, if the input information is a question asking for a solution to a device malfunction, then the multiple steps in the output information are the multiple operational steps required to resolve the device malfunction. As another example, if the input information is a question inquiring about how to start the device, then the multiple steps are the startup steps to start the device.

[0084] This model is an intelligent model used to process input information. Depending on the application scenario, the model can vary and is not specifically limited. For example, it can be a large language model or other machine learning models.

[0085] As described above, after obtaining multiple text blocks based on the input information, this application determines a first prompt word and inputs it along with the input information and multiple text blocks into the model. Since this first prompt word reflects the order of the multiple text blocks, it guides the model to maintain consistency between the order of the multiple generation steps and the order of the text blocks used in the generation steps. This reduces the possibility of the order of the multiple generation steps being disordered with the order of the text blocks on which each step depends, thus reducing the likelihood of low accuracy in the generated output information due to disordered order of the text blocks on which the generation steps depend, and improving the accuracy of the output information generated by the model based on the input information.

[0086] In this application, there are several possible implementations for determining the first prompt word. For example, when generating text blocks, a sequence identifier can be added to the text blocks based on their position or extraction order. This sequence identifier represents the sorting order of the text blocks among all text blocks in the database. However, this approach may require additional analysis of the order between all text blocks, resulting in relatively high complexity.

[0087] For example, in another possible implementation, this application can determine the order of multiple text blocks based solely on their attribute information after obtaining them, thereby reducing data processing volume and lowering the complexity of determining the order of multiple text blocks. The following section discusses this possible implementation in conjunction with... Figure 2 Please provide an explanation. For example... Figure 2 This illustration shows another flowchart of an information processing method provided in this application. The method in this embodiment may include:

[0088] S201, Obtain input information.

[0089] S202, obtain first data based on input information, the first data including multiple text blocks.

[0090] S203, obtain the function type and source location information of each text block.

[0091] The function type of the text block is used to indicate whether the text block belongs to the step content.

[0092] For example, the function type of a text block can characterize the category of the function that the content of the text block plays in its source document. Therefore, based on this function category, it can be reflected whether the text block belongs to the content of the steps recorded in the source document.

[0093] For example:

[0094] The functions of text blocks can be categorized into several types, such as describing steps, titles, and notes.

[0095] If the function type of the text block is "describe step", it means that the function of the text block is to describe the action implementation of the step. Therefore, the content of the text block belongs to the text of a step, that is, the text block belongs to the step content.

[0096] If the text block is designated as a heading, it indicates that the text block contains content that belongs to the heading category in the source document.

[0097] If the function type of the text block is "Notes", it means that the text block is the content of the notes in the source document used for reminders or warnings.

[0098] The source location information of a text block is used to characterize the position of the text block within its source document.

[0099] For example, the source location information of a text block may include, but is not limited to, at least one of the following: the page number of the page in the source document, the chapter to which the text block belongs, the step number to which the text block belongs, and the page coordinates of the text block.

[0100] Understandably, in many scenarios, text blocks available for retrieval in a database may originate from the same document. For example, in equipment repair or maintenance, to accurately answer user questions about equipment, it might be necessary to split the equipment maintenance manual document and save the resulting text blocks. In this case, the text blocks originate from the same document. However, if the text blocks stored in the database originate from multiple related documents, such as the upper and lower volumes of a repair manual, then the source location information of the text block can also include the document identifier of the source document to which the text block belongs.

[0101] The function type and source location information of a text block can be stored as attribute information associated with the text block. Therefore, after obtaining the text block, the function type and source location information associated with the text block can be queried.

[0102] For example, in this application, the text block has text metadata. For instance, when storing the text block in the database, the text metadata associated with the text block is also stored. The text metadata of the text block can include at least the type of function and source location information of the text block, and may also include the document identifier of the source document to which the text block belongs, etc., without limitation.

[0103] S204, based on the source location information of each text block and the function type of each text block, determine the order of at least one target text block belonging to the step content among the multiple text blocks.

[0104] Based on the function type of each text block, the text blocks that belong to the step can be identified. For ease of distinction, the text blocks that belong to the step content are called target text blocks.

[0105] It is understandable that, for each target text block that belongs to the step content, the order of the step content described by each target text block can be determined based on the source location information of each target text block, and the order of the step content corresponding to the at least one target text block is actually the order of the at least one target text block.

[0106] For example, assuming that the source location information of each target text block indicates that each target text block comes from the same chapter, and that the source location information of each target text block includes the step number corresponding to each target text block, then the order of each target text block can be determined according to the step number corresponding to the target text block.

[0107] Of course, if the source location information of the target text blocks indicates that the target text blocks do not come from the same chapter, the page number of the target text block in the source document, the paragraph in the page, and the chapter to which it belongs can be combined to comprehensively determine the order in which the steps of each target text block appear in the source document, and the sorting of each target text block can be determined based on the order in which the steps of each target text block appear.

[0108] S205, Generate the first prompt word.

[0109] The first prompt word is used to characterize the sorting of the at least one target text block, and the order of the multiple steps used to guide the model generation is consistent with the order of the target text blocks used in the generation steps in the sorting.

[0110] For example, suppose there are three target text blocks belonging to the step content among multiple text blocks, and their order is as follows: text block b, text block a, and text block c. Then the first prompt word can be: When reasoning about the output result, it is necessary to keep the order of text block b, text block a, and text block c unchanged in the step.

[0111] S206, input the input information, the first data and the first prompt word into the model to obtain the output information.

[0112] The output information includes at least several steps.

[0113] In this embodiment, after obtaining multiple text blocks, the order of the target text blocks belonging to the step content can be determined based on the function type and source location information of each text block, and the first prompt word can be generated based on the order. This means that when generating the first prompt word, only the relevant data of the obtained multiple text blocks need to be analyzed, without having to pay attention to and process the order of other unrelated text blocks, which can reduce the amount of data processing and improve the efficiency of generating the first prompt word.

[0114] Understandably, in many scenarios, in order to make the output information generated from the input information more intuitive and efficient in expressing the information, this application can also determine the image associated with the text block so that the output information of the model can contain the image, making the information expression of the output information more intuitive.

[0115] For example, in scenarios where it is necessary to address user-raised questions regarding equipment maintenance, repair, or installation, this application can extract text blocks from documents used for equipment repair, maintenance, or installation, while simultaneously extracting images from the documents. These images can describe machine part positioning, cable routing, or disassembly examples, and these images can help understand the information described in the text blocks.

[0116] The following is combined Figure 3 Please provide an explanation. For example... Figure 3 This illustrates another flowchart of the information processing method provided in this application. The method in this embodiment may include:

[0117] S301, obtain input information.

[0118] S302, based on the input information, first data is obtained, the first data including multiple text blocks.

[0119] S303, For each text block, obtain the graph identifier bound to that text block.

[0120] Graph identifiers are used to identify images. A graph identifier is a unique identifier for an image; therefore, each graph identifier is used to identify one image.

[0121] In this application, the graph identifier bound to the text block is used to identify an image that can help understand the content of the text block.

[0122] In this process, obtaining the graph identifier associated with a text block can involve obtaining the graph identifier of an image that is related to the text block. For example, the graph identifier of the image associated with the text block can be pre-configured based on information such as the layout position of the text block and the image in their respective source documents. Therefore, the graph identifier associated with the text block can be obtained after obtaining the text block.

[0123] It should be noted that in practical applications, since a text block in the source document may not be a configured image, it may also not have an associated image. Therefore, the image identifier bound to the text block can be empty. If the text block is associated with multiple images, then the text block can be bound to multiple image identifiers.

[0124] S304, Determine the first prompt word.

[0125] The first prompt word reflects the order of the multiple text blocks, and the order of the multiple steps generated by the model is consistent with the order of the text blocks used to generate that step in the sorting process.

[0126] For example, this application can determine the order of at least one target text block belonging to the step content, and the first prompt word can indicate the order of the at least one target text block, guiding the model to maintain consistency with the order of the target text blocks used to generate the steps in that order.

[0127] In this embodiment, the specific implementation of determining the first prompt word can be found in the relevant descriptions of other embodiments of this application, and no limitations are imposed here.

[0128] It should be noted that the order of steps S303 and S304 is not limited to... Figure 3 As shown, in practical applications, the order of steps S303 and S304 can be interchanged or executed synchronously.

[0129] S305, Identify the second prompt word.

[0130] The second prompt word is used to indicate the graph identifier bound to each text block.

[0131] Understandably, the second cue word can indicate the graph identifiers of the images associated with each text block in the model, guiding the model to determine the images that can be added in the generation step based on the graph identifiers associated with the text blocks on which the step depends.

[0132] S306, input the input information, the first data, the first prompt word, and the second prompt word into the model to obtain the output information.

[0133] The output information includes: multiple steps and graph identifiers bound to the steps, which are used to represent the image to be inserted in that step.

[0134] It is understandable that since the second prompt indicates the graph identifier bound to each text block, after the model generates a step based on the text block, it can necessarily determine that the graph identifier bound to the text block is the graph identifier bound to that step, so as to accurately determine the graph identifier of the image to be inserted in each step by combining the graph identifier bound to the text block.

[0135] This application's research found that after retrieving images associated with text blocks using retrieval enhancement techniques, directly inputting these images into the model would inevitably lead to an excessive amount of data processing required by the model due to the relatively large data volume of the images. This not only consumes resources but also reduces the model's inference efficiency. Furthermore, directly inputting the retrieved images and text blocks into the model can easily lead to confusion in the relationship between text blocks and images because the model cannot accurately determine the association between each text block and the images. This can result in the insertion of unrelated images in steps based on a particular text block, or the absence of necessary images in some important steps, ultimately leading to lower accuracy of the output information.

[0136] Based on this, after obtaining multiple text blocks based on the input information, this application does not directly obtain the images associated with the multiple text blocks, but only obtains the graph identifiers of the images bound to the text blocks, and generates a second prompt word to indicate the graph identifiers bound to each text block. Based on this, this application only needs to input the second prompt word along with the input information, the first data, and the first prompt word into the model, without needing to input the images bound to each text block into the model, thereby reducing the amount of data the model needs to process, reducing resource consumption, and improving inference efficiency.

[0137] Furthermore, since the first prompt indicates the order of each text block and the second prompt indicates the graph identifier bound to each text block, the model can avoid blindly organizing text blocks and images. This ensures that the multiple steps generated by the model based on the text blocks are consistent with the order of the multiple text blocks. It also enables the model to determine the images to be inserted in each step based on the graph identifiers bound to the text blocks on which each step depends, reducing the possibility of image insertion errors and improving the accuracy of the output information determined by the model.

[0138] It is understandable that, in order to reduce the amount of data processing in the model, this application only prompts the graph identifiers associated with each text block through the second prompt word. Accordingly, the output information of the model includes the graph identifiers bound to the steps, rather than using the images indicated by the graph identifiers as the content of the output information. Based on this, in order for the candidate to be able to display the images to be inserted in each step of the output information, after obtaining the output information, the electronic device (such as an intelligent program such as an intelligent agent in the electronic device) can also obtain the target image to be inserted in that step based on the graph identifiers bound to each step in the output information, and insert the target image into that step of the output information.

[0139] Furthermore, to accurately locate the position where the image needs to be inserted in a step, if the step is bound to a graph identifier, then the output information generated by the model includes a graph placeholder for that step. This placeholder includes the graph identifier bound to the step and represents the insertion position of the target image corresponding to the graph identifier bound to the step. Accordingly, after obtaining the target image from the database based on the graph identifier bound to the step, the target image can be inserted into the position of the graph placeholder in the output information for that step.

[0140] To better understand the benefits of this embodiment, examples are provided:

[0141] Assuming that, based on the input information, the text blocks obtained from the database include the following text blocks:

[0142] Text block c: "Step 2, open the device back cover";

[0143] Text block b: "Step 1, disconnect the power";

[0144] Text block e: "Step 4, refer to..." Figure 2 "Install new batteries";

[0145] Text block d: "Caution: Do not reverse the battery polarity...";

[0146] Text block f: "Step 3, remove the old battery".

[0147] It is clear that the order of steps expressed by these text blocks is disordered.

[0148] Suppose text block c is associated with image s1, and the graph identifier of image s1 is Fs1; text block e is associated with image s2, and the graph identifier of image s2 is Fs2. Here, image s2 is a text block from the source document... Figure 2 This refers to the image referenced in the step corresponding to text block e.

[0149] If these text blocks, along with images s1 and s2, are directly input into the model, the model may produce incorrect output results, as shown below, because it cannot accurately organize the order of these text blocks or accurately correlate them with the various images:

[0150] Step 1: Open the back cover of the device;

[0151] Step 2, remove the old battery;

[0152] Step 3, disconnect the power;

[0153] Step 4, install the new battery, as shown in the picture below:

[0154] “Image s1”;

[0155] “Image s2”.

[0156] Please note: Do not reverse the battery polarity when performing the above operations.

[0157] It is evident that the order of "disconnecting the power", "opening the device back cover" and "removing the old battery" in the multiple steps output by the model is incorrect. Moreover, the step "opening the device back cover" is a step generated based on text block c, and this step should insert image s1, but the model incorrectly uses "image s1" as a schematic diagram of the step "installing the new battery".

[0158] After adopting the solution of this application, based on the function type in each text block and the step number associated with each text block, this application can determine the order of the text blocks belonging to the step content in these multiple text blocks as follows: text block b, text block c, text block f, and text block e. Among them, text block d does not belong to the step content and does not need to participate in the sorting. Of course, the sorting order of text block d in these multiple text blocks can also be indicated.

[0159] Based on this, this application generates a first prompt word and a second prompt word. The first prompt word indicates the text block order as follows: text block b, text block c, text block f, and text block e. The second prompt word indicates that the image bound to text block c is labeled Fs1, and the image bound to text block e is labeled Fs2. Based on this, this application only needs to input the input information, each text block, and the first and second prompt words into the model. Based on this, the model will determine the order of the multiple generation steps based on the first prompt word and determine the image to be inserted in each step based on the second prompt word. Accordingly, the model's output information can be as follows:

[0160] Step 1: Disconnect the power supply;

[0161] Step 2, open the back cover of the device, as shown in Figure [Fs1];

[0162] Step 3, remove the old battery;

[0163] Step 4, install the new battery, as shown in Figure [Fs2]:

[0164] Please note: Do not reverse the battery polarity when performing the above operations.

[0165] As can be seen, the order of each step is consistent with the order in which the text blocks for each step are generated. For example, step 1 is generated based on text block b, which is the first text block in the order of the first prompt word. Therefore, the step generated based on the text block is also the first step. Similarly, step 2 is generated based on text block c, which is after text block b and before text block f. Therefore, step 2 is also after step 1 corresponding to text block b and before step 3 corresponding to text block b.

[0166] Furthermore, since text block c is bound to graph identifier Fs1, the model will use the graph represented by this graph identifier as the image to be inserted in step 2 corresponding to the text block, based on the second prompt word. This serves as a placeholder for graph identifier Fs1 in step 2. Consequently, when displaying the output information, the electronic device can insert the image s1 corresponding to graph identifier Fs1 into the placeholder position. The processing of graph identifier Fs2 is similar and will not be elaborated further.

[0167] In this embodiment, there are several ways to obtain the graph identifiers bound to the text block. For example, all pre-configured graph identifiers associated with the text block can be used as the graph identifiers bound to the text block.

[0168] In one possible implementation, to minimize the data processing volume of the model and reduce the number of less important images in the output information generated by the model, this application can also pre-set a maximum number of images allowed to be included in the output information. Based on this, this application can obtain second data, which includes candidate image identifiers associated with each of multiple text blocks. For example, based on the text blocks included in the first data, the candidate image identifiers associated with each text block can be queried from a database.

[0169] Accordingly, if the total number of candidate graph identifiers associated with multiple text blocks exceeds a set number, the graph identifier to be bound to the text block is selected from the candidate graph identifiers associated with the text block based on the set number. If the total number of candidate graph identifiers associated with the multiple text blocks does not exceed the set number, each candidate graph identifier associated with the text block is determined as the graph identifier to be bound to the text block.

[0170] The specific implementation of filtering the graph identifiers to be bound to the text block from the candidate graph identifiers associated with the text block can be unrestricted.

[0171] In one possible implementation, considering that multiple text blocks have different functions and varying importance, the meaning of binding images to these text blocks also differs. Furthermore, for the same text block, the degree of association between the image and the text block also varies. Therefore, this application can combine the attribute information of the image and the text block to reasonably select the image identifiers for the images to be bound to the text blocks. Regarding this possible implementation, the following section will discuss... Figure 4 Please provide an explanation.

[0172] like Figure 4 This illustrates another flowchart of the information processing method provided in this application. The method in this embodiment may include:

[0173] S401, obtain input information.

[0174] S402, obtain first data based on input information, the first data including multiple text blocks.

[0175] S403, determine the first prompt word.

[0176] The first prompt word is used to reflect the order of multiple text blocks, and the order of the multiple steps generated by the model is consistent with the order of the text blocks used in the generation steps.

[0177] The above steps can be found in the descriptions of other embodiments of this application, and will not be repeated here.

[0178] S404, Obtain second data, which includes candidate graph identifiers associated with each of the multiple text blocks.

[0179] Among them, the candidate image identifier associated with the text block is used to characterize the image associated with the text block.

[0180] For example, a database can store text blocks and the graph identifiers associated with those text blocks. Correspondingly, after retrieving multiple text blocks that match the input information from the database, the graph identifiers associated with each of those text blocks can be retrieved from the database separately.

[0181] In order to distinguish it from the graph identifiers that need to be bound to subsequent text blocks, this application refers to the graph identifiers associated with the text block obtained based on the text block as candidate graph identifiers.

[0182] It should be noted that in practical applications, not every text block may be associated with a candidate graph identifier. Therefore, if a text block among multiple text blocks does not have an associated graph identifier, then the candidate graph identifier associated with that text block can be empty.

[0183] S405, if the total number of candidate image identifiers associated with the multiple text blocks exceeds the set number, determine the function type of the text block associated with each candidate image identifier, the image function category corresponding to the image represented by each candidate image identifier, and the degree of correlation between each candidate image identifier and the input information.

[0184] The function type of text block association is used to indicate whether the text block belongs to the step content. For details, please refer to the previous introduction, which will not be repeated here.

[0185] Candidate image identifiers (i.e., identifiers) represent the image's functional category, which characterizes the image's function or purpose. For example, image functional categories can be categorized as schematic diagrams for illustrating content, location diagrams for indicating location, detail diagrams for showcasing details, and decorative diagrams for decorative purposes. In practical applications, the classification method and types of image functional categories will vary depending on the application scenario, and there are no specific restrictions.

[0186] The image function category of the image associated with the candidate image identifier can be obtained from the image metadata of the image associated with the candidate image identifier. Of course, it can also be obtained through other methods, without any restrictions.

[0187] The correlation between candidate image identifiers and input information reflects the relevance between the image represented by the candidate image identifier and the input information. Specifically, the correlation between candidate image identifiers and input information can be considered as the similarity between the image features (such as vector encoding) of the image represented by the candidate image identifier and the information features (such as vector encoding) of the input information.

[0188] It is understandable that the more relevant an image is to the input information, the more important it is, and the greater its role in solving the problem in the input information or in understanding the output information in response to that input information.

[0189] S406, for each candidate image identifier, the importance score of the candidate image identifier is determined based on the function type of the text block associated with the candidate image identifier, the image function category corresponding to the candidate image identifier, and the degree of relevance.

[0190] The importance score is used to characterize the importance of the image represented by the candidate image identifier to the output information generated from the input information.

[0191] For example, if the function type of the text block associated with the candidate image identifier indicates that the text block does not belong to the step type, then the necessity for the text block to be bound to the image is low, and correspondingly, the importance score of the candidate image identifier will also be low; while if the function type of the text block associated with the candidate image identifier indicates that the text block is a step type, then the importance score of the candidate image identifier will be relatively high.

[0192] Among them, the image function category corresponding to the candidate image identifier indicates that if the image represented by the candidate image identifier is a positioning image, a schematic diagram, or a disassembly image, then the importance of the candidate image identifier will be relatively high; while if the image function category corresponding to the candidate image identifier indicates that the image represented by the candidate image identifier is a decorative image, then the importance score corresponding to the candidate image identifier will be relatively low.

[0193] Similarly, the higher the relevance between the candidate image identifier and the input information, the higher its importance score will be.

[0194] In one example, type scores corresponding to different function types, function scores corresponding to different image function categories, and relevance scores corresponding to different degrees of relevance can be predefined.

[0195] Based on this, this application can determine the target type score corresponding to the function type associated with the candidate image identifier, the target function score corresponding to the image function category associated with the candidate image identifier, and the target relevance score corresponding to the degree of relevance associated with the candidate image identifier. An importance score is determined based on the target type score, the target function score, and the target relevance score. For example, according to the first weight corresponding to the function type, the second weight corresponding to the image function category, and the third weight corresponding to the degree of relevance, a weighted sum of the target function score, the target type score, and the target relevance score is calculated, and this weighted sum is determined as the importance score.

[0196] Of course, there are other ways to implement the importance score for candidate graph labels, and there are no restrictions on this.

[0197] S407, based on the importance score of each candidate graph identifier, select a set number of candidate graph identifiers from the candidate graph identifiers associated with multiple text blocks.

[0198] For example, a set number of candidate image identifiers can be selected based on their importance scores, from highest to lowest.

[0199] S408, the selected candidate graph identifier is determined as the graph identifier bound to the text block associated with the candidate graph identifier, so as to obtain the graph identifier bound to each text block.

[0200] As can be seen, the graph identifier bound to the text block is the candidate graph identifier selected from the candidate graph identifiers associated with the text block.

[0201] Furthermore, in addition to limiting the maximum number of image identifiers that can be bound to multiple text blocks, this application can also set a target number of image identifiers that can be bound to each text block. The target number can be set as needed, such as two. Based on this, after filtering out a set number of candidate image identifiers, this application can first determine the filtered candidate image identifiers as candidate image identifiers for the text blocks associated with those candidate image identifiers. If the total number of candidate image identifiers bound to a text block exceeds the target number, then based on the importance score of each candidate image identifier bound to the text block, the target number of image identifiers can be determined from the candidate image identifiers bound to the text block as the image identifiers for that text block.

[0202] For example, from the candidate graph identifiers bound to the text block, select the top number of candidate graph identifiers with the highest importance as the graph identifiers bound to the text block.

[0203] Of course, if the total number of candidate graph identifiers bound to the text block does not exceed the target number, all candidate graph identifiers bound to the text block can be determined as graph identifiers bound to the text block.

[0204] S409, if the total number of candidate graph identifiers associated with the multiple text blocks does not exceed the set number, the candidate graph identifier associated with the text block is determined as the graph identifier bound to the text block.

[0205] Specifically, if the total number of candidate graph identifiers associated with the multiple text blocks does not exceed the set number, this application may first determine the candidate graph identifiers associated with the text blocks as the candidate graph identifiers bound to the text blocks. If the total number of candidate graph identifiers bound to the text blocks exceeds the target number, then based on the importance score of each candidate graph identifier bound to the text blocks, it is necessary to determine the target number of graph identifiers as the graph identifiers bound to the text blocks from the candidate graph identifiers bound to the text blocks.

[0206] S410, determine the second prompt word.

[0207] The second prompt word is used to indicate the graph identifier bound to the text block, which is used to identify the image.

[0208] S411, input the input information, the first data, the first prompt word, and the second prompt word into the model to obtain the output information.

[0209] The output information includes multiple steps and graph identifiers bound to those steps. The graph identifiers bound to the steps are used to represent the image to be inserted within each step.

[0210] The above steps S410 and S411 can be found in the relevant descriptions of the previous embodiments, and will not be repeated here.

[0211] As can be seen, through the solution of this embodiment, this application can also reasonably control the total number of graph identifiers associated with multiple text blocks, and can control the number of images that need to be inserted in the model generation steps. This can not only reduce the amount of data that the model needs to process, but also effectively reduce the situation where too many images are inserted in unimportant steps, and reduce the number of unnecessary images in the final output.

[0212] In addition, this embodiment can also control the number of graph identifiers associated with a single text block, thereby reasonably controlling the maximum number of images that can be inserted in various steps, so as to insert only important images into the steps and simplify the complexity of the output information.

[0213] In this embodiment, the graph identifier (i.e., candidate graph identifier) ​​associated with the text block in the second data obtained by this application can be pre-configured before obtaining the input information. For example, if the second data is obtained from a database, then the graph identifier associated with the text block is pre-configured and stored in the database. The following is in conjunction with... Figure 5 Please provide an explanation. For example... Figure 5 This illustration shows a schematic diagram of an implementation process for determining candidate graph identifiers associated with text blocks in this application. The method in this embodiment may include:

[0214] S501, obtain the text metadata of the text block, the candidate image identifiers of multiple candidate images, and the image metadata of the candidate images.

[0215] It is understood that this application can obtain multiple text blocks and multiple candidate images in advance by extracting text blocks and images from one or more source documents, and determine the text metadata of each text block and the candidate metadata of each candidate image.

[0216] For example, in equipment installation or maintenance scenarios, the installation and maintenance manual can be broken down into text blocks and images can be extracted to obtain multiple text blocks and multiple candidate images.

[0217] For each text block, text metadata can be generated based on its position and layout in the source document, as well as the text content it contains. Similarly, image metadata can be generated based on the position and layout of the candidate image in the source document, the image number, and the image content features.

[0218] In one possible implementation, this application can store the split text blocks and candidate images in a database, and store the text metadata of each text block and the image metadata of the candidate images in the database.

[0219] In order to quickly identify the candidate image associated with each text block during the real-time processing of input information, this application can determine the image identifier of the candidate image associated with each text block based on the text metadata of the text block and the image metadata of the candidate image.

[0220] The text metadata of a text block includes at least one of the following:

[0221] The page location of the text block in the source document, such as the page number of the page containing the text block and / or the chapter within the page;

[0222] The page coordinates of the text block in the source document can represent the distribution area of ​​the text block in the source document;

[0223] The image number referenced by this text block reflects the image directly referenced by the text block;

[0224] The semantic features of the text block, such as obtaining the semantic features of the text block through semantic analysis;

[0225] The function type of the text block is used to indicate whether the text block is for steps, notes, or a title, as detailed in the previous section.

[0226] Of course, the text metadata of a text block can also include other information, which will not be elaborated here.

[0227] The image metadata of the candidate image may include at least one of the following:

[0228] The page location of the candidate image in the source document, such as the page number and / or the chapter within the page;

[0229] The page coordinates of the candidate image in the source document can characterize the distribution area of ​​the candidate image in the source document;

[0230] The image number corresponding to the candidate image can reflect the sequence number of the candidate image in the document, as well as whether it is referenced by other text blocks, etc.

[0231] Semantic features of candidate images.

[0232] In this embodiment, there are various possible implementations for determining the semantic features of candidate images, and no restrictions are imposed on them.

[0233] In one possible implementation, the semantic features of the candidate image can be generated from at least one of the following: vector encoding based on the candidate image, title content associated with the candidate image, and text identified from the candidate image.

[0234] Among them, technologies such as Optical Character Recognition (OCR) can be used to identify the text contained in the candidate image. If the text is identified, it can be used as the semantic feature of the candidate image.

[0235] In the source document containing the candidate image, if there is title content within a set distance range of the candidate image, then the title can be used as the title content associated with the candidate image, and the semantic features of the title content can be used as the semantic features of the candidate image.

[0236] The vector encoding of the candidate image can be obtained by encoding the candidate image using an encoding model, or by semantic encoding of the candidate image. Vector encoding can characterize the semantics expressed by the candidate image.

[0237] Of course, the image metadata of candidate images can also include the image function category of the candidate images, without any specific restrictions.

[0238] S502, for each candidate image, determine the degree of association between the candidate image and the text block.

[0239] The degree of association between the candidate image and the text block can be determined based on at least one of the following:

[0240] The positional difference between the page position of the candidate image and the page position of the text block;

[0241] The coordinate difference between the page coordinates of the candidate image and the page coordinates of the text block;

[0242] Does the image number of the candidate image belong to the image number referenced in this text block?

[0243] Does the function type of this text block indicate that the text block belongs to the step content?

[0244] The similarity between the semantic features of the candidate image and the semantic features of the text block.

[0245] The smaller the positional difference between the candidate image and the text block on the page, the higher the relevance between the content described by the text block and the candidate image, or the higher the probability that the candidate image is an image that provides supplementary explanation of the text block. Therefore, the smaller the positional difference, the higher the degree of correlation between the candidate image and the text block.

[0246] Similarly, the smaller the coordinate distance between the candidate image and the text block, the higher the correlation between them.

[0247] If the image number of a candidate image belongs to the image number referenced by a text block, it indicates that the candidate image must have a high degree of correlation with the text block, and the degree of correlation between the candidate image and the text block must also be high.

[0248] The higher the semantic feature similarity between the candidate image and the text block, the higher the probability that the content described by the text block is related to the candidate image, and the higher the degree of correlation between the two.

[0249] Specifically, if the function type of a text block indicates that the text block is part of a step, then the candidate image will have a relatively high degree of correlation with the text block. Conversely, if the function type of a text block indicates that the text block is not part of a step, then the likelihood of the text block being associated with an image is low, and therefore, the correlation between the candidate image and the text block is also relatively low.

[0250] In this application, there are no restrictions on the specific implementation of determining the degree of correlation by combining the above-mentioned information.

[0251] S503, the candidate image identifier of the candidate image that meets the correlation requirement is determined as the candidate image identifier associated with the text block.

[0252] For example, the candidate image identifiers of the top specified number of candidate images with the highest degree of association can be determined as the candidate image identifiers for text block association.

[0253] For example, candidate image identifiers of candidate images whose relevance exceeds a set threshold are identified as candidate image identifiers associated with the text block.

[0254] In this embodiment, the similarity or difference between candidate images and text blocks in multiple aspects such as semantic features, page position, and page distance is combined to comprehensively determine the degree of association between candidate images and text blocks. This allows for more accurate identification of candidate images associated with text blocks, thereby reducing the likelihood of irrelevant images being associated with text blocks.

[0255] This application also provides an electronic device in its embodiments. For example... Figure 6 As shown, a schematic diagram of the composition structure of the electronic device is presented. The electronic device includes at least an input unit 601, a processor 602, and a display unit 603.

[0256] The input unit 601 is used to obtain input information.

[0257] The processor 602 is used to obtain first data based on input information; the first data includes multiple text blocks; determine a first prompt word, the first prompt word is used to reflect the order of the multiple text blocks, and guides the multiple steps generated by the model to maintain the same order as the text blocks used for the generation steps in the order of sorting; input the input information, the first data and the first prompt word into the model to obtain output information; the output information includes at least multiple steps;

[0258] The display unit 603 is used to display output information.

[0259] Furthermore, the electronic device may also include a memory 604 for storing programs required by the processor to perform operations.

[0260] Of course, the electronic device can also have more than Figure 6 There are no restrictions on the number of components, whether more or fewer.

[0261] Furthermore, the processor is also used to obtain the graph identifier bound to the text block; determine a second prompt word, the second prompt word being used to indicate the graph identifier bound to the text block, the graph identifier being used to identify an image;

[0262] When the processor inputs the input information, the first data, and the first prompt word into the model, it is specifically used to: input the input information, the first data, the first prompt word, and the second prompt word into the model;

[0263] Correspondingly, the output information also includes: the graph identifier bound to the step, which is used to represent the image to be inserted within the step.

[0264] It is understood that the specific operations performed by the processor, input unit and display unit in the electronic device can be found in the relevant descriptions of the preceding method embodiments, and will not be repeated here.

[0265] This application also provides a computer program product, including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the information processing methods provided in this application.

[0266] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the information processing methods provided in this application.

[0267] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0268] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0269] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0270] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. An information processing method, comprising: Obtain input information; First data is obtained based on the input information; The first data includes multiple text blocks; A first prompt word is determined, which is used to reflect the order of the multiple text blocks, and the order of the multiple steps generated by the model is consistent with the order of the text blocks used to generate the steps in the order of the sorting. The input information, the first data, and the first prompt word are input into the model to obtain output information; the output information includes at least the aforementioned steps.

2. The information processing method according to claim 1 further includes: Obtain the graph identifier bound to the text block; A second prompt word is determined, which is used to indicate the image identifier bound to the text block, and the image identifier is used to identify an image; The step of inputting the input information, the first data, and the first prompt word into the model includes: Input the input information, the first data, the first prompt word, and the second prompt word into the model; The output information further includes: the graph identifier bound to the step, which is used to characterize the image to be inserted in the step.

3. The information processing method according to claim 2, wherein obtaining the graph identifier bound to the text block includes: Obtain second data, which includes: candidate image identifiers associated with each of the plurality of text blocks; If the total number of candidate graph identifiers associated with the multiple text blocks exceeds a set number, the graph identifiers to be bound to the text blocks are selected from the candidate graph identifiers associated with the text blocks based on the set number. If the total number of candidate graph identifiers associated with the multiple text blocks does not exceed the set number, the candidate graph identifier associated with the text block is determined as the graph identifier bound to the text block.

4. The information processing method according to claim 3, wherein filtering the graph identifier to be bound to the text block from at least one candidate graph identifier associated with the text block based on the set number includes: The function type of the text block associated with the candidate image identifier, the image function category corresponding to the image represented by the candidate image identifier, and the degree of correlation between the candidate image identifier and the input information are determined, wherein the function type is used to characterize whether the text block belongs to the step content; Based on the function type, image function category, and relevance of the candidate image identifier, an importance score for the candidate image identifier is determined. Based on the importance score of the candidate image identifiers, the set number of candidate image identifiers are selected. The selected candidate graph identifiers are determined as the graph identifiers bound to the text blocks associated with the candidate graph identifiers.

5. The information processing method according to claim 1, wherein determining the first prompt word includes: Obtain the function type and source location information of the text block, wherein the function type is used to characterize whether the text block belongs to the step content, and the source location information is used to characterize the position of the text block in its source document; Based on the source location information of the text blocks and the function type of the text blocks, determine the order of at least one target text block belonging to the step content among the plurality of text blocks; Generate a first prompt word, which is used to characterize the order of the at least one target text block.

6. The information processing method according to claim 3, wherein the candidate image identifier associated with the text block is obtained in the following manner: Obtain the text metadata of the text block, the candidate image identifiers of multiple candidate images, and the image metadata of the candidate images. The text metadata includes: The text block includes at least one of the following: its page position in the source document, its layout coordinates on the page of the source document, the image number referenced by the text block, the function type of the text block, and the semantic features of the text block; the image metadata includes at least one of the following: its page position in the source document, its layout coordinates on the page of the source document, the image number corresponding to the candidate image, and the semantic features of the candidate image. Determine the degree of association between the candidate image and the text block; The candidate image identifiers of candidate images that meet the correlation requirements are determined as the candidate image identifiers associated with the text block; The degree of correlation is determined based on at least one of the following: The positional difference between the page position of the candidate image and the page position of the text block; The coordinate difference between the page coordinates of the candidate image and the page coordinates of the text block; Whether the image number of the candidate image belongs to the image number referenced in the text block; Does the function type of the text block indicate whether the text block belongs to the step content? The similarity between the semantic features of the candidate image and the semantic features of the text block.

7. The information processing method according to claim 6, wherein the semantic features of the candidate image are generated based on at least one of the vector encoding of the candidate image, the title content associated with the candidate image, and the text identified from the candidate image.

8. The information processing method according to claim 2, further comprising: Based on the graph identifier bound in the above steps, the target image to be inserted in the above steps is obtained; The target image is inserted into the step of adding the output information.

9. An electronic device, comprising: Input unit, processor, and display unit; The input unit is used to obtain input information; The processor is configured to obtain first data based on the input information; the first data includes multiple text blocks; determine a first prompt word, the first prompt word being used to reflect the order of the multiple text blocks, guiding the multiple steps generated by the model to maintain consistency with the order of the text blocks used to generate the steps in the order; input the input information, the first data, and the first prompt word into the model to obtain output information; the output information includes at least the multiple steps; The display unit is used to display the output information.

10. The electronic device of claim 9, wherein the processor is further configured to: obtain an image identifier bound to the text block; determine a second prompt word, the second prompt word being used to indicate the image identifier bound to the text block, the image identifier being used to identify an image; When the processor inputs the input information, the first data, and the first prompt word into the model, it is specifically used to: input the input information, the first data, the first prompt word, and the second prompt word into the model; in, The output information also includes: the graph identifier bound to the step, which is used to characterize the image to be inserted in the step.

Images