Information processing method and device, interaction method and device, electronic equipment and storage medium

Abstract

The invention provides an information processing method and device, an interaction method and device, electronic equipment and a storage medium, and relates to the field of computers. The information processing method comprises the following steps: acquiring reply information corresponding to input information of a user; based on a label in the reply information, identifying first information conforming to a specified grammar and second information not conforming to the specified grammar in the reply information; analyzing the first information into one or more first nodes according to the specified grammar; based on the label, generating a second node corresponding to the label for the second information; and fusing the one or more first nodes and the second node to form structured data, so that a client renders the reply information based on the structured data.

Description

Technical Field

[0001] The present disclosure relates to the field of computers, and in particular, to an information processing method, an interaction method, a device, an electronic device, and a storage medium. Background Art

[0002] With the continuous improvement of the capabilities of large models, more and more client AI (Artificial Intelligence) products can output rich and diverse content to users. For example, most AI products can output rich modal data such as pictures, videos, and audio. Incorporating multimodal data into paragraphs with natural grammar can bring a better search and question-and-answer experience to users. Summary of the Invention

[0003] According to some embodiments of the present disclosure, there is provided an information processing method, including: obtaining a reply message corresponding to user input information; based on tags in the reply message, identifying first information that conforms to a specified grammar and second information that does not conform to the specified grammar in the reply message; parsing the first information into one or more first nodes according to the specified grammar; generating, based on the tags, second nodes corresponding to the tags for the second information; and fusing the one or more first nodes and the second nodes to form structured data, so that a client can render the reply message based on the structured data.

[0004] According to some embodiments of the present disclosure, there is provided an interaction method, including: in response to user input information, obtaining a reply message corresponding to the input information; calling an information processing device to parse the reply message, the information processing device being configured to perform the above information processing method; and rendering the reply message based on the parsing result.

[0005] According to some embodiments of the present disclosure, there is provided an information processing device, including: an information acquisition module configured to obtain a reply message corresponding to user input information; a first parsing module configured to, based on tags in the reply message, identify first information that conforms to a specified grammar and second information that does not conform to the specified grammar in the reply message; a second parsing module configured to parse the first information into one or more first nodes according to the specified grammar and generate, based on the tags, second nodes corresponding to the tags for the second information; and a node fusion module configured to fuse the one or more first nodes and the second nodes to form structured data, so that a client can render the reply message based on the structured data.

[0006] According to some embodiments of the present disclosure, there is provided an information processing apparatus, including: a processor; and a memory coupled to the processor for storing instructions, which when executed by the processor, cause the processor to execute the information processing method or the interaction method of any one of the embodiments described in the present disclosure.

[0007] According to some embodiments of the present disclosure, there is provided a computer-readable storage medium having computer instructions stored thereon, wherein when the computer instructions are executed by a processor, the information processing method or the interaction method described above is implemented.

[0008] According to some embodiments of the present disclosure, there is provided a computer program product, including: computer instructions, which when executed by a processor, implement the information processing method or the interaction method described above.

[0009] Other features, aspects, and advantages of the present disclosure will become apparent from the following detailed description of the exemplary embodiments of the present disclosure with reference to the accompanying drawings. Description of the Drawings

[0010] Embodiments of the present disclosure will be described below with reference to the accompanying drawings. It should be understood that the drawings in the following description only relate to some embodiments of the present disclosure and do not constitute a limitation to the present disclosure. In the drawings:

[0011] Figure 1 A flowchart showing an information processing method according to some embodiments of the present disclosure;

[0012] Figure 2 A flowchart showing a process of identifying second information that does not conform to a specified syntax according to some embodiments of the present disclosure;

[0013] Figure 3 A flowchart showing a process of generating a second node corresponding to a tag for second information according to some embodiments of the present disclosure;

[0014] Figure 4 A flowchart showing a process of parsing a mixed tag according to some embodiments of the present disclosure;

[0015] Figure 5 A flowchart showing some embodiments of an interaction method according to some embodiments of the present disclosure;

[0016] Figure 6 A flowchart showing some embodiments of an interaction method according to other embodiments of the present disclosure;

[0017] Figure 7A A schematic diagram showing a rendering interface according to some embodiments of the present disclosure;

[0018] Figure 7BShows a schematic diagram of a rendering interface according to other embodiments of the present disclosure;

[0019] Figure 8 Shows a block diagram of an information processing apparatus according to some embodiments of the present disclosure;

[0020] Figure 9 Shows a block diagram of an electronic device according to some embodiments of the present disclosure;

[0021] Figure 10 Shows a block diagram of an electronic device according to other embodiments of the present disclosure.

[0022] It should be understood that, for the sake of convenience of description, the dimensions of the various parts shown in the drawings are not necessarily drawn according to the actual proportional relationship. The same or similar reference numerals are used in the various drawings to represent the same or similar components. Therefore, once an item is defined in one drawing, it may not be further discussed in subsequent drawings. Detailed Embodiments

[0023] Next, the technical solutions in the embodiments of the present disclosure will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present disclosure. It should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein.

[0024] It should be understood that the various steps recorded in the method embodiments of the present disclosure can be executed in different orders and / or executed in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this regard. Unless otherwise specifically stated, the relative arrangements, numerical expressions and numerical values of the components and steps set forth in these embodiments should be construed as merely exemplary and do not limit the scope of the present disclosure.

[0025] The term "comprising" and its variants used in the present disclosure mean an open term that includes at least the subsequent elements / features, but does not exclude other elements / features, that is, "including but not limited to". The term "based on" means "at least partially based on".

[0026] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order of the functions executed by these devices, modules or units or their interdependent relationships. Unless otherwise specified, the concepts such as "first" and "second" are not intended to imply that the objects so described must be in a given order in terms of time, space, ranking or any other way.

[0027] It should be noted that the modifications of "one" and "multiple" mentioned in this disclosure are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly specified in the context, it should be understood as "one or more".

[0028] The names of the messages or information exchanged between multiple devices in the embodiments of this disclosure are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0029] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. And the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0030] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings, but the present disclosure is not limited to these specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. In addition, in one or more embodiments, specific features, structures, or characteristics can be combined in any suitable manner that will be clear to those of ordinary skill in the art from the present disclosure.

[0031] In the related art, for the multi-modal data mixing scenario, considering the complexity of the parser, the usual practice is not to intrude into the parser logic, but to directly send the text data and multi-modal data to the client respectively, and then the client parses and renders the data in sequence and combines them together. For example, the text data and image data belonging to the same reply information are displayed through two message bodies.

[0032] The above solution has a low cost and is easy to implement, but it is difficult to form a standard. For example, each client and each business scenario need to design their own mixing protocols and process the data by themselves, that is, each client needs to parse the interface data by itself. And the above solution will cause a syntactic break between the front and back paragraphs of the multi-modal data, and cannot bring a coherent reading experience to the user.

[0033] The present disclosure provides an information processing method, which can shield the specific parsing details from the business side, reduce the understanding cost and parsing cost of each business end using the mixing protocol, and can bring a coherent reading experience to the user. The solution of the present disclosure will be introduced below in combination with specific embodiments.

[0034] Figure 1 A flowchart showing an information processing method according to some embodiments of the present disclosure.

[0035] As Figure 1 shown, the information processing method includes: Step S11, obtaining a response message corresponding to the user's input message; Step S12, based on the tags in the response message, identifying a first message that conforms to a specified grammar and a second message that does not conform to the specified grammar in the response message; Step S13, parsing the first message into one or more first nodes according to the specified grammar; Step S14, generating a second node corresponding to the tag for the second message based on the tag; Step S15, fusing the one or more first nodes and the second node to form structured data, so that the client can render the response message based on the structured data.

[0036] The information processing method of this embodiment is executed by a parser. Specifically, for example, it is executed by an information processing device in the parser.

[0037] For example, the large model obtains a response message by analysis according to the user's input message, and feeds back the response message to the client in the form of a specified grammar. The client sends the response content to the parser, and the parser parses the response content. After the parser parses the response message, the renderer renders the parsed content.

[0038] The parser identifies the content other than the tag as the first message. The first message is, for example, basic information such as text information. The second message is, for example, custom information such as images, videos, and interactive cards. The first node is, for example, a basic node such as a text node, and the second node is, for example, a custom node.

[0039] In the above embodiment, the first message that conforms to the specified grammar and the second message that does not conform to the specified grammar in the response content are respectively generated into corresponding nodes, and the nodes are fused to form structured data, so that each client can render based on the structured data, realizing the display of multimodal data in one message body, reducing the understanding cost and parsing cost of the client, and the rendering result can bring a coherent reading experience to the user, improving the richness and expandability of the multimodal Q&A scenario.

[0040] Next, in combination with Figures 2 to 4 , the information processing method of the embodiments of the present disclosure will be further described.

[0041] Figure 2 A flowchart showing the process of identifying a second message that does not conform to a specified grammar according to some embodiments of the present disclosure.

[0042] In this embodiment, the tag includes a specified character pair, a tag name between the specified character pairs, and a key-value pair. As Figure 2As shown, the recognition of the second information that does not conform to the specified syntax in step S12 includes: step S121, in response to the reply information including the specified character pair and the tag name satisfying the specified form, identifying the key-value pair after the tag name as the second information.

[0043] The tag is, for example, an Html (HyperText Markup Language) tag, and the specified character pair is, for example, "<>". The parser can traverse the characters in the reply information. If the character "<" is encountered, it tries to parse the characters after "<" to find the end position of the tag, such as finding the character ">". Parse the information inside the tag. If the tag name satisfies the specified form, it means that the key-value pair inside the tag is custom information. At this time, the second information can be identified.

[0044] The parser can identify the information before "<>" and the information after "<>" as the first information.

[0045] After identifying the first information and the second information, it is necessary to parse the first information and the second information into corresponding nodes. Taking the Markdown syntax as an example for the specified syntax, for the first information that conforms to the Markdown syntax, it is parsed into one or more first nodes in the syntax tree according to the Markdown syntax structure.

[0046] The Markdown syntax tree is a data structure that describes the structure and content of a Markdown document. It decomposes the information that conforms to the Markdown syntax into a series of nodes, and each node represents a syntax element in the document, such as paragraphs, headings, lists, links, etc. This tree structure enables programs to more easily analyze, operate on, and transform Markdown documents.

[0047] The parser can parse the first information into two types of syntax tree nodes. One type is the Block Node, which is usually a structured and independent block-level element that occupies an entire line or multiple lines. The other type is the Inline Node, which is usually a small element embedded in a Block element and does not occupy a separate line.

[0048] For the second information that does not conform to the Markdown syntax, such as pictures, videos, audios, interactive cards, etc., based on the tag, generate a second node corresponding to the tag for the second information. Taking Figure 3 as an example, introduce the generation of a second node corresponding to the tag for the second information.

[0049] Figure 3A schematic flowchart showing the generation of a second node corresponding to a tag for second information according to some embodiments of the present disclosure.

[0050] As Figure 3 shown, step S14 includes: step S141, determining the type of the second node based on the tag name; step S142, identifying the modality and data in the second information based on the key-value pair; step S143, generating the second node based on the type and associating the second node with the modality and the data.

[0051] Corresponding to the type of the first node, the second node can also be divided into two types of nodes. For example, by matching the prefix of the tag name, an attempt is made to generate a custom node of the corresponding type, and then the key-value pair in the tag is further parsed to identify the modality type and custom data, and then the custom node to be generated is associated with the modality type and custom data. The custom node is mounted on the syntax tree, and based on this syntax tree, the reply information can be rendered to display the rendered reply content in a message body.

[0052] In some embodiments, the determining the type of the second node based on the tag name includes: identifying the tag name; in response to the tag name matching a block-level mixed layout tag, determining that the second node is a block-level mixed layout node, and the tag name of the block-level mixed layout tag includes a data-block parameter.

[0053] For example, the mixed layout syntax usually has several business-customized data, and the custom data can be carried by means of Html tags. Corresponding to the standard Markdown syntax elements in the related art, that is, taking into account both block-level styles and inline styles, two new Markdown syntax elements can be defined. For example, a new Markdown syntax element is defined as a block-level mixed layout tag, and the block-level mixed layout tag is, for example, <data-block type = "video" url = "xxx">. data-block is the tag name, and if the tag name in the second information matches data-block, the type of the second node is a block-level mixed layout node.

[0054] In some embodiments, the tag name of the block-level mixed layout tag further includes a first extension parameter, and the first extension parameter is used to determine the layout of the block-level mixed layout node in the rendering interface.

[0055] The business side can add a suffix to the standard custom mixed layout tag to further subdivide the mixed layout scenario. For example, "data-block" can be extended to "data-block-full-mode", or "data-block-single_column". By adding a suffix after "data-block", when rendering, the renderer can determine the layout of the block-level mixed layout node in the rendered interface according to the first extension parameter. For example, an image is displayed with its left and right sides against the page margins, that is, it fills the screen, or multiple images are displayed vertically or horizontally, etc. However, at the parsing level, the parser will still parse the tag with the added suffix as the standard mixed layout type, that is, as long as the tag name contains "data-block", the generated second node type is the block-level mixed layout node.

[0056] In some embodiments, determining the type of the second node based on the tag name includes: identifying the tag name; in response to the tag name matching an inline mixed layout tag, determining that the second node is an inline mixed layout node, and the tag name of the inline mixed layout tag includes a data-inline parameter.

[0057] For example, a new Markdown syntax element is defined as an inline mixed layout tag. The inline mixed layout tag is, for example, <data-inline type="image"url="xxx">. "data-inline" is the tag name. If the tag name in the second information matches "data-inline", the type of the second node is an inline mixed layout node.

[0058] In some embodiments, the tag name of the inline mixed layout tag further includes a second extension parameter, and the second extension parameter is used to determine the layout of the inline mixed layout node in the rendered interface.

[0059] For example, based on the specific mixed layout scenario, a suffix can be added after "data-inline". When rendering, the renderer can determine the layout of the inline mixed layout node in the rendered interface according to the second extension parameter. However, at the parsing level, the parser will still parse the tag with the added suffix as the standard mixed layout type, that is, as long as the tag name contains "data-inline", the generated second node type is the inline mixed layout node.

[0060] For the identified tag, based on the key-value pairs within the tag, determine the modality and data in the second information. For example, based on the key parameter in the key-value pair, obtain the modality; based on the value parameter in the key-value pair, obtain the data.

[0061] Take the tag <data-block type="video"url="xxx"> as an example. The parameter after type is the modality corresponding to the second information, and the parameter corresponding to url is the specific custom data corresponding to the second information. Through this tag, it can be recognized that the type of the custom data is video, and url corresponds to the address of this video.

[0062] Take the tag <data-inline type="image"url="xxx"> as an example. The parameter after type is the modality corresponding to the second information, and the parameter corresponding to url is the specific custom data corresponding to the second information. Through this tag, it can be recognized that the type of the custom data is image, and url corresponds to the address of this image.

[0063] Multiple key-value pairs can be carried in the tag to adapt to various complex scenarios.

[0064] Through the design of the above-mentioned mixed layout protocol, new Markdown syntax elements are provided, which support business-customized embedded mixed layout data, can specify the modality type and specific data of the mixed layout, enabling various businesses and scenarios to achieve the free mixed layout effect of multiple modality data through Markdown syntax.

[0065] In some embodiments, the tag name and the key-value pair are obtained by calling an Html parsing library to parse the content between the specified character pairs.

[0066] For example, the data in the mixed layout tag is parsed by calling the Gumbo parsing library. After the parser encounters the "<" character in the parsing process, it intercepts the parsing process and calls the Gumbo parsing library to parse the tag name. After parsing out the tag name, it attempts to generate a mixed layout node based on the tag name, and further calls the Gumbo parsing library to parse the data in the tag, and then associates the mixed layout node with the corresponding data.

[0067] After associating the mixed layout node with the corresponding data, it is also necessary to mount the mixed layout node to the corresponding parent node. For example, in step S15, the fusing of the one or more first nodes and the second node to form structured data includes: mounting the one or more first nodes and the second node to the syntax tree.

[0068] In this embodiment, the first node is mounted to the syntax tree according to the related art. For the second node, first determine the parent node corresponding to the second node, and then mount the second node to the parent node to form a complete syntax tree. The mixed layout nodes can be directly given in the syntax tree, which is convenient for the business side to render. For example, the parser outputs the syntax tree to the client, and the client can render the reply information according to the syntax tree. For example, according to the type of the second node, it can be determined whether the second information is presented in a block style or an inline style in the rendered page, as well as the specific presentation position. In addition, the presentation size, dimensions, arrangement method, etc. can also be determined according to the extension parameters in the tag name corresponding to the second node.

[0069] Next, taking a specific example as an illustration, the mixed layout parsing of multi-modal data will be further introduced.

[0070] Figure 4 The flowchart showing the process of parsing mixed layout tags according to some embodiments of the present disclosure is shown. This embodiment is executed by a parser. Based on CMark-GFM, two new Cmark nodes are added to the parser, for example, CMARK_NODE_DATA_BLOCK (block-level mixed layout node) and CMARK_NODE_DATA_INLINE (inline mixed layout node). At the same time, the Gumbo parsing library is used to parse the data in the mixed layout tag. This embodiment includes steps S41 to S410.

[0071] In step S41, receive the reply information sent by the client.

[0072] In step S42, parse line by line.

[0073] The block-level mixed layout tag and the inline mixed layout tag belong to Html syntax tags. Therefore, from the parsing level, both of these tags are parsed inline. The present disclosure does not further limit how to perform the inline parsing process.

[0074] In step S43, in the process after inline parsing, obtain the current character.

[0075] In step S44, determine whether the current character is "<". If so, execute step S45; otherwise, execute step S43.

[0076] In step S45, determine the position where the Html tag ends. That is, find the character ">".

[0077] In step S46, call the Gumbo parsing library to parse the tag name of the Html tag.

[0078] In step S47, determine whether the tag name prefix meets the mixed layout protocol. If so, execute step S48; otherwise, end the mixed layout parsing process.

[0079] For example, identify whether the tag name starts with data-inline or data-block. If so, execute step S48; otherwise, end the mixed layout parsing process and execute the existing parsing process.

[0080] In step S48, generate the corresponding mixed layout node.

[0081] For example, if the tag name starts with data-inline, generate a CMARK_NODE_DATA_INLINE node; if the tag name starts with data-block, generate a CMARK_NODE_DATA_BLOCK node.

[0082] In step S49, call the Gumbo parsing library to parse the parameters of the Html tag and associate them with the mixed layout node.

[0083] In step S410, mount the mixed layout node to the parent node, and then end the mixed layout parsing process.

[0084] In this step, by mounting the mixed layout node to the parent node, a syntax tree is generated.

[0085] In the above embodiment, the parser supports mixed layout syntax parsing, shields the specific parsing details from the business side, and can directly provide the mixed layout node through the syntax tree for the business side to render, reducing the understanding cost and parsing cost of using the mixed layout protocol on each end.

[0086] The parsing logic of the above custom mixed layout protocol is an extension of the standard Markdown syntax. Considering that some scenarios do not require this extension, to avoid unnecessary performance loss, the above logic can be made pluggable, and the business side can control whether to start this logic. The following takes Figure 5 as an example to introduce this scenario.

[0087] Figure 5 Shows a flowchart of some embodiments of an interaction method according to some embodiments of the present disclosure.

[0088] As Figure 5 shown, the interaction method includes: step S51, in response to the user's input information, obtain the reply information corresponding to the input information; step S52, call the information processing device to parse the reply information, and the information processing device is configured to execute the above information processing method; step S53, render the reply information based on the parsing result.

[0089] This embodiment is executed by the business side, and the business side is the client.

[0090] The information processing device exists in the form of a plugin. If the user starts the information processing device, the parser executes the above-mentioned information processing method. If the user does not start the information processing device, the parser parses information according to the existing process. This embodiment can achieve flexible configuration of the parser's mixing ability.

[0091] For example, when the user starts the information processing device, the large model analyzes the user's input information to obtain a reply message, and feeds back the reply message to the client in a specified syntax form. The client calls the information processing device and sends the reply message to the information processing device. The information processing device parses the reply message, and then sends the parsing result to the client. The client renders the reply message based on the parsing result so as to display the rendered reply message in a message body, enhancing the richness and expandability of the multi-modal Q&A scenario, and further enhancing the user's favorability and trust in the product.

[0092] Figure 6 The flowchart of some embodiments of the interaction method according to some other embodiments of the present disclosure is shown. As Figure 6 shown, this embodiment includes steps S61 to S67. The parser includes a Core module, a Data-Inline&Data-Block module, and a Gumbo parsing library. The Core realizes the interaction between the parser and the client, and the Data-Inline&Data-Block module and the Gumbo parsing library are two plugins.

[0093] In step S61, the client sends an initialization plugin request to the Core.

[0094] In step S62, the Core sends initialization messages to the Data-Inline&Data-Block module and the Gumbo parsing library respectively. This step can also be the plugin creation process.

[0095] In step S63, the client sends the reply message sent by the server to the Core.

[0096] In step S64, the Core notifies the Data-Inline&Data-Block module to try to parse the reply message into a mixed layout node.

[0097] In step S65, the Data-Inline&Data-Block module calls the Gumbo parsing library to implement Html parsing.

[0098] In step S66, the Data-Inline&Data-Block module generates and mounts the mixed layout node, and sends the generated syntax tree to the Core.

[0099] In step S67, Core sends the parsing result to the client for the client to perform information rendering.

[0100] The interface rendered by the client based on the syntax tree is as Figure 7A and 7B shown. Figure 7A FIG. shows a schematic diagram of a rendered interface according to some embodiments of the present disclosure, Figure 7B FIG. shows a schematic diagram of a rendered interface according to other embodiments of the present disclosure. Figure 7A Image A in is the presentation form of a block-level mixed layout node in the rendered interface. This image A does not follow the text indentation connected to it, but Figure 7A the text and the image are presented in the same message body 71. Figure 7B Image B and FIG. C in are the presentation forms of inline mixed layout nodes in the rendered interface. This image B and image C respectively follow the text indentation connected to them, but Figure 7B the text and the image are presented in the same message body 72.

[0101] The above is the information processing method and interaction method provided by the embodiments of the present disclosure. The following combines Figure 8 to describe an information processing device according to an embodiment of the present disclosure, which is used to execute any one of the above information processing methods.

[0102] Figure 8 FIG. shows a block diagram of an information processing device according to some embodiments of the present disclosure.

[0103] As Figure 8 shown, the information processing device includes an information acquisition module 81, a first parsing module 82, a second parsing module 83, and a node fusion module 84.

[0104] The information acquisition module 81 is configured to acquire a reply message corresponding to the user's input message; the first parsing module 82 is configured to identify first information that conforms to a specified syntax and second information that does not conform to the specified syntax in the reply message based on tags in the reply message; the second parsing module 83 is configured to parse the first information into one or more first nodes according to the specified syntax, and generate second nodes corresponding to the tags for the second information based on the tags; the node fusion module 84 is configured to fuse the one or more first nodes and the second nodes to form structured data, so that the client can render the reply message based on the structured data.

[0105] In some embodiments, the tag includes a specified character pair, a tag name between the specified character pairs, and a key-value pair. The first parsing module 82 is configured to, in response to the reply information including the specified character pair and the tag name satisfying a specified form, identify the key-value pair after the tag name as the second information.

[0106] In some embodiments, the second parsing module 83 is configured to determine the type of the second node based on the tag name; identify the modality and data in the second information based on the key-value pair; generate the second node based on the type, and associate the second node with the modality and the data.

[0107] In some embodiments, the second parsing module 83 is configured to identify the tag name; in response to the tag name matching a block-level mixed layout tag, determine that the second node is a block-level mixed layout node, and the tag name of the block-level mixed layout tag includes a data-block parameter.

[0108] In some embodiments, the tag name of the block-level mixed layout tag further includes a first extension parameter, and the first extension parameter is used to determine the layout of the block-level mixed layout node in the rendering interface.

[0109] In some embodiments, the second parsing module 83 is configured to identify the tag name; in response to the tag name matching an inline mixed layout tag, determine that the second node is an inline mixed layout node, and the tag name of the inline mixed layout tag includes a data-inline parameter.

[0110] In some embodiments, the tag name of the inline mixed layout tag further includes a second extension parameter, and the second extension parameter is used to determine the layout of the inline mixed layout node in the rendering interface.

[0111] In some embodiments, the second parsing module 83 is configured to obtain the modality based on the key parameter in the key-value pair; obtain the data based on the value parameter in the key-value pair.

[0112] In some embodiments, the tag name and the key-value pair are obtained by parsing the content between the specified character pairs by invoking a Hypertext Markup Language (Html) parsing library.

[0113] In some embodiments, the node fusion module 84 is configured to mount the one or more first nodes and the second node to a syntax tree.

[0114] In the above embodiments, the first information that conforms to the specified grammar and the second information that does not conform to the specified grammar in the reply content are respectively generated into corresponding nodes, and the nodes are fused to form structured data, so that each client can render based on the structured data, realizing the display of multi-modal data in one message body, reducing the understanding cost and parsing cost of the client, and the rendering result can bring a coherent reading experience to the user, improving the richness and expandability of the multi-modal question-and-answer scenario.

[0115] It should be noted that the above-mentioned respective units are only logical modules divided according to their specific implemented functions, rather than for restricting specific implementation manners. For example, they can be implemented in software, hardware, or a combination of software and hardware. In actual implementation, the above-mentioned respective units can be implemented as independent physical entities, or can also be implemented by a single entity (for example, a processor (such as a CPU or a DSP, etc.), an integrated circuit, etc.). In addition, the above-mentioned respective units are shown by dashed lines in the drawings indicating that these units may not actually exist, and the operations / functions they implement can be implemented by the processing circuit itself.

[0116] Figure 9 A block diagram of an electronic device according to some embodiments of the present disclosure is shown.

[0117] As Figure 9 shown, the electronic device includes a processor; and a memory coupled to the processor for storing instructions, which when executed by the processor cause the processor to execute the information processing method as described above, or the interaction method as described above.

[0118] The memory 91 is used for storing one or more computer-readable instructions. The memory 91 may include any combination of various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory, including but not limited to random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), flash memory. The memory 91 can store, for example, an operating system, application programs, a boot loader (BootLoader), a database, and other programs, and can also store various application programs and various data, etc.

[0119] The processor 92 is used for running the computer-readable instructions to implement the information processing method described in any one of the foregoing embodiments or the interaction method described in any one of the foregoing embodiments. The specific implementation of each step of the method can be referred to the above embodiments, and the repeated parts are not elaborated herein.

[0120] The processor 92 can be configured to execute Figure 1to the steps in FIGS. 1 to 7. The processor 92 can be embodied as various processing devices, such as a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The central processing unit (CPU) can be of X86 or ARM architecture, etc.

[0121] The processor 92 and the memory 91 can communicate with each other directly or indirectly. For example, the processor 92 and the memory 91 can communicate through a network. The network can include a wireless network, a wired network, and / or any combination of a wireless network and a wired network. The processor 92 and the memory 91 can also communicate with each other through a system bus, and the present disclosure does not limit this.

[0122] It should be noted that Figure 9 The components of the electronic device 9 shown are exemplary rather than restrictive. According to actual application needs, the electronic device 9 can also have other components. The processor 92 can control other components in the electronic device 9 to perform desired functions.

[0123] The electronic device 9 can be implemented in a software, firmware, and / or hardware manner and can be integrated in a device installed with relevant application programs.

[0124] This electronic device can reduce the understanding cost and parsing cost of the client, and the rendering result can bring a coherent reading experience to the user, improving the richness and expandability of the multi-modal question and answer scenario.

[0125] Figure 10 A block diagram of an electronic device according to other embodiments of the present disclosure is shown.

[0126] Figure 10 The electronic device 10 shown can be a computer system with a dedicated hardware structure and can perform corresponding functions when installed with relevant application programs.

[0127] The electronic device includes but is not limited to mobile terminals such as smart phones, laptop computers, personal digital assistants (PDAs), tablet personal computers (Tablet PCs), portable multimedia players (PMPs), vehicle terminals (such as vehicle navigation terminals), wearable devices, etc., and fixed terminals such as digital TVs, desktop computers, etc.

[0128] As Figure 10As shown, the central processing unit (CPU) 101 executes various processes according to the programs stored in the read-only memory (ROM) 102 or the programs loaded from the storage section 108 into the random access memory (RAM) 103. In the RAM 103, data required when the CPU 101 executes various processes and the like is stored as needed. The central processing unit is merely exemplary, and it may also be other types of processors, such as the various processors described above. The ROM 102, the RAM 103, and the storage section 108 may be various forms of computer-readable storage media. It should be noted that although Figure 10 the ROM 102, the RAM 103, and the storage section 108 are shown separately, one or more of them may be combined or located in the same or different memories or storage modules.

[0129] The CPU 101, the ROM 102, and the RAM 103 are connected to each other via a bus 104. An input / output interface 105 is also connected to the bus 104.

[0130] The following components are connected to the input / output interface 105: an input section 106, such as a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output section 107, including a display, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage section 108, including a hard disk, a magnetic tape, etc.; and a communication section 109, including a network interface card such as a LAN card, a modem, etc. The communication section 109 allows communication processing to be performed via a network such as the Internet. It is easily understood that although Figure 10 the parts in the electronic device 10 are shown to communicate via the bus 104, they may also communicate via a network or other means, where the network may include a wireless network, a wired network, and / or any combination of a wireless network and a wired network.

[0131] As needed, a drive 1010 is also connected to the input / output interface 105. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is mounted on the drive 1010 as needed, so that the computer program read therefrom is installed in the storage section 108 as needed.

[0132] In the case where the above series of processes are implemented by software, the programs constituting the software can be installed from a network such as the Internet or a storage medium such as the removable medium 1011.

[0133] According to an aspect of the present disclosure, there is provided a computer-readable storage medium having computer instructions stored thereon, wherein when the instructions are executed by a processor, the above-described information processing method or interaction method is implemented.

[0134] The computer-readable storage medium can enhance the richness and expandability of the multi-modal question-and-answer scenario.

[0135] According to one aspect of the present disclosure, there is provided a computer program product, including computer instructions that, when executed by a processor, implement the information processing method or interaction method described above.

[0136] The computer program product can enhance the richness and expandability of the multi-modal question-and-answer scenario.

[0137] According to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product that, when running on a computer, causes the computer to implement the method described in any of the foregoing embodiments. The computer program product includes computer instructions carried on a computer-readable medium, including program code for executing the method shown in the flowchart. In such an embodiment, the computer instructions can be downloaded and installed from the network through the communication section 109, or installed from the storage section 108, or installed from the ROM 102. When the computer program is executed by the CPU 101, the method of the embodiment of the present disclosure is executed.

[0138] It should be noted that, in the context of the present disclosure, the computer-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0139] The computer-readable medium can be a computer-readable storage medium, or a computer-readable signal medium, or any combination of the two. The computer-readable storage medium is a non-transitory computer storage medium.

[0140] The computer-readable storage medium includes, but is not limited to: electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device. Computer instructions are stored on the computer-readable storage medium, and when executed by a processor, implement the method described in any of the foregoing embodiments.

[0141] A computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, carrying computer-readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.

[0142] The foregoing computer-readable medium may be included in the aforementioned electronic device; or may exist separately without being assembled into the electronic device.

[0143] In some embodiments, a computer program is also provided, including: instructions that, when executed by a processor, cause the processor to execute the method described in any of the foregoing embodiments. For example, the instructions may be embodied as computer program code.

[0144] In embodiments of the present disclosure, computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof. The foregoing programming languages include but are not limited to object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., by using an Internet service provider to connect through the Internet).

[0145] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0146] The functions described above can be performed, at least in part, by one or more hardware logic components. By way of example and not limitation, exemplary hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0147] Although some specific embodiments of the present disclosure have been described in detail by way of example, those skilled in the art should understand that the above examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Those skilled in the art should understand that the above embodiments can be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. An information processing method, comprising: Obtaining a response message corresponding to the user's input message; Based on the tags in the response message, identifying first information that conforms to a specified syntax and second information that does not conform to the specified syntax in the response message; Parsing the first information into one or more first nodes according to the specified syntax; Based on the tags, generating second nodes corresponding to the tags for the second information; Fusing the one or more first nodes and the second nodes to form structured data, so that a client can render the response message based on the structured data.

2. The information processing method according to claim 1, wherein, The tags include specified character pairs, a tag name between the specified character pairs, and key-value pairs. Identifying the second information that does not conform to the specified syntax in the response message includes: In response to the response message including the specified character pair and the tag name satisfying a specified form, identifying the key-value pair after the tag name as the second information.

3. The information processing method according to claim 2, wherein, Generating the second nodes corresponding to the tags for the second information includes: Based on the tag name, determining the type of the second node; Based on the key-value pair, identifying the modality and data in the second information; Generating the second node based on the type and associating the second node with the modality and the data.

4. The information processing method according to claim 3, wherein, Based on the tag name, determining the type of the second node includes: Identifying the tag name; In response to the tag name matching a block-level mixed layout tag, determining the second node as a block-level mixed layout node, and the tag name of the block-level mixed layout tag includes data-block parameters.

5. The information processing method according to claim 4, wherein, The tag name of the block-level mixed layout tag further includes a first extension parameter, and the first extension parameter is used to determine the layout of the block-level mixed layout node in the rendered interface.

6. The information processing method according to claim 3, wherein Based on the tag name, determining the type of the second node includes: Identifying the tag name; In response to the tag name matching an inline mixed layout tag, determining the second node as an inline mixed layout node, and the tag name of the inline mixed layout tag includes data-inline parameters.

7. The information processing method according to claim 6, wherein, The tag name of the inline mixed layout tag further includes a second extension parameter, and the second extension parameter is used to determine the layout of the inline mixed layout node in the rendered interface.

8. The information processing method according to claim 3, wherein, Based on the key-value pair, identifying the modality and data in the second information includes: Based on the key parameter in the key-value pair, obtaining the modality; Based on the value parameter in the key-value pair, obtaining the data.

9. The information processing method according to claim 2, wherein The tag name and the key-value pair are obtained by calling a HyperText Markup Language (Html) parsing library to parse the content between the specified character pairs.

10. The information processing method according to any one of claims 1 to 9, wherein, Fusing the one or more first nodes and the second nodes to form structured data includes: Mounting the one or more first nodes and the second nodes to a syntax tree.

11. An interaction method, comprising: In response to the user's input message, obtaining a response message corresponding to the input message; Invoking an information processing device to parse the response message, and the information processing device is configured to execute the information processing method according to any one of claims 1 to 10. Render the reply information based on the parsing result.

12. An information processing apparatus, comprising: an information acquisition module configured to acquire a reply information corresponding to a user's input information; a first parsing module configured to identify, based on tags in the reply information, a first information that conforms to a specified syntax and a second information that does not conform to the specified syntax in the reply information; a second parsing module configured to parse the first information into one or more first nodes according to the specified syntax, and generate second nodes corresponding to the tags for the second information based on the tags; a node fusion module configured to fuse the one or more first nodes and the second nodes to form structured data, so that a client can render the reply information based on the structured data.

13. An electronic device, comprising: a processor; and a memory coupled to the processor for storing instructions, which when executed by the processor, cause the processor to execute the information processing method according to any one of claims 1 to 10, or the interaction method according to claim 11.

14. A computer-readable storage medium having computer instructions stored thereon, wherein, When the computer instructions are executed by the processor, the information processing method according to any one of claims 1 to 10, or the interaction method according to claim 11 is implemented.

15. A computer program product, comprising: Including computer instructions, which when executed by the processor, implement the information processing method according to any one of claims 1 to 10, or the interaction method according to claim 11.

Images