Data processing method and device, equipment and storage medium

By obtaining and rendering plug-in definition information, the basic rendering unit is used to render the plug-in return results into an easy-to-understand format, solving the readability and accuracy of plug-in results display in large language model scenarios, and achieving user-friendly display effects.

CN119088488APending Publication Date: 2024-12-06THE FOURTH PARADIGM BEIJING TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411126289.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-15
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

In the application scenario where the large language model (LLM) is combined with plug-ins, how to effectively display the results returned by plug-ins is a key problem, and the existing technology is difficult to ensure the readability and accuracy of the results.

Method used

By obtaining the plug-in definition information and the call results returned by the plug-in, the call results are rendered into content data in a format that is easy for users to understand based on the plug-in definition information, and rendered using basic rendering units (such as pictures, tables, text rendering units, etc.), and in If necessary, perform hierarchical disassembly and rendering, and output data in the form of markdown.

Benefits of technology

It realizes the readability and reliability of plug-in call results, improves the user experience, and ensures the accuracy and readability of presentation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119088488A_ABST
    Figure CN119088488A_ABST
Patent Text Reader

Abstract

The invention relates to a data processing method and device, equipment and a storage medium. Obtaining plug-in definition information and a calling result returned by the plug-in; rendering the calling result into content data in a format easy to understand by a user based on the plug-in definition information; and outputting the content data. Therefore, the plug-in calling result (namely the finally output content data) finally displayed to the user has readability and reliability.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a data processing method, device, equipment and storage medium. Background Art

[0002] With the continuous development of Large Language Model (LLM) technology, how to automatically apply large language models to solve problems in engineering scenarios has become a technical focus. Large language models themselves lack the ability to perform actions or actively obtain information, so plug-ins are needed to perform standardized engineering tasks, such as weather query tasks and hotel reservation tasks. In this context, OpenAI ChatGPT launched a plug-in mechanism.

[0003] Generally, plugins in LLM scenarios are defined based on the OpenAPI YAML standard, and common data types of their return bodies include application / json, text / event-stream, text / html, etc. Directly displaying this data results in poor readability and is difficult for general users to understand. Applying the LLM model to summarize and output this data is an alternative solution, but its effect is often limited by the model's capabilities and may cause information bias. While incurring additional model inference costs, the output of the LLM model is difficult to guarantee accuracy in scenarios involving data query or serious numerical calculations.

[0004] Therefore, in the application scenarios where large language models are combined with plug-ins, how to effectively display the results returned by the plug-ins is a key challenge in realizing related application scenarios. Summary of the invention

[0005] A technical problem to be solved by the present disclosure is how to effectively display the results returned by the plug-in.

[0006] According to a first aspect of the present disclosure, a data processing method is provided, comprising: obtaining plug-in definition information and a call result returned by the plug-in; rendering the call result into content data in a format that is easy for a user to understand based on the plug-in definition information; and outputting the content data.

[0007] Optionally, based on the plug-in definition information, the call result is rendered into content data in a format that is easy for a user to understand, including: determining a pattern object to be rendered in the call result according to a pattern object defined in the plug-in definition information, the pattern object being used to define a data structure of a plug-in return body; and rendering the pattern object in the call result into content data in a format that is easy for a user to understand.

[0008] Optionally, rendering the pattern object in the call result into content data in a format that is easy for the user to understand includes: determining whether the pattern object meets the rendering conditions of any basic rendering unit in a set of basic rendering units; if the pattern object meets the rendering conditions of a basic rendering unit, using the basic rendering unit to render the pattern object into content data that is easy for the user to understand, wherein each of the basic rendering units corresponds to a rendering method.

[0009] Optionally, rendering the pattern object in the call result into content data in a format that is easy for the user to understand also includes: if the pattern object does not meet the rendering conditions of all basic rendering units, determining whether the pattern object contains a multi-layer nested relationship of pattern objects; if the pattern object contains a multi-layer nested relationship of pattern objects, decomposing the pattern object layer by layer to obtain at least one sub-pattern object; determining whether each sub-pattern object meets the rendering conditions of any basic rendering unit in the basic rendering unit set; if a sub-pattern object meets the rendering conditions of a basic rendering unit, using the basic rendering unit to render the sub-pattern object, and using the rendering result as the hierarchical rendering result of the sub-pattern object.

[0010] Optionally, rendering the pattern object in the call result into content data in a format that is easy for the user to understand also includes: if the pattern object does not contain a multi-layer nested relationship of pattern objects, rendering the pattern object using a basic rendering method; and / or if a sub-pattern object does not meet the rendering conditions of all basic rendering units and cannot be disassembled, rendering the sub-pattern object using a basic rendering method.

[0011] Optionally, the basic rendering mode is a text rendering mode.

[0012] Optionally, determining whether the pattern object contains multiple layers of nested pattern objects includes: determining whether the pattern object is of array type or object type; if the pattern object is of array type or object type, determining that the pattern object contains multiple layers of nested pattern objects.

[0013] Optionally, the method further includes: if the current mode object to be rendered is of array type or object type, determining whether to render the level title according to description information of the mode object.

[0014] Optionally, the basic rendering unit set includes at least one of the following basic rendering units: a picture rendering unit, a picture list rendering unit, a table rendering unit, an embedded page rendering unit, and a text rendering unit.

[0015] Optionally, the rendering conditions of the image rendering unit include: the type of the pattern object is string; the title field or description field of the pattern object contains prompt words related to the image; the string type data has a picture suffix; the rendering method of the image rendering unit is to render the string type data into a markdown image format, and use the title field or description field as the image prompt word; and / or the rendering conditions of the image list rendering unit include: the type of the pattern object is array, and the type under its items is string; the title field or description field of the pattern object contains prompt words related to the image; the string type data has a picture suffix; the rendering method of the image list rendering unit is to render all the images in the list into a markdown image format, and use the title field or description field as the image prompt word; and / or the rendering conditions of the table rendering unit include: the type of the pattern object is array or object; the title field or description field of the pattern object contains prompt words related to the table; the table The rendering method of the rendering unit is: if the type of the pattern object is object, the data defined by the pattern object is rendered as a single-row table; if the type of the pattern object is array, and the type under its items is object, the data defined by the pattern object is rendered as a two-dimensional table; if the type of the pattern object is array, and the type under its items is different from object, the data defined by the pattern object is rendered as a single-column table; and / or the rendering conditions of the embedded page rendering unit include: the type of the pattern object is string; the title field or description field of the pattern object contains prompt words related to the embedded page; the rendering method of the embedded page rendering unit is to render the data defined by the pattern object into the embedded page format; and / or the rendering conditions of the text rendering unit include: the type of the pattern object is one of number, integer, boolean, and string; the rendering method of the text rendering unit is to render the data defined by the pattern object into markdown text format, and use the title field as the prompt word.

[0016] Optionally, the plug-in definition information is written based on the following principles: a renderable schema object should contain a description field; and / or a title field in a schema object will be rendered; and / or target data that is expected to be displayed with a predetermined effect needs to have prompt words related to the target data added to the description field or title field of the schema object.

[0017] Optionally, the method further includes: acquiring the modified plug-in definition information and the call result returned by the plug-in after the modification; and rendering the call result returned by the plug-in after the modification based on the modified plug-in definition information.

[0018] Optionally, the content data is in markdown format.

[0019] According to the second aspect of the present disclosure, a data processing device is provided, including: an acquisition module for acquiring plug-in definition information and a call result returned by the plug-in; a rendering module for rendering the call result into content data in a format that is easy for a user to understand based on the plug-in definition information; and an output module for outputting the content data.

[0020] Optionally, the rendering module determines the pattern object to be rendered in the call result based on the pattern object defined in the plug-in definition information, and the pattern object is used to define the data structure of the plug-in return body; the rendering module renders the pattern object in the call result into content data in a format that is easy for users to understand.

[0021] Optionally, the rendering module determines whether the pattern object meets the rendering conditions of any basic rendering unit in the basic rendering unit set; if the pattern object meets the rendering conditions of a basic rendering unit, the basic rendering unit is used to render the pattern object into content data that is easy for the user to understand, wherein each basic rendering unit corresponds to a rendering method.

[0022] Optionally, if the pattern object does not meet the rendering conditions of all basic rendering units, the rendering module determines whether the pattern object contains a multi-layer nested relationship of pattern objects; if the pattern object contains a multi-layer nested relationship of pattern objects, the rendering module decomposes the pattern object layer by layer to obtain at least one sub-pattern object; determines whether each sub-pattern object meets the rendering conditions of any basic rendering unit in the basic rendering unit set; if a sub-pattern object meets the rendering conditions of a basic rendering unit, the rendering module uses the basic rendering unit to render the sub-pattern object, and uses the rendering result as the hierarchical rendering result of the sub-pattern object.

[0023] Optionally, if the pattern object does not contain a multi-layer nested relationship of pattern objects, the rendering module uses a basic rendering method to render the pattern object; and / or if a sub-pattern object does not meet the rendering conditions of all basic rendering units and cannot be disassembled, the rendering module uses a basic rendering method to render the sub-pattern object.

[0024] Optionally, the basic rendering mode is a text rendering mode.

[0025] Optionally, the rendering module determines whether the pattern object is of array type or object type; if the pattern object is of array type or object type, it is determined that the pattern object contains a multi-layer pattern object nesting relationship.

[0026] Optionally, if the current pattern object to be rendered is of array type or object type, the rendering module determines whether to render the level title according to the description information of the pattern object.

[0027] Optionally, the basic rendering unit set includes at least one of the following basic rendering units: a picture rendering unit, a picture list rendering unit, a table rendering unit, an embedded page rendering unit, and a text rendering unit.

[0028] Optionally, the rendering conditions of the image rendering unit include: the type of the pattern object is string; the title field or description field of the pattern object contains prompt words related to the image; the string type data has a picture suffix; the rendering method of the image rendering unit is to render the string type data into a markdown image format, and use the title field or description field as the image prompt word; and / or the rendering conditions of the image list rendering unit include: the type of the pattern object is array, and the type under its items is string; the title field or description field of the pattern object contains prompt words related to the image; the string type data has a picture suffix; the rendering method of the image list rendering unit is to render all the images in the list into a markdown image format, and use the title field or description field as the image prompt word; and / or the rendering conditions of the table rendering unit include: the type of the pattern object is array or object; the title field or description field of the pattern object contains prompt words related to the table; the table The rendering method of the rendering unit is: if the type of the pattern object is object, the data defined by the pattern object is rendered as a single-row table; if the type of the pattern object is array, and the type under its items is object, the data defined by the pattern object is rendered as a two-dimensional table; if the type of the pattern object is array, and the type under its items is different from object, the data defined by the pattern object is rendered as a single-column table; and / or the rendering conditions of the embedded page rendering unit include: the type of the pattern object is string; the title field or description field of the pattern object contains prompt words related to the embedded page; the rendering method of the embedded page rendering unit is to render the data defined by the pattern object into the embedded page format; and / or the rendering conditions of the text rendering unit include: the type of the pattern object is one of number, integer, boolean, and string; the rendering method of the text rendering unit is to render the data defined by the pattern object into markdown text format, and use the title field as the prompt word.

[0029] Optionally, the plug-in definition information is written based on the following principles: a renderable schema object should contain a description field; and / or a title field in a schema object will be rendered; and / or target data that is expected to be displayed with a predetermined effect needs to have prompt words related to the target data added to the description field or title field of the schema object.

[0030] Optionally, the acquisition module further acquires the modified plug-in definition information and the call result returned by the plug-in after the modification; and the rendering module renders the call result returned by the plug-in after the modification based on the modified plug-in definition information.

[0031] Optionally, the content data is in markdown format.

[0032] According to a third aspect of the present disclosure, a computing device is provided, comprising: a processor; and a memory on which executable codes are stored, and when the executable codes are executed by the processor, the processor executes the method described in the first aspect above.

[0033] According to a fourth aspect of the present disclosure, a computer program product is provided, comprising an executable code, which, when executed by a processor of an electronic device, causes the processor to execute the method described in the first aspect above.

[0034] According to a fifth aspect of the present disclosure, a non-transitory machine-readable storage medium is provided, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor executes the method described in the first aspect above.

[0035] Therefore, the present invention obtains the plug-in definition information and the call result returned by the plug-in, renders the call result into content data in a format that is easy for the user to understand based on the plug-in definition information, and outputs the content data, so that the plug-in call result (i.e., the final output content data) finally displayed to the user is both readable and reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components in the exemplary embodiments of the present disclosure.

[0037] Figure 1 is a schematic flow chart showing a data processing method according to an embodiment of the present disclosure.

[0038] Figure 2 It is a schematic diagram showing the working principle of the basic rendering unit.

[0039] Figure 3It is a schematic diagram showing the principle of a nested rendering framework with basic rendering units as units.

[0040] Figure 4 It is a schematic diagram showing a data structure of a return body to be rendered.

[0041] Figure 5 It is a schematic diagram showing the structure of a data processing device according to an embodiment of the present disclosure.

[0042] Figure 6 A schematic diagram of the structure of a computing device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0043] The preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the preferred embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0044] The plugin return body is usually data of the application / json, text / event-stream, text / html type, etc. Directly displaying this data results in poor readability and is difficult for general users to understand.

[0045] In view of this, the present disclosure proposes that the plug-in return body (i.e., the call result returned by the plug-in) can be parsed (i.e., rendered) into content data in a user-friendly format. For example, the plug-in return body can be rendered as content in markdown format.

[0046] Figure 1 is a schematic flow chart showing a data processing method according to an embodiment of the present disclosure. Figure 1 The method shown can be implemented in software by a computer program, or can be executed by a computing device with a specific configuration. Figure 1 The functional module of the method shown is used to support rendering the result obtained by calling the plug-in into user-friendly content data.

[0047] See also Figure 1 , in step S110, the plug-in definition information and the calling result returned by the plug-in are obtained.

[0048] The plug-in in the present disclosure may refer to a plug-in in an LLM application scenario.

[0049] The plug-in in the LLM scenario is defined based on the OpenAPI YAML standard. Therefore, the plug-in definition information may refer to an OpenAPI YAML file obtained by defining the plug-in based on the OpenAPI YAML standard.

[0050] The call result returned by the plugin is the plugin return body obtained by calling the plugin. The call result returned by the plugin corresponds to the Response part Schema in the plugin definition information (such as the OpenAPI YAML file). This part of the Schema can be described as a standard OpenAPI Schema object.

[0051] In the OpenAPI specification, the Schema object is a key concept used to describe the data model of the API (i.e., plugin), that is, to define the data types of input and output. The OpenAPI Schema object is a way to describe the data model and data structure of an API. It provides a unified way to define the data format of requests and responses in an API, as well as information such as the type, format, and constraints of the fields. The main purpose of the Schema object is to ensure that data exchange between different systems can proceed smoothly without ambiguity or errors.

[0052] A basic Schema object may contain the following attributes: type; title; enum; description; default. type refers to the type of the Schema object. title refers to the title of the Schema object. Enum refers to the accepted enumeration value. Description refers to the description of the Schema object. Default refers to the default value. Among them, type is a required value, and the others are optional values. In addition, the Schema object may also contain some other optional attributes.

[0053] When defining a Schema object, you need to specify its type. The defined Schema object inherits the above properties and has corresponding properties according to the type. Common types of Schema objects include string, integer, number, Boolean, array, object, etc.

[0054] String: used to represent text data, which can include characters and numbers.

[0055] Example OpenAPI definition:

[0056] type:string

[0057] Integer: Used to represent whole number values.

[0058] Example OpenAPI definition:

[0059] type:integer

[0060] Number: used to represent numbers with decimal parts.

[0061] Example OpenAPI definition:

[0062] type:number

[0063] Boolean: A value that represents true or false.

[0064] Example OpenAPI definition:

[0065] type:boolean

[0066] Array: used to represent a group of data of the same type, with items attribute, where items are also Schema objects.

[0067] Example OpenAPI definition:

[0068] type:array

[0069] Items:

[0070] type:string

[0071] Object: used to represent complex data structures, containing multiple attributes. It has properties, which is a list. Each Property element contains a name and a corresponding Schema object.

[0072] Example OpenAPI definition:

[0073] type:object

[0074] properties:

[0075] name:

[0076] type:string

[0077] age:

[0078] type:integer

[0079] In step S120, the calling result is rendered into content data in a format that is easy for the user to understand based on the plug-in definition information.

[0080] The plugin definition information includes description information of the plugin return body, which is used to define the plugin return body. Taking the LLM plugin rendering scenario as an example, the description information of the plugin return body needs to be given in the OpenAPI YAML definition file, and the definition of the return body should comply with the definition requirements of the OpenAPI Schema.

[0081] The schema object to be rendered in the call result can be determined according to the schema object (i.e., Schema object) defined in the plug-in definition information, that is, the schema object in the call result is located. The schema object is used to define the data structure of the plug-in return body (i.e., the Response part). The schema object in the call result is then rendered into content data in a format that is easy for users to understand.

[0082] The content data in a format that is easy for users to understand may be, but is not limited to, data in the form of markdown. Markdown is a lightweight markup language that allows people to write documents in an easy-to-read and easy-to-write plain text format and then convert them into valid XHTML (or HTML) documents.

[0083] There may be multiple data types for the Schema object to be rendered, and different data types usually correspond to different rendering methods. In order to better support the rendering of different types of Schema objects, the present disclosure proposes the concept of a basic rendering unit. One or more types of basic rendering units can be designed. In addition, the present disclosure supports users to customize and expand more basic rendering units. In the present disclosure, each basic rendering unit has its own rendering conditions and rendering methods. Rendering conditions can also be called expression conditions or renderable rules. Rendering methods can also be called rendering functions.

[0084] When rendering a Schema object, only if the Schema object meets the rendering conditions of a basic rendering unit can the rendering method corresponding to the basic rendering unit be used to render the Schema object. Exemplarily, each basic rendering unit can correspond to a data type, and the rendering method of the basic rendering unit can be used to render the Schema object of the corresponding data type into corresponding markdown content data.

[0085] The present disclosure provides five basic rendering units in combination with the common data types of Schema objects, namely, picture rendering unit, picture list rendering unit, table rendering unit, embedded page rendering unit, and text rendering unit. Among them, the basic rendering units that may be rendered under the array type schema include the picture list rendering unit and the table rendering unit. The basic rendering units that may be rendered under the object type schema include the table rendering unit. The basic rendering units that may be rendered under the basic schema include the picture rendering unit, the embedded page rendering unit, and the text rendering unit. In addition, other types of basic rendering units may be set as needed, for example, including but not limited to the Chart rendering unit, the Echart rendering unit, the PPT rendering unit, the CAD rendering unit, and the like.

[0086] The rendering conditions of the image rendering unit may include: the type of the schema object is string; the title field (i.e., title) or the description field (i.e., description) of the schema object contains prompt words related to the image (such as picture, image, img, avatar, icon, picture, pic, avatar, icon, etc.); the string type data has a picture suffix. The rendering method of the image rendering unit is to render the string type data into the markdown image format, and use the title field or the description field as the picture prompt word. For example, the definition of the image rendering unit can be described as follows: Expression condition 1: The Schema type should be String; Expression condition 2: The Schema description should contain "picture / image"; Expression condition 3: The corresponding string type data should end with a specific suffix, such as jpg; Rendering function: Convert the corresponding string type data into the markdown format of class [xxx](yyyy.jpg).

[0087] For example, suppose a schema object is:

[0088] type: string

[0089] descrpion: "Image resource list"

[0090] Title: "This is a picture"

[0091] The rendering data is the uniform resource locator of the image, that is, the URL, such as https: / / xxxxx.png.

[0092] The pattern object meets the rendering conditions of the image rendering unit, so the pattern object can be rendered using the image rendering unit. The resulting markdown result can be expressed as: '! [This is a picture](https: / / xxxxx.png)', which is content data that meets the markdown picture format.

[0093] The rendering conditions of the picture list rendering unit may include: the type of the pattern object is array, and the type of its items is string; the title field (i.e., title) or description field (i.e., description) of the pattern object contains prompt words related to the picture (such as picture, image, img, avatar, icon, picture, pic, avatar, icon, etc.); the string type data has a picture suffix. The rendering method of the picture list rendering unit is to render all the pictures in the list into markdown picture format, and use the title field or description field as the picture prompt word.

[0094] For example, suppose a schema object is:

[0095] Items:

[0096] description: "is a picture"

[0097] type: string

[0098] type: array

[0099] The rendering data is a list of images, such as ['image1.jpg', 'image2.png', 'image3.gif'].

[0100] This pattern object meets the rendering conditions of the image list rendering unit, so the pattern object can be rendered using the image rendering unit. The resulting markdown result can be expressed as, ':::images\n-! [is an image](image1.jpg)\n-'! [is an image](image2.png)\n-'! [is an image](image3.gif)\n:::', which is content data that meets the markdown image format.

[0101] The rendering conditions of the table rendering unit may include: the type of the pattern object is array or object; the title field or description field of the pattern object contains table-related prompt words (such as table, table). The rendering method of the table rendering unit is: it can render markdown format described by separators such as | and \n. Specifically, if the type of the pattern object is object, the data defined by the pattern object is rendered as a single-row table; if the type of the pattern object is array, and the type under its items is object, the data defined by the pattern object is rendered as a two-dimensional table; if the type of the pattern object is array, and the type under its items is different from object, the data defined by the pattern object is rendered as a single-column table.

[0102] For example, suppose a schema object is:

[0103] Items:

[0104] type: string

[0105] description: "TODO list"

[0106] type: array

[0107] description: "TODO list table"

[0108] Rendering data is ['buy a house', 'buy things', 'sell a house']

[0109] This pattern object meets the rendering conditions of the table rendering unit, so the pattern object can be rendered using the table rendering unit. The resulting markdown result can be expressed as |TODO list|\n|----|\n|buy a house|\n|buy things|\n|sell a house|\n. The corresponding display format of the rendering result is a single-column table.

[0110] The rendering conditions of the embedded page rendering unit may include: the type of the pattern object is string; the title field or description field of the pattern object contains prompt words related to the embedded page (such as iframe, embedded page, etc.). The rendering method of the embedded page rendering unit is to render the data defined by the pattern object into the embedded page format. Among them, the rendering can be started with a specific identifier, and the page height and width can be specified in the description, for example: "height: 1930px, width: 12%".

[0111] For example, suppose a schema object is:

[0112] description: "is an iframe"

[0113] type: string

[0114] Rendering data is https: / / sxample.com

[0115] The pattern object meets the rendering conditions of the embedded page rendering unit, so the pattern object can be rendered using the embedded page rendering unit. The resulting markdown result can be expressed as, \n:::iframe{src='https: / / sxample.com'width=""height=""}\n:::\n\n".

[0116] The rendering condition of the text rendering unit may include: the type of the pattern object is one of number, integer, boolean, and string. The rendering method of the text rendering unit is to render the data defined by the pattern object into a markdown text format and use the title field as a prompt word.

[0117] Figure 2 It is a schematic diagram showing the working principle of the basic rendering unit.

[0118] See also Figure 2 The input of the basic rendering unit can be the defined OpenAPI Schema (i.e., the definition information for the plug-in return body) and the corresponding format data (i.e., the plug-in return body obtained by calling the plug-in). First, the Schema object to be rendered can be located according to the OpenAPI Schema, and then the Schema object is judged according to the data structure and description information of the Schema object whether it meets the rendering conditions of the basic rendering unit. If it meets the rendering conditions of the basic rendering unit, the rendering function of the basic rendering unit can be directly called for rendering to obtain the rendering result.

[0119] When the form of the Schema object is relatively simple, a simple comparison can be made between the Schema object and various basic rendering units to select a basic rendering unit that meets the rendering conditions for rendering.

[0120] For example, for the following Schema object:

[0121] type:string

[0122] description: picture string

[0123] data:123.jpg

[0124] The Schema object meets the rendering conditions of the image rendering unit, so the Schema object can be rendered using the rendering method of the image rendering unit.

[0125] OpenAPI Schema supports more complex definitions. In actual API definitions, the returned json data structure often involves the use of a large number of object and array structures, or deeply nested objects and arrays. It is difficult to point out the string to be rendered, which is prone to ambiguity. In view of this, the present disclosure proposes a nested rendering framework based on basic rendering units. For any complex Schema data structure, the framework will try to prioritize the analysis of the complex structure, search downward layer by layer, and perform basic rendering element compliance judgment during each analysis. If any basic rendering element that meets the current structure is found, the result of the current overall part will be directly rendered.

[0126] Specifically, for a mode object to be rendered, it can be first determined whether the mode object meets the rendering conditions of any basic rendering unit in the basic rendering unit set. If the mode object meets the rendering conditions of a basic rendering unit, the basic rendering unit can be used to render the mode object into content data that is easy for users to understand.

[0127] If the pattern object does not meet the rendering conditions of all basic rendering units, it can be further determined whether the pattern object contains multiple layers of pattern object nesting relationships. The types of pattern objects containing nested relationships are mainly array type and object type. Therefore, by comparing whether the type of the pattern object is array type or object type, it can be determined whether the pattern object contains multiple layers of pattern object nesting relationships. If the pattern object is array type or object type, it can be determined that the pattern object contains multiple layers of pattern object nesting relationships. Conversely, if the pattern object is neither array type nor object type, it can be determined that the pattern object does not contain multiple layers of pattern object nesting relationships.

[0128] If a pattern object contains multiple layers of nested pattern objects, the pattern object can be decomposed layer by layer to obtain at least one sub-pattern object; determine whether each sub-pattern object meets the rendering conditions of any basic rendering unit in the basic rendering unit set; if a sub-pattern object meets the rendering conditions of a basic rendering unit, use the basic rendering unit to render the sub-pattern object, and use the rendering result as the hierarchical rendering result of the sub-pattern object. If a sub-pattern object does not meet the rendering conditions of all basic rendering units and cannot be decomposed any further, the sub-pattern object can be rendered using the basic rendering method. The basic rendering method can refer to the text rendering method.

[0129] If the mode object does not meet the rendering conditions of all basic rendering units and does not contain a multi-layer nested relationship of mode objects, the basic rendering method can be used to render the mode object.

[0130] Figure 3 It is a schematic diagram showing the principle of a nested rendering framework with basic rendering units as units.

[0131] See also Figure 3 ,The rendering process for complex schema objects (i.e., Complex Schema) can be described as follows.

[0132] 1. Determine whether the current Schema conforms to the rendering rules of any basic rendering unit under the current schema type. If so, render the result directly.

[0133] 2. If the current Schema does not conform to the rendering rules of any basic rendering unit under the current schema type, further determine whether the current Schema is an array or object type that may contain sub-Schemas. If not, render the result according to the basic rendering method. If yes, proceed to the next step.

[0134] 3. If the current Schema is of array type, decide whether to render the level title based on the description information, and return to the first step of rendering for the Schema (i.e., sub-schema object) under the items field, and add the sub-level rendering result to the current level rendering result. The description information mainly includes the description and title of the Schema.

[0135] 4. If the current Schema is of object type, decide whether to render the level title based on the description information, and return to the first step of rendering for the Schema of all properties under the properties field (that is, the sub-schema object), and add the sub-level rendering results to the current level rendering results.

[0136] Figure 4 It is a schematic diagram showing a data structure of a return body to be rendered.

[0137] for Figure 4 The Schema shown can perform the following rendering process.

[0138] 1) The outermost ExampleSchema is an object type schema and does not belong to the basic rendering unit. Consider rendering the elements under the inner properties. The outermost ExampleSchema does not contain title, so the title of this level is not rendered. When the child element (that is, the child schema object) has renderable content, the child element content is rendered.

[0139] 2) There is only one property named answer under the inner layer properties. An attempt is made to make a rendering judgment on answer and add the rendering result of answer to the rendering content of ExampleSchema.

[0140] 3) answer is an object type schema and does not conform to the renderable rules of any basic rendering unit. It does not contain title, so the logic of step 1) is repeated for the answer schema.

[0141] 4) There is only one property named resource under the inner layer properties. Repeat the logic of step 2).

[0142] 5) If resource is an array type schema and does not conform to the renderable rules of any basic rendering unit, consider rendering the schema under the inner items. The schema under the inner items contains title, so the title of this level "generated image resources" is rendered, and the child element rendering content is used as the level rendering content.

[0143] 6) The inner items have an object type schema and do not conform to the renderable rules of any basic rendering unit, and do not contain title. Repeat the logic of step 1).

[0144] 7) The inner layer properties contains two fields, type and value. Rendering judgment is performed on type and value respectively, and the rendering results of the two are successively added to the rendering content of the schema of this layer.

[0145] 8) type is a STRING type schema that conforms to the renderable rules of the basic rendering element text_render, and is added to the rendering result after applying the text_render rendering type.

[0146] 9) value is of ARRAY type schema, which complies with the renderable rules of the basic rendering element image_list_render. The inner items are no longer split. Image_list_render is applied to render value and then the rendering result is added.

[0147] In step S130, the content data is output.

[0148] After obtaining the content data in a format that is easy for the user to understand, the content data can be output in a form that is easy for the user to understand. For example, when the content data obtained is in a markdown format, the markdown result can be output.

[0149] As mentioned above, different basic rendering units have corresponding rendering conditions, which can be reflected in the definition information of OpenAPI Schema. Therefore, the plug-in definition information can determine the markdown format to be rendered. In other words, relevant personnel can determine the rendering logic of the schema object (such as whether the basic rendering unit can be used and the specific basic rendering unit to be used) by writing the plug-in definition information. As a result, relevant personnel (such as users or operators) can control the markdown content returned by the rendering module obtained by the front end by changing the plug-in definition information.

[0150] Exemplarily, when relevant personnel (such as operators) write the configured OpenAPI YAML file, if they want to control the display effect, they need to follow the following principles: the renderable schema object should contain a description field (description); and / or, the title field (title) in the schema object will be rendered; and / or, the target data expected to be displayed with a predetermined effect needs to add prompt words related to the target data in the description field or title field of the schema object. For example, for special content such as pictures and tables, the corresponding prompt words should be included in the description or title field so that the corresponding basic rendering unit can be called for rendering. Therefore, the plug-in definition information can be written based on the above principles on the basis of following the OpenAPI YAML specification.

[0151] The present disclosure can also obtain the modified plug-in definition information and the call result returned by the plug-in after the plug-in definition information is modified, and render the call result returned by the plug-in after the modification based on the modified plug-in definition information. In this way, the user can change the display effect of the call result by changing the plug-in definition information.

[0152] In summary, the present disclosure proposes a method for rendering the return body of an LLM plug-in. Aiming at the rendering problem of the return body in the application scenario of the LLM plug-in, the present disclosure proposes a rendering unit definition method based on OpenAPI Schema, and a rendering framework for obtaining results according to the corresponding rendering unit. The technical fields involved in the present disclosure include large language model technology, OpenAPI Schema definition parsing, etc. The present disclosure can be applied to a question-and-answer service scenario based on LLM to execute the rendering process in the question-and-answer service scenario. After completing the plug-in call and obtaining the return result, the LLM plug-in return body is parsed and processed, and readable markdown content is returned to improve the user experience.

[0153] The data processing method disclosed in the present invention may also be implemented as a data processing device. Figure 51 is a schematic diagram showing the structure of a data processing device according to an embodiment of the present disclosure. The functional units of the data processing device can be implemented by hardware, software, or a combination of hardware and software that implements the principles of the present disclosure. It can be understood by those skilled in the art that Figure 5 The functional units described may be combined or divided into sub-units to implement the principles of the above invention. Therefore, the description herein may support any possible combination, division, or further limitation of the functional units described herein. The following is a brief description of the functional units that a data processing device may have and the operations that each functional unit may perform. For the details involved, please refer to the relevant description above, which will not be repeated here.

[0154] See also Figure 5 , the data processing device 500 may include an acquisition module 510, a rendering module 520 and an output module 530. The acquisition module 510 is used to acquire the plug-in definition information and the call result returned by the plug-in. The rendering module 520 is used to render the call result into content data in a format that is easy for the user to understand based on the plug-in definition information. The output module 530 is used to output the content data.

[0155] The rendering module 520 can determine the pattern objects to be rendered in the call result according to the pattern objects defined in the plug-in definition information, and render the pattern objects in the call result into content data in a format that is easy for users to understand.

[0156] In some embodiments, the rendering module 520 may first determine whether the pattern object meets the rendering conditions of any basic rendering unit in the basic rendering unit set. If the pattern object meets the rendering conditions of a basic rendering unit, the basic rendering unit is used to render the pattern object into content data that is easy for the user to understand, wherein each basic rendering unit corresponds to a rendering method. If the pattern object does not meet the rendering conditions of all basic rendering units, the rendering module determines whether the pattern object contains a nested relationship of multiple layers of pattern objects; if the pattern object contains a nested relationship of multiple layers of pattern objects, the rendering module disassembles the pattern object layer by layer to obtain at least one sub-pattern object; determines whether each sub-pattern object meets the rendering conditions of any basic rendering unit in the basic rendering unit set; if a sub-pattern object meets the rendering conditions of a basic rendering unit, the rendering module uses the basic rendering unit to render the sub-pattern object, and uses the rendering result as the hierarchical rendering result of the sub-pattern object. If the pattern object does not contain a nested relationship of multiple layers of pattern objects, the rendering module uses the basic rendering method to render the pattern object; and / or if a sub-pattern object does not meet the rendering conditions of all basic rendering units and cannot be disassembled, the rendering module uses the basic rendering method to render the sub-pattern object.

[0157] Exemplarily, the rendering module 520 can determine whether the pattern object is of array type or object type; if the pattern object is of array type or object type, it is determined that the pattern object contains a multi-layer pattern object nesting relationship. If the pattern object to be rendered is of array type or object type, the rendering module can determine whether to render the hierarchical title according to the description information of the pattern object.

[0158] In some embodiments, the acquisition module 510 acquires the modified plug-in definition information and the call result returned by the modified plug-in; the rendering module 520 may render the call result returned by the modified plug-in based on the modified plug-in definition information.

[0159] Figure 6 A schematic diagram of the structure of a computing device that can be used to implement the above data processing method according to an embodiment of the present disclosure is shown.

[0160] See also Figure 6 , the computing device 600 includes a memory 610 and a processor 620 .

[0161] The processor 620 may be a multi-core processor or may include multiple processors. In some embodiments, the processor 620 may include a general-purpose main processor and one or more special coprocessors, such as a graphics processing unit (GPU), a digital signal processor (DSP), etc. In some embodiments, the processor 620 may be implemented using a customized circuit, such as an application-specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0162] The memory 610 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, ROM can store static data or instructions required by the processor 620 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory 610 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 610 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (e.g., DVD-ROM, double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. The computer-readable storage medium does not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0163] The memory 610 stores executable codes, and when the executable codes are processed by the processor 620 , the processor 620 can execute the data processing method mentioned above.

[0164] The data processing method, apparatus and device according to the present disclosure have been described above in detail with reference to the accompanying drawings.

[0165] In addition, the method according to the present disclosure may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing the above steps defined in the above method of the present disclosure.

[0166] Alternatively, the present disclosure may also be implemented as a non-temporary machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) on which executable code (or computer program, or computer instruction code) is stored. When the executable code (or computer program, or computer instruction code) is executed by a processor of an electronic device (or computing device, server, etc.), the processor executes the various steps of the above-mentioned method according to the present disclosure.

[0167] Those skilled in the art will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or a combination of both.

[0168] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the system and method according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0169] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A data processing method, characterized in that: include: Get the plugin definition information and the call result returned by the plugin; Rendering the call result into content data in a format that is easy for a user to understand based on the plug-in definition information; The content data is output.

2. The method according to claim 1, characterized in that Rendering the call result into content data in a format that is easy for a user to understand based on the plug-in definition information includes: Determine the pattern object to be rendered in the call result according to the pattern object defined in the plug-in definition information, wherein the pattern object is used to define the data structure of the plug-in return body; The schema object in the call result is rendered into content data in a format that is easy for a user to understand.

3. The method according to claim 2, characterized in that Rendering the schema object in the call result into content data in a format that is easy for a user to understand, including: Determine whether the mode object meets the rendering condition of any basic rendering unit in the basic rendering unit set; If the mode object meets the rendering condition of a basic rendering unit, the mode object is rendered into content data that is easy for users to understand using the basic rendering unit, wherein each basic rendering unit corresponds to a rendering mode.

4. The method according to claim 3, characterized in that Rendering the schema object in the call result into content data in a format that is easy for users to understand, further comprising: If the mode object does not meet the rendering conditions of all basic rendering units, determining whether the mode object contains a multi-layer mode object nesting relationship; If the pattern object contains multiple layers of nested pattern objects, the pattern object is decomposed layer by layer to obtain at least one sub-pattern object; whether each sub-pattern object meets the rendering condition of any basic rendering unit in the basic rendering unit set is determined; if a sub-pattern object meets the rendering condition of a basic rendering unit, the basic rendering unit is used to render the sub-pattern object, and the rendering result is used as the hierarchical rendering result of the sub-pattern object.

5. The method according to claim 3, characterized in that: The basic rendering unit set includes at least one of the following basic rendering units: a picture rendering unit, a picture list rendering unit, a table rendering unit, an embedded page rendering unit, and a text rendering unit.

6. The method according to claim 5, characterized in that The rendering conditions of the image rendering unit include: the type of the pattern object is string; the title field or description field of the pattern object contains prompt words related to the image; the string type data has a picture suffix; the rendering method of the image rendering unit is to render the string type data into a markdown image format, and use the title field or description field as the image prompt word; and / or The rendering conditions of the picture list rendering unit include: the type of the pattern object is array, and the type of its items is string; the title field or description field of the pattern object contains prompt words related to the picture; the string type data has a picture suffix; the rendering method of the picture list rendering unit is to render all the pictures in the list into markdown picture format, and use the title field or description field as the picture prompt word; and / or The rendering conditions of the table rendering unit include: the type of the pattern object is array or object; the title field or description field of the pattern object contains prompt words related to the table; the rendering method of the table rendering unit is: if the type of the pattern object is object, the data defined by the pattern object is rendered as a single-row table; if the type of the pattern object is array, and the type under its items is object, the data defined by the pattern object is rendered as a two-dimensional table; if the type of the pattern object is array, and the type under its items is different from object, the data defined by the pattern object is rendered as a single-column table; and / or The rendering conditions of the embedded page rendering unit include: the type of the mode object is string; the title field or description field of the mode object contains prompt words related to the embedded page; the rendering method of the embedded page rendering unit is to render the data defined by the mode object into the embedded page format; and / or The rendering conditions of the text rendering unit include: the type of the pattern object is one of number, integer, boolean, and string; the rendering method of the text rendering unit is to render the data defined by the pattern object into a markdown text format and use the title field as a prompt word.

7. A data processing device, characterized in that: include: The acquisition module is used to obtain the plug-in definition information and the call result returned by the plug-in; A rendering module, used for rendering the call result into content data in a format that is easy for a user to understand based on the plug-in definition information; An output module is used to output the content data.

8. A computing device comprising: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method according to any one of claims 1 to 6.

9. A computer program product, comprising executable codes, which, when executed by a processor of an electronic device, causes the processor to perform the method according to any one of claims 1 to 6.

10. A non-transitory machine-readable storage medium having executable codes stored thereon, which, when executed by a processor of an electronic device, causes the processor to execute the method according to any one of claims 1 to 6.