Data reasoning method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a data reasoning method and device, electronic equipment and a storage medium. Comprising the following steps: in response to an obtained reasoning request, determining a data source, scene information and reasoning result prompt information corresponding to the reasoning request; a target reasoning link is determined according to the scene information, the reasoning link is a logic link composed of at least one large language model and used for data reasoning, and output data of the previous large language model in the reasoning link is input data of the next large language model; obtaining to-be-reasoned data from a data source, and generating model input data matched with a first large language model in the target reasoning link according to the scene information, the reasoning result prompt information and the to-be-reasoned data; the reasoning result prompt information is used for indicating a reasoning result output by the target reasoning link; and inputting the model input data into the target reasoning link for reasoning, and obtaining a reasoning result output by the target reasoning link.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of computer network technology, and specifically to a data inference method, device, electronic device, and storage medium. Background Art

[0002] Data reasoning refers to the process of using existing data to derive new information, explain phenomena, predict future trends, or make decisions.

[0003] In the related art, data reasoning is implemented based on a large language model. A large language model can be a deep learning model trained with a large amount of text data, which can generate natural language text, understand the meaning of language text, and process various natural language tasks such as text classification, question answering, and dialogue processing. However, different large language models are applicable to different scenarios. If a large language model is incorrectly used in a scenario where it is not applicable, the reasoning results output by the model will be inaccurate. Summary of the invention

[0004] The embodiments of this specification provide a data reasoning method, device, electronic device, and storage medium, and the technical solutions thereof are as follows:

[0005] In a first aspect, an embodiment of this specification provides a data reasoning method, which is applied to a reasoning system, where multiple large language models are running on the reasoning system, including:

[0006] In response to the acquired reasoning request, determine the data source, scene information and reasoning result prompt information corresponding to the reasoning request;

[0007] Determine a target reasoning link according to the scenario information, wherein the reasoning link is a logical link for data reasoning composed of at least one large language model, and output data of a previous large language model in the reasoning link is input data of a next large language model;

[0008] Acquire the data to be inferred from the data source, and generate model input data matching the first large language model in the target reasoning link according to the scenario information, the reasoning result prompt information and the data to be inferred; wherein the reasoning result prompt information is used to indicate the reasoning result output by the target reasoning link;

[0009] The model input data is input into the target reasoning link for reasoning, and the reasoning result output by the target reasoning link is obtained.

[0010] In a second aspect, an embodiment of the present specification provides a data reasoning device, which is applied to a reasoning system, and a plurality of large language models are running on the reasoning system, including:

[0011] A first determining unit: in response to the acquired reasoning request, determining the data source, scene information and reasoning result prompt information corresponding to the reasoning request;

[0012] A second determining unit: determining a target reasoning link according to the scene information, wherein the reasoning link is a logical link for data reasoning composed of at least one large language model, and output data of a previous large language model in the reasoning link is input data of a next large language model;

[0013] An acquisition unit: acquires the data to be inferred from the data source, and generates model input data matching the first large language model in the target inference link according to the scenario information, the inference result prompt information and the data to be inferred; wherein the inference result prompt information is used to indicate the inference result output by the target inference link;

[0014] Reasoning unit: inputs the model input data into the target reasoning link for reasoning, and obtains the reasoning result output by the target reasoning link.

[0015] In a third aspect, an embodiment of the present specification provides an electronic device, comprising a processor and a memory; the processor is connected to the memory; the memory is used to store executable program code; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the steps of the data inference method described in the first aspect of the above embodiment.

[0016] In a fourth aspect, an embodiment of the present specification provides a computer storage medium, wherein the computer storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the steps of the data inference method described in the first aspect of the above embodiment.

[0017] The beneficial effects brought by the technical solutions provided by some embodiments of this specification include at least:

[0018] On the one hand, the reasoning system determines the reasoning link based on the scenario information corresponding to the reasoning request, so that the large language model used for data reasoning can be applicable to the scenario corresponding to the reasoning request, thereby improving the accuracy of data reasoning. In addition, since the reasoning link can contain multiple large language models, data reasoning based on the reasoning link can further improve the accuracy of reasoning. On the other hand, the model input data input to the reasoning link is generated based on the reasoning result prompt information, and the reasoning result prompt information can be used to indicate the reasoning result output by the reasoning link, which can prevent the reasoning result from deviating from expectations, thereby improving the accuracy of data reasoning. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of this specification, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is a schematic diagram of an inference system in a related technology provided in an embodiment of this specification.

[0021] Figure 2 It is an architecture diagram of an inference system provided in an embodiment of this specification.

[0022] Figure 3 It is a flowchart of a data reasoning method provided in an embodiment of this specification.

[0023] Figure 4 It is a schematic diagram of an inference link provided in an embodiment of this specification.

[0024] Figure 5 It is a block diagram of a data inference device provided in an embodiment of this specification.

[0025] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of this specification will be described clearly and completely below in conjunction with the drawings in the embodiments of this specification.

[0027] The terms "first", "second", etc. in the description and claims of this specification and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any variation thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0028] The data reasoning method is described in detail below in conjunction with one or more embodiments.

[0029] Data reasoning refers to the process of using existing data to derive new information, explain phenomena, predict future trends, or make decisions.

[0030] In the related art, data reasoning is implemented based on a large language model. A large language model can be a deep learning model trained with a large amount of text data, which can generate natural language text, understand the meaning of language text, and process various natural language tasks such as text classification, question answering, and dialogue processing.

[0031] See also Figure 1 , Figure 1 A schematic diagram of an inference system in a related technology provided by an embodiment of this specification is shown, such as Figure 1 As shown, the reasoning system includes: a mobile phone 11 and a server 12.

[0032] The mobile phone 11 is a type of electronic device that can be used by the user. Of course, the electronic devices used by the user are not limited to this. For example, they can also include: tablet devices, laptops, PDAs (Personal Digital Assistants), wearable devices (such as smart glasses, smart watches, etc.), etc. This specification does not limit this. During operation, the client program of the reasoning system runs on the mobile phone 11, so that the mobile phone 11 is configured as a client of the reasoning system. The client program can generate an inference request according to the user's instructions, and send the inference request to the server 12 so that the server 12 processes the inference request.

[0033] The server 12 may be a physical server including an independent host, or the server 12 may be a virtual server hosted by a host cluster. A server program of the reasoning system is run on the server 12, so that the server 12 is configured as the server of the communication system, and the server program includes a software program corresponding to a large language model for data reasoning. The server program may cooperate with the above-mentioned client, for example, it may receive a reasoning request sent by the client, process the reasoning request, and return the reasoning result to the client.

[0034] The interaction between the mobile phone 11 and the server 12 may include various types of wired or wireless interactions, which are not limited in this specification.

[0035] However, different large language models are applicable to different scenarios. If a large language model is incorrectly used in a scenario where it is not applicable, the reasoning results output by the model will be inaccurate.

[0036] To solve this problem, this specification proposes a data reasoning method, device, electronic device, and storage medium.

[0037] Combine the following Figure 2 This paper introduces the architecture involved in this data reasoning method in detail. Figure 2 , Figure 2Schematic diagram of an inference system provided by an embodiment of this specification is shown. Figure 2 As shown, the system may include: a mobile phone 21 and a server 22.

[0038] The functions of mobile phone 21 and server 22 are similar to those of mobile phone 11 and server 12. The difference is that multiple large language models for data reasoning are running on server 22, and the multiple large language models (such as large language models 221-223) can form multiple reasoning links for data reasoning (such as reasoning link 1 composed of large language model 221 and large language model 222 and reasoning link 2 composed of large language model 223 and large language model 221), and the output data of the previous large language model can be used as the input data of the next large language model, and finally the reasoning result is output. In addition, based on the scene information corresponding to the reasoning request, the reasoning link matching the scene information can be determined.

[0039] In the above-mentioned data reasoning system, on the one hand, the reasoning system determines the reasoning link based on the scenario information corresponding to the reasoning request, so that the large language model used for data reasoning can be applicable to the scenario corresponding to the reasoning request, thereby improving the accuracy of data reasoning. In addition, since the reasoning link can contain multiple large language models, data reasoning based on the reasoning link can further improve the accuracy of reasoning. On the other hand, the model input data input to the reasoning link is generated based on the reasoning result prompt information, and the reasoning result prompt information can be used to indicate the reasoning result output by the reasoning link, which can prevent the reasoning result from deviating from expectations, thereby improving the accuracy of data reasoning.

[0040] See also Figure 3 , Figure 3 A flow chart of a data reasoning method provided in an embodiment of this specification is shown, such as Figure 3 As shown, the data reasoning method is applied to a reasoning system, and a plurality of large language models are running on the reasoning system, and may at least include the following steps:

[0041] Step 302: In response to the acquired reasoning request, determine the data source, scene information, and reasoning result prompt information corresponding to the reasoning request.

[0042] The data source is the source of the data to be inferred, which can be specified by the user in the inference request and can be a website, a system or a data stream, and this specification does not limit this.

[0043] The scene information may be description information of the scene, such as scene name, scene ID, etc.

[0044] The inference result prompt information can be used to indicate the inference result output by the inference link. For example, the inference result prompt information can be "output product features", then the inference result is the product features of the output product; or the inference result prompt information can be "I have a table, add the field values ​​of two fields named A and B in the table and output", then the inference result is the field value sequence of the two fields added. The inference result prompt information is used to guide the large language model to output the inference result desired by the user.

[0045] Step 304: determine a target reasoning link according to the scenario information, wherein the reasoning link is a logical link for data reasoning composed of at least one large language model, and the output data of the previous large language model in the reasoning link is the input data of the next large language model.

[0046] The correspondence between scene information and reasoning links can be stored in the reasoning system. For example, each reasoning link corresponds to a link ID, the scene information can be a scene ID, and the reasoning system stores a data table recording the correspondence between the scene ID and the link ID. According to the scene ID included in the reasoning request, the corresponding link ID is determined from the data table, and then the target reasoning link is determined.

[0047] The reasoning chain can include one or more large language models. When the reasoning chain includes multiple large language models, the output data of the previous large language model can be used as the input data of the next large language model. However, the data structures of different large language models may be different, so the reasoning chain can be configured before performing data reasoning.

[0048] In one embodiment, after determining the target reasoning link, it also includes: determining configuration information corresponding to the target reasoning link according to the scene information, and configuring the large language model contained in the target reasoning link according to the configuration information; wherein the reasoning system stores the correspondence between the scene information, the reasoning link and the configuration information, and the configuration information includes at least one of the following: configuration data for indicating the data structure of the data output by each large language model in the reasoning link, verification rules for verifying the reasoning result, and a timeout duration of the reasoning link.

[0049] The configuration data for indicating the data structure of the data output by each large language model in the reasoning chain can make the data structure of the output data of the previous large language model match the data structure of the input data of the next large language model. Figure 4 As shown, the large model handler (inference link) and configuration information can be determined based on the scene code (scene information).

[0050] Of course, the configuration information includes more than the above, for example, it may also include service version information for determining each large language model, verification rules for verifying the inference results, timeout duration of the inference link, etc. This specification does not limit this.

[0051] In this embodiment, the data structure of the input and output data of each large language model is configured by using configuration data for indicating the data structure of the output data of each large language model in the reasoning link, thereby solving the compatibility problem of the large language model and ensuring smooth reasoning of the reasoning link.

[0052] Step 306: Obtain the data to be inferred from the data source, and generate model input data that matches the first large language model in the target reasoning link based on the scenario information, the reasoning result prompt information and the data to be inferred; wherein the reasoning result prompt information is used to indicate the reasoning result output by the target reasoning link.

[0053] Due to different data sources, the formats of the data to be inferred are also different. The reasoning of the data link needs to generate model input data in a unified format, such as "{"bizPrincipalId":"usage scenario id", "batchNo":"batch number", "data":"data to be inferred", "promptContent":"prompt word"}" is the model input data in a fixed format.

[0054] In one embodiment, when the scenario information is used to indicate a product recommendation scenario, the data to be inferred includes at least one of the following: offline data, log data, Flink stream data, peripheral data, and message data. In the product recommendation scenario, offline data may be an offline data table of a shopping platform, log data may be obtained from a log system of a shopping platform, peripheral data may be a customer's purchase record, click behavior, etc., and message data may be a customer's chat record.

[0055] Step 308: input the model input data into the target reasoning link for reasoning, and obtain the reasoning result output by the target reasoning link.

[0056] In this embodiment, on the one hand, the reasoning system determines the reasoning link based on the scenario information corresponding to the reasoning request, so that the large language model used for data reasoning can be applicable to the scenario corresponding to the reasoning request, thereby improving the accuracy of data reasoning. In addition, since the reasoning link can contain multiple large language models, data reasoning based on the reasoning link can further improve the accuracy of reasoning. On the other hand, the model input data input to the reasoning link is generated based on the reasoning result prompt information, and the reasoning result prompt information can be used to indicate the reasoning result output by the reasoning link, which can prevent the reasoning result from deviating from expectations, thereby improving the accuracy of data reasoning.

[0057] In one embodiment, the inference result is evaluated based on a preset evaluation criterion, and the evaluation criterion includes a preset regular expression.

[0058] During the inference process, model inference may produce "hallucinations", which means that the inference results do not meet expectations. Specifically, the inference results can be verified based on a preset regular expression. If they do not meet the regular expression, the verification fails.

[0059] Further, such as Figure 4 As shown, it also includes: if the evaluation is passed, the model input data and the inference result are input into the model to be trained as training data to train a dedicated large language model for the scene information.

[0060] Specifically, each training sample in the training data may include a set of model input data and corresponding real reasoning results; the training data is input into the model to be trained, so that the data to be trained outputs predicted reasoning results according to the model input data; the real reasoning results and the predicted reasoning results are compared, and the model to be trained is optimized according to the comparison results until the similarity between the two reaches a preset threshold.

[0061] In this embodiment, by training a dedicated large language model, only the dedicated large language model needs to be called to perform data reasoning in specific scenarios, without calling multiple large language models, thereby reducing the reasoning cost.

[0062] Furthermore, it also includes: if the evaluation fails, re-inputting the model input data into the target reasoning link for secondary reasoning, and sending an alarm if the result evaluation of the secondary reasoning fails.

[0063] like Figure 4 As shown in the figure, when the inference result evaluation fails, the "hallucination" problem may be caused by the legacy impact of the previous data inference on the model. Therefore, the "hallucination" problem caused by the model can be solved through the retry mechanism.

[0064] Furthermore, the inference result is evaluated based on preset evaluation criteria, including: evaluating the sub-inference results of each large language model in the target inference link; if the evaluation fails, re-inputting the model input data into the target inference link for secondary inference, including: if any sub-inference result fails the evaluation, re-using the large language model corresponding to any sub-inference result for reasoning.

[0065] During the inference process, timeouts or exceptions may occur due to graphics card bottlenecks. In the inference chain formed by multiple models, if one model fails to infer, the inference cannot fail directly. You need to retry at the point where the failed model is called.

[0066] In this embodiment, on the one hand, a retry mechanism is used to solve the "hallucination" problem generated by the model; on the other hand, a retry mechanism is executed for a single large language model in the reasoning link to avoid restarting the entire reasoning link and reduce resource consumption.

[0067] In one embodiment, the reasoning system is built based on software as a service technology.

[0068] SaaS (Software as a Service) is a cloud computing service model in which software applications are hosted by the provider on servers in the cloud and provided to customers via the Internet. This model allows customers to access and use software through the Internet without having to purchase, install, and maintain the software, reducing costs and increasing flexibility.

[0069] Building an inference system based on SAAS technology allows users to pay for the service fees of large language models on demand and expand or reduce services as needed. It also allows service providers of large language models to centrally manage and update software, ensuring the latest and security of the service.

[0070] See also Figure 5 , Figure 5 A block diagram of a data inference device provided in an embodiment of this specification. The device is applied to an inference system, and multiple large language models are running on the inference system, including:

[0071] A first determining unit 502 is used to determine, in response to the acquired reasoning request, data source, scene information and reasoning result prompt information corresponding to the reasoning request;

[0072] A second determining unit 504 is used to determine a target reasoning link according to the scene information, wherein the reasoning link is a logical link for data reasoning composed of at least one large language model, and the output data of the previous large language model in the reasoning link is the input data of the next large language model;

[0073] An acquisition unit 506 is used to acquire the data to be inferred from the data source, and generate model input data matching the first large language model in the target reasoning link according to the scenario information, the reasoning result prompt information and the data to be inferred; wherein the reasoning result prompt information is used to indicate the reasoning result output by the target reasoning link;

[0074] The reasoning unit 508 is used to input the model input data into the target reasoning link for reasoning, and obtain the reasoning result output by the target reasoning link.

[0075] Optionally, after determining the target reasoning link, the method further includes:

[0076] A configuration unit 510 is used to determine the configuration information corresponding to the target reasoning link according to the scenario information, and configure the large language model contained in the target reasoning link according to the configuration information; wherein the reasoning system stores the correspondence between the scenario information, the reasoning link and the configuration information, and the configuration information includes at least one of the following: configuration data for indicating the data structure of the data output by each large language model in the reasoning link, verification rules for verifying the reasoning result, and a timeout duration of the reasoning link.

[0077] Optionally, also include:

[0078] The evaluation unit 512 is used to evaluate the inference result based on preset evaluation criteria, where the evaluation criteria include coverage evaluation criteria, accuracy evaluation criteria, and bias evaluation criteria.

[0079] Optionally, also include:

[0080] The training unit 514 is used to input the model input data and the inference result as training data into the model to be trained when the evaluation is passed, so as to train and obtain a dedicated large language model for the scene information.

[0081] Optionally, also include:

[0082] The retry unit 516 is used to re-input the model input data into the target reasoning link for secondary reasoning if the evaluation fails, and send an alarm if the result evaluation of the secondary reasoning fails.

[0083] Optional,

[0084] The evaluation unit 512 is specifically used to: evaluate the sub-inference results of each large language model in the target reasoning link;

[0085] The retry unit 516 is specifically used for: if any sub-inference result fails the evaluation, reusing the large language model corresponding to any sub-inference result for reasoning.

[0086] Optionally, the reasoning system is built based on software as a service technology.

[0087] Optionally, when the scenario information is used to indicate a product recommendation scenario, the data to be inferred includes at least one of the following: offline data, log data, Flink stream data, peripheral data, and message data.

[0088] In this embodiment, on the one hand, the reasoning system determines the reasoning link based on the scenario information corresponding to the reasoning request, so that the large language model used for data reasoning can be applicable to the scenario corresponding to the reasoning request, thereby improving the accuracy of data reasoning. In addition, since the reasoning link can contain multiple large language models, data reasoning based on the reasoning link can further improve the accuracy of reasoning. On the other hand, the model input data input to the reasoning link is generated based on the reasoning result prompt information, and the reasoning result prompt information can be used to indicate the reasoning result output by the reasoning link, which can prevent the reasoning result from deviating from expectations, thereby improving the accuracy of data reasoning.

[0089] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the data reasoning device embodiment, since it is basically similar to the data reasoning method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0090] See also Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification is shown.

[0091] like Figure 6 As shown, the electronic device 600 may include: at least one processor 601 , at least one network interface 604 , a user interface 603 , a memory 605 , and at least one communication bus 602 .

[0092] The communication bus 602 may be used to realize the connection and communication among the above-mentioned components.

[0093] The user interface 603 may include buttons, and the optional user interface may also include a standard wired interface or a wireless interface.

[0094] The network interface 604 may include, but is not limited to, a Bluetooth module, an NFC module, a Wi-Fi module, etc.

[0095] Among them, the processor 601 may include one or more processing cores. The processor 601 uses various interfaces and lines to connect the various parts of the entire electronic device 600, and executes various functions and processes data of the electronic device 600 by running or executing instructions, programs, code sets or instruction sets stored in the memory 605, and calling data stored in the memory 605. Optionally, the processor 601 can be implemented in at least one hardware form of DSP, FPGA, and PLA. The processor 601 can integrate one or a combination of CPU, GPU, modem, etc. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to handle wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 601, and it can be implemented separately through a chip.

[0096] Among them, the memory 605 may include RAM or ROM. Optionally, the memory 605 includes a non-transitory computer-readable medium. The memory 605 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 605 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 605 may also be at least one storage device located away from the aforementioned processor 601. The memory 605 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a data reasoning application. The processor 601 can be used to call the data reasoning application stored in the memory 605 and execute the data reasoning steps mentioned in the aforementioned embodiments.

[0097] The embodiment of the present specification also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a computer or a processor, the computer or the processor executes the above Figure 3 , Figure 4 One or more steps in the illustrated embodiment. If the components of the electronic device described above are implemented in the form of software functional units and sold or used as independent products, they can be stored in the computer-readable storage medium.

[0098] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of this specification is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, data center, etc. that contains one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital versatile disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0099] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes. In the absence of conflict, the technical features in this embodiment and the implementation scheme can be combined arbitrarily.

[0100] The embodiments described above are merely preferred embodiments of this specification and are not intended to limit the scope of this specification. Without departing from the design spirit of this specification, various modifications and improvements made to the technical solutions of this specification by ordinary technicians in this field should fall within the scope of protection determined by the claims of this specification.

Claims

1. A data reasoning method, applied to a reasoning system, wherein a plurality of large language models are running on the reasoning system, comprising: In response to the acquired reasoning request, determine the data source, scene information and reasoning result prompt information corresponding to the reasoning request; Determine a target reasoning link according to the scenario information, wherein the reasoning link is a logical link for data reasoning composed of at least one large language model, and output data of a previous large language model in the reasoning link is input data of a next large language model; Acquire the data to be inferred from the data source, and generate model input data matching the first large language model in the target reasoning link according to the scenario information, the reasoning result prompt information and the data to be inferred; wherein the reasoning result prompt information is used to indicate the reasoning result output by the target reasoning link; The model input data is input into the target reasoning link for reasoning, and the reasoning result output by the target reasoning link is obtained.

2. A data reasoning method according to claim 1, after determining the target reasoning link, further comprising: Determine configuration information corresponding to the target reasoning link according to the scenario information, and configure the large language model contained in the target reasoning link according to the configuration information; wherein the reasoning system stores the correspondence between the scenario information, the reasoning link and the configuration information, and the configuration information includes configuration data for indicating the data structure of the data output by each large language model in the reasoning link.

3. The data reasoning method according to claim 1, further comprising: The inference result is evaluated based on a preset evaluation criterion, wherein the evaluation criterion includes a preset regular expression.

4. A data reasoning method according to claim 3, further comprising: If the evaluation is passed, the model input data and the inference result are input into the model to be trained as training data to obtain a dedicated large language model for the scene information.

5. The data reasoning method according to claim 3, further comprising: If the evaluation fails, the model input data is re-input into the target reasoning link for secondary reasoning, and if the result of the secondary reasoning fails to be evaluated, an alarm is sent.

6. A data reasoning method according to claim 5, The evaluating the inference result based on a preset evaluation criterion includes: Evaluating the sub-inference results of each large language model in the target reasoning link; If the evaluation fails, the model input data is re-input into the target reasoning link for secondary reasoning, including: if any sub-inference result fails the evaluation, the large language model corresponding to any sub-inference result is re-used for reasoning.

7. According to a data reasoning method according to claim 1, the reasoning system is built based on software as a service technology.

8. According to a data inference method according to claim 1, when the scenario information is used to indicate a product recommendation scenario, the data to be inferred includes at least one of the following: offline data, log data, Flink stream data, peripheral data, and message data.

9. A data inference device, applied to an inference system, wherein a plurality of large language models are run on the inference system, comprising: A first determining unit: in response to the acquired reasoning request, determining the data source, scene information and reasoning result prompt information corresponding to the reasoning request; A second determining unit: determining a target reasoning link according to the scene information, wherein the reasoning link is a logical link for data reasoning composed of at least one large language model, and output data of a previous large language model in the reasoning link is input data of a next large language model; An acquisition unit: acquires the data to be inferred from the data source, and generates model input data matching the first large language model in the target inference link according to the scenario information, the inference result prompt information and the data to be inferred; wherein the inference result prompt information is used to indicate the inference result output by the target inference link; Reasoning unit: inputs the model input data into the target reasoning link for reasoning, and obtains the reasoning result output by the target reasoning link.

10. The data reasoning device according to claim 9, after determining the target reasoning link, further comprising: Configuration unit: Determine the configuration information corresponding to the target reasoning link according to the scenario information, and configure the large language model contained in the target reasoning link according to the configuration information; wherein the reasoning system stores the correspondence between the scenario information, the reasoning link and the configuration information, and the configuration information includes at least one of the following: configuration data for indicating the data structure of the data output by each large language model in the reasoning link, verification rules for verifying the reasoning result, and the timeout duration of the reasoning link.

11. The data reasoning device according to claim 9, further comprising: Evaluation unit: evaluates the inference result based on preset evaluation criteria, wherein the evaluation criteria include coverage evaluation criteria, accuracy evaluation criteria, and bias evaluation criteria.

12. The data reasoning device according to claim 11, further comprising: Training unit: If the evaluation is passed, the model input data and the inference result are input as training data into the model to be trained to obtain a dedicated large language model for the scene information.

13. The data reasoning device according to claim 11, further comprising: Retry unit: if the evaluation fails, re-input the model input data into the target reasoning link for secondary reasoning, and send an alarm if the result evaluation of the secondary reasoning fails.

14. A data inference device according to claim 13, The evaluation unit is specifically used to: evaluate the sub-inference results of each large language model in the target reasoning link; The retry unit is specifically used for: if any sub-inference result fails the evaluation, reusing the large language model corresponding to any sub-inference result for reasoning.

15. A data reasoning device according to claim 9, wherein the reasoning system is constructed based on software as a service technology.

16. According to a data inference device according to claim 9, when the scenario information is used to indicate a product recommendation scenario, the data to be inferred includes at least one of the following: offline data, log data, Flink stream data, peripheral data, and message data.

17. An electronic device comprising a processor and a memory; The processor is connected to the memory; The memory is used to store executable program code; The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the method according to any one of claims 1 to 8.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images