Large model-based affair handling process verification method, apparatus and device, and storage medium

Abstract

The invention discloses an affair handling process verification method and device based on a large model, equipment and a storage medium, and relates to the technical field of artificial intelligence, and the affair handling process verification method based on the large model comprises the steps: inputting a target government affair handling guide into a target large model, and obtaining affair handling process information corresponding to the target government affair handling guide; performing accuracy verification on a plurality of step information in the affair handling process information through a target verifier to obtain respective verification results of the step information, the target verifier inputting data by taking the sample affair handling process information as a model, and verifying the accuracy of the sample affair handling process information to obtain verification results of the step information; training by taking a real handling result of handling government affair items based on the sample handling process information as a model training label; and determining the accuracy rate of the affair handling process information based on each verification result, and outputting the affair handling process information to guide the handling of government affair items when the accuracy rate is greater than a preset threshold value. The success rate of handling government affairs based on the affair handling process can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and storage medium for verifying a service process based on a large model. Background Art

[0002] With the rapid development of artificial intelligence technology, large models (such as Transformer, GPT, etc.) have achieved remarkable results in the field of natural language processing.

[0003] At present, when manually combing through the service guides in the government affairs field to obtain easy-to-understand service processes, there is a problem of waste of human resources and time consumption due to the need for cross-departmental collaborative combing and maintenance. In other words, the method of relying on manual combing to obtain service processes is inefficient. Therefore, in order to solve this problem, the industry has begun to use the language understanding and language generation capabilities of large models to convert service guide information into service process information. However, when large models are processing tasks in the government affairs field, in order to ensure the coherence of the language, the generated service process information may contain false information, resulting in low accuracy of the service process and affecting the efficiency of users.

[0004] Therefore, how to improve the success rate of handling government affairs based on the service process is a problem that needs to be solved urgently. Summary of the invention

[0005] The main purpose of this application is to provide a large-model-based service process verification method, device, equipment and storage medium, aiming to improve the success rate of handling government affairs based on the service process.

[0006] To achieve the above objectives, the present application provides a service process verification method based on a large model, and the service process verification method based on a large model includes:

[0007] Input the target government affairs service guide into the target macro model to obtain the service process information corresponding to the target government affairs service guide;

[0008] The accuracy of multiple step information in the service process information is verified by a target verifier to obtain the verification results of each step information, wherein the target verifier is trained by using the sample service process information as model input data and the actual handling results of government affairs handled based on the sample service process information as model training labels;

[0009] The accuracy of the service process information is determined based on each of the verification results, and when the accuracy is greater than a preset threshold, the service process information is output to guide the handling of government affairs.

[0010] In one embodiment, the step of verifying the accuracy of multiple step information in the process information by a target verifier to obtain the verification result of each step information includes:

[0011] Decomposing the service process information into a plurality of step information by a target validator, and determining the guide information corresponding to each step information, wherein the guide information is a government service guide related to the keywords in the step information;

[0012] The target verifier compares each step information with the guide information corresponding to the step information to obtain the verification result of each step information.

[0013] In one embodiment, the step of determining the guide information corresponding to each of the step information includes:

[0014] Determine the keywords of each step information, call the search engine to retrieve the keywords in the preset guide knowledge base, and obtain the guide information corresponding to each step information, wherein the preset guide knowledge base is constructed based on the real-time government service guide.

[0015] In one embodiment, the step of comparing each of the step information with the guide information corresponding to the step information by the target verifier to obtain the verification result of each of the step information includes:

[0016] Inputting each of the step information and each of the guide information into a pre-trained language model respectively to obtain first semantic information of each of the step information and second semantic information of each of the guide information, wherein the pre-trained language model has a semantic analysis function;

[0017] The first semantic information of each step information is respectively compared with the second semantic information of the guide information corresponding to the step information to obtain the verification result of each step information.

[0018] In one embodiment, the step of determining the accuracy of the service process information based on each of the verification results includes:

[0019] Determining a first number of target verification results that are accurate among the verification results;

[0020] The first number is divided by the second number of each verification result to obtain the accuracy rate of the service process information.

[0021] In one embodiment, the target large model is obtained by training using a sample government service guide as model input data and using the actual service process information corresponding to the sample government service guide as model training labels.

[0022] In one embodiment, after the step of determining the accuracy of the service process information based on each of the verification results, the method further includes:

[0023] When the accuracy rate is less than or equal to a preset threshold, the service process information is determined to be unreliable information, and a prompt message is output to remind the user to adopt other sorting methods to obtain guidance on handling government affairs.

[0024] In addition, to achieve the above purpose, the present application also provides a service process verification device based on a large model, and the service process verification device based on a large model includes:

[0025] An acquisition module, used to input the target government service guide into the target large model to obtain the service process information corresponding to the target government service guide;

[0026] A verification module, used to verify the accuracy of multiple step information in the service process information through a target verifier to obtain the verification results of each step information, wherein the target verifier is trained by using the sample service process information as model input data and the actual handling results of government affairs handled based on the sample service process information as model training labels;

[0027] The output module is used to determine the accuracy of the service process information based on each of the verification results, and when the accuracy is greater than a preset threshold, output the service process information to guide the handling of government affairs.

[0028] In addition, to achieve the above-mentioned purpose, the present application also provides a storage medium, which is a computer-readable storage medium, and the computer-readable storage medium stores a program for implementing the service process verification method based on a big model, and the program for implementing the service process verification method based on a big model is executed by a processor to implement the steps of the service process verification method based on a big model as described above.

[0029] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned large model-based service process verification method.

[0030] The present application provides a service process verification method based on a large model. The present application inputs a target government service guide into a target large model to obtain service process information corresponding to the target government service guide. The target verifier verifies the accuracy of multiple step information in the service process information to obtain verification results for each step information. The target verifier uses sample service process information as model input data and is trained using actual results of government affairs handled based on the sample service process information as model training labels. Finally, the accuracy of the service process information is determined based on the verification results of each step information. When the accuracy is greater than a preset threshold, the service process information is output to guide users in handling government affairs.

[0031] In summary, this application takes into account the problem that the service process output by the large model contains false information in order to ensure language coherence. The accuracy of the service process information output by the large model is verified by a pre-trained target verifier to obtain the accuracy of the service process information, and when the accuracy is greater than the preset threshold, that is, when the accuracy of the service process information is high, the service process information is output to guide the handling of matters. In this way, compared with the traditional method of directly using the large model to handle government affairs based on the service process sorted out by the target government affairs service guide, this application ensures that the service process information is output when the accuracy of the service process information is high to guide the handling of matters, avoiding the problem of low success rate of users handling matters based on false information output by the large model, that is, improving the success rate of handling government affairs based on the service process. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0033] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0034] Figure 1 This is a flowchart of the first embodiment of the service process verification method based on a large model of the present application;

[0035] Figure 2 A schematic diagram of a verification process involved in an embodiment of a service process verification method based on a large model of the present application;

[0036] Figure 3 This is a schematic diagram of the module structure of the service process verification device based on the large model of this application;

[0037] Figure 4 Schematic diagram of the device structure of the hardware operating environment involved in the large model-based service process verification method in the embodiment of the present application.

[0038] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0039] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0040] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0041] The main solution of the present application is: input the target government service guide into the target big model to obtain the service process information corresponding to the target government service guide; verify the accuracy of multiple step information in the service process information through the target verifier to obtain the verification results of each step information, wherein the target verifier uses the sample service process information as the model input data, and uses the actual handling results of government affairs handled based on the sample service process information as the model training label for training; determine the accuracy of the service process information based on each of the verification results, and when the accuracy is greater than a preset threshold, output the service process information to guide the handling of government affairs.

[0042] At present, when manually combing through the service guides in the government affairs field to obtain easy-to-understand service processes, there is a problem of waste of human resources and time consumption due to the need for cross-departmental collaborative combing and maintenance. In other words, the method of relying on manual combing to obtain service processes is inefficient. Therefore, in order to solve this problem, the industry has begun to use the language understanding and language generation capabilities of large models to convert service guide information into service process information. However, when large models are processing tasks in the government affairs field, in order to ensure the coherence of the language, the generated service process information may contain false information, resulting in low accuracy of the service process and affecting the efficiency of users.

[0043] This application takes into account the problem that the service process output by the large model contains false information in order to ensure language coherence. The accuracy of the service process information output by the large model is verified through a pre-trained target verifier to obtain the accuracy of the service process information. When the accuracy is greater than a preset threshold, that is, when the accuracy of the service process information is high, the service process information is output to guide the handling of matters. In this way, compared to the traditional method of directly using the large model to handle government affairs based on the service process sorted out by the target government affairs service guide, this application ensures that the service process information is output to guide the handling of matters when the accuracy of the service process information is high, avoiding the problem of low success rate of users handling matters based on false information output by the large model, that is, improving the success rate of handling government affairs based on the service process.

[0044] It should be noted that the execution subject of the method in each embodiment of the service process verification method based on a large model of the present application can be a verification system, or a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or a service process verification device based on a large model that can realize the above functions, etc., and this embodiment does not specifically limit this. The following takes the verification system as an example of the execution subject to illustrate this embodiment and the following embodiments.

[0045] Based on this, this application proposes a first embodiment of a large model-based service process verification method, please refer to Figure 1 The service process verification method based on the big model includes steps S10 to S30:

[0046] Step S10, inputting the target government affairs service guide into the target macro model to obtain the service process information corresponding to the target government affairs service guide;

[0047] It should be noted that the government affairs service guide is a guide for handling various government affairs matters involved in the government affairs field. The service process information corresponding to the government affairs service guide is a user operation process provided to the user group, which is used to guide the handling of government affairs matters.

[0048] In this embodiment, the target large model is trained using a sample government service guide as model input data and using the actual service process information corresponding to the sample government service guide as model training labels.

[0049] It should be noted that the embodiment of the present application uses a pre-collected government service guide (hereinafter referred to as a sample government service guide for distinction) as model input data, and uses the service process information corresponding to the sample government service guide (hereinafter referred to as the real service process information for distinction) as the model training label to train the pre-trained large prediction model to obtain the above-mentioned target large model.

[0050] The government affairs service guide to be sorted out (hereinafter referred to as the target government affairs service guide for distinction) is input into the target big model to obtain the service process information corresponding to the target government affairs service guide.

[0051] Step S20, verifying the accuracy of multiple step information in the service process information through a target verifier to obtain verification results of each step information, wherein the target verifier is trained by using sample service process information as model input data and using actual results of handling government affairs based on the sample service process information as model training labels;

[0052] It should be noted that the target model is pre-trained with the service process information (hereinafter referred to as sample service process information for distinction) output by the target large model based on the government service guide as the model input data, and the actual handling results of government affairs based on the sample service process information as the model training label, to train the initial SAFE (Search-Augmented Factual Eval uator) to obtain the target verifier. Among them, the actual handling result is successful or failed.

[0053] The target verifier verifies the accuracy of multiple step information in the service process information output by the target large model, and obtains the verification results of each step information. It should be understood that the service process information is a process composed of multiple steps. The target verifier is used to verify the accuracy of each step information after determining the multiple step information in the service process information, and obtains the verification results of each step information. The verification results represent the accuracy of the step information. In addition, the accuracy of the service process information can be judged based on the verification results of each step information, that is, each verification result can also represent the handling results of government affairs based on the service process information.

[0054] Step S30, determining the accuracy of the service process information based on each of the verification results, and when the accuracy is greater than a preset threshold, outputting the service process information to guide the handling of government affairs.

[0055] It should be noted that the threshold representing the accuracy of the service process information is set in advance, that is, the above-mentioned preset threshold.

[0056] The accuracy of the entire service process information is determined based on the verification results of each step in the service process information. When the accuracy is greater than a preset threshold, the service process information is wired in the user operation interface of the verification system to guide users to handle corresponding government affairs in accordance with the service process information.

[0057] In a specific implementation, the step S30 may include:

[0058] Step S301, determining a first number of target verification results that are accurate among the verification results;

[0059] Step S302: Divide the first number by the second number of each verification result to obtain the accuracy rate of the service process information.

[0060] It should be noted that the above verification results may be accurate or contain errors.

[0061] Determine the number (hereinafter referred to as the first number for distinction) of verification results that represent accuracy (hereinafter referred to as target verification results for distinction) among each verification result, and determine the total number of each verification result (hereinafter referred to as the second number for distinction), and determine the result value of dividing the first number by the second number as the accuracy rate of the service process information.

[0062] Exemplarily, if the number of step information in the current service process information is 10, the accuracy of each step information is verified through the target verifier to obtain 10 verification results, among which, all 10 verification results are accurate, then the accuracy of the service process information is determined to be 100%. If the preset threshold is 95%, then the service process information is determined to be reliable, and the service process information is output on the user operation interface of the verification system to guide users to handle corresponding government affairs.

[0063] In this embodiment, after step S30, the service process verification method based on the big model of this application further includes:

[0064] Step A10, when the accuracy rate is less than or equal to a preset threshold, the service process information is determined to be unreliable information, and a prompt message is output to remind the user to adopt other sorting methods to obtain guidance on handling government affairs.

[0065] When the accuracy of the service process information is less than or equal to a preset threshold, the service process information is determined to be unreliable information, and a prompt message is output to the user operation interface to remind the user to adopt other sorting methods to obtain handling guidance for the corresponding government affairs.

[0066] The embodiment of the present application takes into account the problem that the service process output by the large model contains false information in order to ensure language coherence. The accuracy of the service process information output by the large model is verified by a pre-trained target verifier to obtain the accuracy of the service process information, and when the accuracy is greater than a preset threshold, that is, when the accuracy of the service process information is high, the service process information is output to guide the handling of matters. In this way, compared to the traditional method of directly using the large model to handle government affairs based on the service process sorted out by the target government affairs service guide, the embodiment of the present application ensures that the service process information is output to guide the handling of matters when the accuracy of the service process information is high, thereby avoiding the problem of low success rate of services caused by users handling matters based on false information output by the large model, that is, improving the success rate of handling government affairs based on the service process.

[0067] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. On this basis, the step S20 may include:

[0068] Step S201, decomposing the service process information into a plurality of step information through a target validator, and determining the guide information corresponding to each step information, wherein the guide information is a government service guide related to a keyword in the step information;

[0069] The target validator decomposes the service process information generated by the target large model into multiple step information, and determines the guide information corresponding to each step information, wherein the guide information of the step information is the government service guide related to the keywords in the step information, that is, the guide information is the accurate standard service guide related to the step information. It should be understood that the guide information corresponding to a step information can be one or more factual descriptions related to the government service guide.

[0070] Exemplarily, after obtaining the service process information output by the target large model, the service process information is preprocessed, including data cleaning and formatting, and then the preprocessed service process information is input into the target validator so that the target validator can split the entire process into multiple specific steps (i.e., the above-mentioned step information). For each step information, natural language processing technology is used to extract keywords, which will be used to search and match the corresponding guide information in the preset government service guide database. Through the verification and association process, it is ensured that each step has highly relevant guide information. Finally, the step information is integrated with the corresponding guide information and output to form a detailed, easy-to-understand service process guide for users to refer to and follow.

[0071] Step S202: The target verifier compares each step information with the guide information corresponding to the step information to obtain a verification result of each step information.

[0072] After obtaining the guide information corresponding to each step information, the target verifier compares each step information with the guide information corresponding to the step information to obtain the verification result of each step information.

[0073] In a feasible implementation, each step information obtained by splitting the entire process is decomposed into atomic-scale fact descriptions through a large model with language understanding and language generation, ensuring that each fact description contains only one fact. For each fact description, it is compared and verified with the content in the preset guide knowledge base to obtain the accuracy of each step information. Then, based on the accuracy of each step information, the reliability of the entire process is calculated by calculating the average or weighted average.

[0074] In another feasible implementation, the step S201 may include:

[0075] Step S2011, determine the keywords of each step information, call the search engine to retrieve the keywords in the preset guide knowledge base, and obtain the guide information corresponding to each step information, wherein the preset guide knowledge base is constructed based on the real-time government service guide.

[0076] For each step information, extract the keywords of the step information, call the search engine through the target validator, enter the keywords into the search box of the search engine, and retrieve the keywords in the preset guide knowledge base (i.e. the above-mentioned government service guide database) to obtain the guide information corresponding to the step information. It should be noted that the preset guide knowledge base is constructed based on the government service guide published in real time, that is, the information in the preset guide knowledge base has a high timeliness. It can be understood that the government service guide published in real time is the latest government service guide.

[0077] In this embodiment, the target verifier conducts in-depth analysis on each step information to identify and extract keywords reflecting the core content of the step, and then uses natural language processing technology to further optimize the extraction of keywords. After obtaining the keywords of the step information, the target verifier calls the search engine API (Application Programming Interface) to search for these keywords in the preset guide knowledge base built based on the real-time government service guide. The search result is the guide information related to the step information.

[0078] In this implementation, the step S202 may include:

[0079] Step S2021, inputting each of the step information and each of the guide information into a pre-trained language model respectively, to obtain first semantic information of each of the step information and second semantic information of each of the guide information, wherein the pre-trained language model has a semantic analysis function;

[0080] It should be noted that the above-mentioned pre-trained language model can be a BERT model or a GPT model, etc. The pre-trained language model has language understanding ability and language generation ability.

[0081] Each step information and each guide information is input into the pre-trained language model respectively, and semantic analysis is performed on each step information and each guide information through the pre-trained language model to obtain the semantic information of each step information (hereinafter referred to as the first semantic information for distinction) and the semantic information of each guide information (hereinafter referred to as the second semantic information for distinction).

[0082] Step S2022: respectively compare the first semantic information of each step information with the second semantic information of the guide information corresponding to the step information to obtain the verification result of each step information.

[0083] After obtaining the first semantic information of each step information and the second semantic information of each guide information, the first semantic information of each step information and the second semantic information of the guide information corresponding to the step information are compared respectively to obtain the comparison result corresponding to each step information. When the comparison result is consistent, the verification result of the step information is determined to be accurate. When the comparison result is inconsistent, the verification result of the step information is determined to have an error.

[0084] For example, Figure 2 The figure shows a schematic diagram of the verification process. First, the service process information output by the target large model is obtained; the service process information is decomposed into multiple step information through the target verifier; the keywords in each step information are extracted; the keywords are searched in the preset guide knowledge base to obtain the guide information corresponding to each step information; the semantic information of each step information and each guide information is determined; the first semantic information of each step information and the second semantic information of the guide information corresponding to the step information are compared to obtain the verification result of each step information; the accuracy of the current service process information is calculated based on each verification result; it is determined whether the accuracy is greater than a preset threshold; if so, the service process information is output; if not, a prompt message is output to remind the user to adopt other sorting methods to obtain the service process.

[0085] In this way, the embodiment of the present application utilizes the target validator to decompose the steps of the service process information output by the target big model to obtain multiple step information, and then obtains the standard government service guide related to each step information, and compares the semantic information of each step information with the semantic information of the standard government service guide corresponding to the step information. When the comparison result is consistent, the accuracy verification result of the step information is determined to be accurate, otherwise there is an error, thereby realizing the verification of the service process information output by the target big model, and after the accuracy verification of the service process information is passed, it is output to guide the handling of government affairs, thereby improving the success rate of handling government affairs based on the service process.

[0086] The present application also provides a service process verification device based on a large model, please refer to Figure 3 , the service process verification device based on the large model includes:

[0087] The acquisition module 10 is used to input the target government affairs service guide into the target large model to obtain the service process information corresponding to the target government affairs service guide;

[0088] The verification module 20 is used to verify the accuracy of multiple step information in the service process information through a target verifier to obtain the verification results of each step information, wherein the target verifier is trained by using the sample service process information as model input data and the actual handling results of government affairs handled based on the sample service process information as model training labels;

[0089] The output module 30 is used to determine the accuracy of the service process information based on each of the verification results, and when the accuracy is greater than a preset threshold, output the service process information to guide the handling of government affairs.

[0090] Optionally, the verification module 20 is further used for:

[0091] Decomposing the service process information into a plurality of step information by a target validator, and determining the guide information corresponding to each step information, wherein the guide information is a government service guide related to the keywords in the step information;

[0092] The target verifier compares each step information with the guide information corresponding to the step information to obtain the verification result of each step information.

[0093] Optionally, the verification module 20 is further used for:

[0094] Determine the keywords of each step information, call the search engine to retrieve the keywords in the preset guide knowledge base, and obtain the guide information corresponding to each step information, wherein the preset guide knowledge base is constructed based on the real-time government service guide.

[0095] Optionally, the verification module 20 is further used for:

[0096] Inputting each of the step information and each of the guide information into a pre-trained language model respectively to obtain first semantic information of each of the step information and second semantic information of each of the guide information, wherein the pre-trained language model has a semantic analysis function;

[0097] The first semantic information of each step information is respectively compared with the second semantic information of the guide information corresponding to the step information to obtain the verification result of each step information.

[0098] Optionally, the output module 30 is further used for:

[0099] Determining a first number of target verification results that are accurate among the verification results;

[0100] The first number is divided by the second number of each verification result to obtain the accuracy rate of the service process information.

[0101] Optionally, the target large model is obtained by training using a sample government service guide as model input data and using real service process information corresponding to the sample government service guide as model training labels.

[0102] Optionally, the large model-based service process verification device further includes:

[0103] The reminder module is used to determine that the service process information is unreliable information when the accuracy rate is less than or equal to a preset threshold, and output a prompt message to remind the user to adopt other sorting methods to obtain guidance on the handling of government affairs.

[0104] The service process verification device based on a large model provided in the embodiment of the present application adopts the service process verification method based on a large model in the above embodiment, which can solve the technical problem of how to improve the success rate of handling government affairs based on the service process. Compared with the prior art, the beneficial effects of the service process verification device based on a large model provided in the embodiment of the present application are the same as the beneficial effects of the service process verification method based on a large model provided in the above embodiment, and the other technical features in the service process verification device based on a large model are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0105] The present application provides a service process verification device based on a big model, and the service process verification device based on a big model includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the service process verification method based on the big model in the above-mentioned embodiment one.

[0106] Reference below Figure 4 , which shows a schematic diagram of the structure of a large-model-based service process verification device suitable for implementing the embodiment of the present application. The large-model-based service process verification device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Multimedia Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 4 The large model-based service process verification device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0107] like Figure 4As shown, the service process verification device based on the large model may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read On ly Memory) 1002 or the program loaded from the storage device 1003 to the random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the service process verification device based on the large model are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the large-scale model-based service process verification device to communicate wirelessly or wired with other devices to exchange data. Although the large-scale model-based service process verification device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have alternatively.

[0108] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0109] The service process verification device based on a large model provided by the present application adopts the service process verification method based on a large model in the above-mentioned embodiment, which can solve the technical problem of how to improve the success rate of handling government affairs based on the service process. Compared with the prior art, the beneficial effects of the service process verification device based on a large model provided by the present application are the same as the beneficial effects of the service process verification method based on a large model provided by the above-mentioned embodiment, and the other technical features in the service process verification device based on a large model are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0110] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0111] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0112] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the large model-based service process verification method in the above-mentioned embodiment.

[0113] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM: Erasable Programmab le Read On ly Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read On ly Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0114] The computer-readable storage medium may be included in the service process verification device based on the large model; or may exist independently without being assembled into the service process verification device based on the large model.

[0115] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the service process verification device based on the big model, the service process verification device based on the big model: inputs the target government service guide into the target big model to obtain the service process information corresponding to the target government service guide; verifies the accuracy of multiple step information in the service process information through the target verifier to obtain the verification results of each step information, wherein the target verifier uses the sample service process information as the model input data, and is trained with the actual handling results of government affairs based on the sample service process information as the model training label; determines the accuracy of the service process information based on each of the verification results, and when the accuracy is greater than a preset threshold, outputs the service process information to guide the handling of government affairs.

[0116] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0117] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0118] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0119] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned large-model-based service process verification method, and can solve the technical problem of how to improve the success rate of handling government affairs based on service processes. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the large-model-based service process verification method provided by the above-mentioned embodiment, and will not be repeated here.

[0120] An embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned large model-based service process verification method.

[0121] The computer program product provided in this application can improve the success rate of handling government affairs based on the process. Compared with the prior art, the beneficial effects of the computer program product provided in the embodiment of this application are the same as the beneficial effects of the process verification method based on the large model provided in the above embodiment, which will not be repeated here.

[0122] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

Claims

1. A method for verifying a service process based on a large model, characterized in that: The service process verification method based on the large model includes: Input the target government affairs service guide into the target macro model to obtain the service process information corresponding to the target government affairs service guide; The accuracy of multiple step information in the service process information is verified by a target verifier to obtain the verification results of each step information, wherein the target verifier is trained by using the sample service process information as model input data and the actual handling results of government affairs handled based on the sample service process information as model training labels; The accuracy of the service process information is determined based on each of the verification results, and when the accuracy is greater than a preset threshold, the service process information is output to guide the handling of government affairs.

2. The method according to claim 1, characterized in that The step of verifying the accuracy of multiple step information in the process information by using a target verifier to obtain the verification result of each step information includes: Decomposing the service process information into a plurality of step information by a target validator, and determining the guide information corresponding to each step information, wherein the guide information is a government service guide related to the keywords in the step information; The target verifier compares each step information with the guide information corresponding to the step information to obtain the verification result of each step information.

3. The method according to claim 2, characterized in that The step of determining the guide information corresponding to each of the step information comprises: Determine the keywords of each step information, call the search engine to retrieve the keywords in the preset guide knowledge base, and obtain the guide information corresponding to each step information, wherein the preset guide knowledge base is constructed based on the real-time government service guide.

4. The method according to claim 2, characterized in that The step of comparing each of the step information with the guide information corresponding to the step information by the target verifier to obtain the verification result of each of the step information comprises: Inputting each of the step information and each of the guide information into a pre-trained language model respectively to obtain first semantic information of each of the step information and second semantic information of each of the guide information, wherein the pre-trained language model has a semantic analysis function; The first semantic information of each step information is respectively compared with the second semantic information of the guide information corresponding to the step information to obtain the verification result of each step information.

5. The method according to claim 1, characterized in that The step of determining the accuracy of the service process information based on each of the verification results includes: Determining a first number of target verification results that are accurate among the verification results; The first number is divided by the second number of each verification result to obtain the accuracy rate of the service process information.

6. The method according to claim 1, characterized in that The target large model is trained by using a sample government service guide as model input data and using the actual service process information corresponding to the sample government service guide as model training labels.

7. The method according to claim 1, characterized in that After the step of determining the accuracy of the service process information based on each of the verification results, the method further includes: When the accuracy rate is less than or equal to a preset threshold, the service process information is determined to be unreliable information, and a prompt message is output to remind the user to adopt other sorting methods to obtain guidance on handling government affairs.

8. A service process verification device based on a large model, characterized in that: The service process verification device based on the large model includes: An acquisition module, used to input the target government affairs service guide into the target large model, and obtain the service process information corresponding to the target government affairs service guide; A verification module, used to verify the accuracy of multiple step information in the service process information through a target verifier to obtain the verification results of each step information, wherein the target verifier is trained by using the sample service process information as model input data and the actual handling results of government affairs handled based on the sample service process information as model training labels; The output module is used to determine the accuracy of the service process information based on each of the verification results, and when the accuracy is greater than a preset threshold, output the service process information to guide the handling of government affairs.

9. A service process verification device based on a large model, characterized in that: The large model-based service process verification device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the large model-based service process verification method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the large-model-based service process verification method as described in any one of claims 1 to 7 are implemented.

Images