A decision model training method, small program inspection method and device

Through training decision models and strengthening learning optimization, the mini program inspection path is automatically generated, which solves the problem of high cost of customized scripts for existing mini program inspections, and achieves a fast and efficient inspection process.

CN117725975BActive Publication Date: 2024-12-06ALIPAY (HANGZHOU) INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410177223.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-02-08
Publication Date
2024-12-06
Estimated Expiration
2044-02-08

AI Technical Summary

Technical Problem

The existing mini-program inspection requires customized inspection scripts according to different business needs, resulting in high time costs and lack of automation and efficiency.

Method used

By training the decision model, using user intention tags and web page interaction data, an automated inspection path is generated, and combined with reinforcement learning to optimize model parameters, the inspection scripts are quickly generated.

Benefits of technology

Automatic inspection of different mini programs has been realized, saving a lot of time and cost, and improving inspection efficiency and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117725975B_ABST
    Figure CN117725975B_ABST
Patent Text Reader

Abstract

One or more embodiments of this specification disclose a method for training a decision model, including: obtaining a user's intent label, and a sequence of interactive actions in which the user interacts with a web page based on the intent described by the intent label; obtaining web page image samples that the user interacts with in the process of executing the interactive action sequence; inputting the web page image samples and the intent label into a generation model to obtain a decision result; and determining a loss function based on the decision result and the decision label to update the parameters of the generation model to obtain a target decision model. The decision model provided by this method can make automated decisions for different applets to quickly generate inspection paths. Accordingly, this specification also discloses a training device for a decision model, and a method and device for inspecting applets.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to a decision model training method, a mini-program inspection method and a device. Background Art

[0002] Mini Program inspection refers to the process from entering the Mini Program to finding the target page. Through Mini Program inspection, risks in the Mini Program can be discovered. Currently, Mini Program inspection requires customized inspection scripts for different Mini Programs based on different business needs, which is very time-consuming and costly to implement. Summary of the invention

[0003] One or more embodiments of this specification describe a decision model training method, a mini-program inspection method and a device. The decision model provided by the method can make automated decisions for different mini-programs to quickly generate inspection paths.

[0004] In a first aspect, a decision model training method is provided, comprising:

[0005] Acquire the user's intention tag and a sequence of interaction actions of the user to interact with the webpage based on the intention described by the intention tag;

[0006] Acquire web page image samples interacted by the user in the process of executing the interactive action sequence;

[0007] Inputting the web page image sample and the intent label into a generation model to obtain a decision result;

[0008] A loss function is determined based on the decision result and the decision label to update the parameters of the generation model to obtain a target decision model; the decision label is determined based on each interactive action in the interactive action sequence and the operation object of the interactive action in the web page image sample.

[0009] As an optional manner of the method described in the first aspect, the training method further includes:

[0010] Inputting the user's intention instruction and the page image of the real applet into the decision model;

[0011] Based on the decision result of the decision model for the page image, performing an interactive operation with the applet and acquiring interactive data;

[0012] A path that successfully reaches the target page specified by the intention instruction is selected from the interaction data as positive feedback, and the remaining paths are used as negative feedback. Based on the strategy of reinforcement learning, the parameters of the target decision model are updated.

[0013] As an optional manner of the method of the first aspect, the generation model includes an encoder and a generator; the web page image sample and the intent label are input into the generation model to obtain a decision result, specifically including:

[0014] Inputting the web page image sample into the encoder to obtain an environment representation;

[0015] The environment representation and the intention label are input into a generator to obtain the decision result.

[0016] Specifically, the encoder is a multimodal large model, and the generator is a large language model; the environment representation and the intention label are input into the generator to obtain the decision result, which specifically includes:

[0017] Converting the environment representation into text features in natural language form;

[0018] The text features and the intention labels are input into the generator to obtain a decision result in text form; the decision result in text form is used to characterize the operation object in the web page image sample and the interactive action performed on the operation object.

[0019] In a second aspect, a small program inspection method is provided, including:

[0020] Inputting the page image of the target applet and the intention instruction set by the user into the decision model; the decision model is trained using the above-mentioned decision model training method;

[0021] Based on each decision result of the decision model for the page image of the target applet, an interactive operation with the target applet is performed until the target page specified by the intention instruction is reached.

[0022] As an optional implementation of the method of the second aspect, based on each decision result of the decision model for the page image of the target applet, performing an interactive operation with the target applet specifically includes:

[0023] For each decision result output by the decision model, the decision result is matched with a preset business logic rule with a higher priority. When the decision result conflicts with the business logic rule, the operation specified by the business logic rule is executed.

[0024] As an optional implementation of the method described in the second aspect, the inspection method further includes:

[0025] Saving each inspection path determined based on the decision result output by the decision model during the inspection process;

[0026] For inspection paths with the same start page and target page, redundant steps are removed through pruning operations to obtain at least one shortest inspection path.

[0027] Specifically, the above inspection method may also include:

[0028] Generate an inspection script according to the shortest inspection path.

[0029] In a third aspect, a training device for a decision model is provided, the training device comprising:

[0030] A first data acquisition module is configured to acquire a user's intention tag, an interaction action sequence of the user interacting with the webpage based on the intention described by the intention tag, and a webpage image sample interacted by the user in the process of executing the interaction action sequence;

[0031] The processing module is configured to input the web page image sample and the intention label into the generation model to obtain a decision result; determine the loss function based on the decision result and the decision label to update the parameters of the generation model to obtain a target decision model; the decision label is determined according to each interactive action in the interactive action sequence and the operation object of the interactive action in the web page image sample.

[0032] As an optional implementation of the device described in the third aspect, the training device may further include:

[0033] A second data acquisition module is configured to acquire the user's intended instruction and the page image of the real mini program;

[0034] The optimization module is configured to input the intention instruction and the page image obtained by the second data acquisition module into the decision model; based on the decision result of the decision model for the page image, perform interactive operations with the mini-program and obtain interactive data; select the path that successfully reaches the target page specified by the intention instruction from the interactive data as positive feedback, and use the remaining paths as negative feedback, and update the parameters of the target decision model based on the reinforcement learning strategy.

[0035] In a fourth aspect, a small program inspection device is provided, the inspection device comprising:

[0036] A third data acquisition module is configured to acquire a page image of a target applet and an intention instruction set by a user;

[0037] A decision generation module is configured to input the page image of the target applet and the intention instruction into a decision model to obtain a decision result; the decision model is trained using the above-mentioned decision model training method;

[0038] The interaction module is configured to perform an interaction operation with the target applet based on each decision result of the decision model for the page image of the target applet until the target page specified by the intention instruction is reached.

[0039] As an optional implementation of the device described in the fourth aspect, the interaction module is specifically used to match each decision result output by the decision model with a pre-set business logic rule with a higher priority, and when the decision result conflicts with the business logic rule, execute the operation specified by the business logic rule.

[0040] As an optional implementation of the device described in the fourth aspect, the inspection device may also include:

[0041] The inspection path generation module is configured to save each inspection path determined based on the decision results output by the decision model during the inspection process; for inspection paths with the same starting page and target page, redundant steps are removed through pruning operations to obtain at least one shortest inspection path.

[0042] Specifically, the inspection device may also include:

[0043] The inspection script generation module is configured to generate an inspection script according to the shortest inspection path.

[0044] In a fifth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the training method of the above-mentioned decision model is implemented, or the mini-program inspection method is implemented.

[0045] In a sixth aspect, an electronic device is provided, including:

[0046] one or more processors; and

[0047] A memory associated with the one or more processors, the memory being used to store program instructions, which, when read and executed by the one or more processors, execute the above-mentioned decision model training method, or the above-mentioned mini-program inspection method.

[0048] One or more embodiments of this specification provide a method for training a decision model, which collects a large amount of behavioral data of users interacting with mini-programs with intent labels, trains a large model, and uses it as a decision model in the inspection process, thereby realizing automated inspection. The decision model can automatically and quickly generate inspection decisions for different inspection intents and different mini-programs, which can save a lot of time costs.

[0049] The decision model training device, mini-program inspection method and device provided by one or more embodiments of this specification also have the above-mentioned beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate one or more embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0051] Figure 1 The flowchart of a decision model training method provided by one or more embodiments of this specification is exemplarily shown.

[0052] Figure 2 A network structure diagram of a decision model provided by one or more embodiments of this specification is exemplarily shown.

[0053] Figure 3 A flowchart of a small program inspection method provided by one or more embodiments of this specification is exemplarily shown.

[0054] Figure 4 The schematic diagram of the structure of a training device for a decision model provided by one or more embodiments of this specification is exemplarily shown.

[0055] Figure 5 The schematic diagram of the structure of a decision model optimization module in a decision model training device provided in one or more embodiments of the present specification is exemplarily shown.

[0056] Figure 6 The schematic diagram of the structure of a small program inspection device provided by one or more embodiments of this specification is exemplarily shown.

[0057] Figure 7 The schematic diagram of the structure of an electronic device provided by one or more embodiments of this specification is exemplarily shown. DETAILED DESCRIPTION

[0058] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0059] Mini Program inspection refers to the intermediate process from entering the Mini Program to finding the target page. Through Mini Program inspection, risks in the Mini Program can be discovered. Currently, the inspection of Mini Programs requires customized inspection scripts for different Mini Programs according to different business needs, and obtains the Mini Program pages specified by the business to meet business purposes (such as checking whether the payment process is normal, whether the page is complete, etc.). However, this solution of writing different inspection processes for different Mini Programs to achieve targeted inspection of specified Mini Programs is very time-consuming to implement.

[0060] Therefore, there is an urgent need for a new inspection script construction solution that can automatically and quickly generate inspection scripts according to different business needs.

[0061] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0062] The following will further describe in detail a decision model training method, mini-program inspection method and device described in the embodiments of this specification in combination with the drawings and specific embodiments of the specification, but this detailed description does not constitute a limitation on the embodiments of this specification.

[0063] In some embodiments, a decision model training method is provided, please refer to Figure 1 , the training method includes steps S100-S104.

[0064] S100: Acquire a user's intention tag, an interaction action sequence of the user interacting with a web page based on the intention described by the intention tag, and a web page image sample interacted with by the user in the process of executing the above interaction action sequence.

[0065] The above-mentioned intent label refers to the purpose of the user's interaction with the web page. For example, if a set of interactive action sequences is to complete the payment operation, then the payment operation is the intent label of the set of interactive action sequences. For example, if a target product is selected in a store and added to the shopping cart, then adding the product to the shopping cart is the intent label of the set of interactive action sequences.

[0066] The above-mentioned interactive actions refer to the user's operation actions on the operation object in the web page. For example, if the user address is filled in the address bar, the address bar is the operation object, and the operation of filling in / entering the user address is the operation action on the operation object. For another example, if a detailed introduction of a target product is opened in the product page, the icon of the target product (clicking the icon can jump to the detailed information page of the target product) or the jump button of the detailed information page is the operation object, and the click operation is the operation action on the operation object.

[0067] The web page image samples mentioned above refer to all web page images from the start page to the end page during the execution of the above interaction action sequence.

[0068] In this step, the web page image samples corresponding to each group of interactive action sequences can be input into the generation model as training samples, the intention labels can be input into the generation model as prompt information, and the operation objects and operation actions corresponding to each interactive action in the group of interactive action sequences can be used as decision labels for the corresponding web page image samples.

[0069] S102: Input the web page image samples and intent labels into the generation model to obtain the decision results.

[0070] After the web page image samples and corresponding intention labels corresponding to each set of interactive action sequences are input into the generation model, the generation model will output at least one decision result for each web page image sample. The decision result is used to characterize the operation object and operation action predicted by the generation model for the web page image sample.

[0071] S104: Determine a loss function based on the decision result and the decision label to update the parameters of the generated model to obtain a target decision model.

[0072] The above-mentioned decision label is determined according to each interactive action in the interactive action sequence and the operation object of the interactive action in the corresponding web page image sample. Specifically, for a group of interactive actions, if the group of interactive actions has n interactive actions, and each interactive action will generate a web page jump, then including the start page, the group of interactive actions will generate a total of n+1 web pages. For the first n web pages, each web page has a decision label, which is used to characterize the operation object (such as button, address bar, etc.) and operation action (such as click, slide, input / fill, etc.) on the web page.

[0073] Therefore, when the generative model outputs a decision result, each decision result will have a corresponding decision label. By regressing the gap between the decision label and the decision result, the parameters of the above generative model can be updated to obtain a decision model that meets the preset conditions.

[0074] A loss function may be used to describe the gap between a decision label and a decision result. The loss function may be selected adaptively according to demand, and this embodiment does not impose any limitation on this.

[0075] In addition, the above-mentioned satisfying the preset condition may specifically refer to the number of updates to the above-mentioned generation model reaching a preset threshold of iterations, or may refer to the value of the loss function being less than a preset threshold.

[0076] It can be seen from the above model training method that the decision model training method described in the embodiment of this specification collects a large amount of behavioral data of users interacting with mini-programs with intent labels, trains a large model, and uses it as a decision model in the inspection process, thereby realizing automated inspection. The decision model can automatically and quickly generate inspection decisions for different inspection intentions and different mini-programs, which can save a lot of time costs.

[0077] In order to better illustrate the training method of the above decision-making model, the following will be explained in combination with a specific implementation scenario.

[0078] Please refer to Figure 2 , Figure 2 A structural diagram of a decision model is schematically provided, and the decision model includes an encoder 201 and a generator 202. The encoder 201 is used to encode the input web page image sample into an environment representation. The generator 202 is used to generate a decision result based on the environment representation and the intention label.

[0079] In some possible implementations, the encoder 201 may be implemented using a pre-trained multimodal large model (MLLM), and the generator 202 may be implemented using a large language model (LLM).

[0080] Specifically, the multimodal large model can be pre-trained through massive amounts of pre-collected web page data, such as web page screenshots, DOM structures, etc.

[0081] When pre-training the above-mentioned multimodal large model, a generative network model can be connected to the output end of the multimodal large model. The training process can be carried out in the following manner:

[0082] Input the original web page image into the multimodal large model to obtain the predicted environment representation;

[0083] Input the environmental representation predicted by the multimodal large model into the generative network model to obtain a generated image;

[0084] A loss function is constructed based on the gap between the generated image and the original web page image, and the parameters of the multimodal large model are updated through the loss function, so that the feature encoding of the original web page image by the multimodal large model is closer to the actual environmental representation of the original web page image.

[0085] After completing the pre-training of the above-mentioned multimodal large model, an environment representation model is obtained. This environment representation model can analyze the web page screenshot data and convert the unstructured web page screenshot data into environment representation.

[0086] Since the input of the large language model (LLM) needs to be text data, it is also necessary to convert the environmental representation output by the multimodal large model into text features in natural language form. Similarly, it is also necessary to convert the intent label into intent description text in natural language form. For example, the intent description text can be: The purpose of this operation is to buy a random item and reach the final payment page. The text in natural language form converted from the environmental representation can be: The current page is the product page, and there are product buttons 1, product buttons 2, product buttons 3 and confirmation buttons on the page; the login button on the login page and the coffee button on the product page have been clicked in the history. So this step should be __". The decision result of the large language model (LLM) is to click a button given in the prompt.

[0087] For Figure 2 The decision model shown can be trained using the following method:

[0088] Input web page image samples into the multimodal large model to obtain the environment representation;

[0089] Convert the environmental representation into a descriptive text in natural language form;

[0090] Input the intent label and description text in natural language into the large language model to obtain the decision result in text form;

[0091] The decision results in text form are mapped to the feature space of the decision label to obtain the gap between the decision results in text form and the corresponding decision label. The loss function constructed based on the gap is used to fine-tune the parameters of the large language model until the target decision model is obtained.

[0092] From the above content, we can see that Figure 2 During the training process of the decision model shown, the image modality is accurately described in language to a large language model (such as LLaMA, chatGLM, etc.) through a pre-trained multimodal large model (MLLM), and then fine-tuned with the corresponding intent labels and training sample data. A decision model with decision-making capabilities can be implemented at a lower cost.

[0093] In some more specific implementations, the parameters of the decision model can be optimized and the model performance can be improved through the results of the interaction between the decision model and the real environment.

[0094] by Figure 2Taking the structure of the decision model shown as an example, in specific implementation, an interface or processing module can be constructed as an intelligent agent to execute the decision results output by the decision model. During the interaction process, the user's intention instructions and the page image of the real applet can be input into the fine-tuned decision model, and then the text-based decision results output by the decision model are converted into interactive instructions that can be understood and executed by the above-mentioned processing module or interface, and then the interactive instructions are transmitted to the above-mentioned processing module or interface, so that the processing module or interface executes the interactive instructions and interacts with the real applet.

[0095] Specifically, a text translation model can be trained to convert the textual decision results output by the decision model into interactive instructions that can be understood and executed by the above-mentioned processing module or interface. This text translation model can adopt but is not limited to a regular matching rule model, a large prediction model LLM, etc.

[0096] In the process of interaction between the agent and the real applet, there may be some wrong, repeated, and redundant decisions, which may lead to failure to reach the target page specified by the user's intention instruction. Therefore, the path that successfully reaches the target page specified by the intention instruction can be selected from the interaction data as positive feedback, and the remaining paths can be used as negative feedback. Based on the strategy of reinforcement learning, the parameters of the above decision model are updated to complete the parameter optimization of the decision model.

[0097] In some embodiments, a small program inspection method is also provided, please refer to Figure 3 , the training method comprises steps S300-S302:

[0098] S300: Inputting the page image of the target applet and the intention instruction set by the user into the decision model.

[0099] S302: Based on each decision result of the decision model for the page image of the target applet, perform an interactive operation with the target applet until the target page specified by the intention instruction is reached.

[0100] In the above-mentioned mini-program inspection method, the decision model is obtained by adopting the above-mentioned decision model training method.

[0101] In some scenarios, after the decision model completes the inspection process specified by the user's intent instruction, the following problems may still exist:

[0102] Question 1: During the inspection process, the path taken based on the decision results of the decision model is not the optimal path, resulting in a waste of resources.

[0103] Question 2: If the inspection process is completely handed over to the decision-making model, the entire process may be uncontrollable. Once the decision-making model makes a wrong decision, it cannot be corrected.

[0104] Problem 3: The exploration results cannot be reproduced stably.

[0105] For problem 1 above, pruning can be used to optimize the inspection path. Specifically, the following methods can be used:

[0106] Save each inspection path determined by the decision results output by the decision model during the inspection process;

[0107] For inspection paths with the same start page and target page, redundant, erroneous, or repeated steps are removed through pruning operations to obtain at least one shortest inspection path.

[0108] For the above problem 2, you can set high-priority business logic rules, for example:

[0109] For each decision result output by the decision model, the decision result can be matched with a pre-set business logic rule with a higher priority. When the decision result conflicts with the business logic rule, the operation specified by the business logic rule is executed. These business logic rules can be adaptively set according to needs, and this embodiment does not limit this. For example, the business logic rules can be set to: when the interface reports an error, stop execution; when sensitive information (user name) needs to be entered, report the behavior and stop execution; if the mini program requires authorization to run, the authorization conditions must be met first, and so on.

[0110] For the above question 3, you can adopt the following methods:

[0111] The shortest inspection path after pruning operation is generated as an inspection script and saved for reuse.

[0112] In some embodiments, a training device for a decision model is also provided. Figure 4 , Figure 4 The exemplary embodiment shows a training device for a decision model, which can be used to implement the training method for the decision model. It should be noted that the training method for the decision model described in one or more embodiments of the present application can rely on Figure 4 The training device of the decision model shown implements, but is not limited to, the training device of the decision model.

[0113] like Figure 4 As shown, the training device of the decision model includes:

[0114] The first data acquisition module 401 is configured to acquire the user's intention tag, the user's interaction action sequence with the web page based on the intention described by the intention tag, and the web page image sample interacted by the user in the process of executing the interaction action sequence.

[0115] The processing module 402 is configured to input the web page image sample and the intent label into the generation model to obtain a decision result; determine the loss function based on the decision result and the decision label to update the parameters of the generation model to obtain a target decision model.

[0116] For the first data acquisition module 401, the intent tag it acquires is used to characterize the purpose of the user's interaction with the web page. For example, if a set of interactive action sequences is to complete a payment operation, then the payment operation is the intent tag of the set of interactive action sequences. For another example, if a target product is selected in a store and added to a shopping cart, then adding the product to the shopping cart is the intent tag of the set of interactive action sequences.

[0117] The interactive action sequence obtained refers to a series of interactive actions that the user performs with the mini program in order to complete a certain intention. Specifically, the above-mentioned interactive actions refer to the user's operation actions on the operation objects in the web page. For example, if the user address is filled in the address bar, the address bar is the operation object, and the operation of filling in / entering the user address is the operation action on the operation object. For another example, if a detailed introduction of a target product is opened on the product page, the icon of the target product (clicking the icon can jump to the detail information page of the target product) or the jump button of the detail information page is the operation object, and the click operation is the operation action on the operation object.

[0118] The web page image samples mentioned above refer to all web page images from the start page to the end page during the execution of the above interaction action sequence.

[0119] The processing module 402 is mainly used to complete the training of the generation model to obtain the decision model. Specifically, after the web page image samples and the corresponding intention labels corresponding to each set of interactive action sequences are input into the generation model, the generation model will output at least one decision result for each web page image sample, and the decision result is used to characterize the operation object and operation action predicted by the generation model for the web page image sample.

[0120] The decision label used in the training process is determined based on each interaction action in the interaction action sequence and the operation object of the interaction action in the corresponding web page image sample. Specifically, for a group of interaction actions, if the group of interaction actions has n interaction actions, and each interaction action will generate a web page jump, then including the start page, the group of interaction actions will generate a total of n+1 web pages. For the first n web pages, each web page has a decision label, which is used to characterize the operation object (such as button, address bar, etc.) and operation action (such as click, slide, input / fill, etc.) on the web page.

[0121] After the generation model outputs the decision results, each decision result will have a corresponding decision label. By regressing the gap between the decision label and the decision result, the processing module 402 can update the parameters of the above generation model to obtain a decision model that meets the preset conditions.

[0122] In some embodiments, the training device of the decision model may further include a decision model optimization module. Figure 5 , the decision model optimization module may include:

[0123] The second data acquisition module 501 is configured to acquire the user's intended instruction and the page image of the real mini-program.

[0124] The optimization module 502 is configured to input the intent instruction and page image obtained by the second data acquisition module 501 into the decision model; based on the decision result of the decision model for the page image, perform interactive operations with the real applet and obtain interactive data; select the path that successfully reaches the target page specified by the intent instruction from the interactive data as positive feedback, and use the remaining paths as negative feedback, and update the parameters of the target decision model based on the reinforcement learning strategy.

[0125] The above-mentioned optimization module 502 can be understood as an intelligent agent, which can be implemented by building an interface or processing module to execute the decision result output by the decision model. Figure 2 Taking the decision model shown as an example, during the interaction between the optimization module 502 and the real mini-program, the user's intention instructions and the page image of the real mini-program can be input into the decision model, and then the decision result in text form output by the decision model is converted into an interactive instruction that can be understood and executed by the optimization module 502, and then the interactive instruction is transmitted to the optimization module 502, so that the optimization module 502 executes the interactive instruction and interacts with the real mini-program.

[0126] More specifically, a text translation model can be trained to convert the textual decision results output by the decision model into interactive instructions that can be understood and executed by the optimization module 502. This text translation model can adopt but is not limited to a regular matching rule model, a large prediction model LLM, etc.

[0127] In the process of interaction between the optimization module 502 and the real applet, there may be some wrong, repeated, and redundant decisions, resulting in failure to reach the target page specified by the user's intent instruction. Therefore, the path that successfully reaches the target page specified by the intent instruction can be selected from the interaction data as positive feedback, and the remaining paths can be used as negative feedback. Based on the strategy of reinforcement learning, the parameters of the above decision model are updated to complete the parameter optimization of the decision model.

[0128] The following takes the above-mentioned training device including the first data acquisition module 401 and the processing module 402 as an example to explain the implementable method of the training device. In actual operation, the first data acquisition module 401 and the processing module 402 can be implemented by software, or can be implemented by hardware. Exemplarily, the implementation method of the first data acquisition module 401 is introduced below by taking the first data acquisition module 401 as an example. Similarly, the implementation method of the processing module 402 can refer to the implementation method of the first data acquisition module 401.

[0129] As an example of a software functional unit, the first data acquisition module 401 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the first data acquisition module 401 may include code running on multiple hosts / virtual machines / containers. Multiple hosts / virtual machines / containers for running the code may be distributed in the same region (region) or in different regions. Furthermore, multiple hosts / virtual machines / containers for running the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple data centers with close geographical locations. Among them, usually a region may include multiple AZs.

[0130] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0131] As an example of a hardware functional unit, the first data acquisition module 401 may include at least one computing device, such as a server, etc. Alternatively, the first data acquisition module 401 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0132] The multiple computing devices included in the first data acquisition module 401 can be distributed in the same region or in different regions. The multiple computing devices included in the first data acquisition module 401 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the first data acquisition module 401 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0133] In other embodiments, the first data acquisition module 401 can be used to execute any step in the above-mentioned decision model training method, and the processing module 402 can be used to execute any step in the above-mentioned decision model training method.

[0134] The steps that the first data acquisition module 401 and the processing module 402 are responsible for implementing can be specified as needed. The first data acquisition module 401 and the processing module 402 respectively implement different steps in the training method of the above-mentioned decision model to realize all the functions of the training device of the decision model.

[0135] In this implementation, the training device of the decision model can also be applied to computing devices such as computers and servers, or to a computing device cluster including at least one computing device, to realize the training function of the decision model.

[0136] In some embodiments, a small program inspection device is also provided. Figure 6 , Figure 6The schematic diagram of the structure of a small program inspection device is shown as an example, which can be used to implement the small program inspection method described above. It should be noted that the small program inspection method described in one or more embodiments of the present application can rely on Figure 6 The mini-program inspection device shown is implemented, but not limited to, the mini-program inspection device.

[0137] like Figure 6 As shown, the small program inspection device includes:

[0138] The third data acquisition module 601 is configured to acquire the page image of the target applet and the intention instruction set by the user.

[0139] The decision generation module 602 is configured to input the page image and the intention instruction of the target applet into the decision model to obtain a decision result; the decision model here is trained by the above-mentioned decision model training method.

[0140] The interaction module 603 is configured to perform an interaction operation with the target applet based on each decision result of the page image of the target applet according to the decision model, until the target page specified by the intention instruction is reached.

[0141] In some scenarios, after the decision model completes the inspection process specified by the user's intent instruction, the following problems may still exist:

[0142] Question 1: During the inspection process, the path taken based on the decision results of the decision model is not the optimal path, resulting in a waste of resources.

[0143] Question 2: If the inspection process is completely handed over to the decision-making model, the entire process may be uncontrollable. Once the decision-making model makes a wrong decision, it cannot be corrected.

[0144] Problem 3: The exploration results cannot be reproduced stably.

[0145] In response to the above-mentioned question 1, in some embodiments, the above-mentioned mini-program inspection device may also include an inspection path generation module, configured to save each inspection path determined by the decision results output by the decision model during the inspection process; for inspection paths with the same starting page and target page, redundant steps are removed through pruning operations to obtain at least one shortest inspection path.

[0146] In response to the above question 2, in some embodiments, the above-mentioned interaction module 603 is also specifically used to match each decision result output by the decision model with a pre-set business logic rule with a higher priority, and when the decision result conflicts with the business logic rule, the operation specified by the business logic rule is executed. These business logic rules can be adaptively set according to needs, and this embodiment does not limit this. For example, the business logic rules can be set to: when the interface reports an error, stop execution; when sensitive information (user name) needs to be entered, report the behavior and stop execution; if the mini program requires authorization to run, the authorization conditions must be met first.

[0147] In response to the above question 3, in some implementations, the above inspection device may further include:

[0148] The inspection script generation module is configured to generate an inspection script according to the shortest inspection path and save it for reuse.

[0149] As an example of a software functional unit, the third data acquisition module 601 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the third data acquisition module 601 may include code running on multiple hosts / virtual machines / containers. Multiple hosts / virtual machines / containers for running the code may be distributed in the same region or in different regions. Furthermore, multiple hosts / virtual machines / containers for running the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple data centers with similar geographical locations. Among them, usually a region may include multiple AZs.

[0150] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0151] As an example of a hardware functional unit, the third data acquisition module 601 may include at least one computing device, such as a server, etc. Alternatively, the third data acquisition module 601 may also be a device implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0152] The multiple computing devices included in the third data acquisition module 601 can be distributed in the same region or in different regions. The multiple computing devices included in the third data acquisition module 601 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the third data acquisition module 601 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0153] In other embodiments, the third data acquisition module 601 can be used to execute any step in the above-mentioned mini-program inspection method, the decision generation module 602 can be used to execute any step in the above-mentioned mini-program inspection method, and the interaction module 603 can be used to execute any step in the above-mentioned mini-program inspection method.

[0154] The steps that the third data acquisition module 601, decision generation module 602 and interaction module 603 are responsible for implementing can be specified as needed. The third data acquisition module 601, decision generation module 602 and interaction module 603 respectively implement different steps in the above-mentioned mini-program inspection method to realize all the functions of the mini-program inspection device.

[0155] In this implementation, the mini-program inspection device can also be applied to computing devices such as computers and servers, or to a computing device cluster including at least one computing device, to implement the mini-program inspection function.

[0156] In some embodiments, an electronic device is also provided. Figure 7, the electronic device includes: a bus 701, a processor 702, a memory 703 and a communication interface 704. The processor 702, the memory 703 and the communication interface 704 communicate with each other through the bus 701. The electronic device can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the electronic device.

[0157] The bus 701 may be a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 The use of only one line does not mean that there is only one bus or one type of bus. Bus 701 may include a path for transmitting information between various components of the electronic device (eg, processor 702, memory 703, and communication interface 704).

[0158] The processor 702 may include any one or more of a processor such as a CPU, a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0159] The memory 703 may include a volatile memory, such as a random access memory (RAM). The memory 703 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0160] The memory 703 stores executable program codes, and the processor 702 executes the executable program codes to respectively implement the functions of the aforementioned first data acquisition module 401 and the processing module 402, that is, to implement the functions of the aforementioned training device for the decision model, thereby implementing the aforementioned training method for the decision model. Alternatively, the memory 703 stores executable program codes, and the processor 702 executes the executable program codes to respectively implement the functions of the aforementioned third data acquisition module 601, the decision generation module 602, and the interaction module 603, that is, to implement the functions of the aforementioned mini-program inspection device, thereby implementing the aforementioned mini-program inspection method.

[0161] That is, the memory 703 stores instructions for executing the training method of the decision model or the mini-program inspection method.

[0162] The communication interface 704 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the electronic device and other devices or a communication network.

[0163] In some embodiments, a computer-readable storage medium is also provided, which stores a computer program. When the computer program runs on an electronic device, the electronic device executes the above-mentioned decision model training method, or implements the above-mentioned mini-program inspection method.

[0164] The computer-readable storage medium may be any available medium that can be stored by the electronic device or a data storage device such as a data center containing one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the electronic device to execute the model training method.

[0165] It is to be understood that the structure illustrated in the embodiments of this specification does not constitute a specific limitation on the system of the embodiments of this specification. In other embodiments of the specification, the above system may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0166] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0167] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0168] It should be noted that the above examples are only specific embodiments of the present invention, and the present invention is obviously not limited to the above examples, and there are many similar variations. All variations directly derived or associated from the contents disclosed by the technicians in this field should fall within the protection scope of the present invention.

Claims

1. A decision model training method, comprising: Acquire the user's intention tag and a sequence of interaction actions of the user to interact with the webpage based on the intention described by the intention tag; Acquire web page image samples interacted by the user in the process of executing the interactive action sequence; Inputting the web page image sample and the intent label into a generation model to obtain a decision result; The web page image sample and the intention label are input into the generation model to obtain the decision result, which specifically includes: inputting the web page image sample into an encoder to obtain an environment representation; inputting the environment representation and the intention label into a generator to obtain the decision result; A loss function is determined based on the decision result and the decision label to update the parameters of the generation model to obtain a target decision model; the decision label is determined based on each interactive action in the interactive action sequence and the operation object of the interactive action in the web page image sample.

2. The method of claim 1, further comprising: Inputting the user's intention instruction and the page image of the real applet into the decision model; Based on the decision result of the decision model for the page image, performing an interactive operation with the applet and acquiring interactive data; A path that successfully reaches the target page specified by the intention instruction is selected from the interaction data as positive feedback, and the remaining paths are used as negative feedback. Based on the strategy of reinforcement learning, the parameters of the target decision model are updated.

3. The method according to claim 1, wherein the encoder is a large multimodal model and the generator is a large language model; inputting the environment representation and the intent label into the generator to obtain the decision result specifically comprises: Converting the environment representation into a description text in a natural language form; The description text and the intention label are input into the generator to obtain a decision result in text form; the decision result in text form is used to characterize the operation object in the web page image sample and the interactive action performed on the operation object.

4. A small program inspection method, comprising: Inputting the page image of the target applet and the intention instruction set by the user into the decision model; The decision model is trained using the method described in any one of claims 1 to 3; Based on each decision result of the decision model for the page image of the target applet, an interactive operation with the target applet is performed until the target page specified by the intention instruction is reached.

5. The method according to claim 4, performing an interactive operation with the target applet based on each decision result of the decision model for the page image of the target applet, specifically comprising: For each decision result output by the decision model, the decision result is matched with a preset business logic rule with a higher priority. When the decision result conflicts with the business logic rule, the operation specified by the business logic rule is executed.

6. The method of claim 4, further comprising: Saving each inspection path determined based on the decision result output by the decision model during the inspection process; For inspection paths with the same start page and target page, redundant steps are removed through pruning operations to obtain at least one shortest inspection path.

7. The method of claim 6, further comprising: Generate an inspection script according to the shortest inspection path.

8. A training device for a decision model, comprising: A first data acquisition module is configured to acquire a user's intention tag, an interaction action sequence of the user interacting with the webpage based on the intention described by the intention tag, and a webpage image sample interacted by the user in the process of executing the interaction action sequence; A processing module configured to input the web page image sample and the intention label into a generation model to obtain a decision result; Determining a loss function based on the decision result and the decision label to update the parameters of the generation model to obtain a target decision model; The decision label is determined according to each interactive action in the interactive action sequence and the operation object of the interactive action in the web page image sample; in Inputting the web page image sample and the intention label into a generation model to obtain a decision result, specifically comprising: inputting the web page image sample into an encoder to obtain an environment representation; The environment representation and the intention label are input into a generator to obtain the decision result.

9. The apparatus of claim 8, further comprising: A second data acquisition module is configured to acquire the user's intended instruction and the page image of the real mini program; an optimization module configured to input the intention instruction and the page image acquired by the second data acquisition module into the decision model; Based on the decision result of the decision model for the page image, performing an interactive operation with the applet and acquiring interactive data; A path that successfully reaches the target page specified by the intention instruction is selected from the interaction data as positive feedback, and the remaining paths are used as negative feedback. Based on the strategy of reinforcement learning, the parameters of the target decision model are updated.

10. A small program inspection device, comprising: A third data acquisition module is configured to acquire a page image of a target applet and an intention instruction set by a user; A decision generation module, configured to input the page image of the target applet and the intention instruction into a decision model to obtain a decision result; the decision model is trained using the method described in any one of claims 1 to 3; The interaction module is configured to perform an interaction operation with the target applet based on each decision result of the decision model for the page image of the target applet until the target page specified by the intention instruction is reached.

11. In the device as described in claim 10, the interaction module is specifically used to match each decision result output by the decision model with a pre-set business logic rule with a higher priority, and when the decision result conflicts with the business logic rule, execute the operation specified by the business logic rule.

12. The apparatus of claim 10, further comprising: An inspection path generation module, configured to save each inspection path determined based on the decision result output by the decision model during the inspection process; For inspection paths with the same start page and target page, redundant steps are removed through pruning operations to obtain at least one shortest inspection path.

13. The apparatus of claim 12, further comprising: The inspection script generation module is configured to generate an inspection script according to the shortest inspection path.

14. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the computer program implements the method according to any one of claims 1 to 3, or implements the method according to any one of claims 4 to 7.

15. An electronic device, comprising: one or more processors; as well as A memory associated with the one or more processors, the memory being used to store program instructions, wherein when the program instructions are read and executed by the one or more processors, the method according to any one of claims 1 to 3 is executed, or the method according to any one of claims 4 to 7 is executed.