Monitoring case generation and language model training method, device, equipment and medium

Abstract

The present disclosure provides a monitoring case generation and language model training method, device, equipment and medium, relating to the fields of NLP, LLM, deep learning, etc. The specific implementation scheme is: in response to monitoring a first event, it is inquired whether there is a first monitoring case associated with a first access address of a target page, if not, the markup language text of the target page is obtained according to the first access address; the page function description information of the target page is obtained; the first core element is determined from the markup language text according to the page function description information; and the second monitoring case is generated according to the first core element. Thus, when it is monitored that a certain page is added or a certain page is updated, the monitoring case for monitoring the page can be automatically and timely generated according to the core element in the page without manually creating the monitoring case of the page, which can improve the generation efficiency of the monitoring case and thus reduce the risk of online problem leakage.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of AI (Artificial Intelligence), in particular to the technical field of NLP (Natural Language Processing), LLMs (Large Language Models) deep learning, and the like, and more particularly to a method and device for generating monitoring cases and training a language model, equipment and a medium. BACKGROUND

[0002] After an APP (Application) page goes online, the page can be monitored by a monitoring case to check whether page elements are loaded and displayed at corresponding positions in the page, and the like. If a page element is not displayed at a corresponding position, an alarm can be raised to repair the page, thereby reducing the risk of online problem leakage.

[0003] Therefore, how to generate a monitoring case corresponding to a page is very important. SUMMARY

[0004] The present disclosure provides a method and device for generating monitoring cases and training a language model, equipment and a medium.

[0005] According to an aspect of the present disclosure, a method for generating a monitoring case is provided, comprising:

[0006] In response to monitoring a first event, it is determined whether a first monitoring case associated with a first access address of a target page exists; wherein the first event is used to indicate that the target page is newly added or the target page is updated;

[0007] When the first monitoring case does not exist, a markup language text of the target page is obtained according to the first access address, and page function description information of the target page is obtained;

[0008] According to the page function description information, a first core element is determined from the markup language text; wherein the first core element has a higher importance than other page elements in the target page;

[0009] According to the first core element, a second monitoring case associated with the first access address is generated; wherein the second monitoring case is used to monitor the target page.

[0010] According to another aspect of the present disclosure, a method for training a language model is provided, comprising:

[0011] obtain a training sample; wherein the training sample comprises markup language text and page function description information of the sample page;

[0012] input the training sample into a language model for processing to obtain a predicted core element output by the language model;

[0013] train the language model according to a difference between the predicted core element and a labeled core element corresponding to the training sample to obtain a trained language model;

[0014] The labeled core element is used to generate a sample monitoring use case associated with the sample page, and the sample monitoring use case is used to monitor the sample page.

[0015] According to still another aspect of the present disclosure, a monitoring use case generation apparatus is provided, comprising:

[0016] The query module is configured to query whether a first monitoring use case associated with a first access address of a target page exists in response to monitoring a first event; wherein the first event is used to indicate that the target page is newly added or the target page is updated;

[0017] The acquisition module is configured to acquire markup language text of the target page and page function description information of the target page according to the first access address when the first monitoring use case does not exist;

[0018] The determination module is configured to determine a first core element from the markup language text according to the page function description information; wherein the first core element has a higher importance than other page elements in the target page;

[0019] The generation module is configured to generate a second monitoring use case associated with the first access address according to the first core element; wherein the second monitoring use case is used to monitor the target page.

[0020] According to still another aspect of the present disclosure, a language model training apparatus is provided, comprising:

[0021] The acquisition module is configured to acquire a training sample; wherein the training sample comprises markup language text and page function description information of a sample page;

[0022] The processing module is configured to input the training sample into a language model for processing to obtain a predicted core element output by the language model;

[0023] The training module is configured to train the language model according to a difference between the predicted core element and a labeled core element corresponding to the training sample to obtain a trained language model;

[0024] The core element is used to generate a sample monitoring use case associated with the sample page, and the sample monitoring use case is used to monitor the sample page.

[0025] According to still another aspect of the present disclosure, an electronic device is provided, comprising:

[0026] at least one processor; and

[0027] a memory in communication with the at least one processor; wherein

[0028] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the monitoring use case generation method according to the above-mentioned aspect of the present disclosure, or perform the language model training method according to the above-mentioned another aspect of the present disclosure.

[0029] According to still another aspect of the present disclosure, a non-transitory computer-readable storage medium of computer instructions is provided, the computer instructions being used to cause the computer to perform the monitoring use case generation method according to the above-mentioned aspect of the present disclosure, or perform the language model training method according to the above-mentioned another aspect of the present disclosure.

[0030] According to still another aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the monitoring use case generation method according to the above-mentioned aspect of the present disclosure, or when executed, implements the language model training method according to the above-mentioned another aspect of the present disclosure.

[0031] It should be understood that the contents described in this part are not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0032] The accompanying drawings are used to better understand the present scheme and do not limit the present disclosure. Among them:

[0033] Figure 1 a flowchart of the monitoring use case generation method provided by the first embodiment of the present disclosure;

[0034] Figure 2 a flowchart of the monitoring use case generation method provided by the second embodiment of the present disclosure;

[0035] Figure 3 a flowchart of the monitoring use case generation method provided by the third embodiment of the present disclosure;

[0036] Figure 4 A flowchart of a method for generating a monitoring use case provided in the fourth embodiment of the present disclosure;

[0037] Figure 5 A flowchart of a method for generating a monitoring use case provided in the fifth embodiment of the present disclosure;

[0038] Figure 6 A flowchart of the language model training method provided in the sixth embodiment of the present disclosure;

[0039] Figure 7 A flowchart of a language model training method provided in Embodiment 7 of the present disclosure;

[0040] Figure 8 A flowchart of the language model training method provided in the eighth embodiment of the present disclosure;

[0041] Figure 9 A schematic diagram of the implementation principle of an embodiment of the present disclosure;

[0042] Figure 10 This is a schematic diagram of the working mode of the existing online registration system;

[0043] Figure 11 A schematic diagram of the workflow of the online registration system provided in an embodiment of the present disclosure;

[0044] Figure 12 This is a workflow diagram of the existing front-end pipeline;

[0045] Figure 13 A schematic diagram of the workflow of the front-end pipeline provided by an embodiment of the present disclosure;

[0046] Figure 14 A schematic diagram of the structure of a device for generating a monitoring use case provided in the ninth embodiment of the present disclosure;

[0047] Figure 15 A schematic diagram of the structure of a language model training device provided in the tenth embodiment of the present disclosure;

[0048] Figure 16 A schematic block diagram of an example electronic device that can be used to implement embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0049] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0050] Automatic generation of monitoring case refers to that the front-end monitoring case can be updated in real time and automatically with the iteration of business requirements (or product requirements, front-end requirements), and no longer needs manual maintenance.

[0051] Currently, there is no technology for automatically generating front-end monitoring cases, and all monitoring cases are created or updated manually by testers. Among them, the following three schemes are mainly used to create or update monitoring cases:

[0052] First, monitoring alarm. After updating the page according to the business requirements (or product requirements, front-end requirements), if the changed elements in the updated page involve the elements that need to be checked in the monitoring case, the monitoring case still checks the page before the change after the updated page goes online, so the monitoring starts to alarm. After receiving the alarm notification, if it is found that the monitoring false alarm is caused by business requirement change, the monitoring person in charge is contacted to modify the monitoring case.

[0053] Second, periodic update. The monitoring person in charge periodically checks whether there are new pages that have not added monitoring cases, and if so, adds the corresponding monitoring cases; checks whether there are some pages that have been greatly changed and the monitoring cases have not been updated, and if so, modifies the corresponding monitoring cases; checks whether there are some pages that have been offline but the monitoring cases have not been deleted, and if so, deletes the corresponding monitoring cases.

[0054] Third, online registration system. Before the front-end requirement goes online, the tester registers the changed pages and the development branch in the system and sends a notification to the monitoring person in charge. The monitoring person in charge updates the monitoring cases of the changed pages within a week after receiving the notification.

[0055] The use scenarios of the above three schemes are different, and the problems and deficiencies they bring are also different:

[0056] The use scenario of the first scheme is that the updated elements in the changed page involve the detection items of the monitoring case, the application range is relatively limited, and the monitoring false alarm also needs to consume human resources to investigate, which cannot achieve real-time updating of the monitoring case.

[0057] The implementation of the second scheme is relatively simple, but it cannot achieve timely updating of the monitoring case. When a new business requirement appears online, the monitoring case cannot perceive and alarm in time, and there is a risk of missing a large online problem.

[0058] The third solution can timely generate a change notification, but the update of the monitoring case depends on the manpower of the monitoring person in charge, and the update time of the monitoring case is up to one week, which still cannot update the monitoring case in real time, and there is a risk of missing online problems.

[0059] In summary, the above solutions cannot realize real-time update of the monitoring case, and the update of the monitoring case depends on manual implementation, which has high labor cost and long update period.

[0060] Therefore, in view of at least one of the above problems, the present disclosure proposes a monitoring case generation and language model training method, device, equipment and medium.

[0061] The monitoring case generation and language model training method, device, equipment and medium of the embodiments of the present disclosure are described below with reference to the drawings. Before specifically describing the embodiments of the present disclosure, in order to facilitate understanding, first introduce the commonly used technical terms:

[0062] LLMs are a class of natural language processing models based on deep learning, which mainly have the characteristics of large model parameters and complex neural network structure, and have strong language understanding ability, context perception ability and language generation ability. It can automatically learn useful feature representations from input data and generate related text.

[0063] Figure 1 The flowchart of the monitoring case generation method provided by Embodiment One of the present disclosure.

[0064] The embodiments of the present disclosure take the monitoring case generation method configured in the monitoring case generation device as an example, which can be applied to any electronic device, so that the electronic device can perform the monitoring case generation function.

[0065] Among them, the electronic device can be any device with computing power, such as a personal computer, a mobile terminal, a server, etc. The mobile terminal may, for example, be a vehicle-mounted device, a mobile phone, a tablet computer, a personal digital assistant, a wearable device, etc. Hardware devices with various operating systems, touch screens and / or display screens.

[0066] As shown in Figure 1 The monitoring case generation method can include the following steps:

[0067] Step S101, in response to monitoring a first event, querying whether there is a first monitoring case associated with the first access address of the target page.

[0068] Among them, the first event is used to indicate that a target page is added or a target page is updated.

[0069] In an embodiment of the present disclosure, each page can be monitored to determine whether there is a newly added target page, or whether there is an updated target page. If so, a first event is generated based on the access address of the target page (referred to as the first access address in this disclosure, such as a URL (Uniform Resource Locator) address) and the page function description information.

[0070] Alternatively, the first event can be provided by relevant personnel. For example, when there is a new target page in each page, or when there is an updated target page, the tester can provide the first access address and page function description information of the target page to generate the first event based on the first access address and page function description information.

[0071] In an embodiment of the present disclosure, it is possible to query various stored monitoring use cases to see whether there is a monitoring use case associated with the first access address in the first event (referred to as the first monitoring use case in the present disclosure).

[0072] Step S102 : when the first monitoring use case does not exist, obtaining the markup language text of the target page according to the first access address, and obtaining the page function description information of the target page.

[0073] In an embodiment of the present disclosure, when there is no first monitoring use case associated with the first access address in the stored monitoring use cases, it can be determined that the target page is a newly added page. At this time, the markup language text of the target page (such as HTML (Hyper Text Markup Language, Hypertext Markup Language) text, or called HTML document, HTML file) can be obtained based on the first access address, and the page function description information of the target page can be obtained. For example, the page function description information of the target page can be obtained from the first event.

[0074] Step S103 : determining a first core element from the markup language text according to the page function description information; wherein the first core element is more important than other page elements on the target page.

[0075] In an embodiment of the present disclosure, at least one core page element (referred to as the first core element in the present disclosure) can be determined from the markup language text based on the page function description information, wherein the first core element is more important in the target page than other page elements.

[0076] Step S104: Generate a second monitoring use case associated with the first access address based on the first core element; wherein the second monitoring use case is used to monitor the target page.

[0077] In the embodiments of the present disclosure, the second monitoring case associated with the first access address can be generated according to the first core element, and the second monitoring case is used for monitoring the target page.

[0078] As an example, the check item can be generated according to the class name of the first core element, and the second monitoring case can be generated according to the check item. The second monitoring case can check whether the first core element exists in the target page based on the check item. If not, an alarm can be performed. For example, when a service is abnormal, some page elements in the target page can not be loaded. Through the check item in the second monitoring case, it can be checked whether the target page has abnormal page elements. If yes, an alarm can be performed.

[0079] The method for generating the monitoring case provided in the embodiments of the present disclosure can query whether the first monitoring case associated with the first access address of the target page exists in response to monitoring the first event. If not, the markup language text of the target page can be obtained according to the first access address, and the page function description information of the target page can be obtained. The first core element can be determined from the markup language text according to the page function description information. The second monitoring case associated with the first access address can be generated according to the first core element. Thus, when a new page is monitored or a page is updated, the monitoring case for monitoring the page can be automatically and timely generated according to the core element in the page without manually creating the monitoring case of the page, which can improve the generation efficiency of the monitoring case and reduce the risk of missing online problems.

[0080] It should be noted that in the technical solutions of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information are all carried out on the premise of obtaining the consent of the user and in accordance with relevant laws and regulations, and do not violate public order and good customs.

[0081] In order to clearly illustrate how the first core element is determined from the markup language text according to the page function description information in the above embodiments, the present disclosure further provides a method for generating a monitoring case.

[0082] Figure 2 The flowchart of the method for generating a monitoring case provided in Embodiment Two of the present disclosure.

[0083] As Figure 2 shown, the method for generating a monitoring case can include the following steps:

[0084] In step S201, in response to monitoring the first event, it is queried whether the first monitoring case associated with the first access address of the target page exists.

[0085] The first event is used to indicate that a target page is added or the target page is updated.

[0086] In step S202, when the first monitoring case does not exist, the markup language text of the target page is obtained according to the first access address, and the page function description information of the target page is obtained.

[0087] The explanation of steps S201 to S202 can refer to the related description in any embodiment of the present disclosure, which will not be repeated here.

[0088] In step S203, the markup language text is parsed to obtain a first element list.

[0089] The first element list includes at least one leaf element, and the leaf element does not nest any page element.

[0090] In the embodiments of the present disclosure, the markup language text can be parsed to obtain at least one leaf element, and the first element list can be generated according to the at least one leaf element. For example, the first element list can include the class name of each leaf element.

[0091] As an example, taking the markup language text as an HTML text as an example, for the HTML text or file, when a page element nests other page elements, the div container corresponding to the page element will contain the div container of the other page element, therefore, in the present disclosure, the detection logic of the leaf element can be, for example, traversing each div container in the HTML text, if the current traversed div container does not contain other containers, it is considered that the page element corresponding to the current traversed div container is a leaf element, at this time, the class name of the current traversed div container can be extracted as the class name of the leaf element.

[0092] In step S204, the first core element is determined from the first element list according to the markup language text and the page function description information.

[0093] The first core element is more important than other leaf elements in the target page.

[0094] In the embodiments of the present disclosure, the first core element can be determined from each leaf element in the first element list according to the markup language text and the page function description information, wherein the first core element is more important than other leaf elements in the target page.

[0095] In step S205, the second monitoring case associated with the first access address is generated according to the first core element, wherein the second monitoring case is used to monitor the target page.

[0096] The explanation of step S205 can refer to the related description in any embodiment of the present disclosure, which is not described here.

[0097] The generation method of the monitoring use case provided in the third embodiment of the present disclosure can improve the accuracy and reliability of the determination of the first core element.

[0098] In order to clearly illustrate how the first core element is determined from the first element list according to the markup language text and the page function description information in the above embodiments, the present disclosure further provides a generation method of a monitoring use case.

[0099] Figure 3 The flowchart of the generation method of the monitoring use case provided in the third embodiment of the present disclosure.

[0100] As Figure 3 shown, the generation method of the monitoring use case can include the following steps:

[0101] Step S301, in response to monitoring a first event, querying whether there is a first monitoring use case associated with a first access address of a target page.

[0102] The first event is used to indicate that a target page is newly added or the target page is updated.

[0103] Step S302, when there is no first monitoring use case, obtaining the markup language text of the target page according to the first access address, and obtaining the page function description information of the target page.

[0104] Step S303, parsing the markup language text to obtain a first element list.

[0105] The first element list includes at least one leaf element, and the leaf element does not nest any page element.

[0106] The explanation of steps S301 to S303 can refer to the related description in any embodiment of the present disclosure, which is not described here.

[0107] Step S304, obtaining prompt information; wherein the prompt information is used to indicate the task information to be processed by the language model and the number of core elements output by the language model.

[0108] In order to improve the prediction accuracy of the model, the language model may, for example, be LLMs.

[0109] In the embodiments of the present disclosure, the prompt information (Prompt) can be used to indicate the task information to be processed by the language model and the number of core elements output by the language model.

[0110] As an example, the prompt information can be "select N (exemplarily, N is 5 for example) most core leaf elements from the leaf element list (denoted as the first element list in the disclosure) and output", "select N most core leaf elements from the leaf element list (i.e. the first element list) and output, and output their positions and / or roles in the page (or webpage)", and the like.

[0111] In step S305, the first element list, the markup language text and the page function description information are processed according to the prompt information by using the language model to obtain a second element list output by the language model.

[0112] The first core element has a higher importance in the target page than other leaf elements.

[0113] In the embodiments of the disclosure, the first element list, the markup language text and the page function description information can be processed according to the prompt information by using the trained language model to obtain a second element list output by the language model. The second element list includes each first core element, the first core element has a higher importance in the target page than other leaf elements, and the number of the first core elements is consistent with the number indicated by the prompt information.

[0114] It should be noted that the number of characters contained in the markup language text (such as HTML text) can be relatively large. In order to improve the effectiveness of model prediction, in any one of the embodiments of the disclosure, when the number of characters contained in the markup language text is relatively large, the markup language text can be processed by fragmentation to obtain a plurality of text segments, and the plurality of text segments are sequentially input into the language model for processing to obtain a second element list output by the language model.

[0115] As an example, the number of characters contained in the markup language text can be counted, and it is determined whether the number of characters is higher than the upper limit of the input requirement defined by the language model. If the number of characters is not higher than the upper limit, the markup language text does not need to be processed by fragmentation, and the markup language text can be input into the language model for processing in its entirety.

[0116] If the number of characters is higher than the upper limit, the markup language text is processed by fragmentation to obtain a plurality of text segments, each of which contains a number of characters less than or equal to the upper limit. In the disclosure, the prompt information, the first element list and the page function description information can be input into the language model, and the plurality of text segments are sequentially input into the language model. The language model processes the first element list, the plurality of text segments and the page function description information according to the prompt information to obtain a second element list.

[0117] Thus, the number of characters contained in the data input to the language model can be matched with the input requirement of the language model, and the effectiveness and accuracy of the language model prediction can be improved.

[0118] In step S306, a second monitoring use case associated with the first access address is generated according to each first core element in the second element list.

[0119] The second monitoring use case is used to monitor the target page.

[0120] The explanation of step S306 can be referred to the related description in any embodiment of the present disclosure, which will not be repeated here.

[0121] The monitoring use case generation method of the embodiments of the present disclosure can predict each core element in the target page based on deep learning technology, which can improve the accuracy of the prediction result. At the same time, the prompt information, the first element list, the markup language text and the page function description information are used as the input of the language model, which can improve the prediction effect of the language model. That is, the prompt information can be used as prior information or task information to indicate what kind of information the language model needs to extract, which can improve the expression ability of the language model.

[0122] In order to clearly illustrate how the second monitoring use case associated with the first access address is generated according to the first core element in any embodiment of the present disclosure, the present disclosure further provides a monitoring use case generation method.

[0123] Figure 4 A flowchart of the monitoring use case generation method provided in the fourth embodiment of the present disclosure.

[0124] As shown in the flowchart, the monitoring use case generation method can include the following steps: Figure 4 In step S401, in response to monitoring a first event, it is queried whether there is a first monitoring use case associated with the first access address of the target page.

[0125] The first event is used to indicate that a target page is added or the target page is updated.

[0126] In step S402, when there is no first monitoring use case, the markup language text of the target page is obtained according to the first access address, and the page function description information of the target page is obtained.

[0127] In step S403, the first core element, the position and / or role of the first core element in the target page are determined from the markup language text according to the page function description information.

[0128]

[0129] ​The first core element has a higher importance than other page elements in the target page.

[0130] The explanations of steps S401-S403 can refer to the related descriptions in any embodiment of the present disclosure, which will not be repeated here.

[0131] In any embodiment of the present disclosure, for step S403, first, the markup language text can be parsed to obtain a first element list, wherein the first element list includes at least one leaf element, and each leaf element is not nested with any page element. Then, the first element list, the markup language text and the page function description information can be input into the language model for processing to obtain a second element list output by the language model, wherein the second element list includes at least one first core element. In addition, target prediction information output by the language model can also be obtained, wherein the target prediction information is used to indicate the position and / or role of each first core element in the target page.

[0132] Step S404: Obtain a use case template for generating a monitoring use case.

[0133] In the embodiments of the present disclosure, a related API (Application Programming Interface) can be called to obtain a use case template (or a standard template) for generating a monitoring use case.

[0134] Step S405: Generate a first check item according to the second element list and the target prediction information.

[0135] In the embodiments of the present disclosure, for any first core element in the second element list, a check item corresponding to the first core element can be generated according to the class name of the first core element and the position and / or role of the first core element in the target page in the target prediction information. For example, the check item can be used to check the first core element in the target page, to check whether the corresponding position in the target page exists the first core element.

[0136] Step S406: Update the use case template according to the first check item to obtain a second monitoring use case associated with the first access address.

[0137] In the embodiments of the present disclosure, the use case template can be updated according to the check item corresponding to each first core element to obtain a second monitoring use case associated with the first access address. The second monitoring use case is used to monitor the target page.

[0138] The monitoring case generation method of the embodiments of the present disclosure can generate check items for checking corresponding elements according to the positions and / or roles of each first core element and corresponding element in the target page, and update the case template according to the check items corresponding to each first core element, to obtain a second monitoring case associated with the first access address, which can avoid the situation that the core elements in the target page are missed in monitoring, and reduce the risk of missing online problems.

[0139] To clearly illustrate the above-mentioned embodiments, the present disclosure further provides a monitoring case generation method.

[0140] Figure 5 A flowchart of the monitoring case generation method provided by the fifth embodiment of the present disclosure.

[0141] As Figure 5 shown, the monitoring case generation method can include the following steps:

[0142] Step S501, in response to monitoring a first event, obtaining a markup language text of a target page and obtaining page function description information of the target page according to a first access address of the target page.

[0143] Step S502, determining a first core element from the markup language text according to the page function description information.

[0144] Among them, the importance of the first core element in the target page is higher than that of other page elements.

[0145] Step S503, query whether there is a first monitoring case associated with the first access address, if not, execute step S504, if yes, execute step S505.

[0146] Among them, the first event is used to indicate that the target page is added or the target page is updated.

[0147] It should be noted that step S504 and step S505 are two parallel implementation methods, and only one of them needs to be executed in actual application.

[0148] Step S504, generating a second monitoring case associated with the first access address according to the first core element.

[0149] Among them, the second monitoring case is used to monitor the target page.

[0150] The explanation of steps S501 to S504 can be referred to the related description in any embodiment of the present disclosure.

[0151] In any embodiment of the present disclosure, after generating a second monitoring use case associated with the first access address, the access address of the second monitoring use case (referred to as the second access address in the present disclosure) can also be output so that relevant personnel can access the second monitoring use case according to the second access address, and / or modify or update the second monitoring use case.

[0152] That is, in the present disclosure, the second access address is used to access the second monitoring use case, and / or update the second monitoring use case according to a modification instruction received from a user.

[0153] Thus, relevant personnel can access the second monitoring use case according to the second access address, and / or update the second monitoring use case to meet the actual application needs or business needs of different personnel.

[0154] Step S505: Update the first monitoring use case according to the first core element.

[0155] In an embodiment of the present disclosure, when there is a first monitoring use case associated with the first access address of the target page in the stored monitoring use cases, it can be determined that the target page is the updated page. At this time, the first monitoring use case can be updated according to the first core element.

[0156] In a first possible implementation of the present disclosure, the first core element may be a newly added page element in the target page. In this case, a new inspection item may be generated based on the newly added first core element and added to the first monitoring use case.

[0157] As an example, based on the second core element indicated by each second check item in the first monitoring use case, it can be determined whether a third core element exists in each first core element, where the third core element is different from the second core element, that is, the third core element is a newly added core element in the target page. If the third core element exists in each first core element, a third check item is generated based on the third core element, and the first monitoring use case is updated based on the third check item to obtain an updated first monitoring use case.

[0158] For example, a third check item corresponding to the third core element can be generated based on the third core element and its location and / or function in the target page. The third check item is used to check the third core element in the target page to determine whether the third core element exists at a corresponding location in the target page. The third check item can then be added to the first monitoring use case to obtain an updated first monitoring use case.

[0159] Therefore, when a third core element is added in the target page, a third checking item for checking the third core element can be automatically generated and added to the first monitoring case in time, so as to improve the timeliness and effectiveness of monitoring case updating.

[0160] In a second possible implementation of the present disclosure, the deleted core element in the target page can be determined according to the first core element, and the checking item corresponding to the deleted core element can be deleted from the first monitoring case.

[0161] As an example, whether the fourth core element exists in the second core element indicated by each second checking item in the first monitoring case can be determined according to each first core element; wherein the fourth core element is different from the first core element, that is, the fourth core element is a deleted core element in the target page. If the fourth core element exists, the second checking item corresponding to the fourth core element in the first monitoring case is deleted to obtain the updated first monitoring case.

[0162] Therefore, when the fourth core element is deleted in the target page, the second checking item for checking the fourth core element in the first monitoring case can be automatically deleted, so as to improve the timeliness and effectiveness of monitoring case updating.

[0163] In a third possible implementation of the present disclosure, the core element with changed position and / or function in the target page can be determined according to the first core element, and the checking item corresponding to the changed core element in the first monitoring case can be updated.

[0164] As an example, whether the fourth checking item exists in each second checking item in the first monitoring case can be determined according to the first core element and the target prediction information; wherein the target prediction information is used to indicate the position and / or function of the first core element in the target page.

[0165] Wherein the second core element indicated by the fourth checking item is the same as the first core element; and the position of the second core element indicated by the fourth checking item is different from the position indicated by the target prediction information, and / or the function of the second core element indicated by the fourth checking item is different from the function indicated by the target prediction information. That is, the second core element indicated by the fourth checking item is a core element with changed position and / or function in the target page.

[0166] If the fourth checking item exists, the fourth checking item can be updated according to the target prediction information (such as the position and / or function of the second core element indicated by the fourth checking item can be updated), to obtain the updated first monitoring case.

[0167] Therefore, when the position and / or role of the core element in the target page changes, the first monitoring case can be updated according to the changed position and / or role of the core element, and the timeliness and effectiveness of the monitoring case update can be improved.

[0168] It should be noted that the above only takes one of the following update methods as an example: adding a core element in a page, deleting a core element in a page, and updating the position and / or role of a core element in a page during a page update process. In actual application, the first monitoring case can be updated by combining the above-mentioned multiple methods.

[0169] For example, when a core element is not only added in the target page but also the original core element is deleted, the first monitoring case can be updated by combining the first and second possible implementation manners.

[0170] In summary, considering that actual page update scenarios can be different, in the present disclosure, the monitoring case can be updated by combining multiple methods to improve the accuracy of monitoring case update, thereby further reducing the risk of online problem leakage.

[0171] It should be noted that in actual application, the changed page can include not only added pages and updated pages, but also offline pages (i.e., deleted pages). At this time, in order to reduce resource occupation, the monitoring case of the offline page can be deleted.

[0172] That is, in any one of the embodiments of the present disclosure, when the second event is monitored, the second monitoring case or the first monitoring case can be deleted according to the first access address, wherein the second event is used to indicate that the target page has been deleted. Therefore, the monitoring case of the offline page is deleted in time, which can reduce the occupation of resources.

[0173] The monitoring case generation method of the embodiments of the present disclosure can realize timely updating of the existing monitoring case according to the updated core element in the page, and further reduce the risk of online problem leakage.

[0174] The above is the corresponding embodiment of the application method of the language model, and the present disclosure provides a language model training method.

[0175] Figure 6 The flowchart of the language model training method provided in Embodiment Six of the present disclosure.

[0176] As Figure 6 shown, the language model training method can include the following steps:

[0177] Step S601: Acquire a training sample; wherein the training sample includes the markup language text and page function description information of the sample page.

[0178] In the embodiment of the present disclosure, the sample page may be any page.

[0179] In an embodiment of the present disclosure, the markup language text (such as HTML text) of the sample page can be obtained based on the access address of the sample page, and the page function description information of the sample page can also be obtained. For example, the page function description information can be provided by relevant personnel, or the page function description information of the sample page can be automatically extracted using a relevant algorithm. The embodiment of the present disclosure does not impose any restrictions on this.

[0180] As an example, the page content of the sample page can be input into a trained page function recognition model to obtain page function description information output by the page function recognition model, wherein the page function recognition model has learned the correspondence between the input page content and the page function description information.

[0181] In the embodiment of the present disclosure, a training sample may be generated based on the markup language text and page function description information of the sample page.

[0182] Step S602: Input the training sample into the language model for processing to obtain the predicted core element output by the language model.

[0183] In an embodiment of the present disclosure, a training sample may be input into a language model for processing to obtain a core element output by the language model (referred to as a predicted core element in the present disclosure).

[0184] Step S603 : training the language model based on the difference between the predicted core elements and the labeled core elements corresponding to the training samples to obtain a trained language model.

[0185] Among them, the marked core elements are used to generate sample monitoring use cases associated with the sample page, and the sample monitoring use cases are used to monitor the sample page. The generation method of the sample monitoring use cases is similar to the generation method of the second monitoring use cases, which will not be repeated here.

[0186] The annotated core elements corresponding to the training samples may be manually annotated core elements.

[0187] In the embodiment of the present disclosure, the language model may be trained based on the difference between the predicted core elements and the labeled core elements corresponding to the training samples to obtain a trained language model.

[0188] For example, the value of the loss function (referred to as the loss value) can be determined based on the difference between the predicted core elements and the labeled core elements corresponding to the training samples. The loss value is positively correlated with the difference, i.e., the smaller the difference, the smaller the loss value, and conversely, the larger the difference, the larger the loss value. Therefore, in the present disclosure, the model parameters in the language model can be adjusted based on the loss value to minimize the loss value.

[0189] It should be noted that the above only uses the termination condition of model training as the minimization of the loss value as an example. In actual application, other termination conditions can also be set. For example, the termination conditions can also include: the training time reaches the set time, the number of training times reaches the set number, etc. The embodiments of the present disclosure do not limit this.

[0190] The language model training method of the disclosed embodiment comprises obtaining a training sample, wherein the training sample includes markup language text and page function description information of a sample page; inputting the training sample into the language model for processing to obtain predicted core elements output by the language model; and training the language model based on the differences between the predicted core elements and the labeled core elements corresponding to the training sample to obtain a trained language model. Thus, by training the language model, the prediction effect of the language model can be improved, that is, the accuracy of the language model's prediction results can be improved.

[0191] In order to clearly illustrate the above embodiments, the present disclosure further proposes a language model training method.

[0192] Figure 7 This is a flowchart of the language model training method provided in Example 7 of the present disclosure.

[0193] like Figure 7 As shown, the language model training method may include the following steps:

[0194] Step S701: Acquire a training sample; wherein the training sample includes a markup language text of a sample page, page function description information, and a first sample element list.

[0195] For explanation of step S701, please refer to the relevant description in any embodiment of the present disclosure, and will not be repeated here.

[0196] In an embodiment of the present disclosure, the markup language text may be parsed to obtain a first sample element list, wherein the first sample element list may include at least one leaf element, and each leaf element does not have any page element nested therein.

[0197] It should be noted that the method for obtaining the first sample element list is similar to the method for obtaining the first element list in the aforementioned embodiment and will not be described in detail here. In addition, it is also possible to extract each leaf element from the markup language text through manual recognition and generate the first sample element list based on each leaf element, which is not limited to this embodiment of the present disclosure.

[0198] Step S702: input the training sample into the language model, so as to use the language model to determine the predicted core element from the first sample element list according to the markup language text and the page function description information.

[0199] In an embodiment of the present disclosure, a training sample may be input into a language model, so as to use the language model to determine at least one predicted core element from the first sample element list according to the markup language text and page function description information.

[0200] In any embodiment of the present disclosure, the method for determining the predicted core element may be, for example:

[0201] 1. Obtain prompt information, where the prompt information is used to indicate the task information to be processed by the language model and the number of core elements output by the language model.

[0202] It should be noted that the explanation of the prompt information in the aforementioned embodiment is also applicable to this embodiment and will not be repeated here.

[0203] 2. Use the language model to process the training samples according to the prompt information to obtain the predicted core elements.

[0204] Therefore, using prompt information, markup language text and page function description information as input to the language model can improve the prediction effect of the language model. That is, the prompt information can be used as prior information or task information to indicate what kind of information the language model needs to extract, which can improve the expressive ability of the language model.

[0205] Step S703 : training the language model based on the difference between the predicted core elements and the labeled core elements corresponding to the training samples to obtain a trained language model.

[0206] The annotated core elements are used to generate sample monitoring use cases associated with the sample pages, and the sample monitoring use cases are used to monitor the sample pages.

[0207] For explanation of step S703, please refer to the relevant description in any embodiment of the present disclosure, and will not be repeated here.

[0208] The language model training method of the embodiment of the present disclosure can determine the first core element with relatively high importance from the various leaf elements of the sample page based on the context information of the markup language text of the sample page and the page function description information, which can improve the accuracy and reliability of the prediction of the core element.

[0209] In order to clearly illustrate how the language model is trained according to the difference between the predicted core elements and the labeled core elements corresponding to the training samples in any of the above embodiments, the present disclosure also proposes a language model training method.

[0210] Figure 8 This is a flowchart of the language model training method provided in Example 8 of the present disclosure.

[0211] like Figure 8 As shown, the language model training method may include the following steps:

[0212] Step S801: Acquire a training sample; wherein the training sample includes the markup language text and page function description information of the sample page.

[0213] Step S802: Input the training sample into the language model for processing to obtain the prediction core elements and target prediction information output by the language model.

[0214] The target prediction information is used to indicate the position and / or function of the predicted core element in the sample page.

[0215] For explanations of steps S801 to S802 , reference may be made to the relevant descriptions in any embodiment of the present disclosure, and will not be repeated here.

[0216] Step S803: Generate a first loss value based on the difference between the predicted core element and the labeled core element corresponding to the training sample.

[0217] The annotated core elements are used to generate sample monitoring use cases associated with the sample pages, and the sample monitoring use cases are used to monitor the sample pages.

[0218] In an embodiment of the present disclosure, a first loss value can be generated based on the difference between the predicted core element and the labeled core element corresponding to the training sample, wherein the first loss value is positively correlated with the above-mentioned difference, that is, the smaller the difference, the smaller the first loss value, and conversely, the greater the difference, the greater the first loss value.

[0219] Step S804: Generate a second loss value based on the difference between the target prediction information and the target annotation information corresponding to the training sample.

[0220] The target annotation information is used to indicate the position and / or function of the annotation core element in the sample page. For example, the target annotation information may be manually annotated.

[0221] In the disclosed embodiment, a second loss value may be generated based on the difference between the target prediction information and the target annotation information corresponding to the training sample. The second loss value is positively correlated with the difference, i.e., the smaller the difference, the smaller the second loss value, and conversely, the larger the difference, the larger the second loss value.

[0222] Step S805: Train the language model according to the first loss value and the second loss value to obtain a trained language model.

[0223] In the disclosed embodiment, the language model may be trained based on the first loss value and the second loss value.

[0224] As an example, a target loss value can be determined based on the first loss value and the second loss value. For example, the sum, mean, weighted sum, etc. of the first loss value and the second loss value can be used as the target loss value, so that the model parameters in the language model can be adjusted according to the target loss value to minimize the target loss value.

[0225] It should be noted that the above only uses the termination condition of model training as an example of minimizing the target loss value. In actual application, other termination conditions can also be set. For example, the termination conditions can also include: the training time reaches the set time, the number of training times reaches the set number, etc. The embodiments of the present disclosure do not limit this.

[0226] The language model training method of the embodiment of the present disclosure can train the language model based on multiple loss values, thereby further improving the prediction accuracy of the language model.

[0227] In any embodiment of the present disclosure, leveraging the advantages of LLMs (such as contextual awareness and generation capabilities), front-end monitoring cases can be synchronized and updated with business changes. The monitoring case generation method provided in this disclosure can be applied to all mobile apps and websites that include front-end pages. Once integrated during the application page change phase, monitoring cases can be automatically created and updated.

[0228] As an example, take the markup language text as HTML text and the language model as LLMs as an example. According to the URL address of a certain page, the monitoring case of the page can be automatically generated. The implementation principle can be as follows: Figure 9 As shown, it mainly includes the following steps:

[0229] 1. The URL address of the page and simple page function description information can be input by relevant personnel.

[0230] 2. Access LLMs. By calling the API interface provided by LLMs, relevant answers can be generated according to input instructions. Here, a large language model with better context processing capability and text generation capability can be selected for access.

[0231] 3. Obtain the HTML text of the page according to the URL address, and input the HTML text after fragmentation to LLMs. The fragmentation refers to the segmentation of the HTML text into multiple text segments, because the number of characters contained in the complete HTML text of the page is much larger than the number of characters supported by LLMs for input, and the complete HTML text cannot be input at one time.

[0232] 4. Analyze the leaf elements in the page according to the HTML text to obtain a leaf element list (referred to as a first element list in the present disclosure). If a page element is no longer embedded with other page elements inside, the page element is referred to as a leaf element.

[0233] 5. Input the leaf element list to LLMs.

[0234] 6. After inputting the complete HTML text and the leaf element list to LLMs, LLMs will output a core element list (referred to as a second element list in the present disclosure) in the page according to the prompt information prompt, as well as the position and / or role of each core element in the core element list in the page.

[0235] 7. If the output format is correct (i.e., non-random code), an API of a monitoring platform is called to create a monitoring case, the core element list is configured to the check item of the monitoring case, and monitoring execution information is configured. The check item is used to check whether the core element is normal.

[0236] 8. Output the access address of the monitoring case (hereinafter referred to as the monitoring address).

[0237] Optionally, in order to make the core elements output by LLMs more accurate, the LLMs are set as a front-end R&D engineer in the system-level instruction, which needs to analyze the core elements in the page according to the input HTML text and leaf element list, the HTML text and leaf element list are input in fragments in the user-level instruction, and finally the format of the core elements output by LLMs is told. Through the prompt information prompt, LLMs can accurately output the core elements in the page and their positions and / or roles in the page.

[0238] It should be noted that during the front-end requirement iteration process, there are three types of changes: adding a page, updating a page, and decommissioning a page.

[0239] 1. For the demand of adding a new page, the technology of automatically generating a monitoring case provided by the present disclosure will add a monitoring case according to the input URL address and name of the new page, and bind the URL address with the identification information (such as Id) of the monitoring case.

[0240] 2. For the demand of updating a page, the technology of automatically generating a monitoring case provided by the present disclosure will find the corresponding monitoring case according to the URL address of the updated page, and replace the core elements indicated by the check items in the monitoring case with the core elements in the updated page.

[0241] 3. For the demand of offline page, the technology of automatically generating a monitoring case provided by the present disclosure will find the corresponding monitoring case according to the URL address of the offline page, and then close the monitoring case.

[0242] It should be noted that before the introduction of LLMs, the process of creating a monitoring case for a page was a manual operation, which required manual viewing of the HTML text of the page, finding the class name corresponding to the core element, creating a monitoring case on the monitoring platform, and configuring each core element as a check item and configuring the execution information of the monitoring case. The technology of automatically generating a monitoring case provided by the present disclosure only requires manual input of the URL address and functional description information of the page in the process of automatically generating a monitoring case, and other links can be realized through Python (a computer programming language) script, so this technology can greatly reduce the difficulty of creating a monitoring case and the maintenance manpower. By applying this technology to the change stage of the system, real-time and automatic updating of the monitoring case of the page can be realized.

[0243] Next, the application scenario of this technology will be introduced by example. In order to realize real-time and automatic updating of the monitoring case, the technology of automatically generating a monitoring case based on LLMs provided by the present disclosure can be closely combined with product changes. When the product is changed online, the technology is used to create a monitoring case to ensure the quality of online services, mainly including the following two use methods:

[0244] The first use method is to access through the online registration system. The current online registration system is built to improve the timeliness of monitoring case updating, which can ensure that the monitoring case is updated within a week and requires manual operation for monitoring case addition or update. In the present disclosure, after the technology of automatically generating a monitoring case based on LLMs is connected to the online registration system, the monitoring case can be automatically updated in real time with the business online, releasing manpower.

[0245] The working mode of the current online registration system can be as shown in the following figure: Figure 10 First, the user needs to register the page information of this change, and then submit, and then notify the monitoring person in charge to update the monitoring case. This mode cannot determine whether the monitoring case is really created or updated. Therefore, the disclosure puts the technology of automatically generating the monitoring case in the process after submission, and updates the monitoring case according to the submitted change information. The overall working process after access can be as shown in the following figure: Figure 11 The solid line box is the working mode before the online registration system. After submission, the technology of automatically generating the monitoring case based on LLMs updates the monitoring case, and after the monitoring case is updated, the access address of the updated monitoring case (i.e., the monitoring address) is displayed in the online registration system. After accessing this technology, the tester only needs to register the change information (such as the URL address of the changed page and the page function description information), and can automatically generate the monitoring case and view the monitoring case.

[0246] The working process of the front-end pipeline can be as shown in the following figure: Figure 12 After the developer submits the code, the code needs to be compiled, scanned and tested in the pipeline, and after the test is passed, the code is released in the pipeline. The working process of the pipeline after accessing the technology of automatically generating the monitoring case can be as shown in the following figure: Figure 13 The solid line box is the execution step before the pipeline. In order to realize automatic updating of the monitoring case with business changes, a step of registering the name and URL address of the changed page is added before the release of the change, and after the release, the technology of automatically generating the monitoring case based on LLMs is added, and the monitoring case is automatically updated according to the page name and URL address. Combining monitoring update with the pipeline can realize automatic real-time updating of the monitoring case with business changes.

[0247] In summary, the technology of automatically generating the front-end monitoring case based on LLMs provided by the disclosure can realize that the process of manually creating the monitoring case is completed by the machine, and the manpower of the front-end monitoring is released. By adding this technology to the online change stage, real-time automatic creation of the front-end monitoring case is realized, and online problems caused by the lack of monitoring cases will not occur again. The monitoring case can be created synchronously after the demand is online, so that the online service has real-time monitoring, and the quality of the product is effectively guaranteed. At the same time, high-quality and stable services can be provided to users.

[0248] Corresponding to the monitoring case generation method provided in the above Figures 1 to 5 embodiments, the disclosure also provides a monitoring case generation device. Since the monitoring case generation device provided by the embodiments of the disclosure corresponds to the monitoring case generation method provided in the above Figures 1 to 5 The embodiment provides a method for generating a monitoring use case, and therefore the implementation of the method for generating a monitoring use case is also applicable to the device for generating a monitoring use case provided in the embodiment of the present disclosure, and will not be described in detail in the embodiment of the present disclosure.

[0249] Figure 14 This is a structural diagram of the device for generating monitoring use cases provided in the ninth embodiment of the present disclosure.

[0250] like Figure 14 As shown, the monitoring use case generation device 1400 may include: a query module 1410 , an acquisition module 1420 , a determination module 1430 and a generation module 1440 .

[0251] The query module 1410 is used to query whether there is a first monitoring use case associated with the first access address of the target page in response to monitoring the first event; wherein the first event is used to indicate a new target page is added or the target page is updated.

[0252] The acquisition module 1420 is configured to acquire the markup language text of the target page and the page function description information of the target page according to the first access address when the first monitoring use case does not exist.

[0253] The determination module 1430 is configured to determine a first core element from the markup language text according to the page function description information; wherein the first core element is more important than other page elements in the target page.

[0254] The generating module 1440 is configured to generate a second monitoring use case associated with the first access address based on the first core element; wherein the second monitoring use case is used to monitor the target page.

[0255] In a possible implementation of the embodiment of the present disclosure, the determination module 1430 is used to: parse the markup language text to obtain a first element list; wherein the first element list includes at least one leaf element, and no page element is nested in the leaf element; determine the first core element from the first element list based on the markup language text and the page function description information; wherein the first core element is more important in the target page than other leaf elements.

[0256] In a possible implementation of an embodiment of the present disclosure, the determination module 1430 is used to: obtain prompt information; wherein the prompt information is used to indicate the task information to be processed by the language model and the number of core elements output by the language model; use the language model to process the first element list, markup language text and page function description information according to the prompt information to obtain the second element list output by the language model; wherein the second element list includes the number of first core elements.

[0257] In a possible implementation of an embodiment of the present disclosure, the determination module 1430 is used to: obtain the number of characters contained in the markup language text; when the number of characters is higher than the upper limit of the number specified by the input requirement of the language model, segment the markup language text to obtain multiple text segments; wherein the number of characters contained in each text segment is less than or equal to the upper limit; input the prompt information, the first element list, the multiple text segments and the page function description information into the language model for processing to obtain the second element list output by the language model.

[0258] In a possible implementation of the embodiment of the present disclosure, the language model also outputs target prediction information, which is used to indicate the position and / or function of each first core element in the second element list in the target page; a generation module 1440 is used to: obtain a use case template for generating a monitoring use case; generate a first check item based on the second element list and the target prediction information; update the use case template based on the first check item to obtain a second monitoring use case.

[0259] In a possible implementation of the embodiment of the present disclosure, the monitoring use case generation device 1400 may further include:

[0260] The updating module is used to update the first monitoring use case according to the first core element when the first monitoring use case exists.

[0261] In a possible implementation of the embodiment of the present disclosure, the update module is used to: when a first monitoring use case exists, determine whether a third core element exists in each first core element based on the second core element indicated by each second check item in the first monitoring use case; wherein the third core element is different from the second core element; if the third core element exists, generate a third check item based on the third core element; and update the first monitoring use case based on the third check item to obtain an updated first monitoring use case.

[0262] In a possible implementation of the embodiment of the present disclosure, the update module is used to: when there is a first monitoring use case, determine, based on each first core element, whether there is a fourth core element in the second core elements indicated by each second check item in the first monitoring use case; wherein the fourth core element is different from the first core element; if the fourth core element exists, delete the second check item corresponding to the fourth core element in the first monitoring use case to obtain an updated first monitoring use case.

[0263] In a possible implementation of an embodiment of the present disclosure, an update module is used to: determine whether there is a fourth check item in each second check item in the first monitoring use case based on the first core element and target prediction information; wherein the target prediction information is used to indicate the position and / or function of the first core element in the target page; if there is a fourth check item, the fourth check item is updated according to the target prediction information to obtain an updated first monitoring use case; wherein the second core element indicated by the fourth check item is the same as the first core element; wherein the position of the second core element indicated in the fourth check item is different from the position indicated by the target prediction information, and / or the function of the second core element indicated in the fourth check item is different from the function indicated by the target prediction information.

[0264] In a possible implementation of the embodiment of the present disclosure, the monitoring use case generation device 1400 may further include:

[0265] The deletion module is used to delete the second monitoring use case according to the first access address in response to monitoring the second event; wherein the second event is used to indicate that the target page has been deleted.

[0266] In a possible implementation of the embodiment of the present disclosure, the monitoring use case generation device 1400 may further include:

[0267] The output module is used to output a second access address of the second monitoring use case; wherein the second access address is used to access the second monitoring use case, and / or update the second monitoring use case according to the received modification instruction.

[0268] The monitoring use case generation device of the embodiment of the present disclosure queries whether there is a first monitoring use case associated with the first access address of the target page in response to monitoring a first event. If not, the device obtains the markup language text of the target page and the page function description information of the target page based on the first access address; determines the first core element from the markup language text based on the page function description information; and generates a second monitoring use case associated with the first access address based on the first core element. In this way, when a new page is detected or an update of a page is detected, a monitoring use case for monitoring the page can be automatically and promptly generated based on the core elements in the page, without the need to manually create a monitoring use case for the page. This can improve the efficiency of generating monitoring use cases, thereby reducing the risk of online problems leaking.

[0269] With the above Figures 6 to 8 Corresponding to the training method of the language model provided in the embodiment, the present disclosure also provides a training device for a language model. Figures 6 to 8The training method of the language model provided in the embodiment corresponds to the embodiment, so the implementation method of the language model training method is also applicable to the training device of the language model provided in the embodiment of the present disclosure, and will not be described in detail in the embodiment of the present disclosure.

[0270] Figure 15 This is a structural diagram of the language model training device provided in Example 10 of the present disclosure.

[0271] like Figure 15 As shown, the language model training device 1500 may include: an acquisition module 1510 , a processing module 1520 and a training module 1530 .

[0272] The acquisition module 1510 is used to acquire a training sample; wherein the training sample includes the markup language text and page function description information of the sample page;

[0273] Processing module 1520, configured to input the training sample into the language model for processing to obtain the predicted core elements output by the language model;

[0274] The training module 1530 is used to train the language model according to the difference between the predicted core elements and the labeled core elements corresponding to the training samples to obtain a trained language model.

[0275] The annotated core elements are used to generate sample monitoring use cases associated with the sample pages, and the sample monitoring use cases are used to monitor the sample pages.

[0276] In a possible implementation of an embodiment of the present disclosure, the training sample also includes: a first sample element list, the first sample element list includes at least one leaf element, and no page element is nested in the leaf element; a language model, used to determine the predicted core element from the first sample element list based on the markup language text and page function description information.

[0277] In a possible implementation of an embodiment of the present disclosure, the processing module 1520 is used to: obtain prompt information; wherein the prompt information is used to indicate the task information to be processed by the language model and the number of core elements output by the language model; use the language model to process the training sample according to the prompt information to obtain the predicted core elements.

[0278] In a possible implementation of the embodiment of the present disclosure, the language model also outputs target prediction information, where the target prediction information is used to indicate the position and / or role of the predicted core element in the sample page; the training module 1530 is configured to: generate a first loss value according to a difference between the predicted core element and a labeled core element corresponding to the training sample; generate a second loss value according to a difference between the target prediction information and target labeled information corresponding to the training sample; where the target labeled information is used to indicate the position and / or role of the labeled core element in the sample page; and train the language model according to the first loss value and the second loss value, to obtain the trained language model.

[0279] The training device of the language model according to the embodiment of the present disclosure, by obtaining a training sample; where the training sample includes markup language text and page function description information of a sample page; inputting the training sample into the language model for processing to obtain a predicted core element output by the language model; and training the language model according to a difference between the predicted core element and a labeled core element corresponding to the training sample, to obtain a trained language model. Thus, by training the language model, the prediction effect of the language model can be improved, that is, the accuracy of the prediction result of the language model can be improved.

[0280] To implement the above-mentioned embodiments, the present disclosure further provides an electronic device, which can include at least one processor; and a memory connected with the at least one processor in communication; where the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method for generating a monitoring use case or the method for training a language model according to any one of the embodiments of the present disclosure.

[0281] To implement the above-mentioned embodiments, the present disclosure further provides a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to enable a computer to perform the method for generating a monitoring use case or the method for training a language model according to any one of the embodiments of the present disclosure.

[0282] To implement the above-mentioned embodiments, the present disclosure further provides a computer program product, which includes a computer program, and the computer program, when executed by a processor, implements the method for generating a monitoring use case or the method for training a language model according to any one of the embodiments of the present disclosure.

[0283] According to the embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium, and a computer program product.

[0284] Figure 16A schematic block diagram of an example electronic device that can be used to implement an embodiment of the present disclosure is shown. The electronic device may include the server and client in the above-mentioned embodiments. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0285] like Figure 16 As shown, the device 1600 includes a computing unit 1601, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 1602 or a computer program loaded from a storage unit 1607 into a RAM (Random Access Memory) 1603. Various programs and data required for the operation of the device 1600 can also be stored in the RAM 1603. The computing unit 1601, the ROM 1602, and the RAM 1603 are connected to each other via a bus 1604. An I / O (Input / Output) interface 1605 is also connected to the bus 1604.

[0286] Various components in device 1600 are connected to I / O interface 1605, including an input unit 1606, such as a keyboard and mouse; an output unit 1607, such as various types of displays and speakers; a storage unit 1608, such as a magnetic disk and optical disk; and a communication unit 1609, such as a network card, a modem, a wireless communication transceiver, etc. Communication unit 1609 allows device 1600 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0287] The computing unit 1601 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 1601 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various special-purpose AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 1601 performs various methods and processes described above, such as the generation method of the monitoring use case or the training method of the language model described above. For example, in some embodiments, the generation method of the monitoring use case or the training method of the language model described above can be implemented as a computer software program, which is tangibly embodied in a machine-readable medium, such as the storage unit 1608. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 1600 via the ROM 1602 and / or the communication unit 1609. When the computer program is loaded onto the RAM 1603 and executed by the computing unit 1601, one or more steps of the generation method of the monitoring use case or the training method of the language model described above can be performed. Alternatively, in other embodiments, the computing unit 1601 can be configured to perform the generation method of the monitoring use case or the training method of the language model described above by any other appropriate means, such as by means of firmware.

[0288] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application-Specific Standard Products), SOCs (System on Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0289] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0290] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include a linearly-programmed electronic storage, a portable computer diskette, a hard disk, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory), or flash memory, an optical fiber, a CD-ROM (Compact Disc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0291] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0292] The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, and a blockchain network.

[0293] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. The client-server relationship is established by computer programs running on the respective computers and establishing a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within the cloud computing service ecosystem that addresses the management difficulties and poor scalability of traditional physical hosts and VPS services. The server may also be a server in a distributed system or a server integrated with blockchain.

[0294] It's important to note that artificial intelligence (AI) is the study of how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). This encompasses both hardware and software technologies. AI hardware technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily encompass computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graphs.

[0295] According to the technical solution of the embodiment of the present disclosure, in response to monitoring a first event, a query is made as to whether there is a first monitoring use case associated with the first access address of the target page. If not, the markup language text of the target page is obtained according to the first access address, and the page function description information of the target page is obtained; according to the page function description information, a first core element is determined from the markup language text; according to the first core element, a second monitoring use case associated with the first access address is generated. In this way, when a new page is detected or an update of a page is detected, a monitoring use case for monitoring the page can be automatically and promptly generated according to the core elements in the page, without the need to manually create a monitoring use case for the page. This can improve the efficiency of generating monitoring use cases, thereby reducing the risk of online problems leaking.

[0296] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0297] The above detailed description does not limit the scope of the disclosure. Various modifications, combinations, sub-combinations and alternatives can be made to the detailed description. Any modification, equivalent replacement and improvement etc. made within the spirit and principle of the disclosure shall be included in the scope of the disclosure.

Claims

1. A method for generating a monitoring use case, comprising: In response to detecting a first event, querying whether there is a first monitoring use case associated with a first access address of a target page; wherein the first event is used to indicate that the target page is newly added or updated; When the first monitoring use case does not exist, obtaining the markup language text of the target page and obtaining page function description information of the target page according to the first access address; Parsing the markup language text to obtain a first element list; obtaining prompt information, the prompt information being used to indicate task information to be processed by a language model and the number of core elements output by the language model; using a language model to process the first element list, the markup language text, and the page function description information according to the prompt information to obtain a second element list and target prediction information output by the language model, wherein the second element list includes the number of first core elements, and the target prediction information is used to indicate the position and / or function of each of the first core elements in the second element list in the target page; wherein the first element list includes at least one leaf element, no page element is nested in the leaf element, and the first core element is more important in the target page than other leaf elements; Automatically generating a second monitoring use case associated with the first access address based on the second element list and the target prediction information; wherein the second monitoring use case is used to monitor the target page; When the first monitoring use case exists, automatically updating the first monitoring use case according to the first core element; When the first monitoring use case exists, automatically updating the first monitoring use case according to the first core element includes: When the first monitoring use case exists, determining, based on each of the first core elements, whether a fourth core element exists in the second core elements indicated by each of the second check items in the first monitoring use case; wherein the fourth core element is different from the first core element; If the fourth core element exists, the second check item corresponding to the fourth core element in the first monitoring use case is deleted to obtain an updated first monitoring use case.

2. The method according to claim 1, wherein The using the language model to process the first element list, the markup language text, and the page function description information according to the prompt information to obtain a second element list output by the language model includes: Obtaining the number of characters contained in the markup language text; When the number of characters exceeds the upper limit of the number specified by the input requirement of the language model, segmenting the markup language text to obtain a plurality of text segments; wherein the number of characters contained in each of the text segments is less than or equal to the upper limit; The prompt information, the first element list, the multiple text segments and the page function description information are input into the language model for processing to obtain the second element list output by the language model.

3. The method according to claim 1, wherein The automatically generating a second monitoring use case associated with the first access address according to the second element list and the target prediction information includes: Obtain a use case template for generating monitoring use cases; generating a first inspection item according to the second element list and the target prediction information; The use case template is automatically updated according to the first inspection item to obtain the second monitoring use case.

4. The method according to claim 1, wherein The method of automatically updating the first monitoring use case according to the first core element when the first monitoring use case exists further includes: When the first monitoring use case exists, determining whether a third core element exists in each of the first core elements based on the second core elements indicated by each second check item in the first monitoring use case; wherein the third core element is different from the second core element; If the third core element exists, generating a third check item according to the third core element; According to the third check item, the first monitoring use case is updated to obtain an updated first monitoring use case.

5. The method according to claim 1, wherein The method of automatically updating the first monitoring use case according to the first core element when the first monitoring use case exists further includes: Determining, based on the first core element and target prediction information, whether a fourth check item exists in each second check item in the first monitoring use case; wherein the target prediction information is used to indicate the position and / or function of the first core element in the target page; If the fourth check item exists, updating the fourth check item according to the target prediction information to obtain an updated first monitoring use case; The second core element indicated by the fourth check item is the same as the first core element; the position of the second core element indicated in the fourth check item is different from the position indicated by the target prediction information, and / or the function of the second core element indicated in the fourth check item is different from the function indicated by the target prediction information.

6. The method according to any one of claims 1 to 3, wherein The method further comprises: In response to detecting a second event, the second monitoring use case is deleted according to the first access address; wherein the second event is used to indicate that the target page has been deleted.

7. The method according to any one of claims 1 to 3, wherein The method further comprises: Output a second access address of the second monitoring use case; wherein the second access address is used to access the second monitoring use case, and / or update the second monitoring use case according to the received modification instruction.

8. The method according to any one of claims 1 to 5, wherein The method further comprises: Obtaining a training sample; wherein the training sample includes markup language text and page function description information of a sample page; Inputting the training sample into a language model for processing to obtain a predicted core element output by the language model; Training the language model based on the difference between the predicted core elements and the labeled core elements corresponding to the training samples to obtain a trained language model, wherein the language model outputs the second element list; The annotation core element is used to generate a sample monitoring use case associated with the sample page, and the sample monitoring use case is used to monitor the sample page.

9. The method according to claim 8, wherein The training sample further includes: a first sample element list, wherein the first sample element list includes at least one leaf element, and no page element is nested in the leaf element; The language model is used to determine the predicted core element from the first sample element list according to the markup language text and the page function description information.

10. The method according to claim 8, wherein The step of inputting the training sample into a language model for processing to obtain a prediction core element output by the language model includes: Obtaining prompt information; wherein the prompt information is used to indicate task information to be processed by the language model and the number of core elements output by the language model; The language model is used to process the training sample according to the prompt information to obtain the predicted core element.

11. The method according to claim 8, wherein The language model further outputs target prediction information, wherein the target prediction information is used to indicate the position and / or function of the predicted core element in the sample page; The training of the language model according to the difference between the predicted core element and the labeled core element corresponding to the training sample to obtain a trained language model includes: generating a first loss value according to a difference between the predicted core element and the labeled core element corresponding to the training sample; generating a second loss value based on a difference between the target prediction information and the target annotation information corresponding to the training sample; wherein the target annotation information is used to indicate the position and / or function of the annotation core element in the sample page; The language model is trained according to the first loss value and the second loss value to obtain a trained language model.

12. A device for generating a monitoring use case, comprising: A query module, configured to query whether there is a first monitoring use case associated with a first access address of a target page in response to detecting a first event, wherein the first event is used to indicate that the target page is newly added or updated; an acquisition module, configured to acquire, according to the first access address, the markup language text of the target page and page function description information of the target page when the first monitoring use case does not exist; A determination module is configured to parse the markup language text to obtain a first element list; obtain prompt information, the prompt information being used to indicate task information to be processed by a language model and the number of core elements output by the language model; use a language model to process the first element list, the markup language text, and the page function description information according to the prompt information to obtain a second element list and target prediction information output by the language model, wherein the second element list includes the number of first core elements, and the target prediction information is used to indicate the position and / or function of each of the first core elements in the second element list in the target page; wherein the first element list includes at least one leaf element, wherein no page element is nested in the leaf element, and the first core element is more important in the target page than other leaf elements; a generating module, configured to automatically generate a second monitoring use case associated with the first access address based on the second element list and the target prediction information; wherein the second monitoring use case is used to monitor the target page; an updating module, configured to automatically update the first monitoring use case according to the first core element when the first monitoring use case exists; When the first monitoring use case exists, automatically updating the first monitoring use case according to the first core element includes: When the first monitoring use case exists, determining, based on each of the first core elements, whether a fourth core element exists in the second core elements indicated by each of the second check items in the first monitoring use case; wherein the fourth core element is different from the first core element; If the fourth core element exists, the second check item corresponding to the fourth core element in the first monitoring use case is deleted to obtain an updated first monitoring use case.

13. The device according to claim 12, wherein The determining module is configured to: Obtaining the number of characters contained in the markup language text; When the number of characters exceeds the upper limit of the number specified by the input requirement of the language model, segmenting the markup language text to obtain a plurality of text segments; wherein the number of characters contained in each of the text segments is less than or equal to the upper limit; The prompt information, the first element list, the multiple text segments and the page function description information are input into the language model for processing to obtain the second element list output by the language model.

14. The device according to claim 12, wherein The generation module is specifically used to: Obtain a use case template for generating monitoring use cases; generating a first inspection item according to the second element list and the target prediction information; The use case template is updated according to the first inspection item to obtain the second monitoring use case.

15. The device according to claim 12, wherein The method of automatically updating the first monitoring use case according to the first core element when the first monitoring use case exists further includes: When the first monitoring use case exists, determining whether a third core element exists in each of the first core elements based on the second core elements indicated by each second check item in the first monitoring use case; wherein the third core element is different from the second core element; If the third core element exists, generating a third check item according to the third core element; According to the third check item, the first monitoring use case is updated to obtain an updated first monitoring use case.

16. The device according to claim 12, wherein The method of automatically updating the first monitoring use case according to the first core element when the first monitoring use case exists further includes: Determining, based on the first core element and target prediction information, whether a fourth check item exists in each second check item in the first monitoring use case; wherein the target prediction information is used to indicate the position and / or function of the first core element in the target page; If the fourth check item exists, updating the fourth check item according to the target prediction information to obtain an updated first monitoring use case; The second core element indicated by the fourth check item is the same as the first core element; the position of the second core element indicated in the fourth check item is different from the position indicated by the target prediction information, and / or the function of the second core element indicated in the fourth check item is different from the function indicated by the target prediction information.

17. The device according to any one of claims 12 to 14, wherein: The device further comprises: A deletion module is used to delete the second monitoring use case according to the first access address in response to monitoring a second event; wherein the second event is used to indicate that the target page has been deleted.

18. The device according to any one of claims 12 to 14, wherein: The device further comprises: An output module is used to output a second access address of the second monitoring use case; wherein the second access address is used to access the second monitoring use case, and / or update the second monitoring use case according to the received modification instruction.

19. The device according to any one of claims 12 to 16, wherein: The device further comprises: An acquisition module is used to acquire training samples; wherein the training samples include markup language text and page function description information of the sample page; A processing module, configured to input the training sample into a language model for processing to obtain a predicted core element output by the language model; a training module, configured to train the language model based on a difference between the predicted core element and the labeled core element corresponding to the training sample to obtain a trained language model, wherein the language model outputs the second element list; The annotation core element is used to generate a sample monitoring use case associated with the sample page, and the sample monitoring use case is used to monitor the sample page.

20. The device according to claim 19, wherein The training sample further includes: a first sample element list, wherein the first sample element list includes at least one leaf element, and no page element is nested in the leaf element; The language model is used to determine the predicted core element from the first sample element list according to the markup language text and the page function description information.

21. The apparatus according to claim 19, wherein The processing module is used to: Obtaining prompt information; wherein the prompt information is used to indicate task information to be processed by the language model and the number of core elements output by the language model; The language model is used to process the training sample according to the prompt information to obtain the predicted core element.

22. The apparatus according to claim 19, wherein The language model further outputs target prediction information, wherein the target prediction information is used to indicate the position and / or function of the predicted core element in the sample page; The training module is used to: generating a first loss value according to a difference between the predicted core element and the labeled core element corresponding to the training sample; generating a second loss value based on a difference between the target prediction information and the target annotation information corresponding to the training sample; wherein the target annotation information is used to indicate the position and / or function of the annotation core element in the sample page; The language model is trained according to the first loss value and the second loss value to obtain a trained language model.

23. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 11.

24. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-11.

25. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 11.

Images