Page detection method and device, equipment, medium and program product

Abstract

The embodiment of the invention discloses a page detection method and device, equipment, a medium and a program product. The method comprises the following steps: acquiring a test page image and a reference page image; processing the test page image and the reference page image according to a page detection strategy indicated by a page detection prompt word through a multi-modal large model to obtain a page detection result corresponding to the test page image; wherein the page detection cue word is used for defining an element exception judgment standard, a detection constraint condition and a detection output specification in the page detection strategy; the detection constraint condition is used for defining a visual presentation difference type of exemption detection.

Description

TECHNICAL FIELD

[0001] The present specification relates to the technical field of page testing, and in particular to a page detection method, device, equipment, medium and program product. BACKGROUND

[0002] Currently, page detection mainly relies on image comparison based on pixel-level matching. This method detects element abnormalities in the test page image by comparing the pixel values of the test page image and the reference page image pixel by pixel.

[0003] However, with the diversified development of terminal devices, the same page will produce visual presentation differences when rendered on different devices. The above-mentioned image comparison based on pixel-level matching is difficult to flexibly cope with visual presentation differences caused by inherent differences such as rendering environment when detecting pages, and a more reliable page detection method needs to be provided. SUMMARY

[0004] The embodiments of the present specification provide a page detection method, device, equipment, medium and program product, which can flexibly cope with visual presentation differences caused by inherent differences such as rendering environment when detecting pages, and improve the reliability of page detection.

[0005] In a first aspect, the embodiments of the present specification provide a page detection method, comprising: obtaining a test page image and a reference page image; processing the test page image and the reference page image according to a page detection strategy indicated by a page detection prompt word through a multi-modal large model to obtain a page detection result corresponding to the test page image; The page detection prompt word is used to define element abnormality judgment criteria, detection constraint conditions and detection output specifications in the page detection strategy; and the detection constraint condition is used to define visual presentation difference types exempted from detection.

[0006] In a possible implementation manner, the element abnormality judgment criteria include at least one of the following: element missing judgment criteria, element increasing judgment criteria, element misplacement judgment criteria, and element change judgment criteria.

[0007] In a possible implementation manner, the visual presentation difference types exempted from detection include at least one of the following: visual presentation differences caused by element semantic changes due to preset business logic, and visual presentation differences caused by element position changes due to different page rendering environments.

[0008] In a possible implementation manner, the detection output specification includes: outputting abnormality details in the case where it is detected that the test page image has element abnormalities; and the abnormality details include: abnormality category, abnormality element name, abnormality description, and abnormality position.

[0009] In a possible implementation, the test page image and the reference page image are processed according to a page detection strategy indicated by the page detection prompt word by the multi-modal large model to obtain a page detection result corresponding to the test page image, including: The following steps are performed according to the page detection strategy indicated by the page detection prompt word by the multi-modal large model: The first elements contained in the test page image and the second elements contained in the reference page image are parsed; An element mapping relationship between the first elements and the second elements is established based on semantic consistency and position consistency; An abnormal situation of the test page image is obtained according to the element mapping relationship, an element abnormality determination criterion, and a detection constraint condition; A page detection result corresponding to the test page image is obtained according to the abnormal situation and a detection output specification.

[0010] In a possible implementation, the method further includes: A training data set is obtained, the training data set including a plurality of groups of sample image pairs, each group of sample image pairs including a sample test page image and a sample reference page image, and each sample test page image carrying abnormality annotation data; Each group of sample image pairs is processed by a basic multi-modal large model to obtain abnormality detection data of the sample test page image in each group of sample image pairs; An abnormality detection loss value is calculated based on the abnormality annotation data and the abnormality recognition data; It is determined whether the abnormality detection loss value is greater than a preset loss value; In the case where the abnormality detection loss value is greater than the preset loss value, the step of processing each group of sample image pairs by the basic multi-modal large model is executed again until the abnormality detection loss value is less than or equal to the preset loss value; The basic multi-modal large model when the abnormality detection loss value is less than or equal to the preset loss value is taken as the multi-modal large model.

[0011] In a second aspect, the embodiments of the present specification provide a page detection method, applied to a terminal, including: In response to a page detection operation of a user, a page detection request including a test page image and a reference page image is generated; The page detection request is sent to a server; receive the page detection result corresponding to the test page image sent by the server; the page detection result corresponding to the test page image is obtained by the server by processing the test page image and the reference page image according to a page detection strategy indicated by a page detection prompt word through a multi-modal large model; the page detection prompt word is used to define element abnormality judgment criteria, detection constraint conditions and detection output specifications in the page detection strategy; the detection constraint condition is used to define a visual presentation difference type exempted from detection; display based on the page detection result corresponding to the test page image.

[0012] In a possible implementation, the displaying based on the page detection result corresponding to the test page image includes: displaying an abnormality of the test page image based on the page detection result corresponding to the test page image; in a case where the abnormality exists, displaying an abnormality detail corresponding to the test page image based on the page detection result corresponding to the test page image; the abnormality detail includes an abnormality category, an abnormality element name, an abnormality description and an abnormality position; displaying the test page image rendered with an abnormality bounding box according to the abnormality position.

[0013] In a third aspect, the embodiments of the present specification provide a page detection apparatus, including: an acquisition module configured to acquire a test page image and a reference page image; a processing module configured to process the test page image and the reference page image according to a page detection strategy indicated by a page detection prompt word through a multi-modal large model, to obtain a page detection result corresponding to the test page image; The page detection prompt word is used to define element abnormality judgment criteria, detection constraint conditions and detection output specifications in the page detection strategy; the detection constraint condition is used to define a visual presentation difference type exempted from detection.

[0014] In a fourth aspect, the embodiments of the present specification provide a page detection apparatus applied to a terminal, including: a response module configured to generate a page detection request including a test page image and a reference page image in response to a page detection operation of a user; a sending module configured to send the page detection request to a server; a receiving module configured to receive a page detection result corresponding to the test page image sent by the server; the page detection result corresponding to the test page image is obtained by the server by processing the test page image and the reference page image according to a page detection strategy indicated by a page detection prompt word through a multi-modal large model; the page detection prompt word is used to define element abnormality judgment criteria, detection constraint conditions and detection output specifications in the page detection strategy; the detection constraint condition is used to define a visual presentation difference type exempted from detection. The display module is configured to display based on the page detection result corresponding to the test page image.

[0015] In a fifth aspect, an electronic device is provided, including a processor and a memory. The memory stores a computer program, and the computer program, when executed by the processor, implements the method steps provided in the first aspect or the second aspect of the present specification.

[0016] In a sixth aspect, a computer storage medium is provided, which stores a plurality of instructions. The instructions are adapted to be loaded and executed by a processor to perform the method steps provided in the first aspect or the second aspect of the present specification.

[0017] In a seventh aspect, a computer program product is provided, including a computer program. The computer program, when executed by a processor, implements the method steps provided in the first aspect or the second aspect of the present specification.

[0018] The above-mentioned page detection method, device, equipment, medium and program product, by acquiring the test page image and the reference page image, based on the multi-modal large model, processing the test page image and the reference page image according to the page detection strategy indicated by the page detection prompt word, obtaining the page detection result of the test page image, on the one hand, can guide the multi-modal large model to perform element-level semantic understanding on the test page image and the reference page image based on the element abnormality judgment standard, the detection constraint condition and the detection output specification in the page detection strategy, improving the accuracy of page detection; on the other hand, it can flexibly cope with the visual presentation difference problem caused by the inherent difference such as rendering environment based on the visual presentation difference exempted from detection in the detection constraint condition, improving the generalization ability of page detection, so as to realize reliable page detection. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present specification, the following will briefly introduce the drawings needed to be used in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present specification, and other drawings can also be obtained by those skilled in the art without creative labor.

[0020] Figure 1 An application environment schematic diagram of a page detection method provided by an exemplary embodiment of the present specification; Figure 2 A flowchart of a page detection method provided by an exemplary embodiment of the present specification; Figure 3A A schematic diagram of a test page image provided by an exemplary embodiment of the present specification; Figure 3B A schematic diagram of a reference page image provided for an exemplary embodiment of the present specification; Figure 3C A schematic diagram of a test page image rendered with an abnormal bounding box provided for an exemplary embodiment of the present specification; Figure 3D A schematic diagram of another test page image provided for an exemplary embodiment of the present specification; Figure 4A A schematic diagram of another reference page image provided for an exemplary embodiment of the present specification; Figure 4B A schematic diagram of another reference page image provided for an exemplary embodiment of the present specification; Figure 5 A schematic diagram of a flow of another page detection method provided for an exemplary embodiment of the present specification; Figure 6 A schematic diagram of a flow of another page detection method provided for an exemplary embodiment of the present specification; Figure 7 A schematic diagram of a structure of a page detection apparatus provided for an exemplary embodiment of the present specification; Figure 8 A schematic diagram of a structure of another page detection apparatus provided for an exemplary embodiment of the present specification; Figure 9 A schematic diagram of a structure of an electronic device provided for an exemplary embodiment of the present specification; Figure 10 A schematic diagram of a structure of another electronic device provided for an exemplary embodiment of the present specification. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical scheme and advantages of the present specification clearer, the present specification will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present specification, and cannot be used to limit the present specification.

[0022] In the description of the present specification, it should be understood that the terms "first", "second", etc. are only for the purpose of description, and cannot be understood as indicating or implying relative importance. For those skilled in the art, the specific meanings of the above terms in the present specification can be understood according to the specific circumstances. In addition, in the description of the present specification, "multiple" means two or more, unless otherwise specified. "And / or", which describes the association relationship of the associated objects, means that there can be three relationships, for example, A and / or B can mean that there are three cases of A alone, A and B together, and B alone. The character " / " generally represents that the associated objects before and after it are in an "or" relationship.

[0023] Currently, the page detection methods can be divided into two categories: One category is the most mainstream pixel-level matching-based image comparison. The RGB value or grayscale value of the test page image and the reference page image is compared pixel by pixel to detect element abnormalities in the test page image. Although this method is simple to implement, it is extremely sensitive to visual presentation changes in the page. For example, on devices with different resolutions, the correspondence of pixels in the test page image and the reference page image will be disrupted. A pixel originally at a certain physical location in the reference page image may correspond to a different physical location in the test page, so that accurate pixel-level comparison cannot be performed, and reliable page detection is difficult to achieve.

[0024] The other category is key feature-based image comparison. For example, key features of the test page image and the reference page image are extracted respectively, and then the consistency of the two is evaluated through similarity calculation to achieve page detection. However, this method lacks in-depth understanding of elements and can only roughly determine whether the test page image has abnormalities, but cannot accurately capture element-level abnormalities in the page, and the page detection is not accurate enough.

[0025] To solve the above problems, the present specification provides a page detection method, device, equipment, medium and program product. By obtaining a test page image and a reference page image, processing the test page image and the reference page image according to a page detection strategy indicated by a page detection prompt word based on a multi-modal large model, and obtaining a page detection result of the test page image, on the one hand, the multi-modal large model can be guided to perform element-level semantic understanding on the test page image and the reference page image based on element abnormality determination standards, detection constraint conditions and detection output specifications in the page detection strategy, thereby improving the accuracy of page detection; on the other hand, visual presentation differences caused by inherent differences such as rendering environment can be flexibly coped with based on visual presentation differences exempted from detection in the detection constraint conditions, thereby improving the generalization ability of page detection, and thus reliable page detection is achieved.

[0026] Please refer to Figure 1 An application environment schematic diagram of a page detection method provided by an example embodiment of the present specification is shown. A terminal 10 logged into a page detection platform communicates with a server 20 corresponding to the page detection platform through a network. A data storage system can store data required to be processed by the server 20. The data storage system can be integrated on the server 20, or placed on a cloud or other network server.

[0027] In some possible embodiments, the page detection method provided by the present specification can be executed by the terminal 10 and the server 20 together. Accordingly, the page detection device can also be arranged in the terminal 10 and the server 20 respectively.

[0028] Optionally, the terminal 10 generates a page detection request including the test page image and the reference page image in response to a page detection operation of the user, and sends the page detection request to the server 20. The server 20 receives the page detection request sent by the terminal 10, processes the test page image and the reference page image according to the page detection strategy indicated by the page detection prompt word through the multi-modal large model, obtains the page detection result corresponding to the test page image, and sends the page detection result to the terminal 10; wherein the page detection prompt word is used to define the element abnormality judgment standard, the detection constraint condition and the detection output specification in the page detection strategy; the detection constraint condition is used to define the visual presentation difference type exempted from detection. The terminal 10 receives the page detection result sent by the server 20, and displays based on the page detection result.

[0029] In some possible embodiments, the page detection method provided by the present specification can be executed by the terminal 10. Accordingly, the page detection device can also be arranged in the terminal 10.

[0030] Optionally, the terminal 10 receives the test page image and the reference page image uploaded by the user; in response to a page detection operation of the user, the test page image and the reference page image are processed according to the page detection strategy indicated by the page detection prompt word through the multi-modal large model, and the page detection result corresponding to the test page image is obtained; wherein the page detection prompt word is used to define the element abnormality judgment standard, the detection constraint condition and the detection output specification in the page detection strategy; the detection constraint condition is used to define the visual presentation difference type exempted from detection; and the display is based on the page detection result corresponding to the test page image.

[0031] In some possible embodiments, the page detection method provided by the present specification can be executed by the server 20 according to a preset strategy automatically and periodically. Accordingly, the page detection device can also be arranged in the server 20.

[0032] Optionally, the server 20 obtains the test page image and the reference page image; the test page image and the reference page image are processed according to the page detection strategy indicated by the page detection prompt word through the multi-modal large model, and the page detection result corresponding to the test page image is obtained; wherein the page detection prompt word is used to define the element abnormality judgment standard, the detection constraint condition and the detection output specification in the page detection strategy; and the detection constraint condition is used to define the visual presentation difference type exempted from detection.

[0033] It should be understood that the terminal 10 can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers and Internet of Things devices. The server 20 can be implemented by an independent server or a server cluster composed of multiple servers.

[0034] In one embodiment, as Figure 2As shown, a page detection method is provided, which is taken as an example of application to the terminal 10 and the server 20, and includes the following steps: S202: The terminal 10 generates a page detection request including a test page image and a reference page image in response to a page detection operation of the user.

[0035] The page detection operation can be, but is not limited to, a customized interactive action such as clicking, sliding, long pressing, etc. The test page image is a page image that needs to be detected in the software development or software testing process. The reference page image is a standard page image that is confirmed to meet the expected display. The test page image and the reference page image are associated with the same business node, that is, the display logic of the two images is completely consistent. By comparing the interface elements (hereinafter referred to as elements) in the test page image and the reference page image, the detection of the test page image can be completed.

[0036] Optionally, the terminal 10 generates a page detection request including a test page image and a reference page image in response to a clicking operation of the user on the page detection control. Subsequently, the server 20 compares the interface elements in the test page image and the reference page image based on the page detection request, thereby completing the page detection of the test page image.

[0037] It is worth noting that the embodiment only describes a preferred implementation, and in actual application, the page detection request can not include the test page image and the reference page image, but include the image identifiers of the test page image and the reference page image. Subsequently, the server can obtain the test page image and the reference page image from the data storage system based on the image identifiers in the page detection request for processing. In addition, the page detection request can not include the test page image and the reference page image, but only include the image identifier of the test page image. Subsequently, the server obtains the image identifier of the reference page image associated with the test page image according to the image identifier of the test page image, and obtains the test page image and the reference page image from the data storage system according to the image identifiers of the test page image and the reference page image for processing.

[0038] S204: The terminal 10 sends the page detection request to the server 20.

[0039] Optionally, the terminal 10 sends the page detection request including the test page image and the reference page image to the server 20 through a network interface.

[0040] S206: The server 20 receives the page detection request sent by the terminal 10.

[0041] Optionally, the server 20 receives the page detection request sent by the terminal 10 through the network interface, parses it, obtains the test page image and the reference page image, and creates a page detection task.

[0042] S208: The server 20 processes the test page image and the reference page image according to the page detection strategy indicated by the page detection prompt word through the multi-modal large model, to obtain the page detection result of the test page image.

[0043] The page detection prompt word is used to define the element abnormality judgment standard, the detection constraint condition and the detection output specification in the page detection strategy.

[0044] Specifically, the element abnormality judgment standard in the page detection strategy is used to define what kind of situation is judged as element abnormality. The element abnormality judgment standard includes at least one of the following: element missing judgment standard, element increasing judgment standard, element misplacement judgment standard, and element change judgment standard. For example, the element missing judgment standard can be set as: for any element in the reference page image, if there is no element in the test page image that is consistent with the element in semantics or position, it is judged that the element in the test page image is missing. The element increasing judgment standard can be set as: for any element in the test page image, if there is no element in the reference page image that is consistent with the element in semantics or position, it is judged that the element in the test page image is increasing. The element misplacement judgment standard can be set as: for any element in the test page image, if there is an element in the reference page image that is consistent with the element in semantics but inconsistent in position, it is judged that the element in the test page image is misplaced. The element change judgment standard can be set as: for any element in the test page image, if there is an element in the reference page image that is consistent with the element in position but inconsistent in semantics, it is judged that the element in the test page image is changed.

[0045] In this embodiment, by defining the element missing, element increasing, element misplacement, element change and other judgment standards in the page detection prompt word, the multi-modal large model can accurately and comprehensively identify the element abnormality in the test page image that meets the judgment standard, thereby improving the efficiency and accuracy of page detection, and further improving the reliability of page detection.

[0046] The detection constraint condition in the page detection strategy is used to define the visual presentation difference type exempted from detection, and the visual presentation difference type exempted from detection includes at least one of the following: a visual presentation difference caused by a change in element semantics due to preset business logic, and a visual presentation difference caused by a change in element position due to a difference in page rendering environment. The preset business logic includes a preset dynamic display rule. For example, the preset business logic can include: displaying different personalized content in a dynamic content area for different user accounts or devices, and / or displaying different interactive controls for different interaction states. The preset business logic can cause a compliant semantic change of an element. If this type of visual presentation difference is not exempted in advance, the compliant semantic change of the element may be determined as an anomaly. The page rendering environment includes a device environment for rendering the page and a software environment for rendering the page. The device environment for rendering the page includes, but is not limited to, the screen size of the device for rendering the page, the screen resolution of the device for rendering the page, and the like. The software environment for rendering the page includes, but is not limited to, the operating system for rendering the page, and the like. For example, the font and other visual rendering styles in the home page of an application program may change in position due to adaptation to different operating systems. Therefore, the page rendering environment can also cause a compliant position change of an element. If this type of visual presentation difference is not exempted in advance, the compliant position change of the element may also be determined as an anomaly.

[0047] In this embodiment, by setting the detection constraint condition in the page detection prompt word, the multimodal large model can be instructed to exclude the visual presentation difference caused by the change in element semantics due to the preset business logic and the visual presentation difference caused by the change in element position due to the difference in page rendering environment in the detection process. This realizes intelligent exemption to avoid false positives, improves the flexibility and compatibility of page detection, and further improves the reliability of page detection.

[0048] The detection output specification in the page detection strategy is used to define the output format and output content of the page detection. The detection output specification includes: outputting the page detection result in JSON format, and the page detection result at least includes the abnormal situation of the test page image, that is, the abnormal situation of the test page. In addition, when the abnormal situation of the test page image is an abnormal situation, the page detection result further includes abnormal details, and the abnormal details include: an abnormal category, an abnormal element name, an abnormal description, and an abnormal position. For example, the abnormal details can be: {'element missing': [{'abnormal element name': 'hot activity area', 'abnormal description': 'the hot activity area in the middle of the test page is missing', 'abnormal position': [89, 1324, 768, 1732]}, 'element increase': [], 'element misplacement': [], 'element change': []}.

[0049] In this embodiment, by setting the detection output specification in the page detection prompt word, the multi-modal large model can be instructed to output the abnormality details including the abnormality category, the abnormal element name, the abnormality description and the abnormality location in the case of detecting that the test page image has an abnormality, to provide a complete information chain for subsequent abnormality repair and abnormality positioning, improve the normativity and completeness of page detection, and further improve the reliability of page detection.

[0050] Optionally, the server 20 performs the following steps according to the page detection strategy indicated by the page detection prompt word through the multi-modal large model: parsing each first element contained in the test page image and each second element contained in the reference page image, establishing an element mapping relationship between each first element and each second element based on semantic consistency and position consistency, obtaining an abnormality condition of the test page image according to the element mapping relationship, the element abnormality determination standard and the detection constraint condition, and obtaining a page detection result corresponding to the test page image according to the abnormality condition and the detection output specification.

[0051] S210: The server 20 sends the page detection result corresponding to the test page image to the terminal 10.

[0052] Optionally, after obtaining the page detection result corresponding to the test page image, the server 20 sends the page detection result of the test page image to the terminal 10 through the network interface. It can be understood that the page detection result at least includes the abnormality condition, that is, whether there is an abnormality. In the case of the abnormality condition being an abnormality, the page detection result further includes the abnormality details.

[0053] It is worth noting that the above only describes one preferred embodiment, and in actual application process, in addition to sending the page detection result corresponding to the test page image to the terminal 10, the server 20 can also send the image link corresponding to the reference page image, the image link corresponding to the test page image, the task identifier corresponding to the page detection request, the processing time length of the page detection request, the initiation time stamp of the page detection request and other information to the terminal 10 for display, and the present embodiment does not limit this.

[0054] S212: The terminal 10 receives the page detection result sent by the server 20.

[0055] Optionally, the server receives the page detection result sent by the terminal 10 through the network interface.

[0056] S214: The terminal 10 displays based on the page detection result.

[0057] Optionally, the terminal 10 displays anomalies in the test page image based on the page detection results. If the anomaly in the test page image is found to be present, the terminal 10 also displays anomaly details corresponding to the test page image based on the page detection results, specifically including information such as anomaly category, anomaly element name, anomaly description, and anomaly location, and displays a test page image with an anomaly bounding box rendered according to the anomaly location in the anomaly details.

[0058] In this embodiment, the terminal 10 displays the abnormalities of the test page image based on the page detection results, which can provide users with comprehensive and accurate visual feedback. This makes it easy for users to accurately locate the page based on the abnormality category, abnormal element name, abnormal description and abnormal location, thereby facilitating subsequent abnormality investigation and improving the convenience and interactivity of page detection.

[0059] Please see Figures 3A to 3C , Figure 3A The test page image shown is compared to Figure 3B The baseline page image shown has an element missing anomaly in the "Hot Activities Zone" section. Server 20 uses a multimodal large model to... Figure 3A The test page image shown and Figure 3B After processing the baseline page image shown, the URL (Uniform Resource Locator) of the baseline page image, the URL of the test page image, the anomaly condition of the test page image (anomaly exists), the anomaly details of the test page image, the processing time of the page detection request, and the timestamp of the page detection request are sent to terminal 10. Based on the content sent by server 20, terminal 10 displays the URL of the baseline page image, the URL of the test page image, the anomaly condition of the test page image (anomaly exists), the anomaly details of the test page image, the processing time of the page detection request, and the timestamp of the page detection request in the result display table shown in Table 1. It also displays... Figure 3C The image shown is a test page image 32 with an abnormal bounding box 31.

[0060] Table 1

[0061] Please see Figure 3B as well as Figure 3D , Figure 3D The test page image shown is compared to Figure 3B The baseline page image shown has differences in the "Hot Activities Zone" and the "View Hot Topics" section. Since exemptions can be pre-defined in the prompts to detect visual differences in element semantics caused by preset business logic, server 20 uses a multimodal large model to... Figure 3BThe reference page image shown and Figure 3D After processing the test page image shown, the terminal 10 is sent the URL of the reference page image, the URL of the test page image, the abnormality of the test page image (no abnormality), the processing duration of the page detection request, and the initiation timestamp of the page detection request. Based on the content sent by the server 20, the terminal 10 displays the URL of the reference page image, the URL of the test page image, the abnormality of the test page image (no abnormality), the abnormality details of the test page image, the request processing duration of the page detection request, and the request initiation timestamp of the page detection request in the result display shown in Table 2.

[0062] Table 2

[0063] See Figures 4A to 4B , Figure 4A The test page image shown has a display difference in the top and bottom of the page due to the difference in screen height compared to Figure 4B The reference page image shown and Figure 4A After processing the test page image shown and Figure 4B the terminal 10 is sent the URL of the reference page image, the URL of the test page image, the abnormality of the test page image (no abnormality), the processing duration of the page detection request, and the initiation timestamp of the page detection request. Similarly, based on the content sent by the server 20, the terminal 10 displays the URL of the reference page image, the URL of the test page image, the abnormality of the test page image (no abnormality), the abnormality details of the test page image, the request processing duration of the page detection request, and the request initiation timestamp of the page detection request in the result display table.

[0064] The page detection method can realize reliable page detection.

[0065] In one embodiment, as shown in FIG. 10, another page detection method is provided, which is applied to the terminal 10 and the server 20 as an example and includes the following steps: Figure 5 S502: The terminal 10 generates a page detection request including a test page image and a reference page image in response to a page detection operation of a user. S504: The terminal 10 sends the page detection request to the server 20.

[0066] Optionally, the page detection request can also include a detection task instruction input by the user, which is used to specify a page detection object and a page detection intention. For example, the detection task instruction can be "picture one is a reference page image, picture two is a test page image, please analyze whether the test page image is abnormal according to the reference page image".

[0067] Specifically, S502 is consistent with S202, which will not be described here.

[0068] S504: The terminal 10 sends the page detection request to the server 20.

[0069] Optionally, the terminal 10 sends the page detection request including the test page image, the reference page image and the detection task target to the server 20 through a network interface.

[0070] S506: The server 20 receives the page detection request sent by the terminal 10.

[0071] Optionally, the server 20 receives the page detection request sent by the terminal 10 through the network interface, parses it, obtains the reference page image, the test page image, and the detection task target, and creates a page detection task.

[0072] S508: The server 20 parses each first element contained in the test page image and each second element contained in the reference page image according to the detection task instruction and the detection task target indicated by the page detection prompt word through the multi-modal large model.

[0073] The page detection prompt word includes a user prompt word (User Prompt) and a system prompt word (System Prompt). The user prompt word is used to indicate the detection task instruction in the page detection strategy, and the system prompt word is used to indicate the detection role and the detection target, the element abnormality judgment standard, the detection constraint condition, and the detection output specification in the page detection strategy.

[0074] It can be understood that the above detection task instruction is used to clearly indicate the page detection object and the page detection intention, for example, the detection task instruction can be "picture one is the reference page image, picture two is the test page image, please analyze whether there is an abnormality in the test page image according to the reference page image". The above detection role and detection target are used to functionally position and guide the capability of the multi-modal large model, for example, the detection role and detection target can be "you are a page consistency detection expert, and need to analyze visual elements and semantic information at the same time".

[0075] Optionally, the server 20 identifies each first element contained in the test page image and each second element contained in the reference page image according to the detection task instruction indicated by the user prompt word through the trained multi-modal large model; and then obtains the first element semantic and the first element position of each first element in the test page image, and the second element semantic and the second element position of each second element in the reference page image according to the detection role and the detection target indicated by the system prompt word.

[0076] S510: The server 20 establishes an element mapping relationship between each first element and each second element based on semantic consistency and position consistency according to the detection role and the detection target indicated by the page detection prompt word through the multi-modal large model.

[0077] Optionally, the server 20 establishes an element mapping relationship between each first element and each second element according to the first element semantic and the first element position of each first element, and the second element semantic and the second element position of each second element, according to the detection role and the detection target indicated by the page detection prompt word through the multi-modal large model, to compare whether the element semantic and the element position of each first element in the test page image and each second element in the reference page image are consistent.

[0078] S512: The server 20 obtains the abnormal situation of the test page image according to the element mapping relationship, the element abnormality determination standard, and the detection constraint condition through the multi-modal large model.

[0079] The element abnormality determination standard is used to define what kind of situation is determined as an element abnormality. The element abnormality determination standard includes: an element missing determination standard, an element increase determination standard, an element misplacement determination standard, and an element change determination standard. The detection constraint condition is used to define the visual presentation difference type exempted from detection. The visual presentation difference type exempted from detection defined by the detection constraint condition includes: a visual presentation difference caused by a preset business logic leading to an element semantic change, and a visual presentation difference caused by a page rendering environment leading to an element position change. The preset business logic includes a preset dynamic display rule. The page rendering environment includes a device environment for rendering a page and a software environment for rendering a page. The device environment for rendering a page includes but is not limited to: a screen size of a device for rendering a page, a screen resolution of a device for rendering a page, etc. The software environment for rendering a page includes but is not limited to: an operating system for rendering a page, etc.

[0080] Optionally, the server 20 first determines whether the first element semantics of each first element in the test page image and the second element semantics of each second element in the reference page image are consistent, and whether the first element position of each first element in the test page image and the second element position of each second element in the reference page image are consistent, according to the element mapping relationship through the multi-modal large model; then, in combination with the element abnormality determination standard, determines suspicious abnormalities according to the consistency of the first element semantics and the second element semantics, and the consistency of the first element position and the second element position; finally, according to the detection constraint condition, screens out the element abnormality existing in the test page image from the suspicious abnormalities to determine the abnormal situation of the test page image.

[0081] The server 20 determines the specific process of the suspicious abnormality by combining the element abnormality determination standard with the multi-modal large model according to the consistency of the first element semantics and the second element semantics and the consistency of the first element position and the second element position as follows: For any second element in the reference page image, if there is no first element in the test page image that is consistent with the second element semantics or the second element position of the second element, the server 20 determines that the test page image has a suspicious element missing abnormality. For any first element in the test page image, if there is no second element in the reference page image that is consistent with the first element semantics or the first position, it is determined that the test page image has a suspicious element increase abnormality. For any first element in the test page image, if there is a second element in the reference page image that is consistent with the first element semantics of the first element but inconsistent with the first element position, it is determined that the test page image has a suspicious element misplacement abnormality. For any first element in the test page image, if there is a second element in the reference page image that is consistent with the first element position of the first element but inconsistent with the first element semantics, it is determined that the test page image has a suspicious element change abnormality.

[0082] The server 20 determines the specific process of the element abnormality of the test page image from the suspicious abnormality according to the detection constraint condition to determine the abnormality of the test page image as follows: excluding the visual presentation difference caused by the change of the element semantics due to the preset business logic (for example, the display difference of the dynamic content area caused by the preset business logic), the visual presentation difference caused by the change of the element position due to the difference of the page rendering environment (for example, the display difference of the top and bottom of the page caused by the difference of the screen height) in the suspicious abnormality, thereby determining the element abnormality in the suspicious abnormality, and then determining whether the test page image has an abnormality according to the number of element abnormalities.

[0083] S514: The server 20 obtains the page detection result corresponding to the test page image according to the abnormality and the detection output specification by the multi-modal large model.

[0084] The detection output specification is used to define the output format and output content of the page detection, for example, the detection output specification can be: outputting the page detection result in JSON format, and the page detection result at least includes the abnormality of the test page image, that is, the abnormality of the test page. In addition, when the abnormality of the test page image is abnormal, the page detection result further includes abnormal details, and the abnormal details include: abnormal category, abnormal element name, abnormal description and abnormal position.

[0085] Optionally, the server 20 outputs, according to the abnormal situation and the detection output specification, a page detection result including the abnormal situation (existence of abnormality) and abnormal details in a case where the test page image exists abnormality, and outputs a page detection result including the abnormal situation (non-existence of abnormality) in a case where the test page image does not exist abnormality, by means of the multi-modal large model.

[0086] S516: The server 20 sends the page detection result corresponding to the test page image to the terminal 10.

[0087] Optionally, the server 20 sends, by means of the network interface, the page detection result of the test page image, the image link corresponding to the reference page image, the image link corresponding to the test page image, the task identifier corresponding to the page detection request, the processing time length of the page detection request, and the initiation timestamp of the page detection request to the terminal 10.

[0088] S518: The terminal 10 receives the page detection result sent by the server 20.

[0089] Optionally, the terminal 10 receives, by means of the network interface, the page detection result sent by the server 20, the image link corresponding to the reference page image, the image link corresponding to the test page image, the task identifier corresponding to the page detection request, the processing time length of the page detection request, and the initiation timestamp of the page detection request.

[0090] S520: The terminal 10 displays based on the page detection result corresponding to the test page image.

[0091] Optionally, the terminal 10 displays based on the above content sent by the server 20.

[0092] Specifically, please refer to S214 described above, which will not be repeated here.

[0093] In this embodiment, the server, by means of the multi-modal large model, first, automatically compares the test page image and the reference page image in two dimensions of semantics and position to determine the suspicious abnormality of the test page image, thereby improving the efficiency and coverage of page detection; second, filters out the element abnormality of the test page image from the suspicious abnormality of the test page image according to the pre-defined visual presentation difference type exempted from detection in the detection constraint condition, thereby improving the flexibility, compatibility and accuracy of page detection; third, obtains the page detection result corresponding to the test page image according to the abnormal situation and the detection output specification, thereby improving the comprehensiveness of the page detection result output. The page detection method provided in this embodiment can effectively improve the reliability of page detection.

[0094] In one embodiment, as Figure 6As shown, another page detection method is provided, and the method is described by taking the server 20 as an example. The process of training the multi-modal large model in the method includes the following steps: S602: Obtain a training data set; the training data set includes a plurality of groups of sample image pairs, each group of sample image pairs includes a sample test page image and a sample reference page image, and each sample test page image carries abnormal annotation data.

[0095] The abnormal annotation data at least includes the abnormal situation of the sample test page image, and in the case that the abnormal situation of the sample test page image exists, the abnormal annotation data further includes sample abnormal details of the sample test page image. The sample abnormal details can include: sample abnormal category, sample abnormal element name, sample abnormal description and sample abnormal position.

[0096] It can be understood that the server 20 can directly obtain the training data set from the preset sample image database, or receive the sample image sent by the developer through the terminal, and then construct the training data set.

[0097] S604: Process each group of sample image pairs by the basic multi-modal large model to obtain abnormal detection data of the sample test page image in each group of sample image pairs.

[0098] The basic multi-modal large model is a deep learning model capable of processing and understanding multiple types of data (such as text, image, etc.) at the same time. The basic multi-modal large model can be trained by a large-scale multi-modal corpus. For example, Qwen2.5VL, which is specially used for visual-linguistic tasks, can be used as the basic multi-modal large model.

[0099] Optionally, the server 20 processes the sample test page image and the sample reference page image in the group of sample image pairs according to the page detection strategy indicated by the page detection prompt word through the basic multi-modal large model, to obtain the abnormal detection data of the sample test page image in each group of sample image pairs. It can be understood that the above-mentioned page detection prompt word for training the multi-modal large model can also be used to define the element abnormality judgment standard, the detection constraint condition and the detection output specification in the page detection strategy. The above-mentioned element abnormality standard can include at least one of the following: element missing judgment standard, element increasing judgment standard, element misplacement judgment standard, and element change judgment standard. The above-mentioned detection constraint condition is used to define the visual presentation difference type exempted from detection. The visual presentation difference type exempted from detection includes at least one of the following: visual presentation difference caused by preset business logic leading to element semantic change, and visual presentation difference caused by different page rendering environments leading to element position change. The above-mentioned detection output specification includes: outputting abnormal details in the case that the test page image is detected to have element abnormality; the abnormal details include: abnormal category, abnormal element name, abnormal description and abnormal bounding box represented in the form of coordinates.

[0100] S606: Calculate an anomaly detection loss value based on the anomaly annotation data and the anomaly recognition data.

[0101] Optionally, the server 20 calculates the anomaly detection loss value based on the anomaly annotation data and the anomaly recognition data in combination with a pre-constructed anomaly loss function. Illustratively, the anomaly loss function can be constructed based on an anomaly classification loss function and a bounding box regression loss function, for example, it can be constructed by weighted summation of the anomaly classification loss and the bounding box regression loss. The anomaly classification loss function is used to measure the difference between the anomaly class in the anomaly recognition data and the anomaly class in the anomaly annotation data, and the bounding box regression loss is used to measure the difference between the bounding box in the anomaly recognition data and the bounding box in the anomaly annotation data in position and size.

[0102] S608: Determine whether the anomaly detection loss value is greater than a preset loss value. If yes, execute S604 again; if no, execute S610.

[0103] Optionally, the server 20 determines whether the anomaly detection loss value calculated based on the anomaly annotation data and the anomaly recognition data is greater than a preset loss value. If the anomaly detection loss value is greater than the preset loss value, the server 20 executes again the step of processing each group of sample image pairs by the base multi-modal large model to obtain the anomaly detection data of the sample test page image in each group of sample image pairs, to start the next round of training. If the anomaly detection loss value is less than or equal to the preset loss value, the server 20 takes the base multi-modal large model at this time as the multi-modal large model.

[0104] S610: Take the base multi-modal large model when the anomaly detection loss value is less than or equal to the preset loss value as the multi-modal large model.

[0105] Optionally, the server 20 ends the model training process and obtains the multi-modal large model when the anomaly detection loss value is less than or equal to the preset loss value.

[0106] In this embodiment, the server 20 obtains the multi-modal large model by fine-tuning the base multi-modal large model, which can improve the performance of the model in the page detection field task without losing the general text understanding ability of the model, so that the multi-modal large model obtained by training can accurately and quickly realize end-to-end anomaly recognition when processing the page detection task. The whole process improves the efficiency and accuracy of page detection, and further improves the reliability of page detection.

[0107] It should be understood that although each step in the flowchart involved in each embodiment as described above is shown in sequence according to the direction of the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless explicitly stated herein, there is no strict order limitation for the execution of these steps, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be alternately or alternately executed with at least part of other steps or steps or stages in other steps.

[0108] Based on the above page detection method, the application concept, as shown in Figure 7 The embodiments of the present specification also provide a page detection device 700 for implementing the above-mentioned page detection method. The page detection device 700 comprises: The acquisition module 701 is configured to acquire a test page image and a reference page image. The processing module 702 is configured to process the test page image and the reference page image according to a page detection strategy indicated by a page detection prompt word through a multi-modal large model to obtain a page detection result corresponding to the test page image. The page detection prompt word is used to define the element abnormality determination standard, the detection constraint condition and the detection output specification in the page detection strategy; and the detection constraint condition is used to define the visual presentation difference type exempted from detection.

[0109] In a possible implementation manner, the element abnormality determination standard comprises at least one of the following: an element missing determination standard, an element increase determination standard, an element misplacement determination standard, and an element change determination standard.

[0110] In a possible implementation manner, the visual presentation difference type exempted from detection comprises at least one of the following: a visual presentation difference caused by a change in element semantics due to a preset business logic, and a visual presentation difference caused by a change in element position due to a difference in page rendering environment.

[0111] In a possible implementation manner, the detection output specification comprises: outputting abnormality details in the case where it is detected that the test page image has element abnormalities; and the abnormality details comprise: an abnormality category, an abnormal element name, an abnormality description, and an abnormality position.

[0112] In a possible implementation, the processing module 702 is specifically configured to perform the following steps on the basis of the page detection strategy indicated by the page detection prompt word through the multi-modal large model: parsing each first element contained in the test page image and each second element contained in the reference page image; establishing an element mapping relationship between each first element and each second element based on semantic consistency and position consistency; obtaining an abnormal situation of the test page image according to the element mapping relationship, the element abnormality determination standard, and the detection constraint condition; and obtaining a page detection result corresponding to the test page image according to the abnormal situation and the detection output specification.

[0113] In a possible implementation, the page detection apparatus 700 further includes a model training module configured to obtain a training data set; the training data set includes a plurality of groups of sample image pairs, each group of sample image pairs including a sample test page image and a sample reference page image, and each sample test page image carries abnormality annotation data; process each group of sample image pairs through a basic multi-modal large model to obtain abnormality detection data of the sample test page image in each group of sample image pairs; calculate an abnormality detection loss value based on the abnormality annotation data and the abnormality recognition data; determine whether the abnormality detection loss value is greater than a preset loss value; in the case where the abnormality detection loss value is greater than the preset loss value, perform again the step of processing each group of sample image pairs through the basic multi-modal large model until the abnormality detection loss value is less than or equal to the preset loss value; and take the basic multi-modal large model when the abnormality detection loss value is less than or equal to the preset loss value as the multi-modal large model.

[0114] Based on the inventive concept of the above-described page detection method applied to a terminal, as shown in Figure 8 The present specification also provides a page detection apparatus 800 for implementing the above-described page detection method applied to a terminal. The page detection apparatus 800 includes: a response module 801 configured to generate a page detection request including a test page image and a reference page image in response to a page detection operation of a user; a sending module 802 configured to send the page detection request to a server; a receiving module 803 configured to receive a page detection result corresponding to the test page image sent by the server; the page detection result corresponding to the test page image is obtained by the server by processing the test page image and the reference page image through a multi-modal large model according to a page detection strategy indicated by a page detection prompt word; the page detection prompt word is used to define an element abnormality determination standard, a detection constraint condition, and a detection output specification in the page detection strategy; and the detection constraint condition is used to define a visual presentation difference type exempted from detection; a display module 804 configured to display based on the page detection result corresponding to the test page image.

[0115] In a possible implementation, the display module 804 is specifically configured to: display an abnormal situation of the test page image based on the page detection result corresponding to the test page image; and in a case where the abnormal situation is an abnormality existing, display abnormality details corresponding to the test page image based on the page detection result corresponding to the test page image, wherein the abnormality details include an abnormality category, an abnormality element name, an abnormality description, and an abnormality position; and display the test page image with an abnormality bounding box rendered according to the abnormality position.

[0116] Each of the modules in the page detection apparatuses 700 and 800 described above can be implemented by software, hardware, or a combination thereof, in whole or in part. Each of the modules described above can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in a computer device in software form, so as to be invoked and executed by a processor to perform operations corresponding to each of the modules.

[0117] The embodiments of the present specification also provide an electronic device, which can be a server, and an internal structure diagram of the electronic device can be as shown in Figure 9 The electronic device includes a processor, a memory, an input / output interface, and a communication interface. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The application database of the electronic device is configured to store test page images, reference page images, sample images, and the like. The input / output interface of the electronic device is configured to exchange information between the processor and external devices. The communication interface of the electronic device is configured to communicate with external terminals through network connection. The processor of the electronic device executes the computer program to implement a page detection method.

[0118] It should be noted that the electronic device can also be a terminal, and an internal structure diagram of the electronic device can be as shown in Figure 10As shown in the figure. The electronic device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through the system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the electronic device is used to provide computing and control capability. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The input / output interface of the electronic device is used to exchange information between the processor and the external device. The communication interface of the electronic device is used to communicate with the external terminal in a wired or wireless manner. The wireless manner can be realized through WIFI, mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by the processor to realize a page detection method. The display unit of the electronic device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the electronic device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the electronic device, or an external keyboard, touchpad or mouse, etc.

[0119] Those skilled in the art can understand that, Figure 9 and Figure 10 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present specification, and does not constitute a limitation on the electronic device to which the scheme of the present specification is applied. A specific electronic device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0120] In a possible implementation, an electronic device is provided, including a processor and a memory, the memory stores a computer program, and the computer program is executed by the processor to implement the steps in the above method embodiments.

[0121] The embodiments of the present specification also provide a computer storage medium, which stores instructions, and when the instructions are run on a computer or a processor, the computer or the processor executes one or more steps in the above embodiments. The constituent modules of the above electronic device, if realized in the form of software function units and sold or used as independent products, can be stored in the above computer storage medium.

[0122] The embodiments of the present specification also provide a computer program product, including a computer program, which is executed by a processor to implement the steps in the above method embodiments.

[0123] In the above embodiments, all or part of the methods can be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the methods can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the specification are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted by the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through a wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, data center, etc. that includes one or more available media sets. The available media can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a digital versatile disc (DVD)), or a semiconductor medium (for example, a solid state disk (SSD)) and the like.

[0124] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by a computer program instructing related hardware, which can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments. The storage medium includes ROM, RAM, magnetic or optical discs, and various media that can store program codes. In the case of no conflict, the technical features in the embodiments and the embodiments can be combined arbitrarily.

[0125] The above-described embodiments are merely described as preferred embodiments of the specification, and do not limit the scope of the specification. Without departing from the design spirit of the specification, various modifications and improvements to the technical solutions of the specification made by those of ordinary skill in the art should fall within the protection scope of the claims.

[0126] It should be noted that the information, data and signals involved in the embodiments of the specification are authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0127] The above described embodiments of the present description have been described. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order and still achieve desirable results. Additionally, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In certain implementations, multitasking and parallel processing can be advantageous.

Claims

1. A page detection method, the method comprising: Obtain the test page image and the baseline page image; Using a multimodal large model, the test page image and the baseline page image are processed according to the page detection strategy indicated by the page detection prompt words to obtain the page detection result corresponding to the test page image; The page detection prompts are used to define the criteria for judging element anomalies, detection constraints, and detection output specifications in the page detection strategy; the detection constraints are used to define the types of visual presentation differences that are exempt from detection.

2. The method as described in claim 1, wherein the element anomaly determination criteria include at least one of the following: element missing determination criteria, element increase determination criteria, element misalignment determination criteria, and element change determination criteria.

3. The method as described in claim 1, wherein the types of visual presentation differences exempted from detection include at least one of the following: visual presentation differences caused by semantic changes of elements due to preset business logic, and visual presentation differences caused by changes in element position due to different page rendering environments.

4. The method as described in claim 1, wherein the detection output specification includes: If an anomaly is detected in the test page image, output anomaly details; The anomaly details include: anomaly category, anomaly element name, anomaly description, and anomaly location.

5. The method as described in claim 1, wherein processing the test page image and the baseline page image using a multimodal large model according to the page detection strategy indicated by the page detection prompt words to obtain the page detection result corresponding to the test page image includes: Using a multimodal large model, perform the following steps according to the page detection strategy indicated by the page detection prompts: Analyze each first element contained in the test page image and each second element contained in the benchmark page image; Establish an element mapping relationship between each first element and each second element based on semantic consistency and positional consistency; Based on the element mapping relationship, the element anomaly judgment criteria, and the detection constraints, the anomaly situation of the test page image is obtained; Based on the aforementioned abnormal situation and the aforementioned detection output specifications, the page detection result corresponding to the test page image is obtained.

6. The method of claim 1, further comprising: Obtain the training dataset; The training dataset includes multiple pairs of sample images. Each pair of sample images includes a sample test page image and a sample baseline page image. Each sample test page image carries anomaly annotation data. By processing the sample image pairs in each group using a basic multimodal large model, anomaly detection data of the sample test page images in each group of sample image pairs are obtained. Based on the anomaly annotation data and the anomaly identification data, calculate the anomaly detection loss value; Determine whether the anomaly detection loss value is greater than a preset loss value; If the anomaly detection loss value is greater than the preset loss value, the step of processing each group of sample image pairs through the basic multimodal large model is executed again until the anomaly detection loss value is less than or equal to the preset loss value. The basic multimodal large model when the anomaly detection loss value is less than or equal to the preset loss value is taken as the multimodal large model.

7. A page detection method, applied to a terminal, the method comprising: In response to the user's page detection operation, a page detection request is generated, including a test page image and a baseline page image; Send the page detection request to the server; Receive the page detection result corresponding to the test page image sent by the server; The page detection result corresponding to the test page image is obtained by the server processing the test page image and the benchmark page image according to the page detection strategy indicated by the page detection prompt words through a multimodal large model; The page detection prompts are used to define the criteria for judging element anomalies, detection constraints, and detection output specifications in the page detection strategy. The detection constraints are used to define the types of visual representation differences that are exempted from detection; The results are displayed based on the page detection results corresponding to the test page image.

8. The method as described in claim 7, wherein displaying the page based on the page detection result corresponding to the test page image includes: Based on the page detection results corresponding to the test page image, display the abnormal situation of the test page image; If an anomaly is found, the anomaly details corresponding to the test page image are displayed based on the page detection results corresponding to the test page image. The anomaly details include: anomaly category, anomaly element name, anomaly description, and anomaly location; Based on the location of the anomaly, display a test page image with the rendered anomaly bounding box.

9. A page detection device, the device comprising: The acquisition module is used to acquire test page images and benchmark page images; The processing module is used to process the test page image and the benchmark page image according to the page detection strategy indicated by the page detection prompt words through a multimodal large model, so as to obtain the page detection result corresponding to the test page image; The page detection prompts are used to define the criteria for judging element anomalies, detection constraints, and detection output specifications in the page detection strategy; the detection constraints are used to define the types of visual presentation differences that are exempt from detection.

10. A page detection device, applied to a terminal, the device comprising: The response module is used to respond to the user's page detection operation and generate a page detection request including a test page image and a baseline page image; The sending module is used to send the page detection request to the server; The receiving module is used to receive the page detection result corresponding to the test page image sent by the server; The page detection result corresponding to the test page image is obtained by the server processing the test page image and the benchmark page image according to the page detection strategy indicated by the page detection prompt words through a multimodal large model; The page detection prompts are used to define the criteria for judging element anomalies, detection constraints, and detection output specifications in the page detection strategy. The detection constraints are used to define the types of visual representation differences that are exempted from detection; The display module is used to display the page detection results corresponding to the test page image.

11. An electronic device, comprising: Processor and memory; The memory stores a computer program, and when the processor executes the computer program, it implements the method steps of any one of claims 1-8.

12. A computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the method steps of any one of claims 1-8.

13. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-8.

Images