Consistency verification method and device

Abstract

The invention discloses a consistency verification method and device, and belongs to the technical field of electronic equipment. The consistency checking method comprises the steps that page information of a first page is obtained, and the page information comprises a first image corresponding to the first page and extensible markup language information; generating automatic operation action information according to the first image, the extensible markup language information and the automatic operation demand information; acquiring a second image corresponding to the second page; according to the automatic operation action information, images with the same area position are intercepted from the first image and the second image, and a third image and a fourth image are obtained; and determining the consistency of the first page and the second page according to the third image and the fourth image.

Description

Technical Field

[0001] This application belongs to the technical field of electronic devices, and particularly relates to a consistency verification method and apparatus thereof. Background Art

[0002] With the development of electronic devices and Internet technologies, people rely more on electronic devices to complete various complex operations. Currently, various electronic device manufacturers and electronic device applications continuously introduce various intelligent assistants to perform the function of automatic operation tasks. For example, automatically complete the ordering task of a certain commodity by the user. During the automatic operation process, various abnormal situations may occur in the electronic device, such as slow network speed or slow page rendering, which may lead to a slow page loading speed corresponding to the automatic operation. During the automatic operation process, the electronic device may have jumped to another page. At this time, if the automatic operation action is generated according to the old page, when the electronic device executes this automatic operation action, it may cause an error in the execution of the automatic operation task.

[0003] In order to ensure the accurate execution of automatic operation tasks, in the related art, a fixed waiting duration is set. After executing an automatic operation action, wait for the set waiting duration and then execute the next automatic operation action. However, by setting the waiting duration, the duration of executing the automatic operation task will be long, and the execution efficiency of the automatic operation task is low. Summary of the Invention

[0004] The purpose of the embodiments of this application is to provide a consistency verification method and apparatus thereof, which can solve the problem of low execution efficiency of automatic operation tasks.

[0005] In a first aspect, the embodiments of this application provide a consistency verification method, including:

[0006] Obtain the page information of the first page, where the page information includes the first image corresponding to the first page and Extensible Markup Language (XML) information;

[0007] Generate automatic operation action information according to the first image, XML information, and automatic operation requirement information;

[0008] Obtain the second image corresponding to the second page;

[0009] According to the automatic operation action information, respectively intercept the images with the same regional positions from the first image and the second image to obtain the third image and the fourth image;

[0010] Determine the consistency between the first page and the second page according to the third image and the fourth image.

[0011] Second aspect, an embodiment of the present application provides a consistency verification device, including:

[0012] A first acquisition module, configured to acquire page information of a first page, where the page information includes a first image corresponding to the first page and extensible markup language information;

[0013] A generation module, configured to generate automatic operation action information according to the first image, the extensible markup language information, and automatic operation requirement information;

[0014] A second acquisition module, configured to acquire a second image corresponding to a second page;

[0015] A truncation module, configured to respectively truncate images with the same regional positions from the first image and the second image according to the automatic operation action information, to obtain a third image and a fourth image;

[0016] A verification module, configured to determine the consistency between the first page and the second page according to the third image and the fourth image.

[0017] Third aspect, an embodiment of the present application provides an electronic device, the electronic device includes a processor and a memory, the memory stores a program or instruction that can run on the processor, and when the program or instruction is executed by the processor, the steps of the consistency verification method in the embodiment of the present application are implemented.

[0018] Fourth aspect, an embodiment of the present application provides a readable storage medium, where a program or instruction is stored on the readable storage medium, and when the program or instruction is executed by a processor, the steps of the consistency verification method in the embodiment of the present application are implemented.

[0019] Fifth aspect, an embodiment of the present application provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to run a program or instruction to implement the steps of the consistency verification method in the embodiment of the present application.

[0020] Sixth aspect, an embodiment of the present application provides a computer program product, the program product is stored in a storage medium, and the program product is executed by at least one processor to implement the steps of the consistency verification method in the embodiment of the present application.

[0021] In the embodiment of the present application, by obtaining the page information of the first page, where the page information includes the first image corresponding to the first page and the extensible markup language information; generating the automatic operation action information according to the first image, the extensible markup language information and the automatic operation requirement information; obtaining the second image corresponding to the second page; respectively intercepting the images with the same regional positions from the first image and the second image according to the automatic operation action information to obtain the third image and the fourth image; determining the consistency between the first page and the second page according to the third image and the fourth image. In this way, the automatic operation action information generated according to the first image, the extensible markup language information and the automatic operation requirement information of the first page can be used to intercept the images with the same regional positions from the first image corresponding to the first page and the second image corresponding to the second page, and the consistency between the first page and the second page can be determined by the intercepted images. Furthermore, when the first page and the second page are consistent, the corresponding automatic operation actions can be executed without setting a waiting duration, which can save the duration of the automatic operation task and improve the execution efficiency of the automatic operation task. Description of the Drawings

[0022] Figure 1 is a schematic flowchart of the consistency verification method provided by the embodiment of the present application;

[0023] Figure 2 is a schematic structural diagram of the consistency verification device provided by the embodiment of the present application;

[0024] Figure 3 is a schematic structural diagram of the electronic device provided by the embodiment of the present application;

[0025] Figure 4 is a schematic hardware structure diagram of the electronic device for implementing the embodiment of the present application. Detailed Embodiments

[0026] Next, the technical solutions in the embodiments of the present application will be clearly described in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0027] The terms "first", "second", etc. in the description and claims of this application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such terms can be interchanged under appropriate circumstances so that the embodiments of this application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are usually of the same category, and the number of objects is not limited. For example, the first object can be one or multiple. In addition, "and / or" in the description and claims means at least one of the connected objects, and the character " / ", generally represents an "or" relationship between the associated objects before and after.

[0028] The following will, with reference to the accompanying drawings, through specific embodiments and their application scenarios, elaborate in detail on the consistency check method and device provided by the embodiments of this application.

[0029] Figure 1 It is a flowchart of the consistency check method provided by the embodiments of this application. The consistency check method may include:

[0030] Step 101: Obtain the page information of the first page, where the page information includes the first image corresponding to the first page and extensible markup language information;

[0031] In some possible implementations of the embodiments of this application, in step 101, the first page may be intercepted to obtain the first image corresponding to the first page.

[0032] In some possible implementations of the embodiments of this application, in step 101, an extensible markup language file corresponding to the first page may be obtained, and the extensible markup language file corresponding to the first page may be parsed to obtain the extensible markup language information corresponding to the first page. The embodiments of this application do not elaborate on the process of parsing the extensible markup language file corresponding to the first page to obtain the extensible markup language information corresponding to the first page, and the specific process may refer to the description in related technologies.

[0033] In some possible implementations of the embodiments of this application, the extensible markup language file corresponding to the first page records information such as the element structure and each control attribute included in the first page.

[0034] Step 102: Generate automatic operation action information according to the first image, extensible markup language information, and automatic operation requirement information;

[0035] In some possible implementations of the embodiments of this application, step 102 may include: inputting the first image, extensible markup language information, and automatic operation requirement information into an automatic operation action generation model, and the automatic operation action generation model outputs automatic operation action information, where the automatic operation action generation model is used to generate automatic operation action information.

[0036] In some possible implementations of the embodiments of the present application, an automatic operation action generation model can be trained in advance using a sample image, the extensible markup language information of the page corresponding to the sample image, the automatic operation requirements corresponding to the page, and the operation actions corresponding to the automatic operation requirements.

[0037] After the automatic operation action generation model is trained, the first image, the extensible markup language information, and the automatic operation requirement information are input into the automatic operation action generation model, and the automatic operation action generation model outputs the automatic operation action information.

[0038] Exemplarily, the page currently displayed on the electronic device is a shopping page for a certain commodity, and the automatic operation requirement is to add to the shopping cart. The screenshot image of the shopping page, the extensible markup language information, and the automatic operation requirement "add to the shopping cart" are input into the automatic operation action generation model, and the automatic operation action information output by the automatic operation action generation model is to click on the shopping cart control.

[0039] In some possible implementations of the embodiments of the present application, the automatic operation actions in the embodiments of the present application include, but are not limited to, clicking, inputting, swiping, etc.

[0040] In some possible implementations of the embodiments of the present application, the automatic operation actions in the embodiments of the present application are shown in Table 1:

[0041] Table 1

[0042]

[0043] Exemplarily, it is assumed that the user's self-operation requirement is to search for a commodity named "XX" in a certain shopping application, and the shopping application is Application A.

[0044] Parse the XML file corresponding to the main interface of application A to obtain the search box in the main interface of application A. Use the coordinates of the upper left corner of the search box as the upper left corner coordinates of the input area range, and the coordinates of the lower right corner of the search box as the lower right corner coordinates of the input area range. Use XX as the input text. Use the coordinates of the upper left corner of the search control corresponding to the search box as the upper left corner coordinates of the click area range, and the coordinates of the lower right corner of the search control corresponding to the search box as the lower right corner coordinates of the click area range. The generated automatic operation action information is: input area: [x1, y1, x2, y2]; text: XX; click area: [x3, y3, x4, y4]. Among them, x1 and y1 in the automatic operation action information are the horizontal and vertical components of the upper left corner coordinates of the input area range respectively, x2 and y2 are the horizontal and vertical components of the upper right corner coordinates of the input area range respectively, x3 and y3 are the horizontal and vertical components of the upper left corner coordinates of the click area range respectively, and x4 and y4 are the horizontal and vertical components of the upper right corner coordinates of the click area range respectively.

[0045] Step 103: Obtain a second image corresponding to the second page;

[0046] In some possible implementations of the embodiments of the present application, in step 103, the second page may be intercepted to obtain a second image corresponding to the second page.

[0047] It can be understood that the first page and the second page in the embodiments of the present application are the page displayed by the electronic device before generating the automatic operation action information and the page displayed by the electronic device after generating the automatic operation information respectively.

[0048] Step 104: According to the automatic operation action information, intercept images with the same regional positions from the first image and the second image respectively to obtain a third image and a fourth image;

[0049] In some possible implementations of the embodiments of the present application, the automatic operation action information may include the action range corresponding to the automatic operation action. Step 104 may include: intercepting the action range corresponding to the automatic operation action from the first image to obtain a third image; intercepting the action range corresponding to the automatic operation action from the second image to obtain a fourth image. It can be understood that the sizes of the third image and the fourth image are the same.

[0050] In some possible implementations of the embodiments of the present application, step 104 may include: when it is determined that there is a first area in the first page according to the automatic operation action information, intercept the images of the first area from the first image and the second image respectively to obtain a third image and a fourth image, where the first area is the area that needs to interact with the user.

[0051] In some possible implementations of the embodiments of the present application, the automatic operation action information may include the action type corresponding to the automatic operation action. When determining whether there is an area that needs to interact with the user in the first page according to the automatic operation action information, it may be determined whether the action type corresponding to the automatic operation action is an interaction action type. When the action type corresponding to the automatic operation action is an interaction action type, it is determined that there is an area that needs to interact with the user in the first page. When the action type corresponding to the automatic operation action is not an interaction action type, it is determined that there is no area that needs to interact with the user in the first page. Among them, the interaction action type may be click, input or swipe.

[0052] Step 105: Determine the consistency between the first page and the second page according to the third image and the fourth image.

[0053] In some possible implementations of the embodiments of the present application, step 105 may include: determining the first number of pixel points with the same color in the third image and the fourth image; when the first number of pixel points with the same color in the third image and the fourth image is greater than the first threshold, determining that the first page and the second page are consistent.

[0054] In some possible implementations of the embodiments of the present application, the third image and the fourth image may be respectively converted into RGB matrices, where the size of the RGB matrix is W*H, where W is the matrix width, and its value is the number of pixel points in the horizontal direction of the third image and the fourth image, and H is the matrix height, and its value is the number of pixel points in the vertical direction of the third image and the fourth image. Each element in the RGB matrix includes 3 values, and the 3 values are respectively the red component (R), the green component (G), and the blue component (B) of the pixel point corresponding to the element. Then, the following formula (1) is used to determine whether the pixel point colors are the same.

[0055]

[0056] Among them, in formula (1), p1ij represents the element value of the element in the i-th column and the j-th row of the RGB matrix corresponding to the third image, and p2ij represents the element value of the element in the i-th column and the j-th row of the RGB matrix corresponding to the fourth image.

[0057] Through the above formula (1), the pixel points with the same color can be determined, and then the first number of pixel points with the same color can be counted. When the first number is greater than the first threshold, it is determined that the third image and the fourth image are consistent, and then it is determined that the first page and the second page are consistent. Among them, the first threshold can be set according to actual needs. For example, the first threshold is 95% of the number of elements in the RGB matrix.

[0058] In some possible implementations of the embodiments of the present application, step 105 may include: calculating the mean square error of the RGB matrix corresponding to the third image and the RGB matrix corresponding to the fourth image; and determining that the first page and the second page are consistent when the mean square error of the RGB matrix corresponding to the third image and the RGB matrix corresponding to the fourth image is less than the second threshold.

[0059] It can be understood that the smaller the mean square error between the RGB matrix corresponding to the third image and the RGB matrix corresponding to the fourth image, the more similar the third image and the fourth image are; the larger the mean square error between the RGB matrix corresponding to the third image and the RGB matrix corresponding to the fourth image, the less similar the third image and the fourth image are.

[0060] In some possible implementations of the embodiments of the present application, the mean square error of the RGB matrix corresponding to the third image and the RGB matrix corresponding to the fourth image can be calculated by the following formula (2):

[0061]

[0062] Wherein, in formula (2), MSE is the mean square error of the two RGB matrices, p1ij represents the element value of the element in the i-th column and the j-th row of the RGB matrix corresponding to the third image, p2ij represents the element value of the element in the i-th column and the j-th row of the RGB matrix corresponding to the fourth image, W is the width of the RGB matrix, and H is the height of the RGB matrix.

[0063] After calculating the mean square error of the RGB matrix corresponding to the third image and the RGB matrix corresponding to the fourth image through the above formula (2), compare the mean square error with the second threshold. When the mean square error is less than the second threshold, determine that the third image and the fourth image are consistent, and further determine that the first page and the second page are consistent. Among them, the second threshold can be set according to actual needs.

[0064] In some possible implementations of the embodiments of the present application, step 105 may include: inputting the third image and the fourth image into an image consistency detection model to obtain the consistency of the third image and the fourth image, wherein the image consistency detection model is used to detect the consistency of two images input into the image consistency detection model; and determining the consistency of the first page and the second page according to the consistency of the third image and the fourth image.

[0065] In some possible implementations of the embodiments of the present application, an image consistency detection model for detecting whether two images are consistent can be pre-trained using sample images. The input of the image consistency detection model is two images, and the output of the image consistency detection model is that the two images are consistent or the two images are inconsistent. The embodiments of the present application do not elaborate on the process of training the image consistency detection model, and the specific process can refer to the descriptions in related technologies.

[0066] In some possible implementations of the embodiments of the present application, an image consistency detection model in related technologies can be directly adopted.

[0067] After the third image and the fourth image are input into the image consistency detection model, if the image consistency detection model outputs that the third image and the fourth image are consistent, it is determined that the first page and the second page are consistent; after the third image and the fourth image are input into the image consistency detection model, if the image consistency detection model outputs that the third image and the fourth image are inconsistent, it is determined that the first page and the second page are inconsistent.

[0068] In the embodiments of the present application, based on the page information of the first page, where the page information includes a first image corresponding to the first page and extensible markup language information; according to the first image, the extensible markup language information, and the user's automatic operation requirement information, automatic operation action information is generated; a second image corresponding to the second page is obtained; according to the automatic operation action information, images with the same region positions are intercepted from the first image and the second image respectively to obtain a third image and a fourth image; according to the third image and the fourth image, the consistency between the first page and the second page is determined. In this way, the automatic operation action information generated based on the first image, the extensible markup language information, and the user's automatic operation requirement information can be used to intercept images with the same region positions from the first image corresponding to the first page and the second image corresponding to the second page, and the consistency between the first page and the second page is determined through the intercepted images. Furthermore, when the first page and the second page are consistent, corresponding automatic operation actions can be executed without setting a waiting duration, which can save the duration of the automatic operation task and improve the execution efficiency of the automatic operation task.

[0069] In some possible implementations of the embodiments of the present application, the consistency verification method provided by the embodiments of the present application may further include: when the first page and the second page are consistent, executing the automatic operation actions corresponding to the automatic operation action information.

[0070] Exemplarily, taking the above automatic operation action information "input area: [x1, y1, x2, y2]; text: XX; click area: [x3, y3, x4, y4]" as an example.

[0071] When it is determined that the first page and the second page are consistent, perform the automatic operation actions corresponding to "input area: [x1, y1, x2, y2]; text: XX; click area: [x3, y3, x4, y4]". Among them, the automatic operation actions corresponding to "input area: [x1, y1, x2, y2]; text: XX; click area: [x3, y3, x4, y4]" are to input "XX" within the rectangle with (x1, y1) as the upper left corner coordinate and (x2, y2) as the lower right corner coordinate, and then perform a click operation within the rectangle with (x3, y3) as the upper left corner coordinate and (x4, y4) as the lower right corner coordinate.

[0072] In the consistency verification method provided by the embodiments of the present application, the execution subject may be a consistency verification device. In the embodiments of the present application, taking the consistency verification device as an example to execute the consistency verification method, the consistency verification device provided by the embodiments of the present application is described.

[0073] Figure 2 It is a schematic structural diagram of the consistency verification device provided by the embodiments of the present application. The consistency verification device 200 may include:

[0074] A first acquisition module 201, configured to acquire the page information of the first page, where the page information includes a first image corresponding to the first page and extensible markup language information;

[0075] A generation module 202, configured to generate automatic operation action information according to the first image, the extensible markup language information, and the user's automatic operation requirement information;

[0076] A second acquisition module 203, configured to acquire a second image corresponding to the second page;

[0077] A cropping module 204, configured to crop images with the same regional positions from the first image and the second image respectively according to the automatic operation action information, to obtain a third image and a fourth image;

[0078] A verification module 205, configured to determine the consistency of the first page and the second page according to the third image and the fourth image.

[0079] In the embodiments of the present application, by obtaining the page information of the first page, where the page information includes the first image corresponding to the first page and the extensible markup language information; generating the automatic operation action information according to the first image, the extensible markup language information, and the user's automatic operation requirement information; obtaining the second image corresponding to the second page; respectively intercepting the images with the same regional positions from the first image and the second image according to the automatic operation action information to obtain the third image and the fourth image; determining the consistency between the first page and the second page according to the third image and the fourth image. In this way, the automatic operation action information generated according to the first image, the extensible markup language information, and the user's automatic operation requirement information of the first page can be used to intercept the images with the same regional positions from the first image corresponding to the first page and the second image corresponding to the second page, and the consistency between the first page and the second page can be determined by the intercepted images. Furthermore, when the first page and the second page are consistent, the corresponding automatic operation actions can be executed without setting a waiting duration, which can save the duration of the automatic operation task and improve the execution efficiency of the automatic operation task.

[0080] In some possible implementations of the embodiments of the present application, the generating module 202 is specifically configured to:

[0081] Input the first image, the extensible markup language information, and the automatic operation requirement information into the automatic operation action generation model, and the automatic operation action generation model outputs the automatic operation action information, where the automatic operation action generation model is used to generate the automatic operation action information.

[0082] In some possible implementations of the embodiments of the present application, the verification module 205 is specifically configured to:

[0083] Calculate the mean square error between the RGB matrix corresponding to the third image and the RGB matrix corresponding to the fourth image;

[0084] When the mean square error between the RGB matrix corresponding to the third image and the RGB matrix corresponding to the fourth image is less than the second threshold, determine that the first page and the second page are consistent.

[0085] In some possible implementations of the embodiments of the present application, the verification module 205 is specifically configured to:

[0086] Input the third image and the fourth image into the image consistency detection model to obtain the consistency between the third image and the fourth image, where the image consistency detection model is used to detect the consistency of the two images input into the image consistency detection model;

[0087] Determine the consistency between the first page and the second page according to the consistency between the third image and the fourth image.

[0088] In some possible implementations of the embodiments of the present application, the intercepting module 204 is specifically configured to:

[0089] When it is determined that there is a first area in the first page according to the automatic operation action information, images of the first area are respectively intercepted from the first image and the second image to obtain a third image and a fourth image, where the first area is an area that needs to interact with the user.

[0090] The consistency verification device in the embodiments of the present application may be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device may be a terminal or other devices other than terminals. Exemplarily, the electronic device may be a mobile phone, a tablet computer, a laptop computer, a handheld computer, an in-vehicle electronic device, a Mobile Internet Device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc., and may also be a server, a Network Attached Storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, etc. The embodiments of the present application do not make specific limitations.

[0091] The consistency verification device in the embodiments of the present application may be a device with an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems. The embodiments of the present application do not make specific limitations.

[0092] The consistency verification device provided in the embodiments of the present application can implement Figure 1 each process implemented by the consistency verification method embodiments provided in the embodiments of the present application. To avoid repetition, it will not be elaborated here.

[0093] Optionally, as Figure 3 shown, the embodiments of the present application further provide an electronic device 300, including a processor 301 and a memory 302. A program or instruction that can run on the processor 301 is stored on the memory 302. When the program or instruction is executed by the processor 301, it implements each step of the consistency verification method embodiments provided in the embodiments of the present application and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0094] Figure 4It is a schematic diagram of the hardware structure of the electronic device implementing the embodiments of the present application.

[0095] The electronic device 400 includes, but is not limited to: a radio frequency unit 401, a network module 402, an audio output unit 403, an input unit 404, a sensor 405, a display unit 406, a user input unit 407, an interface unit 408, a memory 409, and a processor 410, etc.

[0096] Those skilled in the art can understand that the electronic device 400 may further include a power supply (such as a battery) for supplying power to each component. The power supply can be logically connected to the processor 410 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. Figure 4 The structure of the electronic device shown does not limit the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0097] Among them, the processor 410 is used to: obtain the page information of the first page currently displayed on the electronic device 400, where the page information includes the first image and extensible markup language information of the first page; generate automatic operation action information according to the first image, extensible markup language information, and user automatic operation requirement information; obtain the second image corresponding to the second page currently displayed on the electronic device 400; respectively intercept images with the same regional position from the first image and the second image according to the automatic operation action information to obtain a third image and a fourth image; determine the consistency between the first page and the second page according to the third image and the fourth image.

[0098] In the embodiments of the present application, by obtaining the page information of the first page currently displayed on the electronic device, where the page information includes the first image and extensible markup language information of the first page; generating automatic operation action information according to the first image, extensible markup language information, and user automatic operation requirement information; obtaining the second image corresponding to the second page currently displayed on the electronic device; respectively intercepting images with the same regional position from the first image and the second image according to the automatic operation action information to obtain a third image and a fourth image; determining the consistency between the first page and the second page according to the third image and the fourth image. In this way, it is possible to intercept images with the same regional position from the first image of the first page and the second image of the second page through the automatic operation action information generated according to the first image, extensible markup language information, and user automatic operation requirement information of the first page, and determine the consistency between the first page and the second page through the intercepted images. Furthermore, when the first page and the second page are consistent, corresponding automatic operation actions can be executed without setting a waiting duration, which can save the duration of the automatic operation task and improve the execution efficiency of the automatic operation task.

[0099] In some possible implementations of the embodiments of the present application, the processor 410 is specifically configured to:

[0100] Input the first image, the extensible markup language information, and the automatic operation requirement information into an automatic operation action generation model, and the automatic operation action generation model outputs automatic operation action information, where the automatic operation action generation model is used to generate the automatic operation action information.

[0101] In some possible implementations of the embodiments of the present application, the processor 410 is specifically configured to:

[0102] Calculate the mean square error of the RGB matrix corresponding to the third image and the RGB matrix corresponding to the fourth image;

[0103] When the mean square error of the RGB matrix corresponding to the third image and the RGB matrix corresponding to the fourth image is less than a second threshold, determine that the first page and the second page are consistent.

[0104] In some possible implementations of the embodiments of the present application, the processor 410 is specifically configured to:

[0105] Input the third image and the fourth image into an image consistency detection model to obtain the consistency of the third image and the fourth image, where the image consistency detection model is used to detect the consistency of two images input into the image consistency detection model;

[0106] Determine the consistency of the first page and the second page according to the consistency of the third image and the fourth image.

[0107] In some possible implementations of the embodiments of the present application, the processor 410 is specifically configured to:

[0108] When it is determined according to the automatic operation action information that there is a need for a first area in the first page, intercept the images of the first area from the first image and the second image respectively to obtain a third image and a fourth image, where the first area is an area that needs to interact with the user.

[0109] It should be understood that in the embodiments of the present application, the input unit 404 may include a Graphics Processing Unit (GPU) 4041 and a microphone 4042. The GPU 4041 processes the image data of static pictures or videos obtained by an image capturing device (such as a camera) in the video capturing mode or the image capturing mode. The display unit 406 may include a display panel 4061, and the display panel 4061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 407 includes at least one of a touch panel 4071 and other input devices 4072. The touch panel 4071 is also referred to as a touch screen. The touch panel 4071 may include two parts: a touch detection device and a touch controller. The other input devices 4072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, power on / off keys, etc.), a trackball, a mouse, and a joystick, which will not be elaborated herein.

[0110] The memory 409 can be used to store software programs and various data. The memory 409 mainly includes a first storage area for storing programs or instructions and a second storage area for storing data. Among them, the first storage area can store an operating system, applications or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 409 may include a volatile memory or a non-volatile memory, or the memory 409 may include both a volatile memory and a non-volatile memory. Among them, the non-volatile memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash memory. The volatile memory may be a Random Access Memory (RAM), a Static RAM (SRAM), a Dynamic RAM (DRAM), a Synchronous DRAM (SDRAM), a Double Data Rate SDRAM (DDR SDRAM), an Enhanced SDRAM (ESDRAM), a Synch link DRAM (SLDRAM), and a Direct Rambus RAM (DRRAM). The memory 409 in the embodiments of the present application includes, but is not limited to, these and any other suitable types of memories.

[0111] The processor 410 may include one or more processing units; optionally, the processor 410 integrates an application processor and a modem processor. Among them, the application processor mainly processes operations related to the operating system, user interface, application programs, etc., and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above-mentioned modem processor may not be integrated into the processor 410 either.

[0112] The embodiment of the present application further provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, it implements each process of the embodiment of the consistency check method provided by the embodiment of the present application, and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0113] Among them, the processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer-readable storage medium. Examples of the computer-readable storage medium include non-transitory computer-readable media, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk, or an optical disc, etc.

[0114] The embodiment of the present application further provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run a program or instruction to implement each process of the embodiment of the consistency check method provided by the embodiment of the present application, and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0115] It should be understood that the chip mentioned in the embodiment of the present application may also be referred to as a system-on-chip, a system chip, a chip system, or a system-on-chip, etc.

[0116] The embodiment of the present application further provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement each process of the embodiment of the consistency check method provided by the embodiment of the present application, and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0117] It should be noted that in this text, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the presence of additional identical elements in the process, method, article or device comprising such element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in a reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0118] From the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present application.

[0119] The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Those of ordinary skill in the art, under the inspiration of the present application and without departing from the spirit of the present application and the scope protected by the claims, can still make many forms, all of which fall within the protection scope of the present application.

Claims

1. A consistency verification method, characterized in that: The method comprises: Acquire page information of a first page, wherein the page information includes a first image and extensible markup language information corresponding to the first page; generating automatic operation action information according to the first image, the extensible markup language information and the automatic operation requirement information; Acquire a second image corresponding to the second page; According to the automatic operation action information, images with the same area position are respectively intercepted from the first image and the second image to obtain a third image and a fourth image; The consistency between the first page and the second page is determined according to the third image and the fourth image.

2. The method according to claim 1, characterized in that The step of generating automatic operation action information according to the first image, the extensible markup language information and the automatic operation requirement information includes: The first image, the extensible markup language information and the automatic operation requirement information are input into an automatic operation action generation model, and the automatic operation action generation model outputs the automatic operation action information, wherein the automatic operation action generation model is used to generate the automatic operation action information.

3. The method according to claim 1, characterized in that The determining, according to the third image and the fourth image, the consistency between the first page and the second page includes: Calculate the mean square error of the RGB matrix corresponding to the third image and the RGB matrix corresponding to the fourth image; When the mean square error is less than a second threshold, it is determined that the first page and the second page are consistent.

4. The method according to claim 1, characterized in that: The determining, according to the third image and the fourth image, the consistency between the first page and the second page includes: Inputting the third image and the fourth image into an image consistency detection model to obtain consistency between the third image and the fourth image, wherein the image consistency detection model is used to detect consistency between two images input into the image consistency detection model; According to the consistency between the third image and the fourth image, the consistency between the first page and the second page is determined.

5. The method according to claim 1, characterized in that The method of capturing images having the same region position from the first image and the second image respectively according to the automatic operation action information to obtain a third image and a fourth image comprises: When it is determined that there is a first area in the first page according to the automatic operation action information, images of the first area are captured from the first image and the second image respectively to obtain a third image and a fourth image, wherein the first area is an area that requires interaction with the user.

6. A consistency checking device, characterized in that: The device comprises: A first acquisition module, configured to acquire page information of a first page, wherein the page information includes a first image and extensible markup language information corresponding to the first page; A generating module, configured to generate automatic operation action information according to the first image, the extensible markup language information and the automatic operation requirement information; A second acquisition module, used to acquire a second image corresponding to the second page; A capture module, configured to capture images having the same region position from the first image and the second image respectively according to the automatic operation action information, to obtain a third image and a fourth image; A verification module is used to determine the consistency between the first page and the second page according to the third image and the fourth image.

7. The device according to claim 6, characterized in that The generation module is specifically used for: The first image, the extensible markup language information and the automatic operation requirement information are input into an automatic operation action generation model, and the automatic operation action generation model outputs the automatic operation action information, wherein the automatic operation action generation model is used to generate the automatic operation action information.

8. The device according to claim 6, characterized in that The verification module is specifically used for: Calculate the mean square error of the RGB matrix corresponding to the third image and the RGB matrix corresponding to the fourth image; When the mean square error is less than a second threshold, it is determined that the first page and the second page are consistent.

9. The device according to claim 6, characterized in that The verification module is specifically used for: Inputting the third image and the fourth image into an image consistency detection model to obtain consistency between the third image and the fourth image, wherein the image consistency detection model is used to detect consistency between two images input into the image consistency detection model; According to the consistency between the third image and the fourth image, the consistency between the first page and the second page is determined.

10. The device according to claim 6, characterized in that The interception module is specifically used for: When it is determined that there is a first area in the first page according to the automatic operation action information, images of the first area are captured from the first image and the second image respectively to obtain a third image and a fourth image, wherein the first area is an area that requires interaction with the user.

Images