Automatic data verification system operation method, device, equipment and readable medium

Abstract

This disclosure provides embodiments of an automatic data verification system operation method, apparatus, device, and readable medium. One specific implementation of the method includes: in response to detecting system operation information corresponding to the automatic data verification system, performing the following operation steps: statically locating the current operation page according to preset system operation configuration information to obtain a detection result; determining the state of the interactive element corresponding to the current operation page based on the detection result; in response to determining that the state of the interactive element meets preset abnormal state conditions, performing a system operation termination operation and a current operation page capture operation; performing failed element detection processing on the current operation page image to obtain failed element detection information; updating the preset system operation configuration information based on the failed element detection information; and executing a system continue operation operation based on the updated system operation configuration information. This implementation can improve the robustness of the automatic data verification system.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of computer technology, and more specifically to an automatic data verification system operation method, apparatus, device, and readable medium. Background Technology

[0002] Automatic data verification systems can automatically operate the data verification system, enabling automated data validation. They have been widely used in finance, healthcare, and other fields. Currently, automatic data verification systems typically operate by using static positioning methods (based on XPath, CSS Selectors, fixed coordinates, AutomationId, etc.) to identify and manipulate page elements. Positioning is achieved by matching specific attributes or hierarchical structures of page elements. Then, interactive operations are executed based on the positioning results to enable the automatic data verification system to run.

[0003] However, when running an automatic data verification system in the above manner, the following technical problems often occur: When the operating system for data verification is upgraded, the UI interface is iterated, or the layout is adjusted, the static attributes change. The static locator is written based on the specific attributes or hierarchical structure of the page elements, which causes the automatic data verification system to be unable to accurately locate the page elements to be interacted with, resulting in the failure of the interactive operation. This leads to a large number of times the automatic data verification system process is interrupted, and eventually, the automatic data verification system crashes.

[0004] The information disclosed in this background section is only intended to enhance the understanding of the background of the present disclosure concept, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0006] Some embodiments of this disclosure provide methods, apparatus, devices, and computer-readable media for operating an automatic data verification system to address the technical problems mentioned in the background section above.

[0007] In a first aspect, some embodiments of this disclosure provide an automatic data verification system operation method, the method comprising: in response to detecting system operation information corresponding to the automatic data verification system, performing the following operation steps: performing static positioning processing of interactive elements on the current operation page according to preset system operation configuration information to obtain detection results; determining the state of the interactive elements corresponding to the current operation page according to the detection results; in response to determining that the state of the interactive elements meets preset state abnormality conditions, performing a system operation suspension operation and a current operation page capture operation to obtain an image of the current operation page; performing a failure element detection processing on the current operation page image to obtain failure element detection information; updating the preset system operation configuration information according to the failure element detection information to obtain system operation configuration update information; and performing a system continue operation operation according to the system operation configuration update information.

[0008] Secondly, some embodiments of this disclosure provide an automatic data verification system operation apparatus. The apparatus includes an execution unit configured to, in response to detecting system operation information corresponding to the automatic data verification system, perform the following operation steps: perform static positioning processing of interactive elements on the current operation page according to preset system operation configuration information to obtain a detection result; determine the state of the interactive element corresponding to the current operation page based on the detection result; in response to determining that the state of the interactive element meets preset state abnormality conditions, perform a system operation termination operation and a current operation page capture operation to obtain an image of the current operation page; perform a failure element detection processing on the current operation page image to obtain failure element detection information; update the preset system operation configuration information according to the failure element detection information to obtain system operation configuration update information; and perform a system continue operation operation according to the system operation configuration update information.

[0009] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.

[0010] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in any implementation of the first aspect.

[0011] The above embodiments of this disclosure have the following beneficial effects: The automatic data verification system operation method of some embodiments of this disclosure can reduce the number of times the automatic data verification system's operation process is interrupted, thus improving the robustness of the automatic data verification system. Specifically, the reason for the large number of times the automatic data verification system's operation process is interrupted and the automatic data verification system crashes is that when the operating system for data verification undergoes version upgrades, UI iterations, or layout adjustments, static attributes change. The static locator relies on specific attributes or hierarchical structures of page elements, causing the automatic data verification system to be unable to accurately locate the page elements to be interacted with, resulting in failed interactive operations, and consequently, a large number of times the automatic data verification system's operation process is interrupted, leading to the automatic data verification system crashing. Based on this, the automatic data verification system operation method of some embodiments of this disclosure firstly, in response to detecting the system operation information corresponding to the automatic data verification system, executes the following operation steps: First, according to preset system operation configuration information, static positioning processing of interactive elements is performed on the current operation page to obtain detection results. Thus, it can be determined whether the page elements to be interacted with can be located statically, thereby determining whether the static attributes of the page elements to be interacted with are abnormal. Then, based on the above detection results, the state of the interactive elements corresponding to the current operation page is determined. Therefore, it can be determined whether the interactive page element can be interacted with normally. Then, in response to the determination that the state of the interactive element meets the preset abnormal state conditions, a system operation stop operation and a current operation page capture operation are performed to obtain the current operation page image. Thus, when an abnormality occurs in the interactive page element, the data verification task can be paused, and the current operation page image can be obtained, which can then be used to re-detect the failed element. Next, the failed element detection processing is performed on the current operation page image to obtain failed element detection information. This allows the determination of the failed element's position and attributes on the operation page, which can then be used to continue the data verification task. Finally, based on the failed element detection information, the preset system operation configuration information is updated to obtain updated system operation configuration information; based on the updated system operation configuration information, the system continues operation. Therefore, the automatic data verification system can achieve self-healing in the event of page element location abnormalities, thereby improving the robustness of the automatic data verification system. Because during the operation of the automatic data verification system, when the static attributes of page elements on the operating page become abnormal, the system can combine image detection algorithms to re-detect the failed page elements to obtain the current more accurate static attributes. This can be used to achieve stable operation of the automatic data verification system, thereby reducing the number of times the automatic data verification system's operation process is interrupted and improving the robustness of the automatic data verification system. Attached Figure Description

[0012] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0013] Figure 1 This is a flowchart of some embodiments of the automatic data verification system operation method according to this disclosure; Figure 2 This is a schematic diagram of the structure of some embodiments of the automatic data verification system operating device according to the present disclosure; Figure 3 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0014] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0015] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0016] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0017] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0018] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0019] Figure 1 A flow 100 of some embodiments of an automatic data verification system operation method according to the present disclosure is shown. The automatic data verification system operation method includes the following steps: Step 101: In response to detecting the system operation information of the corresponding automatic data verification system, the following operation steps are executed: Step 1101: Based on the preset system operation configuration information, perform static positioning processing on the interactive elements of the current operation page to obtain the detection results.

[0020] In some embodiments, the execution entity of the automatic data verification system operation method (e.g., a computing device carrying the automatic data verification system) can perform static positioning processing of interactive elements on the current operation page according to preset system operation configuration information to obtain detection results. The automatic data verification system can be a Robotic Process Automation (RPA) system used to implement data verification at night. For example, the data verification can be reconciliation. The system operation information can indicate that the automatic data verification system has started running. For example, the system operation information can be "start". The preset system operation configuration information can be a pre-set process file configured for the automatic data verification system to perform data verification tasks. The process file can include task configuration information for each task node executed by the automatic data verification system. The task configuration information can include, but is not limited to, task node identifiers and task node attribute information. The task node identifier can be a unique identifier for the corresponding task node. The task node identifier can be represented by numbers or text. For example, the task node can be "download bill". The task node attribute information can represent the attributes of the page elements that need to be interacted with at the corresponding task node. The task node attribute information can include, but is not limited to, preset component identifiers and preset component positions. The aforementioned preset component identifier can be a unique identifier for a pre-defined page element. The aforementioned preset component position can be the pre-defined position of the page element within the page. The aforementioned preset component position can be represented in coordinate form. The aforementioned current operation page can be the page that the automatic data verification system currently requires interactive operation on. In practice, firstly, the aforementioned execution entity can determine the task node attribute information of the corresponding page element to be interacted with in the preset system operation configuration information as the target node attribute information. Then, through a preset static positioning detection algorithm, based on the aforementioned target node attribute information, static positioning detection of interactive elements is performed on the current operation page to obtain the detection result. The page element to be interacted with can be the page element that the current task node requires interactive operation on. The aforementioned preset static positioning detection algorithm can be a pre-defined algorithm used to locate and detect page elements in a webpage. The aforementioned preset static positioning detection algorithm can be at least one of the following: XPath, CSS Selector, and JavaScript Path (JS Path). The aforementioned detection result can indicate whether the page element to be interacted with is found on the current operation page.

[0021] In addressing the technical problems mentioned above, and considering the application scenario where the automatic data verification system performs data verification (e.g., reconciliation) tasks related to the value information (e.g., funds) of virtual items (e.g., stocks), the following technical problem often arises: The automatic data verification system typically uses a fixed system startup time for data verification tasks. However, due to industry fluctuations, server network resource usage varies at different times (e.g., at the end of the month or quarter, the volume of data file downloads is large at night, consuming more network resources), resulting in lower security and robustness during the operation of the automatic data verification system. To address the following requirements for this application scenario: the automatic data verification system needs high security and robustness during operation. Based on joint research and development with universities, the following solution is adopted: Optionally, prior to step 101, the aforementioned execution entity may also perform the following steps: The first step is to determine the estimated startup time information based on the current time meeting the preset system operation time conditions, specifically the preset system startup time information corresponding to the current time in the preset system startup time information set. The preset system operation time conditions can be: the current time being the system operation time; or, the system operation time being a pre-set time indicating when the automatic data verification system can begin operation, such as 6 PM daily. The preset system startup time information in the preset system startup time information set can be a pre-set time indicating the immediate startup of the automatic data verification system. The preset system startup time information can include a preset date and a preset time point. The preset date can be a pre-set date; the preset time point can be a pre-set time point; and each preset system startup time information in the preset system startup time information set corresponds to a different date. For example, if the system's operating time is 6 PM daily, then the preset system startup time information in the preset system startup time information set could be: "{March 1st, 6:00 PM; March 2nd, 6:10 PM; March 3rd, 6:20 PM; March 4th, 6:30 PM; March 5th, 6:40 PM; March 6th, 6:50 PM; March 7th, 7:00 PM; March 8th, 6:00 PM; ... March 31st, 6:20 PM;}". The preset system startup time information corresponding to the current time could be: the preset system startup time information for the date corresponding to the current time.

[0022] The second step is to obtain the historical server network usage information sequence corresponding to the aforementioned automatic data verification system. This historical server network usage information sequence can be a sequence of historical server network usage information arranged in ascending chronological order. The historical server network usage information may include, but is not limited to, historical network bandwidth usage sequences. A historical network bandwidth usage sequence can be: a sequence of historical network bandwidth usages for the same historical date, arranged in ascending order according to the corresponding historical time points. This historical network bandwidth usage can represent the network usage of the system server at the corresponding historical time point on the corresponding date. The system server can be the server of the automatic data verification system. In practice, the executing entity can obtain the historical server network usage information sequence corresponding to the aforementioned automatic data verification system from the database via wired or wireless connection.

[0023] The third step involves inputting the aforementioned current time and historical server network usage information sequence into a pre-trained network usage prediction information generation model to obtain network usage prediction information. This network usage prediction information can be: the network required at the current time; the network required on the current date; a sequence of estimated network bandwidth usage; a sequence arranged in ascending chronological order; a sequence representing the network required by the system server at various points in time on the current date; a sequence of estimated network bandwidth usage; and a time series prediction model that takes the current time and historical server network usage information sequence as input and outputs the network usage prediction information. For example, the time series forecasting model mentioned above can be a SARIMA (Seasonal AutoRegressive Integrated Moving Average) model.

[0024] The fourth step is to obtain the historical system update time series of the corresponding target data verification operating system. The target data verification operating system can be the system required by the automatic data verification system to perform the data verification task. The historical system update time series can be a sequence of historical system update times arranged in ascending order. The historical system update time can be a historical point in time where the target data verification operating system was updated. It should be noted that the historical system update time includes both the date and the specific time. In practice, the executing entity can obtain the historical system update time series of the corresponding target data verification operating system from the database via wired or wireless connection.

[0025] The fifth step involves inputting the aforementioned historical system update time series into the pre-trained system update prediction time generation model to obtain the system update prediction time. This system update prediction time generation model can be a linear function that takes the historical system update time series as input and outputs the system update prediction time. This linear function can be obtained by fitting training samples.

[0026] Step 6: Based on the estimated startup time information, the estimated system update time, and the estimated network usage information mentioned above, determine the system startup time. In practice, firstly, for each preset time point included in the above-mentioned preset system startup time information set, perform the following sub-steps: The first sub-step is to determine the target date as the preset date corresponding to the preset time point in the preset system startup time information set.

[0027] The second sub-step involves determining the time carousel score based on the current time and the target date. In practice, the executing entity may, in response to determining that the current date is the same as the target date, set a preset first date score as the time carousel score. This preset first date score can be a pre-defined indicator of the impact of identical dates on the selection of the preset time point. For example, the preset first date score could be 80. In response to determining that the current date is different from the target date, a preset second date score is determined as the time carousel score. This preset second date score can be a pre-defined indicator of the impact of different dates on the selection of the preset time point. For example, the preset first date score could be 20.

[0028] The third sub-step involves determining the system update score based on the estimated system update time, the target date, and the preset time point. In practice, firstly, the executing entity may, in response to determining that the date corresponding to the estimated system update time differs from the target date, determine the preset no-impact score as the system update score. Then, in response to determining that the date corresponding to the estimated system update time is the same as the target date, the interval between the time point corresponding to the estimated system update time and the preset time point is determined as the system update interval duration. Finally, the product of the system update interval duration and the preset update impact score is determined as the system update score. The preset update impact score can be a pre-defined value representing the impact of each minute interval on the selection of the preset time point. The preset update impact score can be 1.

[0029] The third sub-step involves determining at least one estimated network bandwidth usage corresponding to the aforementioned preset time point, included in the network occupancy estimation information, as the target estimated network bandwidth usage set. Specifically, the estimated network bandwidth usage corresponding to the aforementioned preset time point is defined as follows: the interval between the estimated network bandwidth usage time point and the aforementioned preset time point is less than or equal to a preset interval. The preset interval indicates that the interval has a relatively small impact on network bandwidth usage. For example, the preset interval can be 30 minutes.

[0030] The fourth sub-step involves determining the network bandwidth score based on the aforementioned target estimated network bandwidth usage set. In practice, the executing entity can determine the average value of each target estimated network bandwidth usage in the aforementioned target estimated network bandwidth usage set as the estimated average network bandwidth usage. Then, the product of the aforementioned estimated average network bandwidth usage and the preset unit network impact score is determined as the network bandwidth score. The aforementioned preset unit network impact score can be a pre-defined value representing the impact of network bandwidth usage on the selected preset time point. For example, the aforementioned preset unit network impact score can be -1. It should be noted that since a higher server network bandwidth usage at the corresponding time point makes network lag more likely to occur when completing data verification tasks, the aforementioned preset unit network impact score needs to be a negative value.

[0031] The fifth sub-step involves determining the sum of the aforementioned time-based carousel score, the aforementioned system update score, and the aforementioned network bandwidth score as the time-point score.

[0032] The seventh step is to determine the maximum value among the scores at each determined time point as the system startup time.

[0033] Step 8: In response to determining that the current time is the system startup time, the preset system operation information is determined as the system operation information.

[0034] The above-described technical solution and its related content, as an inventive point of this disclosure, solve the second technical problem mentioned in the background art: "leading to low security and robustness during the operation of the automatic data verification system." Factors leading to low security and robustness during the operation of the automatic data verification system are often as follows: Automatic data verification systems generally use a fixed system startup time when performing data verification tasks. Affected by industry fluctuations, server network resource usage varies at different times (for example, at the end of each month or quarter, the amount of data file downloads is large at night, consuming more network resources), resulting in low security and robustness during the operation of the automatic data verification system. Solving these factors can improve the security and robustness of the automatic data verification system. To achieve this effect, some embodiments of the automatic data verification system operation method of this disclosure firstly, in response to the current time meeting a preset system startup time condition, determine the preset system startup time information corresponding to the current time in the preset system startup time information set as estimated startup time information. The preset system startup time information in the preset system startup time information set includes a preset date and a preset time point. Therefore, different time points can be set for different dates, adopting a dynamic time-based startup approach, which can improve the security of system operation. Secondly, the historical server network usage information sequence corresponding to the aforementioned automatic data verification system is obtained; the current time and the historical server network usage information sequence are input into a pre-trained network usage estimation information generation model to obtain network usage estimation information. This network usage estimation information includes an estimated network bandwidth usage sequence. This allows prediction of server network resource usage at various points in time, which can be used to determine the startup timing of the automatic data verification system. Next, the historical system update time sequence of the corresponding target data verification operating system is obtained; this historical system update time sequence is input into a pre-trained system update estimation time generation model to obtain the system update estimation time. This allows determination of whether the data verification system currently being operated by the automatic data verification system will undergo a system update, which can be used to determine the startup timing of the automatic data verification system. Then, based on the estimated startup time information, the estimated system update time, and the estimated network usage information, the system startup time is determined. Thus, a more accurate startup timing for the automatic data verification system can be obtained by comprehensively considering the impact of system updates and network resources on the system startup time. Finally, in response to determining that the current time is the system startup time, the preset system operation information is determined as the system operation information. Therefore, when the current time reaches the obtained system startup time, system operation information is generated to run the automatic data verification system.Because the startup time of the automatic data verification system is determined by comprehensively considering the impact of system updates and network resources on the operation of the automatic data verification system, a dynamic startup time determination method is adopted, thereby improving the security and robustness of the automatic data verification system.

[0035] Step 1102: Based on the detection results, determine the state of the interactive element corresponding to the current operation page.

[0036] In some embodiments, the execution entity can determine the state of the interactive element corresponding to the current operation page based on the detection results. The interactive element state can indicate whether the interactive page element is abnormal. In practice, the execution entity can determine the preset location anomaly state as the interactive element state corresponding to the current operation page in response to determining that the detection result meets a preset location anomaly condition. The preset location anomaly condition can be a pre-set detection result indicating that the interactive page element was not found on the current operation page. The preset location anomaly state can be a pre-set condition indicating an interaction anomaly of the interactive page element.

[0037] In some optional implementations of certain embodiments, the execution entity can determine the state of the interactive element corresponding to the current operation page based on the detection results in the following ways: The first step involves, in response to the determination that the above detection results meet the preset normal positioning conditions, performing an interactive operation on the page element to be interacted with, based on the preset system operation configuration information. The preset abnormal positioning conditions can be a pre-defined detection result indicating that the page element to be interacted with is found on the current operation page. The interactive operation can be, but is not limited to, at least one of the following: clicking, hovering, and swiping. In practice, firstly, the executing entity can, in response to the determination that the above detection results meet the preset normal positioning conditions, determine the operation type represented by the target node attribute information as the interactive operation type. Then, according to the above interactive operation type, performing the interactive operation on the page element to be interacted with.

[0038] The second step involves, in response to the detection of operation failure information corresponding to the aforementioned interactive operation, determining the preset location anomaly state as the state of the interactive element corresponding to the page element to be interacted with. The aforementioned operation failure information can indicate that the interactive operation has failed. For example, there is no response after the interactive operation.

[0039] Step 1103: In response to determining that the state of the element to be interacted meets the preset abnormal state conditions, execute the system operation stop operation and the current operation page capture operation to obtain the current operation page image.

[0040] In some embodiments, the execution entity may, in response to determining that the state of the interactive element meets a preset abnormal state condition, perform a system shutdown operation and a current operation page capture operation to obtain an image of the current operation page. The preset abnormal state condition may be: the interactive element's state indicates that the interactive element cannot be interacted with on the current operation page. The system shutdown operation may be an operation to pause the operation of the automatic data verification system. For example, the system shutdown operation may be "stop". The current operation page capture operation may be an operation to capture the current operation page. For example, the current operation page capture operation may be "PrtSc".

[0041] Step 1104: Perform failure element detection processing on the current operation page image to obtain failure element detection information.

[0042] In some embodiments, the execution entity may perform failure element detection processing on the current operation page image to obtain failure element detection information.

[0043] In practice, the aforementioned executing entity can perform fault element detection processing on the currently operating page image in various ways to obtain fault element detection information.

[0044] In some optional implementations of certain embodiments, the aforementioned execution entity may perform fault element detection processing on the currently operating page image through the following steps to obtain fault element detection information: The first step is to obtain the preset page template information set corresponding to the current operation page. This preset page template information set includes: a preset page template image and a preset page element information set corresponding to each preset page element. The preset page template information represents the template of the operation page for the pre-defined target data verification operating system. It should be noted that the task node corresponding to each preset page template in the preset page template information set is the same as the task node corresponding to the current operation page. The preset page template image can be a pre-defined image of the corresponding operation page. Each preset page element exists on its corresponding preset page template. Each preset page element corresponds to a preset page element information set. The preset page element can be a pre-defined page element. The preset page element information can include, but is not limited to, page element labels and page element positions. The page element labels identify the page elements. The page element positions represent the location of the corresponding preset page element on the operation page. For example, the page element position can be represented by a coordinate system. In practice, the execution entity can obtain the preset page template information set corresponding to the current operation page from the database via a wired or wireless connection.

[0045] The second step involves performing feature matching processing on each preset page template image included in the aforementioned preset page template information set, and comparing the preset page template image with the currently operating page image to obtain feature similarity. In practice, for each preset page template image included in the aforementioned preset page template information set, the executing entity can perform feature matching processing on the preset page template image and the currently operating page image using a preset feature matching algorithm to obtain feature similarity. The preset feature matching algorithm can be a pre-defined algorithm for matching features between two images. For example, the preset feature matching algorithm can be a global feature matching (Global Descriptors) algorithm.

[0046] The third step is to determine the target feature similarity among the obtained feature similarities that satisfy the preset image matching conditions. The preset image matching conditions can be: the feature similarity is the maximum value among all feature similarities.

[0047] The fourth step is to determine the preset page template information that corresponds to the similarity of the target features in the preset page template information set as the target page template information; The fifth step is to determine the invalid element detection information based on the preset page element information set included in the target page template information. In practice, firstly, the executing entity can determine the preset page element information set included in the target page template information as the target page element information set. Then, the target page element information corresponding to the interactive element in the target page element information set is determined as the invalid element detection information. The target page element information corresponding to the interactive element can be: the included page element tags are the same as the tags corresponding to the interactive element.

[0048] In addressing the technical problems mentioned above, the application scenario—an automatic data verification system performing cross-border data verification (e.g., cross-border reconciliation)—often presents the following technical problem: the pages of the cross-border data verification system are complex, with some page elements densely packed and small, and involve multiple languages, making semantic understanding difficult. Directly detecting the current page through image recognition results in low accuracy in detecting small page elements and low accuracy in text understanding, leading to low accuracy in detecting the information of the interactive elements and causing the automatic data verification system to interrupt operation again. The automatic data verification system also has low self-healing capabilities. Therefore, this application scenario requires the following characteristics: high accuracy in recognizing small page elements and high accuracy in understanding multilingual text. Based on joint research and development with universities, the following solution is adopted: In some optional implementations of certain embodiments, the aforementioned execution entity may perform fault element detection processing on the current operation page image through the following steps to obtain fault element detection information: The first step involves performing page element recognition processing on the current operation page image to obtain a page element information set. The page element information in this set corresponds one-to-one with the page elements within the current operation page. This page element information may include, but is not limited to, page element tags. These tags uniquely identify the corresponding page element. The page element information may also include page element type and page element coordinate information. The page element type may be, but is not limited to, one of the following: button or text box. The page element coordinate information represents the position of the corresponding page element on the current operation page. For example, when the page element type is a text box, the coordinate information may include the vertex coordinates of the four vertices of the corresponding text box.

[0049] In some optional implementations of certain embodiments, the aforementioned execution entity may perform page element recognition processing on the currently operating page image through the following steps to obtain a page element information set: Step one involves performing page element detection processing on the currently operating page image to obtain a set of page element detection information for each current page element. In practice, the executing entity can use a preset page element detection algorithm to perform page element detection processing on the currently operating page image to obtain a set of page element detection information for each current page element. The preset page element detection algorithm can be a pre-defined target detection algorithm aimed at detecting page elements. For example, the preset page element detection algorithm can be a convolutional neural network that takes the currently operating page image as input and the set of page element detection information as output. The page element detection information can include the page element image, the page element detection type, and the page element region coordinate information. The page element detection type can be the type of the detected page element. The page element region coordinate information can represent the region where the detected page element is located. The page element image can represent the page element.

[0050] Step 2: For each page element image included in the above page element detection information set, perform text recognition processing on the page element image to obtain element text information.

[0051] In some optional implementations of certain embodiments, for each page element image included in the above-mentioned page element detection information set, the execution entity may perform the following sub-steps: The first sub-step involves, in response to determining that the size of the page element image is smaller than a preset image detection size, enlarging the page element image to obtain an enlarged image of the page element. The preset image detection size can be a pre-defined size that facilitates normal recognition of the representation image.

[0052] The second sub-step involves extracting text features from the magnified images of the page elements to obtain text feature information. In practice, the executing entity can use a preset feature extraction algorithm to extract text features from the magnified images of the page elements to obtain text feature information. This preset feature extraction algorithm can be local feature descriptors or a deep learning-based text feature extraction algorithm.

[0053] The third sub-step involves classifying the aforementioned text feature information into language types to obtain the text language type. This language type characterizes the language of the text. The text language type may include, but is not limited to, at least one of the following: Chinese, English, Russian, and Arabic. In practice, the executing entity can use a preset language classification algorithm to classify the aforementioned text feature information into language types. This preset language classification algorithm can be a pre-defined classification algorithm for language identification. It may include, but is not limited to, one of the following: Naive Bayes, Support Vector Machine, or a deep learning-based language identification algorithm.

[0054] The fourth sub-step involves performing text recognition processing on the magnified image of the page element based on the aforementioned text language type and text feature information to obtain the element text information. In practice, the executing entity can determine the target text recognition model as a preset text recognition model corresponding to the aforementioned text language type from a preset text recognition model set. Each preset text recognition model in the preset text recognition model set corresponds to one language. The preset text recognition model can be a pre-trained text recognition model used to recognize text in the corresponding language. The aforementioned text recognition model can be a text detection model that takes text feature information as input and text as output.

[0055] Then, the aforementioned text feature information is input into the target text recognition model to obtain text recognition information. It should be noted that the text recognition information represents the text detected by the target text recognition model. Subsequently, in response to determining that the aforementioned text language type is not a preset matching language, a preset translation software interface corresponding to the aforementioned text language type is invoked to translate the aforementioned text recognition information, obtaining element text information. The aforementioned preset matching language can be a pre-defined language that facilitates matching preset system operation configuration information. For example, the aforementioned preset matching language can be English or Chinese. The aforementioned preset translation software interface can be an interface of a pre-defined translation software.

[0056] Step 3: Based on the obtained text information of each element and the aforementioned page element detection information set, generate a page element information set. In practice, firstly, for each page element detection information included in the aforementioned page element detection information set, the executing entity can perform the following sub-steps: The first sub-step is to determine the element text information that corresponds to the above page element detection information among the obtained element text information as the target element text information; The second sub-step involves combining the target element text information and the page element detection information to obtain the page element information.

[0057] Then, the obtained page element information is determined as a page element information set.

[0058] The third step involves matching each page element in the aforementioned page element information set with the obtained identifier of the element to be interacted with, yielding a matching result. In practice, for each page element in the aforementioned page element information set, the executing entity can perform the following sub-steps: The first sub-step involves extracting semantic features from the target element text information included in the above page element information to obtain the first semantic feature vector. The second sub-step involves extracting semantic features from the obtained identifiers of the elements to be interacted with, resulting in a second semantic feature vector. The third sub-step involves determining the similarity between the first semantic feature information and the second semantic feature information as the matching result. This similarity can be cosine similarity.

[0059] The fourth step is to determine the target matching result as the matching result that meets the preset matching conditions among all the matching results. The preset matching conditions can be: the similarity represented by the matching result is the maximum value among all the obtained matching results.

[0060] The fifth step is to determine the page element information that corresponds to the target matching result in the above page element information set as the invalid element detection information.

[0061] The above technical solution and related content, as an inventive point of this disclosure, solve the third technical problem mentioned in the background art: "low self-healing capability of automatic data verification system". Factors leading to low self-healing capability of automatic data verification system are often as follows: the pages of the cross-border data verification system operated by the automatic data verification system are complex, some page elements are densely laid out and small, and multiple languages ​​are involved, making semantic understanding difficult. Directly detecting the current operation page through image recognition results in low detection accuracy of small page elements and low accuracy of text understanding, thus causing low accuracy of the detected information of the interactive elements, leading to the automatic data verification system interrupting operation again, resulting in low self-healing capability. Solving the above factors can improve the self-healing capability of the automatic data verification system. To achieve this effect, some embodiments of the automatic data verification system operation method of this disclosure first perform page element detection processing on the image of the current operation page to obtain a page element detection information set corresponding to each current page element, wherein the page element detection information in the page element detection information set includes page element images. Thus, the page element images of each page element can be obtained, which can then be used to obtain the current static attributes of the interactive elements. Secondly, for each page element image included in the aforementioned page element detection information set, the following steps are performed: In response to determining that the size of the page element image is smaller than a preset image detection size, the page element image is magnified to obtain a magnified page element image; text feature extraction is performed on the magnified page element image to obtain text feature information; language classification is performed on the text feature information to obtain the text language type; based on the text language type and the text feature information, text recognition is performed on the magnified page element image to obtain element text information. Therefore, by prioritizing the magnification of smaller page elements, more accurate positional information of the interactive elements can be obtained, and by identifying the text attributes of page elements through targeted language recognition, more accurate text information of the interactive elements can be obtained. Then, based on the obtained text information of each element and the aforementioned page element detection information set, a page element information set is generated. For each page element information in the aforementioned page element information set, the page element information and the obtained identifier of the element to be interacted with are matched to obtain a matching result. The matching results that meet the preset matching conditions are determined as the target matching results. The page element information in the aforementioned page element information set corresponding to the target matching results is determined as the failed element detection information. Thus, a relatively accurate current static attribute of the element to be interacted with can be obtained, which can then be used to continue running the automatic data verification system.Because when relocating the static attributes of the interactive element by combining the current operation page image, the accuracy of the detected static attributes of the interactive element is improved by processing the details of smaller page elements and identifying text in different languages. This can improve the self-healing capability of the automatic data verification system.

[0062] In some alternative implementations of certain embodiments, the aforementioned execution entity may perform fault element detection processing on the current operation page image through the following steps to obtain fault element detection information: The first step involves inputting the preset failure element detection prompt, the acquired current system operation information, and the image of the current operation page into a pre-trained failure element detection model to obtain the first failure element information. The preset failure element detection prompt can be pre-defined text or speech used to prompt the failure element detection model to detect failure elements. For example, the preset failure element detection prompt could be "Detect the failure location and attributes of the interactive element on the current operation page." The failure element detection model can be a large language model used to detect failure elements. The failure element can be a page element that causes interactive operation errors or location failures. It should be noted that the failure element can be the same as or different from the interactive element. The large language model can be, but is not limited to, one of the following: Doubao-Seed-2.0 large model, Qwen3-30B-A3B. The failure element detection model can be implemented through an interface call. The current system operation information can include, but is not limited to, system resolution, system window status, and system operation logs. The system resolution can be the resolution of the automatic data verification system. The system window status can be the window status of the automatic data verification system. The aforementioned failure element detection information may include, but is not limited to, failure type and the first failure element tag. This information may also include failure element location information. The failure element location information indicates the failure element's position on the page. The failure type indicates whether the failure element exists on the currently accessing page. The failure type can be a failure of a currently accessing element or a failure of a non-currently accessing element.

[0063] As an example, when the data validation system upgrades its interface, it adjusts the language order in the selection text of page components, changing English, which was ranked second, to Arabic. Since the corresponding page position remains unchanged, the current operation page, which should be in English, is incorrectly displayed as an Arabic page. When the interactive element is to fill in text, the different positions of the text filler on the English and Arabic pages cause the interactive text box to malfunction. This type of failure is a failure of a non-current page element. As another example, when the data validation system upgrades its interface, it updates the properties of the "Download Bill" control, changing the click control to a sliding control. Therefore, when interacting according to the pre-set "click" action, the interaction fails, and the detected failure type is a failure of a current page element. The first failure element label can characterize the function of the detected failure element. For example, the first failure element label could be "Download Bill".

[0064] The second step involves performing page element recognition processing on the current operation page image to obtain a page element information set, in response to the determination that the aforementioned failure type meets the preset failure condition for the current page element. The preset failure condition for the current page element can be that the aforementioned failure type is present on the current operation page. Therefore, the failed element can be re-located using a more targeted, computationally less computationally efficient traditional target detection algorithm, compensating for the lower accuracy of large models in target localization and thus improving the accuracy of failed element localization.

[0065] Third, in response to determining that the above failure type meets the preset failure condition for a non-current page element, the associated display device is controlled to display the failure operation page corresponding to the first failure element label. The preset failure condition for a non-current page element can be: the above failure type indicates that it does not exist on the current operation page. The failure operation page can be an operation page where the failure element exists. For example, the failure operation page can be a page for selecting a language system.

[0066] The fourth step is to perform an image capture operation on the aforementioned failed operation page to obtain an image of the failed operation page. This image capture operation can be performed on the currently displayed failed operation page.

[0067] The fifth step involves performing page element recognition processing on the aforementioned failed operation page image to obtain a page element information set. In practice, the executing entity performs secondary failure detection processing on the aforementioned failed operation page image in the same way as the above-mentioned "secondary failure detection processing on the aforementioned current operation page image," and will not be repeated here.

[0068] The sixth step is to determine the page element information corresponding to the first invalid element tag in the above page element information set as the second invalid element information.

[0069] Step 7: Determine the number of each page element information included in the above page element information set as the number of page elements.

[0070] Step 8: Based on the aforementioned number of page elements, the first failed element information, and the second failed element information, generate failed element detection information. In practice, the executing entity can determine the second failed element information as failed element detection information in response to determining that the number of page elements is greater than or equal to a preset page complexity number. The preset page complexity number can be a pre-defined number of page elements representing page complexity, such as 30. Therefore, when the page containing the failed element has a large number of page elements and is complex, the location detection results obtained using traditional object detection algorithms improve the accuracy of the location. In response to determining that the number of page elements is less than the preset page complexity number, determine the first failed element information as failed element detection information. Therefore, when the page containing the failed element has a small number of page elements and is simple, the detection results using a large language model, while maintaining location accuracy, also provide higher semantic accuracy for the obtained failed element detection information.

[0071] Step 1105: Based on the failure element detection information, update the preset system operation configuration information to obtain the system operation configuration update information.

[0072] In some embodiments, the executing entity can update the preset system operation configuration information based on the failure element detection information to obtain system operation configuration update information. In practice, firstly, the executing entity can determine the task node attribute information corresponding to the failure element detection information in the preset system operation configuration information as the target node attribute information. The task node attribute information corresponding to the failure element detection information can be: task node attribute information containing a first failure element tag included in the failure element detection information among the included preset component identifiers. Then, the target node attribute information is replaced with the failure element detection information to obtain the system operation configuration update information.

[0073] Step 1106: Based on the system operation configuration update information, execute the system continue operation operation.

[0074] In some embodiments, the aforementioned execution entity may perform system continue operation operations based on the aforementioned system operation configuration update information. In practice, the aforementioned execution entity may perform system continue operation operations based on the aforementioned system operation configuration update information.

[0075] As an example, if the first failure tag in the failure element detection information is "Download Bill" with the attribute of a sliding control, and the default control for the same function in the preset system operation configuration information has the attribute of a click control, then the attribute of the default control for the same function in the preset system operation configuration information needs to be changed to a sliding control to obtain the aforementioned system operation configuration update information. Then, the aforementioned execution entity can perform a sliding operation on the "Download Bill" sliding control to continue executing the data verification task.

[0076] Optionally, the aforementioned executing entity may also perform the following operational steps: In response to the detection of a success message for the operation corresponding to the system's continued operation, the system's updated operation configuration information is used as the preset system operation configuration information, and the above operation steps continue to be executed. The success message indicates that the interactive operation was successfully completed. For example, the success message could be "success". Therefore, when a success message for the operation corresponding to the system's continued operation is detected, it indicates that the detected failure element detection information is accurate, and the automatic data verification system can achieve a self-healing effect when the static positioning of page elements fails.

[0077] The above embodiments of this disclosure have the following beneficial effects: The automatic data verification system operation method of some embodiments of this disclosure can reduce the number of times the automatic data verification system's operation process is interrupted, thus improving the robustness of the automatic data verification system. Specifically, the reason for the large number of times the automatic data verification system's operation process is interrupted and the automatic data verification system crashes is that when the operating system for data verification undergoes version upgrades, UI iterations, or layout adjustments, static attributes change. The static locator relies on specific attributes or hierarchical structures of page elements, causing the automatic data verification system to be unable to accurately locate the page elements to be interacted with, resulting in failed interactive operations, and consequently, a large number of times the automatic data verification system's operation process is interrupted, leading to the automatic data verification system crashing. Based on this, the automatic data verification system operation method of some embodiments of this disclosure firstly, in response to detecting the system operation information corresponding to the automatic data verification system, executes the following operation steps: First, according to preset system operation configuration information, static positioning processing of interactive elements is performed on the current operation page to obtain detection results. Thus, it can be determined whether the page elements to be interacted with can be located statically, thereby determining whether the static attributes of the page elements to be interacted with are abnormal. Then, based on the above detection results, the state of the interactive elements corresponding to the current operation page is determined. Therefore, it can be determined whether the interactive page element can be interacted with normally. Then, in response to the determination that the state of the interactive element meets the preset abnormal state conditions, a system operation stop operation and a current operation page capture operation are performed to obtain the current operation page image. Thus, when an abnormality occurs in the interactive page element, the data verification task can be paused, and the current operation page image can be obtained, which can then be used to re-detect the failed element. Next, the failed element detection processing is performed on the current operation page image to obtain failed element detection information. This allows the determination of the failed element's position and attributes on the operation page, which can then be used to continue the data verification task. Finally, based on the failed element detection information, the preset system operation configuration information is updated to obtain updated system operation configuration information; based on the updated system operation configuration information, the system continues operation. Therefore, the automatic data verification system can achieve self-healing in the event of page element location abnormalities, thereby improving the robustness of the automatic data verification system. Because during the operation of the automatic data verification system, when the static attributes of page elements on the operating page become abnormal, the system can combine image detection algorithms to re-detect the failed page elements to obtain the current more accurate static attributes. This can be used to achieve stable operation of the automatic data verification system, thereby reducing the number of times the automatic data verification system's operation process is interrupted and improving the robustness of the automatic data verification system.

[0078] Further reference Figure 2As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of an automatic data verification system operating device, which are similar to... Figure 2 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0079] like Figure 2 As shown, the automatic data verification system operation device 200 in some embodiments includes an execution unit 201. The execution unit 201 is configured to, in response to detecting system operation information corresponding to the automatic data verification system, perform the following operation steps: perform static positioning processing of interactive elements on the current operation page according to preset system operation configuration information to obtain detection results; determine the state of the interactive elements corresponding to the current operation page based on the detection results; in response to determining that the state of the interactive elements meets preset abnormal state conditions, perform a system operation suspension operation and a current operation page capture operation to obtain an image of the current operation page; perform failed element detection processing on the current operation page image to obtain failed element detection information; update the preset system operation configuration information according to the failed element detection information to obtain system operation configuration update information; and perform a system continue operation operation according to the system operation configuration update information.

[0080] It is understandable that the various units and references described in the automatic data verification system operating device 200 are... Figure 1 The steps in the described method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method also apply to the device 200 and the units contained therein, and will not be repeated here.

[0081] The following is for reference. Figure 3 This document illustrates a structural schematic of an electronic device 300 suitable for implementing some embodiments of the present disclosure. The electronic devices in some embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0082] like Figure 3As shown, the electronic device 300 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the electronic device 300. The processing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0083] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 An electronic device 300 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 3 Each box shown can represent a device or multiple devices as needed.

[0084] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 309, or installed from storage device 308, or installed from ROM 302. When the computer program is executed by processing device 301, it performs the functions defined in the methods of some embodiments of this disclosure.

[0085] It should be noted that, in some embodiments of this disclosure, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0086] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0087] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: in response to detecting system operation information corresponding to an automatic data verification system, perform the following operational steps: based on preset system operation configuration information, perform static positioning processing of interactive elements on the current operation page to obtain detection results; based on the detection results, determine the state of the interactive elements corresponding to the current operation page; in response to determining that the state of the interactive elements meets preset abnormal state conditions, perform a system operation termination operation and a current operation page capture operation to obtain an image of the current operation page; perform failure element detection processing on the current operation page image to obtain failure element detection information; based on the failure element detection information, update the preset system operation configuration information to obtain system operation configuration update information; and based on the system operation configuration update information, perform a system continue operation operation.

[0088] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0089] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0090] The units described in some embodiments of this disclosure can be implemented in software or in hardware. The described units can also be housed in a processor; for example, a processor may be described as including an execution unit. The names of these units do not necessarily limit the unit itself; for example, an execution unit may also be described as "a unit that performs the following operational steps in response to detecting system operating information corresponding to an automatic data verification system."

[0091] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0092] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A method for operating an automatic data verification system, comprising: Upon detecting system operation information for the corresponding automatic data verification system, the following steps are executed: Based on the preset system operation configuration information, static positioning of interactive elements is performed on the current operation page to obtain the detection results; Based on the detection results, determine the state of the interactive element corresponding to the current operation page; In response to determining that the state of the interactive element meets the preset abnormal state conditions, the system operation is stopped and the current operation page is captured to obtain the image of the current operation page; Perform failure element detection processing on the current operation page image to obtain failure element detection information; Based on the failure element detection information, the preset system operation configuration information is updated to obtain system operation configuration update information; Based on the system operation configuration update information, execute the system continue operation operation.

2. The method according to claim 1, wherein, The operation steps also include: In response to the detection of a successful operation message corresponding to the system continuing to run, the system running configuration update information is used as the preset system running configuration information, and the running steps are continued.

3. The method according to claim 1, wherein, Determining the state of the interactive element corresponding to the current operation page based on the detection result includes: In response to determining that the detection result meets the preset positioning normal conditions, an interactive operation is performed on the page element to be interacted with according to the preset system operation configuration information; In response to the detection of operation failure information corresponding to the interactive operation, the preset positioning abnormal state is determined as the interactive element state of the corresponding interactive page element.

4. The method according to claim 1, wherein, The step of performing fault element detection processing on the current operation page image to obtain fault element detection information includes: Obtain a preset page template information set corresponding to the current operation page, wherein the preset page template information set includes: a preset page template image and a preset page element information set corresponding to each preset page element; For each preset page template image included in the preset page template information set, feature matching processing is performed on the preset page template image and the current operation page image to obtain feature similarity; The feature similarity that satisfies the preset image matching conditions among the obtained feature similarities is determined as the target feature similarity; The preset page template information corresponding to the similarity of the target feature in the preset page template information set is determined as the target page template information; Based on the preset set of page element information included in the target page template information, the failure element detection information is determined.

5. The method according to claim 1, wherein, The step of performing fault element detection processing on the current operation page image to obtain fault element detection information includes: The preset failure element detection prompt information, the acquired current system operation information and the current operation page image are input into the pre-trained failure element detection model to obtain the first failure element information, wherein the first failure element information includes the failure type and the first failure element label. In response to determining that the failure type meets the preset failure condition of the current page element, the current operation page image is subjected to page element recognition processing to obtain a page element information set; In response to determining that the failure type meets the preset failure conditions of non-current page elements, the associated display device is controlled to display the failure operation page corresponding to the first failure element tag; The image of the failed operation page is obtained by performing an image cropping operation on the failed operation page; The page element identification process is performed on the failed operation page image to obtain a page element information set; The page element information corresponding to the first invalid element tag in the page element information set is determined as the second invalid element information; The number of each page element information included in the page element information set is determined as the number of page elements; Based on the number of page elements, the first failed element information, and the second failed element information, failed element detection information is generated.

6. An automatic data verification system operating device, comprising: The execution unit is configured to perform the following steps in response to the detection of system operation information of the corresponding automatic data verification system: Based on the preset system operation configuration information, static positioning of interactive elements is performed on the current operation page to obtain the detection results; Based on the detection results, determine the state of the interactive element corresponding to the current operation page; In response to determining that the state of the interactive element meets the preset abnormal state conditions, the system operation is stopped and the current operation page is captured to obtain the image of the current operation page; Perform failure element detection processing on the current operation page image to obtain failure element detection information; Based on the failure element detection information, the preset system operation configuration information is updated to obtain system operation configuration update information; Based on the system operation configuration update information, execute the system continue operation operation.

7. An electronic device, comprising: One or more processors; Storage device, on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-5.

8. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-5.

Images