Data collection method, device, computer equipment, storage medium and program product

Abstract

The present application discloses a data collection method, apparatus, computer equipment, storage medium and program product, the method comprising: obtaining a control handle in a current interface of a target application; the control handle is at least used to determine a control type; wherein a control whose control type in the current interface belongs to a target type is a target control; the target types include lists, buttons and input boxes; the target controls in the current interface include at least one of list controls, button controls and input box controls; inputting feature data of the target control into a collection strategy generation model to generate a data collection strategy corresponding to the control type of the target control; the feature data includes at least: a control type; and performing real-time monitoring of the target control based on the data collection strategy to capture interactive behavior data of the target control when a change in state is detected.

Description

Technical Field

[0001] The present application relates to the field of Internet of Things technology, and in particular to data acquisition methods, devices, computer equipment, storage media, and program products. Background Art

[0002] With the rapid development of industrial automation, the demand for data collection and analysis of production equipment is increasing.

[0003] Existing technologies mainly rely on the data interface provided by production equipment to collect data through remote communication, or use industrial gateway connections for collection. However, some production equipment does not provide a data interface, or the data interface provides a limited number of collectable data types. This results in some interactive behavior data of controls in industrial scenarios often being unable to be collected. Summary of the Invention

[0004] The purpose of this application is to provide a data collection method, apparatus, computer equipment, storage medium and program product.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a data collection method, comprising:

[0007] Obtaining a control handle in the current interface of the target application; the control handle is used to at least determine the control type; wherein a control in the current interface whose control type is a target type is a target control; the target types include lists, buttons, and input boxes; the target control in the current interface includes at least one of a list control, a button control, and an input box control;

[0008] Inputting the characteristic data of the target control into the acquisition strategy generation model to generate a data acquisition strategy corresponding to the control type of the target control; the characteristic data at least includes: the control type;

[0009] The target control is monitored in real time based on the data collection strategy, so as to capture the interactive behavior data of the target control when a state change is monitored.

[0010] In one embodiment, the control handle of the target control is the target control handle; and the process of acquiring the characteristic data includes:

[0011] Calling a preset API or function to read the text content and storage location of the target control handle;

[0012] Acquire state-related information and control type of the target control according to the target control handle;

[0013] The characteristic data includes a handle value, the text content, the storage location, the control type of the target control, and state-related information.

[0014] In one embodiment, the step of generating a data collection strategy corresponding to the control type of the target control includes:

[0015] Determine, from a preset policy template library, a basic collection rule that matches the control type of the target control;

[0016] Based on the control monitoring configuration corresponding to the target control, the matching basic collection rules are adjusted to obtain a data collection strategy.

[0017] In one embodiment, the step of adjusting the matching basic acquisition rules includes:

[0018] Adjusting a block loading rule for the list control based on a control listening configuration corresponding to the list control, the block loading rule including a scroll trigger condition and a data block size, and collecting currently visible list item data when a scroll event is detected for the list control, and collecting clicked list item data when a click event is detected for the list control;

[0019] Adjusting a monitoring rule for the button control based on a control monitoring configuration corresponding to the button control, the monitoring rule including a baseline monitoring interval and a minimum monitoring interval;

[0020] Based on the control monitoring configuration corresponding to the input box control, a difference comparison rule is adjusted for the input box control, where the difference comparison rule includes a comparison period and a change content extraction rule.

[0021] In one embodiment, after the step of capturing the interactive behavior data of the target control, the method further includes:

[0022] Encoding the interactive behavior data into transmission data in a preset format and transmitting the data to a receiving end via an encryption protocol;

[0023] Based on the feedback result of the receiving end, the data collection strategy corresponding to the target control is optimized.

[0024] In one embodiment, the step of optimizing the data collection strategy corresponding to the target control based on the feedback result of the receiving end includes:

[0025] Calculate the strategy optimization weight corresponding to each target type based on the data integrity score and system load index in the feedback results;

[0026] When the strategy optimization weight corresponding to the list control reaches the list control weight threshold, adjusting the data block size in the block loading rule;

[0027] When the policy optimization weight corresponding to the button control reaches the button control weight threshold, the monitoring mode in the monitoring rule is switched to a hybrid monitoring mode, which includes: an event-driven mode and a polling mode.

[0028] In one embodiment, after the step of optimizing the data collection strategy corresponding to the target control, the method further includes:

[0029] Synchronize the optimized control monitoring configuration to the operating environment of the target application through a hot loading mechanism;

[0030] Verify the compatibility of the optimized control listening configuration with the control handle;

[0031] If the verification is successful, the process re-enters the step of monitoring the target control in real time based on the data collection strategy.

[0032] In one embodiment, the step of encoding the interactive behavior data into transmission data in a preset format includes:

[0033] Encapsulate the interaction behavior data into a JSON structure;

[0034] Perform Base64 encoding on the JSON structure to generate intermediate encoded data;

[0035] Encrypting the intermediate coded data using a symmetric encryption algorithm to generate an encrypted data packet;

[0036] A transmission protocol header is added to the encrypted data packet to generate the transmission data that complies with the preset communication specification.

[0037] In one embodiment, the step of monitoring the target control in real time based on the data collection strategy includes:

[0038] Monitor the target events of the list control through an event listener or a message hook; wherein the target events of the list control include scroll events and selection events;

[0039] Monitoring the target event of the button control according to the monitoring mode in the control monitoring configuration; wherein the monitoring mode in the control monitoring configuration is event-driven mode, polling mode or mixed monitoring mode; the mixed monitoring mode includes: event-driven mode and polling mode; the target event of the button control includes a click event and a long press event;

[0040] Monitoring target events of the input box control in a polling mode; wherein the target events of the input box control include text input events and focus switching events;

[0041] The interactive behavior data includes monitored target events.

[0042] In one embodiment, after the step of capturing the interactive behavior data of the target control, the method further includes:

[0043] Output a data collection report of the target application, wherein the data collection report includes content obtained by at least one of the following methods:

[0044] Counting the operation frequency distribution and timestamp of the interactive behavior data;

[0045] Identifying abnormal operations in the interactive behavior data through a clustering algorithm and generating an alarm event;

[0046] Associating the interaction behavior data with the device operation log based on the timestamp to obtain an association table;

[0047] The analysis results of the interactive behavior data are rendered into a visual chart and exported as a document.

[0048] In one embodiment, after the step of obtaining the control handle in the current interface of the target application, the method further includes:

[0049] If a control fingerprint matching the control handle of the target control is found in a pre-loaded static control fingerprint library, the data collection strategy associated with the matching control fingerprint is determined as the data collection strategy of the target control; the static control fingerprint library includes associated control fingerprints and data collection strategies;

[0050] The step of inputting the characteristic data of the target control into the acquisition strategy generation model is performed after no matching control fingerprint is found in the static control fingerprint library.

[0051] In a second aspect, the present application provides a data acquisition device, comprising:

[0052] An acquisition module, configured to acquire a control handle in the current interface of a target application; the control handle is used to determine at least the control type; a control in the current interface whose control type is a target type is a target control; the target types include lists, buttons, and input boxes; and the target control in the current interface includes at least one of a list control, a button control, and an input box control;

[0053] A processing module, configured to input the characteristic data of the target control into a collection strategy generation model to generate a data collection strategy corresponding to the control type of the target control; the characteristic data at least includes: the control type;

[0054] The acquisition module is used to monitor the target control in real time based on the data acquisition strategy, so as to capture the interactive behavior data of the target control when a state change is monitored.

[0055] In one embodiment, the control handle of the target control is a target control handle; and the acquisition module is further configured to:

[0056] Calling a preset API or function to read the text content and storage location of the target control handle;

[0057] Acquire state-related information and control type of the target control according to the target control handle;

[0058] The characteristic data includes the handle value, the text content, the storage location, the control type of the target control, and state-related information.

[0059] In one embodiment, in the aspect of generating a data collection strategy corresponding to the control type of the target control, the processing module is configured to:

[0060] Determine, from a preset policy template library, a basic collection rule that matches the control type of the target control;

[0061] Based on the control monitoring configuration corresponding to the target control, the matching basic collection rules are adjusted to obtain a data collection strategy.

[0062] In one embodiment, in the aspect of adjusting the matched basic acquisition rules, the processing module is configured to:

[0063] Adjusting a block loading rule for the list control based on a control listening configuration corresponding to the list control, the block loading rule including a scroll trigger condition and a data block size, and collecting currently visible list item data when a scroll event is detected for the list control, and collecting clicked list item data when a click event is detected for the list control;

[0064] Adjusting a monitoring rule for the button control based on a control monitoring configuration corresponding to the button control, the monitoring rule including a baseline monitoring interval and a minimum monitoring interval;

[0065] Based on the control monitoring configuration corresponding to the input box control, a difference comparison rule is adjusted for the input box control, where the difference comparison rule includes a comparison period and a change content extraction rule.

[0066] In one embodiment, the processing module is further configured to:

[0067] Encoding the interactive behavior data into transmission data in a preset format and transmitting the data to a receiving end via an encryption protocol;

[0068] Based on the feedback result of the receiving end, the data collection strategy corresponding to the target control is optimized.

[0069] In one embodiment, in the aspect of optimizing the data collection strategy corresponding to the target control based on the feedback result of the receiving end, the processing module is configured to:

[0070] Calculate the strategy optimization weight corresponding to each target type based on the data integrity score and system load index in the feedback results;

[0071] When the strategy optimization weight corresponding to the list control reaches the list control weight threshold, adjusting the data block size in the block loading rule;

[0072] When the policy optimization weight corresponding to the button control reaches the button control weight threshold, the monitoring mode in the monitoring rule is switched to a hybrid monitoring mode, which includes: an event-driven mode and a polling mode.

[0073] In one embodiment, the processing module is further configured to:

[0074] Synchronizing the optimized data collection strategy to the operating environment of the target application through a hot loading mechanism;

[0075] Verify the compatibility of the optimized control listening configuration with the control handle;

[0076] If the verification is successful, the process re-enters the step of monitoring the target control in real time based on the data collection strategy.

[0077] In one embodiment, in the aspect of encoding the interactive behavior data into transmission data in a preset format, the processing module is configured to:

[0078] Encapsulate the interaction behavior data into a JSON structure;

[0079] Perform Base64 encoding on the JSON structure to generate intermediate encoded data;

[0080] Encrypting the intermediate coded data using a symmetric encryption algorithm to generate an encrypted data packet;

[0081] A transmission protocol header is added to the encrypted data packet to generate the transmission data that complies with the preset communication specification.

[0082] In one embodiment, in the aspect of real-time monitoring of the target control based on the data collection strategy, the collection module is configured to:

[0083] Monitor the target events of the list control through an event listener or a message hook; wherein the target events of the list control include scroll events and selection events;

[0084] Monitoring the target event of the button control according to the monitoring mode in the control monitoring configuration; wherein the monitoring mode in the control monitoring configuration is event-driven mode, polling mode or mixed monitoring mode; the mixed monitoring mode includes: event-driven mode and polling mode; the target event of the button control includes a click event and a long press event;

[0085] Monitoring target events of the input box control in a polling mode; wherein the target events of the input box control include text input events and focus switching events;

[0086] The interactive behavior data includes monitored target events.

[0087] In one embodiment, the acquisition module is further configured to:

[0088] Output a data collection report of the target application, wherein the data collection report includes content obtained by at least one of the following methods:

[0089] Counting the operation frequency distribution and timestamp of the interactive behavior data;

[0090] Identifying abnormal operations in the interactive behavior data through a clustering algorithm and generating an alarm event;

[0091] Associating the interaction behavior data with the device operation log based on the timestamp to obtain an association table;

[0092] The analysis results of the interactive behavior data are rendered into a visual chart and exported as a document.

[0093] In one embodiment, the acquisition module is further configured to:

[0094] If a control fingerprint matching the control handle of the target control is found in a pre-loaded static control fingerprint library, the data collection strategy associated with the matching control fingerprint is determined as the data collection strategy of the target control; the static control fingerprint library includes associated control fingerprints and data collection strategies;

[0095] The step of inputting the characteristic data of the target control into the acquisition strategy generation model is performed after no matching control fingerprint is found in the static control fingerprint library.

[0096] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any one of the above-mentioned data acquisition methods.

[0097] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the above-mentioned data acquisition methods.

[0098] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of any one of the above-mentioned data acquisition methods.

[0099] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0100] This application uses data collection technology based on control handle capture to obtain control handles and generate corresponding data collection strategies for different target controls according to the type of control handles, thereby monitoring the target controls according to the data collection strategies to realize data collection of various interactive data. It does not need to rely on the data interface provided by the production equipment, nor is it restricted by the collectible data type provided by the data interface, thereby improving the applicability of interactive data collection in production equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0101] Figure 1 A flowchart of a data collection method provided in one embodiment of the present application;

[0102] Figure 2 A schematic diagram of a data acquisition method applied to a data acquisition system provided in one embodiment of the present application;

[0103] Figure 3 A schematic diagram of the functional modules of a data acquisition device provided in one embodiment of the present application;

[0104] Figure 4 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0105] Technical term explanation:

[0106] Windows API: Windows API, also known as WinAPI, is the core programming interface of Microsoft's Windows operating system. It is a collection of C language functions that allow developers to interact with the Windows operating system. These APIs cover a large number of functions, including window management, system services, device input and output, file operations, network communication, memory management, etc. Due to their low-level nature, Windows API functions talk directly to the operating system kernel, which means that they run efficiently but may be more complex to use than modern programming frameworks. Over time, Microsoft introduced higher-level programming interfaces and frameworks, such as .NET Framework and Windows Runtime (WinRT), but Windows API remains the cornerstone behind these technologies.

[0107] Base64: Base64 is a method of representing binary data based on 64 printable characters. Because , every 6 bits constitute a unit, corresponding to a printable character. Three bytes have 24 bits, corresponding to four Base64 units, meaning three bytes can be represented by four printable characters. It can be used as a transmission encoding for email. The printable characters in Base64 include the letters A-Z, a-z, and the numbers 0-9, for a total of 62 characters. In addition, the two printable symbols vary between systems. Base64 is often used to represent, transmit, and store binary data in situations where text data is typically processed, including MIME emails and some complex XML data.

[0108] Java: Java is a high-level, object-oriented programming language that draws on the strengths of C++ while eliminating C++'s complex concepts such as multiple inheritance and pointers. This makes it both powerful and easy to use. Java seamlessly implements object-oriented theory, allowing programmers to approach complex programming with a rigorous mindset. Key features of Java include simplicity, object orientation, distribution, robustness, security, platform independence and portability, multithreading, and dynamism.

[0109] Style IDs: A style ID is a constant or flag used to define and specify the appearance and behavior of a window, control, or other object. These IDs are typically a set of predefined values ​​that can be combined to set a specific style or property of an object.

[0110] Monitoring filter (event filter): Event filter is a powerful mechanism that allows an object to monitor and intercept events received by other objects. Through event filters, events can be processed, modified or intercepted before they reach the target object. Its features include: event filters can centrally process events from multiple controls without having to write event processing logic separately in each control (for example, event filters can be used to uniformly monitor click events of multiple buttons without having to rewrite event processing functions for each button separately); perform global event monitoring, for example, it can be used to globally monitor certain types of events, such as keyboard shortcuts or mouse events; through event filters, custom behavior can be added to specific controls while retaining the default event processing logic; it can be used to debug and optimize event processing logic; it can monitor events between multiple components to implement complex interaction logic.

[0111] Event Queue: The event queue is used to store and manage pending events. Its main functions include: 1. Event Storage: When an event occurs, it is first placed in the event queue, awaiting processing; 2. Event Distribution: The event loop (EventLoop) removes events from the event queue and distributes them to the target objects; 3. Event Filtering: Before an event is distributed, event filters have the opportunity to process or intercept the event.

[0112] Bloom filter: In a fingerprint database, a Bloom filter can be used to quickly determine whether a fingerprint is likely to exist in the database. If the Bloom filter returns "possibly in the set," further database queries can be performed; if it returns "not in the set," the fingerprint can be directly excluded.

[0113] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0114] This application claims protection for a data acquisition method, a data acquisition device, a computer device, a storage medium, and a program product.

[0115] The above-mentioned data acquisition device and computer equipment can be used to execute the above-mentioned data acquisition method. The data acquisition device and computer equipment can be collectively referred to as a system, or the equipment / system including the data acquisition device and computer equipment of this application can be referred to as a system.

[0116] like Figure 1 As shown, some embodiments of the present application provide a data collection method, which includes:

[0117] Step 101, obtain the control handle in the current interface of the target application; the above control handle is at least used to determine the control type; wherein, the control whose control type in the current interface belongs to the target type is the target control; the above target type refers to the control category that needs to collect data, exemplarily including lists, buttons and input boxes; these control types may correspond to different interactive behaviors and data collection requirements.

[0118] "Current interface" can refer to the screen or window that the user is currently interacting with. It includes all elements that the user can see and operate, such as buttons, text boxes, menus, icons, etc., or it can refer to the window or view that the program is currently displaying; or, in an operating system, "current interface" can refer to the window or desktop environment that is currently in the foreground.

[0119] Each interface may become the current interface. It should be noted that different interfaces may include only one of the list control, button control, and input box control, or multiple of them. Therefore, the target control in the current interface includes at least one of the list control, button control, and input box control.

[0120] In the embodiment of the present application, a control handle (handle for short) is a unique identifier assigned by the operating system to an interface element (such as a window, button, input box, etc.), which is used to identify and operate the control when the program is running.

[0121] For example, the control handle may be obtained through an application programming interface (API) or a service provided by an operating system, or by using an automated testing tool such as a pywinauto library.

[0122] For example, in Windows, handles are obtained through Win32 APIs (such as FindWindow and FindWindowEx). Specifically, operating system APIs (such as the Windows UI Automation API or the Spy++ tool) can be called to traverse the target application's current interface elements and obtain the handles of all controls within the interface. The system then selects controls of the target type (such as lists, buttons, and input boxes) based on their class name, window name, or other properties. For example, FindWindowEx can be used to recursively search for child window handles, and the control type can be verified using GetClassName. Finally, the system stores the target control's handle in memory for subsequent use.

[0123] Step 102: Input the characteristic data of the target control into a collection strategy generation model to generate a data collection strategy corresponding to the control type of the target control; the characteristic data at least includes: the control type.

[0124] For example, feature data is structured information describing control attributes, including control type, size, location, and hierarchical relationships, and is used to guide the system in generating adaptive data collection strategies. The collection strategy generation model is based on rules or machine learning algorithms and is used to dynamically generate data collection strategies based on control feature data.

[0125] In one example, the system can serialize acquired control feature data into JSON format and input it into a collection strategy generation model. The model first determines the control type using a classifier and then selects the corresponding basic collection rules based on the type. For example, for list controls, a block loading rule based on scroll events is generated; for button controls, a monitoring rule is generated; and for input box controls, a difference comparison rule is used. The data collection strategy output by the model includes parameters such as the monitoring mode and data processing method. The system can then compile these strategies into an executable set of monitoring instructions.

[0126] Specifically, for Linux systems, the control tree in the application can be parsed based on the XQueryTree / XGetWindowProperty interface of the X Window System, and then the D-Bus interface of the GTK+ / Qt framework can be used to capture events corresponding to the controls.

[0127] For macOS systems, you can call the NSAccessibility protocol through Objective-C runtime reflection and use the Core Graphics API to take a screenshot of the control area and match the visual features of the pre-prepared standard control image to obtain changes in the control.

[0128] Furthermore, if the control handle cannot be successfully obtained, the OpenCV template matching algorithm can be used to locate the control position. Specifically, the location of each control is determined by visually matching the preset OpenCV templates corresponding to different controls with the application interface screenshot. The text recognition function of the OCR engine is then used to analyze the screenshot at the corresponding control location to obtain the text content within the control.

[0129] Specifically, the Tesseract OCR engine can be integrated into the control handle capture program to parse the control text content (supporting multi-language recognition).

[0130] In one example, the acquisition strategy generation model may include a connected intelligent optimization layer and an adaptive adjustment layer. The acquisition strategy generation model may be trained in the following manner:

[0131] 1) Data collection and preprocessing: Collect control operation logs from devices on multiple platforms, such as Windows, Android, and iOS. Each log includes but is not limited to: control type, historical state sequence, system parameters, clock deviation, and business tags.

[0132] 2) Data cleaning and standardization: Remove invalid records (e.g., missing key fields, abnormal timestamps). Perform min-max scaling on numerical features (CPU load, network latency) to distribute them in the [0, 1] range. One-hot encode control types (e.g., button → 1, 0, 01, 0, 0, list → 0, 1, 00, 1, 0). Split the time series data into fixed-length windows to form LSTM input samples.

[0133] 3) Intelligent optimization layer training.

[0134] The model architecture includes: input layer: receiving a historical state sequence of length 10 (including features such as control type, state value, system load, etc.); LSTM layer: 2 layers of stacked LSTM units, each with 64 hidden nodes, used to capture long-term dependencies of time series; fully connected layer: mapping the LSTM output into a 3D vector corresponding to the three probability distributions of "high / medium / low"; output layer: Softmax activation function, outputting the probability of state change within the next 3 seconds.

[0135] Model training process: Cross-Entropy Loss can be used as the loss function to measure the difference between the predicted probability and the true label (the manually annotated change state). The Adam optimizer can be used. During training, the training set and validation set are divided into a preset ratio to prevent overfitting. The model is trained using the training set and validated using the validation set.

[0136] 4) Adaptive adjustment layer training.

[0137] The Q-Learning algorithm is used to learn the optimal strategy in an unknown environment and output the predicted action.

[0138] In this embodiment, the state space of the Q-Learning algorithm includes the LSTM prediction results (high / medium / low probability) output by the intelligent optimization layer, real-time system parameters (CPU load, network latency), and business priorities. Its action space can exemplarily include polling mode (checking the control status at fixed intervals), event-driven mode (triggering collection only when the status changes), and hybrid mode (combining polling and event-driven in proportion).

[0139] The reward value is based on data integrity, system load, and energy consumption. The reward function can be seen in the following formula (1):

[0140] reward = (data_completeness * 0.4) + (1 / system_load * 0.3) -(energy_consumption * 0.3) (1)

[0141] Data_completeness represents data integrity, which is calculated by comparing the actual collected data with the actual status. System_load represents system load, which is the inverse of CPU and memory usage and encourages a low-load strategy. Energy_consumption represents energy consumption, which estimates the energy cost of the collection action based on the device power consumption model.

[0142] The training process begins by assigning an initial Q-value (typically 0 or a small random value) to each state-action pair. An ε-greedy strategy (initial ε = 0.9, gradually decaying to 0.1) is then employed to balance random exploration with optimal policy execution. After each action is executed, the current Q-value is updated based on the immediate reward and the maximum Q-value of the next state. Training terminates when the Q-value change falls below a threshold (e.g., 1e-5) or when the accumulated reward stabilizes. Finally, the optimized Q-table is converted into a policy function to select the optimal acquisition action in real time. Online incremental learning is supported: local Q-value updates are triggered when the environment changes (e.g., when a new control type is added).

[0143] After training is complete, the Q-table stores the optimal Q-value for each state-action pair. Based on the Q-value, the optimal action for each state can be determined, thereby obtaining the optimal policy.

[0144] Step 103: monitor the target control in real time based on the data collection strategy, so as to capture the interactive behavior data of the target control when a state change is detected.

[0145] The states of a control usually include but are not limited to: visibility state (visible / hidden / collapsed), enable state (enabled / disabled), focus state (focused / lost focus), selection state (selected / unselected), activation state (activated / inactivated), etc.

[0146] Taking a list control as an example, its states may include one or more of visibility state, enable state, focus state, selection state, detailed state of selected items (single selection / multiple selection / no selected items), scroll state (scroll position / scroll to bottom / scroll to top), etc.

[0147] Taking a button control as an example, its status may include one or more of visibility status, enabled status, focus status, selected status, and button status (normal status / pressed status / hover status / selected status / unselected status).

[0148] Taking the input box control as an example, its status may include one or more of visibility status, enabled status, focus status, selected status, input status (normal input / read-only / password input), data status (data complete / data missing / data validation), selected status (selected content / no selected content), and data binding status (bound / unbound).

[0149] Changes in control state can be caused by a variety of factors, including event triggering, program logic, data binding, timers, system state changes, and window messages.

[0150] In an embodiment of the present application, real-time monitoring is a process in which the system continuously monitors changes in the control state through an event hook, event listener, or polling mechanism, such as using a Windows message hook (WH_CALLWNDPROC) to capture click or input events of a control.

[0151] The aforementioned interaction behavior data is information generated when users interact with controls, including trigger time (which can be represented by a timestamp), trigger method or operation type (click, input), trigger result or operation result (such as input text, list selection), etc.

[0152] Event hooks, event listeners, functions, and so on that perform real-time monitoring can be collectively referred to as monitoring modules. In one example, the system can configure and deploy monitoring modules based on corresponding data collection strategies for different control types: For list controls, register the WM_VSCROLL message hook, call GetScrollInfo to obtain the scroll position when a scroll event is triggered, and use ListView_GetItemText to collect visible item data; for button controls, set the WH_CALLWNDPROC hook to capture the BN_CLICKED message, recording the click time and context information; for input box controls, start a timer to periodically call the GetWindowText or SendMessage function to obtain text content, and use a difference algorithm to filter unchanged content.

[0153] When a state change is detected, the monitoring module encapsulates the interaction data into a predefined format (such as Protocol Buffers), adds a timestamp and session ID, and transmits it to the receiving end via a secure channel. The receiving end can use the interaction data for analysis, and is therefore also called a data analysis module.

[0154] Furthermore, the three functions of control handle identification, data collection, and event monitoring in some embodiments of the present application can be implemented in a plug-in design format at the software design level by defining a standardized plug-in interface (IControlCollectorPlugin). Specifically, the hot-swappable function of plug-in loading can be implemented through the Java SPI mechanism or dynamic link library. This allows the control handle identification, data collection, and event monitoring functions in some embodiments of the present application to be updated in real time during the monitoring process, without having to restart the monitoring process.

[0155] The system can also collect the operation events of the physical buttons on the production equipment through USB HID during the process of collecting the interactive behavior data of the control, and associate the interactive behavior data with the operation events of the physical buttons in the time dimension, so that the user operations when the interactive behavior data are generated can be reflected through the operation events of the physical buttons for later analysis.

[0156] It can be seen that the embodiment of the present application can obtain the control handle through the data acquisition technology based on the control handle capture, and generate corresponding data acquisition strategies for different target controls according to the type of the control handle, so as to monitor the target control according to the data acquisition strategy to realize the data acquisition of various interactive data. It does not need to rely on the data interface provided by the production equipment, nor is it restricted by the type of collectible data provided by the data interface, thereby improving the applicability of interactive data acquisition in production equipment.

[0157] In one embodiment, for convenience, the control handle of the target control may be referred to as the target control handle. The process of acquiring the characteristic data includes:

[0158] Step 201: Call a preset API or function to read the text content and storage location of the target control handle.

[0159] Exemplarily, the preset API or function may refer to a set of standard interface functions provided by the operating system or programming framework for interacting with graphical user interface (GUI) elements. Taking Windows as an example, the Windows platform mainly includes export functions in User32.dll and UIAutomationCore.dll. Text content refers to the readable text information currently displayed by the control, such as the label text on the button, the input text in the input box, the display content of the list item, etc. The storage location refers to the physical address or logical location identifier of the control in the memory, including the window handle, the hierarchical path of the control in the window tree, the screen coordinate position and other information.

[0160] Specifically, the system can call local API functions (such as GetWindowTextW function and GetWindowRect function) through JNI (Java Native Interface) or JNA (Java Native Access) technology to obtain text content and storage location. Among them: for standard Windows controls, use the GetWindowTextW function to read the text content. This function requires the control handle and character buffer to be passed in; use GetWindowRect to obtain the control screen coordinates and calculate the storage location. For complex controls such as ListView (list), you can also call additional macros such as ListView_GetItemText to extract sub-item text. The system converts the obtained text content into UTF-8 encoding and converts the storage location information into a rectangular structure containing left, top, right, and bottom. All data can be stored in a temporary buffer after verification, waiting for the next step of processing.

[0161] For mobile applications, you can use reflection to call internal methods or private fields of AccessibilityNodeInfo to collect data on the mobile terminal.

[0162] Step 202, obtaining the state-related information and control type of the target control according to the target control handle; wherein, in this embodiment, the aforementioned characteristic data includes the handle value, the text content, the storage location, the control type of the target control and state-related information.

[0163] In this embodiment, state-related information may include the current interactive state and display properties of the control, including but not limited to dynamic properties such as enabled / disabled state, visibility, focus state, selected state (indicating whether it is selected), scroll position, value range, etc.

[0164] The control type refers to the functional classification identifier of the control, such as Win32 standard control types such as BUTTON, EDIT, LISTBOX, COMBOBOX, or custom control types defined by third-party UI frameworks.

[0165] The following describes how to obtain status information and control types:

[0166] The system can query the control status by sending messages such as WM_GETTEXT and LB_GETCURSEL through the SendMessage function. Alternatively, the system can use GetWindowLong to obtain the control style identifier to determine whether the control has available styles, and combine it with functions such as IsWindowEnabled (used to determine whether the specified window is enabled) and IsWindowVisible (used to check whether the window is visible) to detect status-related information. It can also confirm whether the control is selected by checking the BM_GETCHECK property value of the button control, confirm the selection status of the detection item through the LB_GETSEL property value of the list box, and obtain the modification flag of the edit box through the EM_GETMODIFY property value (the modification flag is used to confirm the modification of the editing content). Of course, the above-mentioned method of obtaining control status-related information is only an example description, and can be set according to actual needs, which is not limited here.

[0167] The control type can be obtained by using GetClassName to obtain the standard class name, which is then converted to an enumeration type using a predefined mapping table. The system combines this information with the data obtained in step 201 to construct a complete feature dataset, including the handle value (hexadecimal), text content (string), storage location (coordinate structure), control type (enumeration value), and status information (key-value pair set).

[0168] It can be seen that this implementation method obtains the text content, physical location, type identification and real-time status of the control through a standardized interface, and constructs a complete control feature data set. The collected feature data provides an accurate data basis for subsequent automated testing, user behavior analysis, interface monitoring and other applications, enabling the system to determine accurate data collection strategies based on feature data, and then accurately identify changes in the status of interface elements.

[0169] In one embodiment, the step 102 includes:

[0170] Step 1021: Determine a basic collection rule that matches the control type of the target control from a preset policy template library.

[0171] In this embodiment, the policy template library is a collection of predefined policies, stored as a key-value data structure, where the key is a control type enumeration and the value is the corresponding basic collection rule. Each basic collection rule can be considered a template, containing standard fields such as monitoring mode, collection frequency, and data processing method.

[0172] The acquisition frequency field may store an acquisition frequency value or an acquisition interval (ie, the inverse of the acquisition frequency value).

[0173] In an example, if the specific value of the monitoring mode field is the event-driven mode, the corresponding acquisition frequency field may be empty.

[0174] The value stored in the listening mode field can be used to represent event-driven mode, polling mode or mixed listening mode. For example, 1, 2, 3 or other characters can be used to represent event-driven mode, polling mode, mixed listening mode, respectively, or the listening mode field can also directly store the text form of "event-driven mode, polling mode or mixed listening mode".

[0175] A basic collection rule is the smallest policy unit designed for a specific control type. Its standard fields store common collection parameters and default configuration values ​​for that control type. For example, a basic collection rule for a list control includes fixed parameters such as scroll trigger conditions and data block size.

[0176] In one example, the system can retrieve matching items in the policy template library using a hash search algorithm: first, the type identifier of the target control (such as "LIST_VIEW", "BUTTON", etc.) is used as the query key, and the hash value is calculated using the MurmurHash3 algorithm to locate the corresponding control type within O(1) time complexity. The system loads the basic collection rules corresponding to the control type, such as differential comparison rules, monitoring rules, and block loading rules, and also loads additional configuration options applicable to the control type. Additional configuration options can include, for example, the range of loaded list items for list controls, the limit on the amount of data loaded for button controls, and the data collection limit for input controls. All loaded rules (basic collection rules, or basic collection rules + additional configuration options) are stored in the policy buffer, awaiting the next step of optimization and adjustment.

[0177] Step 1022: Based on the control monitoring configuration corresponding to the target control, the matching basic collection rules are adjusted to obtain a data collection strategy (ie, a dynamic adjustment mechanism is used to obtain a final data collection strategy).

[0178] In an embodiment of the present application, the control monitoring configuration may exemplarily include a dynamically generated set of optimization parameters, including collection constraints, current operating parameters and optimization targets for the target control under the current operating environment. The collection constraints may be real-time factors such as the upper limit of CPU occupancy, network delay tolerance, and service priority weights. The optimization target may be, for example, reducing CPU occupancy, shortening network delay, etc. The current operating parameters may exemplarily include the current CPU usage rate. In addition, the control monitoring configuration may exemplarily include statistical data on the interactive behavior of the target control or target type, characteristics of the target control, etc. The data collection strategy is the final generated, immediately executable monitoring collection plan that integrates the core logic of the basic rules and runtime optimization parameters to form a complete control instruction sequence.

[0179] For example, to adjust basic collection rules, the system can create a policy adjustment engine, inputting the basic collection rules and real-time monitoring configuration. The engine first performs a resource assessment, adjusting the collection frequency based on current CPU utilization (obtained via GetSystemTimes) and memory pressure (GlobalMemoryStatusEx). For example, when system load exceeds the CPU usage limit (70%), the polling interval is automatically extended by a set increment, such as 20%. (Of course, for event-driven mode, the collection frequency can be left blank later.) It then applies service priority weights to adjust the monitoring mode. For example, high-priority controls (service weight ≥ weight threshold 0.8) can be event-driven, while low-priority controls remain in polling mode. (Other adjustment methods are possible and will be described later in this article.) Environmental parameters are then injected to set data transmission timeout thresholds based on network latency (ping test results). The system compiles the data collection policy into a binary instruction set and deploys it to the kernel-level monitoring module for execution.

[0180] Those skilled in the art can flexibly design specific values ​​of the CPU usage upper limit, weight threshold, step size, etc., which will not be described in detail here.

[0181] Furthermore, the data collection strategy can also be set by determining the usage scenario of the production equipment. The usage scenario of the production equipment can be set by the user, such as high-frequency data interaction scenario, industrial control scenario, terminal device reduction scenario, etc. For example, for high-frequency data interaction scenarios, a millisecond-level polling monitoring mechanism and a memory write-only method can be used to ensure the accuracy and completeness of the collected interactive behavior data; or when the production equipment belongs to an industrial control scenario with higher safety requirements, the stability of data transmission can be guaranteed by Modbus protocol conversion, and an abnormal fuse mechanism can be introduced to ensure that it will automatically fuse when a data abnormality occurs, avoiding impact on industrial production safety; or for scenarios where the production equipment is a mobile terminal device, considering the limited performance of the mobile terminal device, multi-sensor data fusion can be used: let the data be aggregated and sent together to reduce the mobile terminal transmission load and ensure that the data collection process can be completed smoothly. The embodiment of the present application provides basic collection rules, and configures and adjusts the basic collection rules based on the dynamic adjustment mechanism of the control type and scenario to obtain a data collection strategy, ensuring that the strategy adapts to the real-time operating environment.

[0182] The following describes example operations for adjusting basic collection rules for controls of different control types.

[0183] In one embodiment, for a list control, step 1022 includes:

[0184] Step 10221: Based on the control listening configuration corresponding to the list control, adjust the block loading rules for the list control.

[0185] The block loading rules include scroll trigger conditions and data block size, as well as scroll monitoring operations and click event monitoring operations. The scroll monitoring operation is used to collect the currently visible list item data when a scroll event is detected for the list control, and the click event monitoring operation is used to collect the clicked list item data when a click event is detected for the list control.

[0186] Chunked loading rules can reduce system resource consumption by dividing list data items into multiple small chunks and gradually loading these chunks when the user needs them, rather than loading all data items at once.

[0187] The block loading rule consists of two core parameters: the scroll trigger condition and the data block size. The scroll trigger condition is one of the criteria for triggering data collection for the list control, and includes physical indicators such as the scroll distance threshold and the dwell time threshold. The scroll distance threshold is used to determine whether the list is in a scrolling state. If so, block loading will be performed.

[0188] The data block size is the amount of data contained in each data block. For example, each block contains 10 or 50 list items.

[0189] The dwell time threshold is used to trigger full collection. For example, if the dwell time threshold is 200ms, if the dwell time reaches 200ms, full collection will be triggered to collect data for all currently visible list items.

[0190] It's important to note that scroll events themselves aren't determined by a scroll distance threshold. Scroll events are typically triggered automatically when the user scrolls. The scroll distance threshold can be used to determine the extent or conditions of scrolling to implement specific functionality or interactions, such as triggering certain actions when the scroll distance reaches a threshold, rather than determining whether a scroll event has occurred.

[0191] More specifically, to implement data collection for list controls in scrolling scenarios, the system first analyzes the structural characteristics of the list control, obtains scroll bar parameters through GetScrollInfo, and calculates the ratio of the visible area height to the list item height. Based on current memory usage (obtained through GlobalMemoryStatusEx), the data block size is dynamically set. For example, when available memory exceeds 2GB, the size is set to 50 items; otherwise, it is set to 30 items. The scroll trigger condition is adjusted based on the control's response speed. For example, GetMessageTime is used to measure the scroll event interval. When the average interval is less than 200ms, the scroll trigger threshold is set to 5 pixels; otherwise, it is set to 10 pixels. The system encapsulates these parameters into a SCROLL_PARAMS structure and injects it into the configuration register of the list monitoring thread. The monitoring thread implements intelligent loading based on these parameters: when it detects that the scroll distance exceeds the threshold and the dwell time reaches the dwell time threshold, full data collection is triggered.

[0192] For a button control, step 1022 further includes:

[0193] Step 10222: Based on the control monitoring configuration corresponding to the button control, adjust the monitoring rules for the button control.

[0194] For example, the monitoring rule may include two parameters: a baseline monitoring interval and a minimum monitoring interval.

[0195] In the embodiment of the present application, the monitoring rule belongs to the basic acquisition rule, which can also be considered as the event capture strategy of the button control, defining the frequency and method of the system to check the button state change, including control parameters such as the time interval (which can be stored in the aforementioned acquisition frequency field or other fields) and the mode switching condition (which can be stored in the aforementioned monitoring mode field or other fields). Among them, the time interval includes the baseline monitoring interval and the minimum monitoring interval. The baseline monitoring interval is the default polling time interval for button monitoring, which serves as the benchmark value of the basic sampling frequency. The minimum monitoring interval is the shortest polling period allowed by the system. This parameter ensures that all valid events can still be captured in high-frequency operation scenarios while avoiding excessive consumption of CPU resources. Generally, the baseline monitoring interval is adjusted, and the adjusted baseline monitoring interval is not less than the minimum monitoring interval.

[0196] In this embodiment, the control monitoring configuration corresponding to the button control may include statistical data on the interactive behavior of the button control, such as click frequency distribution, recent click frequency, click interval, etc. In addition, it may also include the aforementioned optimization parameter set.

[0197] The system may include a historical data analysis module that collects statistics on interactive behaviors such as the click frequency distribution of buttons. The recent click frequency can be obtained by performing an exponentially weighted calculation on each point in the click frequency distribution using an exponentially weighted moving average algorithm.

[0198] In one example, when the recent click frequency is higher than the click frequency threshold, step 10222 may be executed. For example, when the recent click frequency is greater than 5 times / second, the dynamic interval adjustment mode (ie, step 10222) is started.

[0199] The initial value of the baseline monitoring interval can be set to 100ms. There are several ways to dynamically adjust the baseline monitoring interval. For example, it can be dynamically adjusted based on CPU usage (obtained through GetSystemTimes):

[0200] When the CPU usage is in the first range (for example, <60%), the current or default baseline monitoring interval is maintained. When the CPU usage is in the second range (for example, 60%-80%), the baseline monitoring interval is extended by 20% based on the current or default baseline monitoring interval. When the CPU usage is in the third range (for example, >80%), the minimum monitoring interval is used.

[0201] Alternatively, the baseline monitoring interval can be dynamically adjusted based on the click interval. For example, if N consecutive clicks (N can be 3 or any other natural number) with an interval less than 50ms are detected, the baseline monitoring interval is automatically set to the minimum monitoring interval and continues monitoring for 300ms. Within this 300ms period, if N consecutive clicks with an interval less than 50ms are still detected, the minimum monitoring interval is maintained; otherwise, the interval is restored to the initial value or the last adjusted value.

[0202] The minimum monitoring interval is determined by hardware performance testing: the CPUID instruction is used to determine the processor model. Intel Core i5 and above processors are set to 10ms, while other processors are set to 20ms. The system writes these parameters to the BUTTON_MONITOR_CONFIG configuration object, which the event monitoring service reads and adjusts in real time.

[0203] For the input box control, the above step 1022 further includes:

[0204] Step 10223: Based on the control monitoring configuration corresponding to the input box control, adjust the difference comparison rules for the input box control. The difference comparison rules include a comparison period and a change content extraction rule.

[0205] In this embodiment, the difference comparison rule is a content change detection strategy for the input box control. By comparing text differences at different time points, only the changed content is collected to optimize data transmission efficiency. The comparison period is the time interval between text difference detection by the system. This parameter affects the real-time performance of content changes and system overhead. The change content extraction rule defines how to identify and record text differences.

[0206] Exemplarily, the control monitoring configuration corresponding to the input box control may include content features of the input box (text length), and interactive behavior statistics (eg, input speed, input frequency).

[0207] Specifically, the system first detects the content characteristics of the input box: it obtains the text length using GetWindowTextLength and then selects a comparison algorithm based on the length: for text <100 characters, an exact LCS-based comparison is used; for text ≥100 characters, a fast hash-based comparison is used. The comparison interval is dynamically adjusted based on input speed: a keyboard hook is used to count input frequency. When the input interval is less than a threshold (e.g., 500ms), the comparison interval is set to the first duration (e.g., 300ms); otherwise, it is set to the second duration (e.g., 500ms), with the first duration being shorter than the second. The change content extraction rule utilizes a three-level process: locating changes within the text using Levenshtein distance; identifying special formats (e.g., phone numbers and email addresses) using regular expressions; and merging consecutive deletions. The system compiles the parameters of the difference comparison rule into DIFF_RULE bytecode, which is interpreted and executed by a dedicated text diffing engine. After each comparison, a DELTA structure is generated, containing fields such as the change location, old value, and new value, which is Base64-encoded for transmission.

[0208] The embodiment of the present application provides basic collection rules and a dynamic adjustment mechanism based on control types and scenarios to configure and adjust the basic collection rules to obtain a data collection strategy, ensuring that the strategy adapts to the real-time operating environment.

[0209] In other embodiments of the present application, after step 103, all the above embodiments further include:

[0210] Step 301: Encode the above-mentioned interactive behavior data into transmission data in a preset format, and transmit it to a receiving end through an encryption protocol.

[0211] In the embodiments of the present application, the transmission data in a preset format refers to a standardized data structure defined by the system. For example, the Protocol Buffers serialization format can be used, and the message body structure containing fixed fields can be included to ensure consistency in data parsing across platforms. This format includes necessary fields such as timestamp, session ID, control path, operation type, and data content. The encryption protocol refers to a transport layer security protocol that complies with the TLS 1.3 standard, using the AES-256-GCM encryption algorithm and the ECDHE key exchange mechanism to provide end-to-end data encryption protection. The protocol configuration includes parameters such as the encryption suite list, certificate verification rules, and session resumption policy.

[0212] The system first encapsulates the interaction behavior data into a PROTOBUF_MSG structure, compresses integer fields using Varint encoding, and applies UTF-8 encoding and Huffman compression to string fields. After data serialization, the system calls the secure transmission module to initialize the TLS session: generates an elliptic curve key pair through BCryptGenerateKeyPair; uses CertOpenSystemStore to obtain the preset CA certificate; and completes the TLS handshake after establishing a TCP connection. After the encrypted channel is established, the system transmits the data in blocks, each with a fixed size of 16KB and an HMAC-SHA256 checksum. The transmission process uses a double buffering mechanism. While data in the foreground buffer is being sent, the background buffer continues to receive new monitoring data to ensure transmission continuity. The system monitors network conditions in real time and automatically switches to the UDP+QUIC protocol when the delay exceeds 500ms.

[0213] Step 302: Based on the feedback result from the receiving end, optimize the data collection strategy (basic collection rule) corresponding to the target control.

[0214] In this embodiment of the present application, the feedback result refers to the data processing status report returned by the receiving end, which can be formatted in JSON or XML. The data processing status report includes performance indicators such as data reception integrity indicators (packet loss rate, out-of-order rate), processing delay statistics, resource usage, and business-level data validity verification results.

[0215] Exemplarily, the data collection strategy may be stored in a shared memory configuration area.

[0216] For example, the system can establish a feedback analysis engine to process receiving-end data: parsing JSON feedback messages to extract key performance indicators; using a sliding window algorithm to calculate the moving average of the packet loss rate of the last 10 received feedback results; and establishing a correlation model between network indicators and collection frequency through regression analysis to determine the correlation coefficient between network latency and collection frequency. The data collection strategy (basic collection rules) can then be optimized based on the correlation coefficient.

[0217] Specifically, there are three ways: 1) modifying the polling interval according to network indicators (for example, extending the polling interval by 5ms for every 1% increase in packet loss rate); 2) reconstructing the event listener based on the data validity verification result (for example, increasing the data collection frequency of the event listener when the proportion of invalid data is greater than the proportion threshold. Those skilled in the art can flexibly design the proportion threshold, such as 10%, 20%, etc.); 3) adjusting the event triggering conditions (for example, the aforementioned rolling triggering conditions. For how to adjust, please refer to the aforementioned records and will not be repeated here).

[0218] The optimized data collection strategy is synchronized to all monitoring threads via a memory-mapped file, using CAS (Compare-And-Swap) operations to ensure strategy updates. The system records strategy changes and rolls back to the previous version if three consecutive optimizations fail to improve performance.

[0219] The embodiments of the present application ensure data security and integrity through efficient encoding and encrypted transmission, and introduce machine learning algorithms into the dynamic optimization process, so that the system can automatically identify the optimal configuration combination, reduce the need for manual intervention, and improve operation and maintenance efficiency.

[0220] The above data validity verification can ensure the accuracy and availability of the data.

[0221] In one embodiment of the present application, data validity verification can be achieved through methods such as data quality inspection and outlier detection.

[0222] Data quality inspection refers to the process of evaluating and verifying data accuracy, completeness, consistency, timeliness, reliability, and availability. Data quality is the foundation of data analysis, data mining, machine learning, and any data-based decision-making.

[0223] Common methods for data quality checks include:

[0224] 1. Data cleaning:

[0225] Handling missing values: Handling missing data by filling (such as mean, median, mode), deletion, or interpolation.

[0226] Correcting Errors: Identify and correct errors or outliers in your data.

[0227] Standardized data format: Unify data formats, such as date format, currency unit, etc.

[0228] 2. Data Validation:

[0229] Range checking: Ensures that data values ​​are within a reasonable range. For example, output should not be a negative number.

[0230] Format check: Verify that the data conforms to the predetermined format.

[0231] Uniqueness check: Ensures that the values ​​of certain fields (such as primary keys) are unique.

[0232] Completeness Check: Make sure all required fields are filled in.

[0233] 3. Data consistency check:

[0234] Cross-table consistency: Check whether the data between different data tables is consistent.

[0235] Time series consistency: Checks whether time series data is continuous and has no duplication.

[0236] 4. Data Deduplication: Check and delete duplicate records to ensure data uniqueness.

[0237] 5. Data integrity check:

[0238] Foreign key integrity: Ensures that the value of the foreign key field exists in the related table.

[0239] Field Completeness: Make sure all required fields are filled.

[0240] 6. Data Quality Monitoring:

[0241] Use data quality monitoring tools (such as Informatica, Talend, etc.) to regularly check data quality;

[0242] Set data quality indicators (such as missing value ratio, error value ratio, etc.) and monitor them.

[0243] Exemplary data quality checking tools may include:

[0244] Excel: Suitable for preliminary inspection of small-scale data, discovering outliers through functions such as sorting, filtering, and conditional formatting.

[0245] SQL: Check the integrity, consistency, and accuracy of data through SQL query statements.

[0246] Python: Use libraries such as Pandas and NumPy for data cleaning and validation.

[0247] R language: Use dplyr, tidyverse and other packages for data processing and quality checking.

[0248] Professional data quality tools: such as Informatica Data Quality and Talend Data Quality, provide powerful data quality checking and repair capabilities.

[0249] Outlier detection can be achieved based on statistical methods (such as standard deviation method) and machine learning methods (such as clustering).

[0250] In another embodiment of the present application, data validity verification at the receiving end may include:

[0251] For the interactive behavior data of the list view (LIST_VIEW) control, adaptive numerical range validation (dynamically adjusting the 0-1000 threshold) and cross-field association validation (checking the logical consistency of the index and content) are applied simultaneously.

[0252] For the interactive behavior data of the button control (BUTTON), considering that the response of the button control has a certain response time, the timing behavior verification is enabled: the interactive behavior data is filtered according to the preset click interval to eliminate invalid and redundant interactive behavior data caused by continuous invalid operations or prevent misoperation. For example, if the response time interval after each press of the button control corresponding to the interactive behavior data is 100ms, the preset click interval can be designed to be 100ms. If the user repeatedly clicks the button control at a frequency of 10ms for up to 100ms, it will cause the interactive behavior data to be repeatedly collected 10 times in 100ms. Then, according to the preset click interval of 100ms, only the interactive behavior data collected when the button control is clicked between 90ms-100ms can be retained, and the interactive behavior data collected when the button control is clicked between 0ms-90ms can be eliminated to avoid the generation of invalid and redundant interactive behavior data.

[0253] Verification results trigger a three-level response mechanism: Single-dimensional anomalies (e.g., values ​​outside a preset range) flag the problematic data and generate a warning log. The data remains in the database but triggers manual review. Double-dimensional violations (e.g., violations of both the numerical range and field association rules) immediately terminate the current data flow, initiate an automatic recollection process, and lock the problematic data source. Complex anomalies (failure of multiple rules, including numerical, association, and time series) trigger an emergency isolation protocol, transferring the anomalous data to a sandbox environment and simultaneously sending an alert notification containing a fault location map to the operations and maintenance terminal.

[0254] Then, by tracing the abnormal links, potential problems such as hardware drift, logic conflicts or interface rendering anomalies are identified. The link identifiers, data types, data indicators and other characteristic data associated with the interactive behavior data corresponding to the potential problems are hot-deployed to the verification engine in the form of binary packages. That is, the characteristics of unknown potential problems are extracted and added to the verification engine, so that the verification engine can quickly identify abnormal problems / abnormal characteristics in future interactive behavior data. The updated strategy supports a hybrid execution mode: event-driven verification (data changes immediately trigger the verification process for interactive behavior data) and periodic polling verification (triggers the verification process for interactive behavior data at periodic time intervals) operate in parallel.

[0255] Ultimately, the system employs a versioned incremental update protocol, with each policy update carrying an incrementing version number. This ensures a smooth transition between old and new rule sets (or old and new data collection policies) with zero service interruption. All verification operations are recorded and encrypted, and a timestamped audit trail is generated during exception handling, enabling policy effectiveness tracking and compliance verification. This closed-loop system, through its self-evolving "collection-verification-analysis-optimization" cycle, reduces the cost of manual intervention while continuously improving data quality, effectively addressing complex and ever-changing data risk scenarios in dynamic business environments.

[0256] Furthermore, when using the optimized data collection strategy to collect interactive behavior data, if a system failure occurs, hierarchical processing can be performed.

[0257] Specifically, fault analysis can be performed based on the frequency of occurrence of the same fault: when the fault frequency is below the fault frequency threshold, it can be determined as a Level 1 error, and the current acquisition channel can be automatically switched to the backup acquisition channel. When the fault frequency is above the fault frequency threshold, it can be confirmed as a Level 2 error, and the system snapshot stored when the data acquisition strategy of the previous stable version was used can be used to restore the current system, allowing the system to resume data acquisition using the data acquisition strategy of the previous stable version. The data acquisition strategy of the previous stable version can be set by the user.

[0258] In one embodiment, the above step 302 includes:

[0259] Step 3021: Calculate the strategy optimization weight corresponding to each target type based on the data integrity score and system load index in the above feedback results.

[0260] In the embodiment of the present application, the data integrity score is an indicator for quantitatively evaluating the quality of data transmission, with a value range of 0-100%. It is calculated based on the verification results at the receiving end and reflects the combined impact of the packet loss rate, disorder rate and verification failure rate.

[0261] The scoring algorithm uses a weighted average. For example, the verification failure rate accounts for 60%, the packet loss rate accounts for 30%, and the disorder rate accounts for 10%.

[0262] The system load metric is a collection of metrics that reflect computer resource usage. It includes four core dimensions: CPU utilization (percentage), memory usage (MB), disk I / O latency (ms), and network bandwidth usage (Mbps). Data for each dimension is collected in real time using operating system performance counters. The system load metric can be derived by normalizing CPU utilization, memory usage, disk I / O latency, and network bandwidth usage to a value between 0 and 1, then summing the values ​​for each core dimension.

[0263] The policy optimization weight is a numerical value calculated for the policy adjustment priority of different control types, ranging from 0 to 1. A larger value indicates that the collection policy of that type of control needs to be adjusted first. The weight calculation takes into account both the importance of business optimization and the system impact.

[0264] The system establishes a multi-dimensional evaluation model to process feedback data to calculate policy optimization weights. The system extracts the data integrity scores of the last five transmissions from the feedback message and calculates a sliding average. The instantaneous value of the current system load indicator is obtained using PerformanceCounter. Each indicator is normalized to a range of 0–1. Weight calculation uses a linear weighting formula: Policy optimization weight = α × (1 - data integrity score) + β × system load indicator, where α and β are type-correlation coefficients (α=0.6, β=0.4 for list controls; α=0.4, β=0.6 for button controls). The calculation process uses SIMD instructions for parallel processing, and the policy optimization weights are updated every 100ms. The system maintains a historical weight queue (containing historical policy optimization weights calculated over a period of time). When a sudden change in weight (a change rate >15%) is detected, an anomaly detection mechanism is triggered. This anomaly detection is performed on the interaction data collected by the control to determine the cause of the anomaly and prompt staff to address it promptly.

[0265] Step 3022: When the strategy optimization weight corresponding to the list control reaches the list control weight threshold, the data block size in the above block loading rule is adjusted.

[0266] In this embodiment, the list control weight threshold is a preset policy adjustment trigger value, illustratively set to 0.75 by default. When the policy optimization weight exceeds this value, the system initiates the list control collection policy optimization process. The threshold can be dynamically adjusted through the management interface based on business needs. For details on data block size, please refer to the previous description and will not be repeated here.

[0267] More specifically, the system monitors the policy optimization weight of the list control in real time. When the weight value exceeds the threshold, in one example, the following adjustments can be made: the correlation curve between the current data block size and the system CPU usage and memory usage (obtained through GlobalMemoryStatusEx) is analyzed to calculate the allowable value of the data block size; considering the network delay (obtained through the ping test), the recommended value of the data block size is determined from the allowable value to complete the adjustment.

[0268] For example, experiments and performance monitoring tools (such as top and iostat) can be used to measure CPU usage at different data block sizes, thereby generating a correlation curve between the two. The corresponding data block size value can then be found in the correlation curve based on the current CPU usage and the upper limit of the CPU usage, thereby determining the data block size range (referred to as the first range).

[0269] At the same time, the value (which may be referred to as the second value) or value range (which may be referred to as the second value range) of the data block size may be determined according to the memory occupancy rate.

[0270] The intersection of the second value / second value range and the first value range can be used as the allowed value for the data block size. If there is no intersection, the range with the smaller upper limit can be used as the allowed value. For example, assuming the second value range is [64, 128] and the first value range is [256, 512], the allowed value can be [64, 128]. For another example, assuming the second value is 64 and the first value range is [256, 512], the allowed value can be 64.

[0271] Exemplarily, the method for determining the data block size according to the memory occupancy rate includes:

[0272] Calculate the current available memory based on the memory usage. The formula is: Available memory = Total memory − Used memory; Used memory = Memory usage * Total memory.

[0273] The size of a data block can be calculated using the following formula:

[0274] Data block size = available memory / number of data blocks.

[0275] For example, suppose the available memory needs to be divided into 5 data blocks, and the size of each data block is:

[0276] The maximum data block size = available memory / 5. The lower limit of the data block size can be 0 or the default setting.

[0277] The recommended value of the data block size can be determined from the allowed values ​​in the following way: a delay threshold is set. When the network delay is greater than the threshold, the minimum value among the allowed values ​​is preferably used as the recommended value; and when it is not greater than the threshold, the maximum value among the allowed values ​​can be used as the recommended value.

[0278] In another example, the data block size adjustment algorithm can use a PID control model: New block number = Current value + Kp × (1-data integrity score) + Ki × Σ (load deviation) + Kd × (weight change rate), where Kp = 8, Ki = 0.5, and Kd = 2 are tuning parameters, and the load deviation is the deviation between the current device system load and the preset standard load. The adjusted value is clamped to upper and lower limits and written to the configuration register, and all monitoring threads are notified to synchronize the new configuration through memory barriers.

[0279] Step 3023: When the policy optimization weight corresponding to the button control reaches the button control weight threshold, the monitoring mode in the monitoring rule is switched to a hybrid monitoring mode, which includes an event-driven mode and a polling mode.

[0280] In an embodiment of the present application, the button control weight threshold is the activation condition value of the hybrid monitoring mode, and is illustratively set to 0.8 by default; further, when the policy optimization weight exceeds the threshold for three consecutive times, the mode switch is triggered. The threshold setting takes into account the performance overhead brought by the mode switch. The hybrid monitoring mode is a composite monitoring strategy that combines event-driven and timed polling, wherein the event-driven mode handles high-frequency operations, and the polling mode ensures the capture of low-frequency operations. The switching conditions of the two modes are dynamically adjusted based on the operating frequency, and the switching delay is controlled within 10ms. High frequency and low frequency can be defined based on the frequency threshold, and those skilled in the art can flexibly design their values, which will not be elaborated here.

[0281] When the button control weight continuously exceeds the threshold, the system switches modes: it saves a snapshot of the current event listener state and initializes the polling thread. For example, the initial interval can be set to (100 - weight value × 80) ms. When operating in hybrid mode: During high-frequency phases (e.g., operation frequency > 5 times / second), the event-driven mode is used primarily, with polling enabled only for missed events. In this case, the baseline listening interval for polling mode is 10 times / second. During low-frequency phases, the polling frequency is automatically reduced to the default baseline listening interval. The system determines the current dominant mode based on three criteria: operation frequency, CPU utilization, and event loss rate. During each switch, the system compares the data capture rate over the previous and next three seconds. If the hybrid mode improves by less than 5%, it reverts to the original dominant mode.

[0282] This embodiment of the application achieves an optimal balance between data collection quality and resource consumption through a quantitative evaluation mechanism, providing highly reliable data collection for intelligent manufacturing scenarios. All optimization operations are logged in detail, supporting post-analysis and algorithm optimization, forming a closed-loop system for continuous improvement.

[0283] In one embodiment, after step 302, the method further includes:

[0284] Step 303: Synchronize the optimized data collection strategy to the operating environment of the target application through a hot loading mechanism.

[0285] In the embodiments of this application, the hot reload mechanism is a technical solution for dynamically updating component configurations during runtime, using memory-mapped files and atomic operations to achieve seamless switching. This mechanism includes three core modules: a configuration parser, a version manager, and a memory synchronization controller. The runtime environment refers to the execution context of the target application and its associated monitoring service, including runtime elements such as process memory space, thread pool status, and loaded dynamic link libraries.

[0286] In one example, the system implements hot reloading through the following process: serializing the optimized data collection policy into binary format and writing it to a temporary memory-mapped file; replacing the data collection policy pointer using the InterlockedCompareExchange atomic operation to ensure thread safety; and notifying the configuration manager of the control monitoring process (exemplarily used to execute step 103) via IPC to reload the data collection policy. In specific implementation, the system creates a double buffer structure: the foreground buffer services the current request, and the backend buffer loads the new data collection policy. During the switch, the control monitoring process is first frozen (for a maximum of 50ms), the configuration references in all thread-local storage (TLS) are updated, and finally, the thread is released to continue execution. Semaphore synchronization is maintained throughout the entire process to ensure the atomicity and visibility of the policy change. Monitoring data generated during hot reloading is marked with a special status bit for subsequent processing.

[0287] Step 304: Verify the compatibility of the optimized data acquisition strategy with the control handle.

[0288] In the embodiments of this application, compatibility verification is a check process to ensure that the new configuration matches the characteristics of the target control. It includes three verification dimensions: handle validity verification, control type matching detection, and operation permission verification. Verification results are divided into three levels: full compatibility, partial compatibility, and incompatibility. The control handle is an interface element identifier assigned by the operating system. During the verification process, it is necessary to check the handle's survival status (IsWindow), access rights (GetSecurityInfo), and associated properties (GetWindowLong).

[0289] In one example, verification can be implemented through multi-layer collaboration: the basic verification layer calls IsWindow to verify handle validity and checks cross-process permissions through GetWindowThreadProcessId; the type verification layer uses GetClassName to match control types and compares the control types corresponding to the old and new configurations; the functional verification layer simulates sending messages such as WM_GETTEXT to test actual operability. The verification process introduces a timeout mechanism (default 300ms) and generates a compatibility score (0-100 points) for each control handle. The system maintains a verification state machine. When the score is ≥80 points, it is judged as passed (corresponding to the aforementioned full compatibility); 60-79 points trigger downgrade adaptation (corresponding to the aforementioned partial compatibility); and scores below 60 trigger a rollback strategy (corresponding to the aforementioned incompatibility). All verification results are recorded in the compatibility matrix as a basis for subsequent optimization.

[0290] Exemplarily, the compatibility matrix is ​​composed of multiple row vectors, each row vector corresponds to a verification, including the verification results of the three verification dimensions of handle validity verification, control type matching detection and operation permission verification in this verification, as well as the final verification result. Each row vector can be expressed as {a1, a2, a3, A}, where a1 represents the result of the handle validity verification, and different characters can be used to indicate whether the handle validity verification passed or failed; a2 represents the result of the control type matching detection, and different characters can be used to indicate a match or mismatch; a3 represents the result of the operation permission verification (actual operability), and different characters can be used to indicate a pass or fail. A is calculated based on a1'-a3'. a1'-a3' is obtained by normalizing a1-a3. Taking a1 as an example, its value can be 1 or 0. A=(ɑa1'+ßa2'+γa3') / 3*100%,where ɑ, ß, and γ represent the importance of each dimension. They can all be equal to 1, or the values ​​of ɑ, ß, and γ can be flexibly set. We will not elaborate on this here.

[0291] Step 305: If the verification is successful, the process re-enters the step of performing real-time monitoring on the target control based on the data collection strategy (ie, the monitoring process is restarted).

[0292] In the embodiments of this application, restarting the monitoring process is a state reset operation for the monitoring service, which includes three standard phases: releasing old resources, initializing a new instance, and restoring data flow. The restart process ensures business continuity, and the loss window is controlled within 3 events. The policy version number is a 64-bit incrementing identifier consisting of a timestamp (high 32 bits) and a sequence number (low 32 bits). The version number also serves as metadata for the data record, supporting policy-based retrospective analysis.

[0293] For example, the monitoring process can be restarted in the following manner: the system sends a pause signal to the monitoring thread and waits for the event being processed to complete (up to 100ms); calls FreeLibrary to unload the old monitoring module and loads the new implementation through LoadLibrary; and uses the new policy parameters to initialize core components such as the monitoring filter and event queue. The version management service generates a new version number: based on GetSystemTimeAsFileTime, it obtains the timestamp and generates a serial number through atomic increment operations. The system writes the version number to the registry hive and log file header, and updates the version index table in memory. After the restart is complete, the monitoring service sends a ready event, and the data processor begins to receive monitoring data in the new format and embeds a version tag in each record.

[0294] The embodiment of the present application implements zero-downtime deployment of policy changes through a hot loading mechanism, shortening the average effective time to less than 200ms. The compatibility verification mechanism ensures the security of policy updates and reduces the monitoring interruption rate due to policy errors to less than 0.1%. Versioning management and atomic operations ensure data consistency and support policy backtracking with millisecond accuracy. This solution enables the system to achieve minute-level policy iteration optimization without affecting business continuity, and improves overall operation and maintenance efficiency by 70%. The adaptive capability of the monitoring service has been significantly enhanced, and it can automatically respond to more than 95% of environmental change scenarios, providing continuous and stable monitoring guarantees for industrial data collection. Detailed audit logs record all operations to meet the security audit requirements of ISO27001.

[0295] In one embodiment, the above step 301 includes:

[0296] Step 3011: Encapsulate the above interaction behavior data into a JSON structure;

[0297] In the embodiments of this application, JSON (JavaScript Object Notation) is a lightweight data exchange format that uses a text format that is completely independent of the programming language to store and transmit data. It consists of key-value pairs and includes two basic structures: objects (denoted by curly braces {}) and arrays (denoted by square brackets []). It supports basic data types such as strings, numbers, and Boolean values.

[0298] The system first parses the interaction behavior data into structured objects, including core fields such as timestamp (ISO 8601 format), control handle (hexadecimal string), operation type (enumeration value), and operation content. The system organizes data using a tree structure: the root node contains metadata (version number, session ID), and child nodes store detailed information categorized by operation type. For complex operations (such as list scrolling), the system treats additional parameters (scroll position, visible item index, etc.) as nested objects. The generated JSON strictly adheres to the RFC 8259 standard, with strings encoded in UTF-8 and values ​​in IEEE 754 double-precision format. The system manages the JSON construction process through a memory pool to avoid frequent memory allocation. After construction is complete, syntax verification (verification via JSON.parse) is performed to ensure the validity of the data structure.

[0299] Step 3012: Base64 encode the JSON structure to generate intermediate encoded data.

[0300] In this embodiment, Base64 encoding is a method for representing binary data based on 64 printable characters. Each 3-byte (24-bit) data is converted into 4 Base64 characters. The encoded character set includes 62 alphanumeric characters (AZ, az, 0-9), as well as the "+" and " / " symbols. "=" is often used as a padding character.

[0301] The system takes a UTF-8-encoded JSON string as input and processes it in 118-byte chunks (meeting the MIME line length limit). Each chunk is converted using a Base64 encoder: 3-byte chunks are grouped, and any chunks less than 3 bytes are padded with zeros. Every 6 bits are converted to the corresponding Base64 character, and the zero-padded bytes are padded with "=". The encoding process is accelerated using SIMD instructions (such as the Intel AVX2 instruction set), improving performance by 40%. The system inserts a CRLF line break after every 76 characters (compliant with RFC 2045), and the final output is in standard Base64 MIME format. After encoding, the system verifies that the output length is a multiple of 4 and performs a decoding loopback test to ensure data integrity.

[0302] Step 3013: Encrypt the intermediate coded data using a symmetric encryption algorithm to generate an encrypted data packet.

[0303] In the embodiment of the present application, a symmetric encryption algorithm refers to a cryptographic algorithm that uses the same key for encryption and decryption. This system adopts the AES-256 (Advanced Encryption Standard) algorithm, uses a 256-bit key, operates in GCM (Galois / Counter Mode) mode, and provides data encryption and integrity verification functions.

[0304] The system generates a 256-bit key using CryptGenRandom; creates a key object using BCryptGenerateSymmetricKey; and generates a 12-byte random IV (initialization vector). The encryption process is performed in three steps: PKCS7 padding is performed on the Base64 data according to the AES block size (16 bytes); BCryptEncrypt is used for GCM encryption, simultaneously generating a 128-bit authentication tag; and the IV, ciphertext, and authentication tag are concatenated in the format [IV(12B)][ciphertext(NB)][tag(16B)]. The system uses hardware acceleration (such as the Intel AES-NI instruction set) to achieve an encryption throughput of 5GB / s. Each data packet carries a key version number, which the receiver uses to obtain the corresponding decryption key from the key management service.

[0305] Step 3014: Add a transmission protocol header to the encrypted data packet to generate the transmission data that complies with the preset communication specification.

[0306] In the embodiment of the present application, the transport protocol header is control information attached to the front of the data packet, including fields such as protocol version (2 bytes), data packet length (4 bytes), timestamp (8 bytes), checksum (4 bytes CRC32), etc., and is encoded in network byte order (big endian).

[0307] The system constructs the protocol header structure: the protocol version field is fixed to 0x0102; the length of the encrypted data packet (including the IV and authentication tag) is calculated; a timestamp accurate to 100ns is obtained using GetSystemTimeAsFileTime; and a CRC32 checksum is calculated for the complete data packet. The protocol header is stored in a memory-aligned (4-byte boundary) structure, and endian conversion is handled using the ntohs / htonl series of functions. The final transmitted data is assembled in the format [protocol header (18B)][encrypted data packet], with a 2-byte END marker (0x0D0A) appended to the end. The system manages data packets using a preallocated send buffer, supporting zero-copy transmission. Each data packet is accompanied by a sequence number for reassembly and packet loss detection at the receiving end.

[0308] The embodiment of the present application ensures data readability through standardized JSON format, Base64 encoding provides binary secure transmission capability, AES-256-GCM encryption ensures data confidentiality, and structured protocol headers achieve reliable transmission. This solution controls the data encapsulation time within 2ms, the transmission efficiency reaches 98% of the theoretical bandwidth upper limit, and the data packet integrity rate reaches 99.999%. The encryption strength meets the FIPS 140-2 Level 3 requirements and supports tens of thousands of encryption operations per second. The design of the protocol header enables the receiving end to quickly verify data integrity and identify and repair transmission errors below 0.01%. The entire mechanism provides high-security and high-reliability transmission guarantees for industrial data acquisition, meeting the security requirements of Level 3 security while ensuring data real-time performance.

[0309] In one embodiment, the step 103 includes:

[0310] Step 1031: monitor target events of the list control through an event listener or a message hook; wherein the target events of the list control include scroll events and selection events.

[0311] In the embodiments of the present application, an event listener is a collection of system-registered callback functions, implemented through Windows message mechanisms (such as WM_NOTIFY) or UI Automation event interfaces (such as UIA_ScrollPattern_ScrollEvent), and used to respond to asynchronous notifications of specific control events. A message hook is a system-level message interception mechanism that installs a WH_CALLWNDPROC-type hook via the SetWindowsHookEx function to capture window messages (such as WM_VSCROLL and LBN_SELCHANGE) sent to the target control. A scroll event is an operation notification triggered when the vertical or horizontal scroll bar position of a list control changes, and includes parameters such as the scroll direction (SB_LINEUP / SB_LINEDOWN) and the current position (the GetScrollInfo return value). A selection event is a state change notification generated when a list item is selected by the user, and includes information such as the selected item index (the wParam of LBN_SELCHANGE) and the selection status (the LB_GETSEL return value).

[0312] The system establishes a two-tiered monitoring system for list controls: ScrollPattern and SelectionPattern event listeners are registered through UI Automation, and event filters (such as UIA_ScrollHorizontallyScrollablePropertyId) are set to limit the monitoring scope. A thread-level message hook is installed using SetWindowsHookEx to capture WM_VSCROLL and LBN_SELCHANGE messages. Event handlers are managed using a priority queue: high-priority UI Automation events (latency < 10ms) are handled, while low-priority message hook events (latency < 50ms) are handled.

[0313] For scroll events, the system records the starting position (GetScrollPos), end position, and duration. For selection events, the currently selected item's text (LB_GETTEXT) and the selection method (mouse / keyboard) are captured. All event data is accompanied by a microsecond-accurate timestamp (obtained using QueryPerformanceCounter) and passed to the data processing thread via a lock-free queue.

[0314] In addition, after capturing the scroll event, the list control can be monitored in real time according to the aforementioned block loading rules to perform block loading, and when the dwell time reaches the dwell time threshold, the full set collection is triggered - all currently visible list item data is collected.

[0315] Step 1032: monitor the target events of the button control according to the monitoring mode in the corresponding data collection strategy; wherein the monitoring mode is specifically an event-driven mode, a polling mode or a mixed monitoring mode; the mixed monitoring mode includes: an event-driven mode and a polling mode; the target events of the button control include click events and long press events.

[0316] In the embodiments of this application, event-driven mode is a passive monitoring method implemented through the Windows message pump (such as WM_COMMAND / BN_CLICKED) or the UI Automation event interface (UIA_InvokePattern_InvokedEvent). It triggers callbacks only when an event occurs, resulting in low CPU usage but potentially missing high-frequency events. Polling mode is an active monitoring method that periodically checks button status (for example, calling GetWindowLong every 50ms to detect style changes). This consumes more resources but ensures a high event capture rate. Hybrid monitoring mode dynamically combines event-driven and polling strategies. Event-driven mode is used by default, and polling is automatically enabled to catch missed events (by measuring the difference between BN_CLICKED and actual clicks). A click event is a standard action triggered by a left-click of a mouse or a keyboard enter, and contains metadata such as the click coordinates (GET_X_LPARAM) and the trigger time. A long-press event is a special action caused by a button being pressed for longer than a threshold (800ms by default). It is calculated by the time difference between WM_LBUTTONDOWN and WM_LBUTTONUP and includes a press duration parameter.

[0317] Specifically, in event-driven mode, you can register a BN_CLICKED message handler and a UIA Invoke event listener. In polling mode, you can start a high-precision timer (timeSetEvent) to periodically check the button status (via BM_GETSTATE). In hybrid monitoring mode, you can use a basic event listener in conjunction with a watchdog timer or timer (default 100ms interval). When the interval between two consecutive events is less than 50ms, polling mode is activated for assistance. Either of the two events can be a click or a long press.

[0318] For long-press event detection, the system records the mouse press duration (WM_LBUTTONDOWN) and starts a long-press detection timer (800ms by default). After the timeout, a long-press event is triggered. If WM_LBUTTONUP is received beforehand, the timer is canceled. All event data is accompanied by a hardware input timestamp (GetMessageTime) and transmitted through a thread-safe queue to ensure that the event sequence is consistent with the actual operation.

[0319] Step 1033: monitor the target events of the input box control in a polling mode; wherein the target events of the input box control include text input events and focus switching events; and the interaction behavior data includes the monitored target events.

[0320] In this embodiment, polling mode is a timed check strategy for input box controls. The detection period is set via SetTimer (default 300ms), and APIs such as GetWindowText and GetFocus are periodically called to obtain control state changes. The polling interval is dynamically adjusted based on CPU load (calculated via GetSystemTimes). When the load exceeds the upper limit of CPU usage (e.g., 70%), the polling interval is automatically extended by a set step size, for example, 20%, 50%, and so on.

[0321] A text input event is a record of changes to the input box's content, including details such as the previous text (obtained through a diff algorithm), the new text, and the change location. A focus switch event is a state transition when an input box gains or loses keyboard focus, including contextual information such as the focus state (WM_SETFOCUS / WM_KILLFOCUS), the switch direction (Tab key / mouse click), and more.

[0322] An exemplary implementation is as follows:

[0323] During initialization, the system retrieves the initial text (GetWindowText) and focus state (GetFocus). A timer is started to periodically (configurable 300-1000ms) to check for changes in text length using GetWindowTextLength and compare text content using memcmp. The system also uses the WH_CALLWNDPROC hook to capture WM_SETFOCUS / WM_KILLFOCUS messages (WM_SETFOCUS and WM_KILLFOCUS are Windows messages used to notify windows when they gain or lose input focus. When a window gains input focus, the system sends a WM_SETFOCUS message to it; when a window loses input focus, the system sends a WM_KILLFOCUS message to it). Both WM_SETFOCUS and WM_KILLFOCUS are WM_ messages.

[0324] Among them, text change detection adopts a three-level optimization: when the lengths are different, it is directly judged as a change; when the lengths are equal, the hash values ​​are compared (MurmurHash3); when the hashes are inconsistent, a character-by-character comparison is performed.

[0325] The focus switching event is accompanied by the type of operation that caused the switch (detected by the Tab key / Shift-Tab key combination through GetAsyncKeyState). All focus switching event data records the complete window message sequence (the last 5 WM_ messages) for analyzing the operation context.

[0326] The embodiments of the present application achieve optimal resource utilization through differentiated monitoring strategies: the list control adopts high-precision event monitoring to improve the capture rate; the button control supports dynamic mode switching, which keeps the CPU usage to a minimum while ensuring the event capture rate; the input box polling mechanism reduces the amount of data transmission through intelligent optimization. All event data is accompanied by nanosecond timestamps and complete operation context, providing a high-fidelity data source for behavioral analysis. The solution supports processing a large number of control events per second, with reduced end-to-end latency and low memory usage. The dynamic switching mechanism of multi-mode monitoring enables the system to adapt to different load scenarios, maintain operational stability in the complex environment of industrial sites, and provide reliable data support for equipment monitoring and operation analysis.

[0327] In one embodiment, after step 103, the method further includes: outputting a data collection report of the target application, wherein the data collection report includes content obtained by at least one of the following methods:

[0328] Collect statistics on the operation frequency distribution and timestamps of the above interactive behavior data.

[0329] Clustering algorithms are used to identify abnormal operations in the above interactive behavior data and generate alarm events.

[0330] The above interaction behavior data is associated with the device operation log based on the timestamp to obtain an association table.

[0331] Render the analysis results of the above interactive behavior data into visual charts (such as heat maps, line charts, scatter plots, etc.) and export them as documents.

[0332] In the embodiments of this application, the operation frequency distribution is a statistical result of the occurrence frequency of various interactive behaviors in the target application over time, including a histogram of the number of operations distributed by minute / hour / day, as well as derived indicators such as peak frequency and average frequency. The timestamp is a record of the time when an event occurred, accurate to the millisecond level. For example, it can be stored in the ISO 8601 extended format (YYYY-MM-DDThh:mm:ss.sssZ) and uniformly converted to the UTC time zone to eliminate the impact of time zone differences.

[0333] After receiving the interaction data, the receiving end can clean the data. Data cleaning methods include but are not limited to deduplication, error correction, and missing value processing. Deduplication can be performed by removing duplicate data based on specific key fields (such as user ID and order number), retaining the latest or most complete records, and using similarity algorithms (such as Levenshtein distance and Jaccard similarity) to process records with similar but not identical spellings. Error correction can be performed by detecting outliers (such as negative age and incorrect date format) based on business rules and correcting text errors using natural language processing (NLP) tools (such as TextBlob and SymSpell). Missing value processing can be performed by directly deleting records or fields with high missing rates (applicable to cases with fewer missing values).

[0334] The interaction data can also be normalized by using Min-Max standardization or Z-score standardization. Format unification can be achieved by unifying the date and time format, text size, encoding type, etc. in the data.

[0335] The interactive behavior data of the same control type or the same target control in the same time period can be clustered using a clustering algorithm to obtain multiple clusters, each cluster including the interactive behavior data corresponding to at least one interactive behavior.

[0336] For example, the above clustering algorithm adopts the improved DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm with parameters set to Eps=0.5 (normalized distance) and MinPts=10, which supports processing density anomalies in high-dimensional time series data.

[0337] The steps of the DBSCAN algorithm include:

[0338] Initialization: mark all points as unvisited; in this embodiment, the interaction behavior data corresponding to one interaction behavior corresponds to one point.

[0339] Select core points: Randomly select an unvisited point and check the number of points in its neighborhood.

[0340] Expand the cluster: If the point is a core point, add the points in its neighborhood to the cluster and expand it recursively.

[0341] Label noise: Points that are not assigned to any cluster are labeled as noise.

[0342] Repeat the above steps until all points have been visited.

[0343] The DBSCAN algorithm can discover clusters of arbitrary shapes without pre-specifying the number of clusters and can handle noisy data.

[0344] In one example, the abnormal operation may refer to the interactive behavior data or operation corresponding to the noise point;

[0345] In another example, abnormal operations refer to interaction behaviors that deviate from historical interaction data by more than 3σ (standard deviation), including predefined patterns such as unconventional operation sequences (such as 10 consecutive rapid clicks) and timeouts without operations (intervals > 30 minutes).

[0346] Alarm events are structured exception records that contain fields such as the abnormal operation's exception ID, trigger time, exception type (enumerated value), associated controls, and confidence level (0-100%), conforming to the Common Event Format standard. For example, the confidence level may refer to the confidence level of the abnormal operation. The exception ID uniquely identifies an abnormal operation, and the exception type can be categorized based on statistical abnormal operations, yielding different types of abnormalities, such as rapid clicks and timeouts.

[0347] Device operation logs are standardized status records generated by industrial equipment. They contain structured data such as the device ID (MAC address), timestamp (synchronized to an NTP server), operating parameters (over 200 indicators such as temperature and speed), and alarm codes. Association tables are structured like relational database tables, with a primary key composed of a composite key (device ID + event timestamp). Fields contain association results such as interaction behavior data, device status parameters, and association confidence scores (0-1).

[0348] A heatmap is a two-dimensional density visualization chart. For example, the x-axis represents time (24-hour format) and the y-axis represents the control type. Color saturation represents the frequency of operation, and frequency intensity is encoded using the HSL color space (H = 120° green to 0° red gradient). As can be seen, a heatmap can be drawn based on the operation frequency distribution and timestamps of interaction behavior data. In other words, the operation frequency distribution and timestamps of interaction behavior data are analysis results.

[0349] A line chart is a time series trend chart with time on the x-axis (supporting granularity from seconds to months) and statistical indicators (frequency / anomaly count, etc.) on the y-axis. It includes confidence interval bands and key point annotations (such as peaks and inflection points). Exported documents are PDF / A-3 files compliant with the ISO 32000 standard, including vector graphics, structured bookmarks, accessibility tags, and digital signature blocks, supporting archiving for long-term preservation. For example, a line chart can be drawn based on the operation frequency distribution and timestamps of interactive behavior data, as well as abnormal operations. This means that abnormal operations are also considered analysis results.

[0350] In addition, the x-axis represents time (supporting granularity from seconds to months), and the y-axis represents statistical indicators (frequency / number of anomalies, etc.), which can also be drawn as a scatter plot.

[0351] For example, the following analysis may be performed to obtain analysis results.

[0352] The system processes raw data using a distributed computing framework: 1) Using a TumblingWindow aggregator, the system counts the number of various operations at a 1-minute granularity; 2) Using a Percentile Calculator, the system generates P50 / P90 / P99 frequency metrics; and 3) Applying a Fourier transform to identify periodic operation patterns. This statistical process utilizes columnar storage optimization, establishing a B+ tree index on the timestamp field to keep frequency query response times under 100ms. Analysis results are stored in a three-column structure (operation type, time bucket, and count value), supporting dynamic range queries and sliding window analysis. The system automatically generates frequency distribution reports, including visualization components such as time series histograms categorized by operation type and frequency comparison radar charts.

[0353] The system executes the following anomaly detection pipeline: 1) Feature engineering extracts features such as operation intervals, durations, and sequential patterns; 2) Normalizing flows are used to map these features into Gaussian space; 3) A clustering algorithm is run to identify outliers, and the Mahalanobis distance is calculated for each cluster. The alarm generation module, for confirmed abnormal operations (confidence level >85%), performs the following: 1) Correlate contextual data (the previous five operations); 2) Match against a predefined rule base (such as industrial operating procedures); and 3) Generate an alarm event with remediation recommendations. Alarm events are distributed via a message queue, supporting multiple notification methods such as email, SMS, and Syslog, and are recorded in an audit database for subsequent analysis.

[0354] The system implements precise time correlation: 1) The two data sources (interaction data and device operation logs) are synchronized using the same time base (PTP protocol synchronization, with an error of <1ms); 2) A time alignment window is established (default is a ±500ms sliding window); 3) Correlation is performed: For each target event, all device logs within the time alignment window are searched and the correlation probability is calculated using a random forest model. The device log with the highest correlation probability is then correlated with the target event.

[0355] The association table is stored in a columnar format (Parquet), sharded by device ID, and indexed by time range for each shard. The system automatically maintains an association quality dashboard, displaying real-time metrics such as match rate (>95% compliance) and latency distribution (P99 <200ms). A drop in match rate triggers a self-optimization process (adjusting the time window or model). The display match rate measures the degree of match between the successfully displayed data and the actual data required for display. Latency distribution refers to the difference between the time when real-time data is generated and when it is displayed.

[0356] The system generates reports through a visualization pipeline: 1) A WebGL-accelerated heatmap renderer processes millions of data points, dynamically adjusting the color scale (0-max frequency); 2) Line charts are smoothed using Bezier curves and trend lines (LOESS local regression); 3) A document assembly engine combines charts, tables, and text analysis into templates, applying corporate styling (fonts and color schemes). The export process includes: 1) PDF / A compliance checking (veraPDF verification); 2) digital signature application (RSA-PSS algorithm); and 3) generation of accompanying machine-readable data (a summary of the original data embedded in JSON-LD format). The final document is distributed via a CDN, supporting versioning and differential downloads (updated and modified portions only).

[0357] In addition to being returned to the industrial equipment side to provide data collection support for subsequent equipment production and operation, the above analysis results can also be used in other application scenarios, such as customer segmentation, product recommendations, risk assessment, market trend forecasting, etc.

[0358] Through automated analysis processes, the embodiment of the present application can process tens of millions of interactive data points per hour and generate comprehensive reports containing multiple analytical dimensions. The report documents meet the GMP data integrity ALCOA+ principles (attributable, clear, synchronized, original, and accurate) and support audit tracking and electronic signature verification. The entire mechanism improves industrial operation analysis efficiency by 80% and shortens problem location time by 90%, providing data support for equipment optimization, personnel training, and quality traceability. The system resource consumption is stable, with a single-node peak memory usage of <8GB and CPU utilization maintained below 30%, making it suitable for long-term operation in industrial sites.

[0359] In one embodiment, after the above step 101, the following further includes: if a control fingerprint matching the control handle of the target control is found in a pre-loaded static control fingerprint library, the data acquisition strategy associated with the matching control fingerprint is determined as the data acquisition strategy of the above target control; the above static control fingerprint library includes associated control fingerprints and data acquisition strategies; the above step of inputting the characteristic data of the above target control into the acquisition strategy generation model (step 102) is performed after no matching control fingerprint is found in the above static control fingerprint library.

[0360] The static control fingerprint library is a database of predefined control features loaded at system startup. For example, it can be stored in a hash table structure. This structure uses a hash function to map a key to a table location for accessing records, enabling fast data insertion, search, and deletion. The key is the control fingerprint (a 64-bit hash value), and the value is the associated data collection strategy. The fingerprint library is stored in memory as a read-only file, enabling fast access via a memory-mapped file. A control fingerprint is a digest value that uniquely identifies a control. It is generated using the SHA-256 algorithm to calculate control properties (including over 20 features such as the window class name, style bits, location hash, and parent window ID), ensuring collision resistance. A data collection strategy is a set of predefined monitoring scheme configurations, encoded in Protocol Buffers format. It includes a complete set of instructions, including the monitoring mode (event-driven, polling, or hybrid), event monitoring configuration (such as message hook type and callback function pointer), data collection parameters (sampling rate, interval, and data format), and transmission settings (compression algorithm and encryption method). Feature data is a structured dataset that describes the dynamic characteristics of controls. It contains over 30 metrics collected in real time (handle value, text content, storage location, control type, state-related information, message response latency, number of sub-controls, drawing frequency, etc.). It is organized in a columnar storage format (Apache Arrow) to support fast vectorized computation. The acquisition strategy generation model is an XGBoost-based decision tree ensemble model, which takes a 200-dimensional feature vector as input and outputs the optimal strategy configuration.

[0361] The system performs an exemplary fingerprint matching process: 1) Extract the complete attribute set of the target control (obtained through APIs such as GetClassName and GetWindowLong); 2) Calculate the fingerprint value using the SIMD-accelerated SHA-256 algorithm, with a processing throughput of 1GB / s; 3) Perform a quick preliminary check in the fingerprint library's Bloom filter (false positive rate <0.1%) to confirm whether a matching fingerprint is possible; 4) If a possible match is determined, perform a precise search in the hash table (time complexity O(1)). The search process uses optimistic concurrency control, allowing 100,000 query requests per second to be made simultaneously. If a match is successful, a read-only view of the policy configuration is returned, including 50+ parameter items such as monitoring mode and collection frequency.

[0362] When a data collection policy is found in the static control fingerprint library, the system further executes the policy loading process: 1) Verify data collection policy version compatibility (check if the major version number of the data collection policy matches); 2) Parse the policy configuration into a memory structure, including a function pointer table (20+ callback functions) and a parameter block (200+ bytes of configuration data); 3) Initialize the policy execution context (allocate thread-local storage and create an event queue); 4) Replace the policy pointer for the current control through an atomic operation, a switching process that takes less than 1ms. After the policy takes effect, the system records a policy binding log (including the control handle, fingerprint hash, and loading timestamp) and updates the runtime policy cache (managed using a least-recently-used algorithm with a capacity of 1000 entries).

[0363] In addition, in other embodiments of the present application, a dynamic data collection strategy can also be used to update the static control fingerprint library: 1) Start the feature collection pipeline to collect control feature data in real time through performance counters (QueryPerformanceCounter) and API hooks (Detours library); 2) Perform feature engineering (standardization, missing value filling, PCA dimensionality reduction); 3) Call the model inference engine (ONNX Runtime) to generate the data collection strategy configuration, which takes less than 5ms; 4) Write the new data collection strategy and its fingerprint to the dynamic strategy cache (ConcurrentDictionary) and start a background thread to perform strategy verification (simulating 100 operations). Data collection strategies that pass verification are periodically (for example, every 24 hours) batch-merged into the static fingerprint library to complete knowledge accumulation.

[0364] The embodiments of this application use a static fingerprint library to achieve sub-millisecond policy matching for 95% of common controls, and a dynamic model to handle 5% of long-tail controls. The system's average policy decision time is reduced from 50ms to 1.2ms, memory usage is reduced by 40% (through fingerprint sharing), and CPU utilization is reduced by 35%. The dynamic policy generation mechanism enables the system to adapt to new controls, and policy accuracy improves by 0.5% per month. The fingerprint library's hot update function supports updating more than 1,000 policies per hour without restarting the service, providing continuously optimized data collection capabilities for industrial sites. The entire mechanism ensures support for 100,000 policy queries per second on an 8-core CPU, with an error decision rate of less than 0.01%, meeting industrial-grade reliability requirements.

[0365] In one achievable embodiment of the present application, referring to Figure 2 ,The data acquisition system can include: production equipment, edge computing, and cloud services.

[0366] Production equipment refers to physical devices in industrial automation scenarios (such as PLCs and CNC machine tools). These devices interact with applications through control handles, generating operational data and production logs. Edge computing refers to distributed computing nodes located close to the data source, responsible for real-time processing of raw data generated by production equipment, reducing the transmission load on the cloud. Cloud services refer to the storage, computing, and analysis capabilities provided by remote servers. They receive edge computing-processed data for in-depth analysis and long-term storage.

[0367] Among them, after the target application is started, the control handle capture program in the production equipment will start to obtain the control handle in the interface of the target application, and then it will monitor the interface elements corresponding to the control handle, analyze the monitored interaction data, and send the interaction data to edge computing.

[0368] Edge computing will clean and assemble the received interaction data and then encrypt and send it to the cloud service.

[0369] The cloud service will decode and parse the encrypted data and insert it into the corresponding business table based on business processing.

[0370] Of course, the above description is only an example embodiment. In actual applications, edge computing and cloud services can be the same device or different devices. Edge computing can also be the same device as production equipment. The specific settings can be based on actual needs and are not limited here.

[0371] Based on the same inventive concept, the present application also provides a data acquisition device for implementing the aforementioned data acquisition method. The implementation solution provided by this device is similar to the implementation solution described in the aforementioned method. Therefore, the specific limitations of one or more data acquisition device embodiments provided below can be found in the above-mentioned limitations of the data acquisition method and will not be repeated here.

[0372] In an exemplary embodiment, Figure 3 As shown, a data acquisition device 50 is provided, comprising:

[0373] Acquisition module 501 is used to obtain the control handle in the current interface of the target application; the above-mentioned control handle is at least used to determine the control type; wherein, the control whose control type in the current interface belongs to the target type is the target control; the above-mentioned target type includes lists, buttons and input boxes; the target control in the above-mentioned current interface includes at least one of a list control, a button control and an input box control.

[0374] The processing module 502 is configured to input the characteristic data of the target control into a collection strategy generation model to generate a data collection strategy corresponding to the control type of the target control; the characteristic data at least includes: the control type.

[0375] The collection module 503 is used to monitor the target control in real time based on the data collection strategy, so as to capture the interactive behavior data of the target control when a state change is detected.

[0376] Please refer to the above records and I will not go into details here.

[0377] In one embodiment, the control handle of the target control is a target control handle; the acquisition module 501 is further configured to:

[0378] Call the preset API or function to read the text content and storage location of the target control handle.

[0379] Get the state-related information and control type of the target control according to the target control handle.

[0380] The characteristic data includes the handle value, the text content, the storage location, the control type of the target control, and state-related information.

[0381] In terms of generating a data collection strategy corresponding to the control type of the target control, the processing module 501 is used to:

[0382] From the preset policy template library, determine the basic collection rules that match the control type of the target control.

[0383] Based on the control monitoring configuration corresponding to the target control, the matching basic collection rules are adjusted to obtain the data collection strategy. Please refer to the above record and will not be repeated here.

[0384] In one embodiment, in terms of adjusting the matched basic acquisition rules, the processing module 502 is configured to:

[0385] Based on the control listening configuration corresponding to the above-mentioned list control, the block loading rules are adjusted for the above-mentioned list control. The above-mentioned block loading rules include scroll trigger conditions and data block size, as well as collecting the currently visible list item data when a scroll event for the above-mentioned list control is detected, and collecting the clicked list item data when a click event for the above-mentioned list control is detected.

[0386] Based on the control monitoring configuration corresponding to the button control, a monitoring rule is adjusted for the button control, where the monitoring rule includes a baseline monitoring interval and a minimum monitoring interval.

[0387] Based on the control monitoring configuration corresponding to the above input box control, the difference comparison rules are adjusted for the above input box control. The above difference comparison rules include the comparison period and the change content extraction rules. Please refer to the above description and will not be repeated here.

[0388] In one embodiment, the processing module 502 is further configured to:

[0389] The above interactive behavior data is encoded into transmission data in a preset format and transmitted to the receiving end through an encryption protocol.

[0390] Based on the feedback results from the receiving end, the data collection strategy corresponding to the target control is optimized. Please refer to the above record and will not be repeated here.

[0391] In one embodiment, in terms of optimizing the data collection strategy corresponding to the target control based on the feedback result of the receiving end, the processing module 502 is configured to:

[0392] Based on the data integrity score and system load indicators in the above feedback results, the strategy optimization weight corresponding to each target type is calculated.

[0393] When the strategy optimization weight corresponding to the list control reaches the list control weight threshold, the data block size in the above block loading rule is adjusted.

[0394] When the policy optimization weight corresponding to the button control reaches the button control weight threshold, the monitoring mode in the above monitoring rule is switched to a hybrid monitoring mode, which includes: event-driven mode and polling mode. Please refer to the above description and will not be repeated here.

[0395] In one embodiment, the processing module 502 is further configured to:

[0396] Synchronize the optimized data collection strategy to the operating environment of the target application through the hot loading mechanism;

[0397] Verify the compatibility of the optimized control listening configuration with the above control handle.

[0398] If the verification is successful, the process will re-enter the step of real-time monitoring of the target control based on the data collection strategy. Please refer to the above record and will not be repeated here.

[0399] In one embodiment, in encoding the interactive behavior data into transmission data in a preset format, the processing module 502 is configured to:

[0400] Encapsulate the above interaction behavior data into a JSON structure.

[0401] Perform Base64 encoding on the above JSON structure to generate intermediate encoded data.

[0402] The intermediate coded data is encrypted using a symmetric encryption algorithm to generate an encrypted data packet.

[0403] Adding a transmission protocol header to the encrypted data packet generates the transmission data that complies with the preset communication specification.

[0404] In one embodiment, in terms of real-time monitoring of target controls based on a data collection strategy, the collection module 503 is configured to:

[0405] The target events of the list control are monitored through an event listener or a message hook; wherein the target events of the list control include scroll events and selection events.

[0406] The target events of the button control are monitored according to the monitoring mode in the control monitoring configuration; wherein the monitoring mode in the control monitoring configuration is event-driven mode, polling mode or mixed monitoring mode; the mixed monitoring mode includes: event-driven mode and polling mode; the target events of the button control include click events and long press events.

[0407] The target events of the input box control are monitored in a polling mode; wherein the target events of the input box control include text input events and focus switching events.

[0408] The above-mentioned interactive behavior data includes the monitored target events. Please refer to the above records and will not be repeated here.

[0409] In one embodiment, the acquisition module 503 is further configured to:

[0410] Output a data collection report for the target application, wherein the data collection report includes content obtained by at least one of the following methods:

[0411] Collect statistics on the operation frequency distribution and timestamps of the above interactive behavior data;

[0412] Identify abnormal operations in the above interactive behavior data through clustering algorithms and generate alarm events;

[0413] Correlate the above interaction behavior data with the device operation log based on the timestamp to obtain a correlation table;

[0414] Render the analysis results of the above interactive behavior data into visual charts (such as heat maps, line charts, scatter plots, etc.) and export them to documents. Please refer to the previous description and will not be repeated here.

[0415] In one embodiment, the acquisition module 503 is further configured to:

[0416] If a control fingerprint that matches the control handle of the target control is found in the pre-loaded static control fingerprint library, the data collection strategy associated with the matching control fingerprint is determined as the data collection strategy of the above-mentioned target control; the above-mentioned static control fingerprint library includes the associated control fingerprints and data collection strategies.

[0417] The step of inputting the characteristic data of the target control into the acquisition strategy generation model is performed after a matching control fingerprint is not found in the static control fingerprint library. Please refer to the above description and will not be repeated here.

[0418] The embodiment of the present application can obtain the control handle through the data collection technology based on the control handle capture, and generate corresponding data collection strategies for different target controls according to the type of the control handle, so as to monitor the target control according to the data collection strategy to realize the data collection of various interactive data. It does not need to rely on the data interface provided by the production equipment, nor is it restricted by the type of collectible data provided by the data interface, thereby improving the applicability of interactive data collection in production equipment.

[0419] In an exemplary embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0420] For example, the computer device may be a server or a terminal, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store data acquisition data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the data acquisition method in the aforementioned embodiments is implemented.

[0421] Those skilled in the art will understand that Figure 4The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0422] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0423] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0424] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0425] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0426] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0427] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0428] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A data collection method, characterized in that: The data collection method comprises: Obtaining a control handle in the current interface of a target application running on a production device in an industrial automation production line; the control handle is used at least to determine the control type; wherein a control in the current interface whose control type belongs to a target type is a target control; the target types include lists, buttons, and input boxes; the target control in the current interface includes at least one of a list control, a button control, and an input box control; Inputting the characteristic data of the target control into the acquisition strategy generation model to generate a data acquisition strategy corresponding to the control type of the target control; the characteristic data at least includes: the control type; Performing real-time monitoring on the target control based on the data collection strategy to capture the interactive behavior data of the target control when a state change is detected; The step of generating a data collection strategy corresponding to the control type of the target control includes: Determine, from a preset policy template library, a basic collection rule that matches the control type of the target control, wherein the basic collection rule is the smallest policy unit designed for a specific control type; Based on the control monitoring configuration corresponding to the target control, adjusting the matching basic collection rules to obtain a data collection strategy; Among them, the control monitoring configuration includes a dynamically generated optimization parameter set, which contains the collection constraints, current operating parameters and optimization goals for the target control under the current operating environment. The collection constraints include the CPU occupancy upper limit, network delay tolerance, and business priority weight. The optimization goals include reducing CPU occupancy and shortening network delay. The current operating parameters include the current CPU usage.

2. The data collection method according to claim 1, characterized in that: The control handle of the target control is the target control handle; the process of acquiring the characteristic data includes: Calling a preset API or function to read the text content and storage location of the target control handle; Acquire state-related information and control type of the target control according to the target control handle; The characteristic data includes a handle value, the text content, the storage location, the control type of the target control, and state-related information.

3. The data collection method according to claim 1, wherein: The step of adjusting the matching basic collection rules includes: Adjusting a block loading rule for the list control based on a control listening configuration corresponding to the list control, the block loading rule including a scroll trigger condition and a data block size, and collecting currently visible list item data when a scroll event is detected for the list control, and collecting clicked list item data when a click event is detected for the list control; Adjusting a monitoring rule for the button control based on a control monitoring configuration corresponding to the button control, the monitoring rule including a baseline monitoring interval and a minimum monitoring interval; Based on the control monitoring configuration corresponding to the input box control, a difference comparison rule is adjusted for the input box control, where the difference comparison rule includes a comparison period and a change content extraction rule.

4. The data collection method according to claim 3, characterized in that: After the step of capturing the interactive behavior data of the target control, the method further includes: Encoding the interactive behavior data into transmission data in a preset format and transmitting the data to a receiving end via an encryption protocol; Based on the feedback result of the receiving end, the data collection strategy corresponding to the target control is optimized.

5. The data collection method according to claim 4, characterized in that: The step of optimizing the data collection strategy corresponding to the target control based on the feedback result of the receiving end includes: Calculate the strategy optimization weight corresponding to each target type based on the data integrity score and system load index in the feedback results; When the strategy optimization weight corresponding to the list control reaches the list control weight threshold, adjusting the data block size in the block loading rule; When the policy optimization weight corresponding to the button control reaches the button control weight threshold, the monitoring mode in the monitoring rule is switched to a hybrid monitoring mode, which includes: an event-driven mode and a polling mode.

6. The data collection method according to claim 5, characterized in that: After the step of optimizing the data collection strategy corresponding to the target control, the method further includes: Synchronizing the optimized data collection strategy to the operating environment of the target application through a hot loading mechanism; Verifying the compatibility of the optimized data acquisition strategy with the control handle; If the verification is successful, the process re-enters the step of monitoring the target control in real time based on the data collection strategy.

7. The data collection method according to claim 5, characterized in that: The step of encoding the interactive behavior data into transmission data in a preset format includes: Encapsulate the interaction behavior data into a JSON structure; Perform Base64 encoding on the JSON structure to generate intermediate encoded data; Encrypting the intermediate coded data using a symmetric encryption algorithm to generate an encrypted data packet; A transmission protocol header is added to the encrypted data packet to generate the transmission data that complies with the preset communication specification.

8. The data collection method according to claim 1, wherein: The step of monitoring the target control in real time based on the data collection strategy includes: Monitor the target events of the list control through an event listener or a message hook; wherein the target events of the list control include scroll events and selection events; Monitoring the target event of the button control according to the monitoring mode in the control monitoring configuration; wherein the monitoring mode in the control monitoring configuration is event-driven mode, polling mode or mixed monitoring mode; the mixed monitoring mode includes: event-driven mode and polling mode; the target event of the button control includes a click event and a long press event; Monitoring target events of the input box control in a polling mode; wherein the target events of the input box control include text input events and focus switching events; The interactive behavior data includes monitored target events.

9. The data collection method according to claim 1, characterized in that: After the step of capturing the interactive behavior data of the target control, the method further includes: Output a data collection report of the target application, wherein the data collection report includes content obtained by at least one of the following methods: Counting the operation frequency distribution and timestamp of the interactive behavior data; Identifying abnormal operations in the interactive behavior data through a clustering algorithm and generating an alarm event; Associating the interaction behavior data with the device operation log based on the timestamp to obtain an association table; The analysis results of the interactive behavior data are rendered into a visual chart and exported as a document.

10. The data collection method according to claim 1, characterized in that: After the step of obtaining the control handle in the current interface of the target application running on the production equipment in the industrial automation production line, the method further includes: If a control fingerprint matching the control handle of the target control is found in a pre-loaded static control fingerprint library, the data collection strategy associated with the matching control fingerprint is determined as the data collection strategy of the target control; the static control fingerprint library includes associated control fingerprints and data collection strategies; The step of inputting the characteristic data of the target control into the acquisition strategy generation model is performed after no matching control fingerprint is found in the static control fingerprint library.

11. A data acquisition device, characterized in that: The data acquisition device comprises: An acquisition module is configured to acquire a control handle in a current interface of a target application running on a production device in an industrial automation production line; the control handle is used to determine at least the control type; a control in the current interface whose control type is a target type is a target control; the target types include lists, buttons, and input boxes; and the target control in the current interface includes at least one of a list control, a button control, and an input box control; A processing module, configured to input characteristic data of the target control into an acquisition strategy generation model to generate a data acquisition strategy corresponding to the control type of the target control; the characteristic data at least includes: the control type; the acquisition strategy generation model is constructed based on rules or a machine learning algorithm; A collection module, configured to monitor the target control in real time based on the data collection strategy, so as to capture the interactive behavior data of the target control when a state change is detected; The processing module is specifically used to: The step of generating a data collection strategy corresponding to the control type of the target control includes: Determine, from a preset policy template library, a basic collection rule that matches the control type of the target control, wherein the basic collection rule is the smallest policy unit designed for a specific control type; Based on the control monitoring configuration corresponding to the target control, adjusting the matching basic collection rules to obtain a data collection strategy; Among them, the control monitoring configuration includes a dynamically generated optimization parameter set, which contains the collection constraints, current operating parameters and optimization goals for the target control under the current operating environment. The collection constraints include the CPU occupancy upper limit, network delay tolerance, and business priority weight. The optimization goals include reducing CPU occupancy and shortening network delay. The current operating parameters include the current CPU usage.

12. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the data acquisition method according to any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the data collection method according to any one of claims 1 to 10 are implemented.

14. A computer program product, characterized in that The computer program product comprises instructions, and when the instructions are executed, the data collection method according to any one of claims 1 to 10 is performed.

Images