Data collection methods, devices, computer equipment and storage media for cross-border platforms

Abstract

This application relates to the field of internet data processing technology, and discloses a data collection method, apparatus, computer device, and storage medium for cross-border platforms. The method includes: when the device fingerprint update condition is met, obtaining corresponding hardware feature statistics and network feature statistics based on the target platform's geographical location; generating hardware feature parameters and creating a virtual device based on the hardware feature statistics; generating network feature parameters and configuring network parameters for the virtual device based on the network feature statistics; inputting the platform user behavior dataset and hardware network parameters into a pre-trained large language model to generate a human-like user interaction sequence; and controlling the virtual device to execute the interaction sequence to achieve data collection from the target platform. This application can effectively reduce the probability of identity recognition and behavior detection in traditional web crawlers, significantly improving the success rate and efficiency of data collection on cross-border platforms.

Description

Technical Field

[0001] This invention relates to the field of Internet data processing technology, and in particular to a data collection method, apparatus, computer equipment, and storage medium for cross-border platforms. Background Technology

[0002] With the continuous development of global e-commerce, e-commerce companies urgently need to gain in-depth insights into competing products in target markets when expanding into overseas markets. Obtaining real-time price dynamics and user reviews of competing products has become a key basis for companies to formulate precise marketing strategies and optimize product layout. To achieve this goal, the industry generally adopts automated data collection programs to crawl target data from major cross-border e-commerce platforms, social media platforms, and other relevant websites.

[0003] However, such traditional data collection programs face significant technical bottlenecks in practical applications. Their operational patterns are often monotonous and repetitive, making them easily identifiable by the traffic control and anti-scraping mechanisms deployed by target overseas platforms, thus triggering corresponding security protection strategies. This severely restricts the continuity and stability of data collection, ultimately resulting in low information acquisition efficiency for enterprises in overseas market competition. Summary of the Invention

[0004] Based on this, it is necessary to address the technical problem of low efficiency in data collection from overseas platforms in existing technologies, and propose a data collection method, device, computer equipment, and storage medium for cross-border platforms.

[0005] Firstly, a data collection method for a cross-border platform is provided, the method comprising:

[0006] When the device fingerprint update conditions are met, the corresponding hardware feature statistics are determined based on the geographical location of the cross-border platform, and hardware feature parameters are generated based on the hardware feature statistics.

[0007] A virtual device is created by calling hardware resources in the hardware resource library based on the hardware characteristic parameters;

[0008] The system determines the corresponding network feature statistics based on the geographical location of the cross-border platform, generates network feature parameters based on the network feature statistics, and configures the network feature parameters for the virtual device.

[0009] The user behavior dataset of the cross-border platform, the hardware feature parameters, and the network feature parameters are input into a pre-trained large language model to generate user interaction sequences.

[0010] The virtual device is instructed to interact with the cross-border platform using the user interaction sequence, and to collect target data from the webpage content returned by the cross-border platform in response to the user interaction sequence.

[0011] Secondly, a data collection device for a cross-border platform is provided, the device comprising:

[0012] The generation module is used to determine the corresponding hardware feature statistics based on the geographical location of the cross-border platform when the device fingerprint update conditions are met, and to generate hardware feature parameters based on the hardware feature statistics.

[0013] A creation module is used to create a virtual device by calling hardware resources in the hardware resource library based on the hardware feature parameters;

[0014] The configuration module is used to determine the corresponding network feature statistics based on the geographical location of the cross-border platform, generate network feature parameters based on the network feature statistics, and configure the network feature parameters for the virtual device.

[0015] The model processing module is used to input the user behavior dataset of the cross-border platform, the hardware feature parameters, and the network feature parameters into a pre-trained large language model to generate user interaction sequences.

[0016] The data acquisition module is used to instruct the virtual device to interact with the cross-border platform using the user interaction sequence, and to acquire target data from the webpage content returned by the cross-border platform in response to the user interaction sequence.

[0017] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the data acquisition method for the cross-border platform described above.

[0018] Fourthly, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the data acquisition method for the cross-border platform described above.

[0019] The beneficial effects of this application are:

[0020] By dynamically generating hardware and network characteristic parameters based on the target platform's geographical location, and constructing virtual devices accordingly, each data collection task can simulate the device and network environment of real users in that region. This approach makes the data collection program appear to the target platform as access from real users using different devices in different regions, rather than a single, fixed crawler program, thus effectively reducing the probability of traffic control triggered by fixed identity characteristics.

[0021] A large language model is introduced to generate user interaction sequences. These sequences are not fixed scripts, but rather dynamically generated operation flows based on real user behavior datasets, combined with the current virtual device performance and network conditions, exhibiting both the randomness and logic of human behavior. This ensures that every operation performed by the virtual device on the platform reflects the behavioral characteristics of a real user, significantly reducing the risk of being identified by anti-scraping systems due to monotonous or mechanical behavior patterns.

[0022] In summary, this application ensures the continuity and stability of data acquisition tasks through a combination of environmental simulation and anthropomorphic behavior. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] in:

[0025] Figure 1 This is an application environment diagram of a data collection method for a cross-border platform in one embodiment;

[0026] Figure 2 This is a flowchart of a data collection method for a cross-border platform in one embodiment;

[0027] Figure 3 This is a structural block diagram of a data acquisition device for a cross-border platform in one embodiment;

[0028] Figure 4 This is a structural block diagram of a computer device in one embodiment. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] The data collection method for cross-border platforms provided in this invention can be applied to, for example... Figure 1 In this application environment, the client communicates with the server via a network. When the device fingerprint update conditions are met, the computer device obtains corresponding hardware feature statistics and network feature statistics based on the target platform's geographical location; it generates hardware feature parameters and creates a virtual device based on the hardware feature statistics; it generates network feature parameters based on the network feature statistics and configures network parameters for the virtual device; it inputs the platform user behavior dataset and hardware network parameters into a pre-trained large language model to generate a human-like user interaction sequence; and it controls the virtual device to execute this interaction sequence to collect data from the server. This effectively reduces the probability of identity recognition and behavior detection in traditional web crawlers, significantly improving the success rate and efficiency of data collection on cross-border platforms. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.

[0031] Please see Figure 2 As shown, Figure 2 A flowchart illustrating a data collection method for a cross-border platform provided in this embodiment of the invention includes the following steps:

[0032] S1: When the device fingerprint update conditions are met, determine the corresponding hardware feature statistics based on the geographical location of the cross-border platform, and generate hardware feature parameters based on the hardware feature statistics.

[0033] In the process of device fingerprint management on computer equipment, the device fingerprint update process is immediately triggered when the device fingerprint update conditions are met. Device fingerprint update conditions refer to specific events that trigger the computer equipment to initiate its virtual identity reset and reconstruction process. These conditions mainly include two typical scenarios: one is periodic updates, where the computer equipment automatically initiates the update process according to a preset time period, such as every 24 hours. This is designed to prevent a single virtual identity from triggering the platform monitoring system's alert due to prolonged continuous activity. The other condition ensures that when a virtual identity becomes invalid, a new usable identity can be quickly generated to maintain the continuity of data collection tasks.

[0034] Computer devices retrieve hardware characteristic statistics matching their location from a pre-stored statistical database based on the geographic location information associated with the cross-border platform. These statistics reflect the distribution of common hardware attributes within a specific region, such as the prevalence of parameters like screen resolution, processor type, or memory capacity. By analyzing these hardware parameter statistics, the computer devices extract key characteristic trends and use mathematical modeling methods to generate hardware characteristic parameters.

[0035] The process by which computer devices generate hardware characteristic parameters based on statistical information about hardware features involves using a series of algorithms to transform macroscopic statistical distribution data into specific parameter values ​​that can be used to construct device fingerprints. This process aims to ensure that the generated parameters not only conform to the overall hardware ecosystem of the target region but also possess appropriate randomness and variability, in order to avoid generating duplicate or overly predictable device fingerprints.

[0036] The sources of hardware characteristic statistics may include:

[0037] I. Data sourced from publicly available global mobile device market reports. For computer equipment, statistical distribution data on key hardware attributes such as screen resolution, operating system version, and memory capacity are extracted from regional equipment sales, market share, and configuration distribution reports published by professional market research institutions.

[0038] Second, the data comes from third-party data service providers that have partnerships with cross-border e-commerce platforms. Through compliant data interfaces, aggregated and anonymized device information of platform visitors is obtained. This information accurately reflects the device composition of users accessing the platform in that region.

[0039] Third, it stems from the long-term accumulation of data from a self-built data collection network. By deploying distributed sensing nodes under legal and compliant conditions, it continuously collects anonymous device information from target areas, forming a first-hand hardware feature sample library.

[0040] Methods for constructing hardware characteristic statistical information for a cross-border region include:

[0041] Invalid records with abnormal device identifiers or contradictory configuration information are removed from the collected multi-source raw data. Heterogeneous hardware descriptions from different data sources are uniformly mapped to a standardized feature coding system. Continuous features (such as memory capacity) are discretized into intervals, and a hierarchical standard is established. Computer devices are statistically calculated on the processed data according to geographical region: the frequency distribution of each enumerated value is calculated for discrete features (such as operating system version), generating a probability quality function; kernel density estimation is performed for continuous features (such as screen size) to determine its core distribution interval and confidence threshold; and a feature correlation model is established (such as the co-occurrence probability of high-resolution screens and large memory capacity). A data freshness evaluation index is set, triggering statistical information recalculation when the source data update rate reaches a threshold. Differentiated weights are configured for different data sources, and a Bayesian update strategy is used to progressively optimize the statistical model, retaining historical version statistical information to support the analysis of regional device evolution trends.

[0042] In one embodiment, the computer device employs a weighted random selection algorithm. This method first takes the categories and their corresponding probability proportions from the hardware feature statistics as input. Then, the algorithm performs random sampling based on this probability distribution, thereby selecting a specific parameter value for each hardware feature item. This method ensures that the generated parameter values ​​are statistically consistent with the actual distribution of the target area, reflecting both commonalities and introducing reasonable randomness.

[0043] In another embodiment, the computer device employs a numerical range generation method applicable to continuous hardware features such as screen size or battery capacity. The computer device first identifies the most prevalent numerical ranges from hardware feature statistics and uses these as a basis to determine a core parameter range. Subsequently, within this core range, the computer device generates a specific value using a random function, which serves as the final hardware feature parameter. The parameters generated by this method both cluster within the mainstream configuration range and exhibit subtle fluctuations, more closely reflecting the actual situation of real-world devices.

[0044] For example, regarding the generation of device resolution parameters in the Southeast Asian market, hardware characteristic statistics show that 1280×720 resolution accounts for 68%, 1920×1080 accounts for 25%, and other resolutions account for 7%. Computer devices use a weighted random selection algorithm, so when generating parameters, there is a 68% probability that the resolution parameter will be set to 1280×720, and a 25% probability that it will be set to 1920×1080, thus making the distribution consistent with statistical patterns.

[0045] For continuous hardware characteristics, such as screen size, statistics show that over 70% of devices in the North American market have screen sizes between 6.1 and 6.7 inches. When using a numerical range generation method, computer devices randomly select from this core range of 6.1 to 6.7 inches to generate specific parameter values ​​with one decimal place, such as 6.3 inches or 6.5 inches. This approach is also applicable to generating battery capacity parameters; for example, a random integer value, such as 3950mAh, is generated within the mainstream range of 3800mAh to 4100mAh, making the generated device characteristics both realistic and diverse.

[0046] S2: Create a virtual device by calling hardware resources in the hardware resource library according to the hardware feature parameters.

[0047] Upon obtaining the hardware characteristic parameters, the computer device initiates a virtual device construction process. This process involves matching and retrieving data from a pre-defined hardware resource library based on the generated hardware characteristic parameters. The hardware resource library is a structured data storage module that systematically archives various types of hardware drivers, firmware versions, device identifiers, and underlying hardware configuration templates. The computer device uses the hardware characteristic parameters as query conditions or index keys to retrieve and integrate corresponding specific hardware resource instances from the hardware resource library. Subsequently, the computer device assembles and configures these retrieved discrete hardware resources according to the logical specifications for device startup and operation, ultimately instantiating a complete virtual device with the attributes defined by the hardware characteristic parameters within the host operating system. This virtual device presents itself externally as a consistent and interactive virtual hardware environment.

[0048] It should be noted that the computer equipment can create multiple virtual devices simultaneously based on the number of data acquisition tasks. These virtual devices can be mobile phones, tablets, desktop computers, laptops, or other devices. The process described in this application allows for the simultaneous configuration of different hardware and network characteristic parameters, as well as different user interaction sequences, for multiple virtual devices, thereby further improving the efficiency of data acquisition tasks.

[0049] For example, the computer device generates hardware characteristic parameters based on the aforementioned steps, including a screen resolution of 1280×720, 6GB of RAM, and a Snapdragon 680 processor. The computer device then uses these parameters to access the hardware resource library. It retrieves the display driver file matching the 1280×720 resolution, the system memory management configuration template optimized for 6GB of RAM, and the kernel image and device tree file corresponding to the Snapdragon 680 processor. Finally, the computer device integrates these resources and loads them into a separate virtualization container, thereby generating a complete virtual device. After startup, the virtual device's system information will accurately report a screen resolution of 1280×720, 6GB of RAM, and a Snapdragon 680 processor, simulating a virtual instance highly similar in hardware characteristics to mainstream devices in the target region.

[0050] S3: Determine the corresponding network feature statistics based on the geographical location of the cross-border platform, generate network feature parameters based on the network feature statistics, and configure the network feature parameters for the virtual device.

[0051] The computer device then performs a subsequent step: determining the corresponding network characteristic statistics based on the geographical location of the cross-border platform. The computer device accesses its stored network environment database, which aggregates typical network connection data from users in different geographical regions. Based on the geographical location of the target platform, the computer device retrieves network characteristic statistics associated with that region, which macroscopically reflects the commonalities in the network environment of the user group in that region.

[0052] Subsequently, the computer equipment generates network characteristic parameters based on the acquired network characteristic statistics. This generation process is achieved through a parametric model, which transforms the distribution patterns reflected in the statistics into specific, configurable parameter values. These parameters are designed to accurately describe the technical attributes of the network connectivity.

[0053] The network characteristic statistics information is mainly obtained through the following compliant channels:

[0054] I. Publicly available statistical reports from the Internet Society of America (ISAA) regularly publish network infrastructure indicators for major countries on various continents, including median average bandwidth, mobile network coverage, and typical network latency.

[0055] Second, access the regional dataset of the global network quality monitoring platform, and obtain network transmission characteristics such as transmission control protocol window size, round-trip time, and packet loss rate by analyzing the network performance data continuously collected by the probe nodes deployed in the target area.

[0056] Third, obtain anonymized user network environment sample data through legitimate local telecommunications operators, focusing on collecting the fluctuation patterns of network connection modes at different times.

[0057] Methods for constructing network characteristic statistics for a cross-border region include:

[0058] Multi-dimensional data collection and alignment are performed, standardizing network metrics from different data sources according to a unified spatiotemporal benchmark. A data aggregation model based on geographical regions is established, statistically analyzing the percentile distribution of network bandwidth metrics by Internet service provider category, and recording typical configuration values ​​of transmission control protocol parameters by network type. Feature association and pattern extraction are conducted to analyze the evolution of transport layer security protocol fingerprints under different network conditions, establishing a correlation model between network latency and Hypertext Transfer Protocol (HTTP) connection multiplexing rate. Clustering algorithms are used to identify typical network environment combinations in target areas, such as identifying "high latency-large window-multiplexing" as a cross-border access characteristic of a specific region. Quantifiable statistical descriptions are generated, calculating the core numerical ranges and confidence levels of each network feature item, forming a complete statistical profile including basic connection parameters, transmission performance parameters, and security protocol parameters. This dynamic statistical information update mechanism includes setting a data validity verification window, automatically triggering statistical model reconstruction when regional network infrastructure upgrades are detected.

[0059] Finally, the computer device configures the virtual device with the generated network characteristic parameters. This configuration process involves writing the parameters into the virtual device's network protocol stack and related software-defined network interfaces, enabling the virtual device to exhibit technical characteristics highly consistent with the real user network environment in the target area when initiating a network connection.

[0060] For example, network characteristic statistics obtained by computer equipment targeting a cross-border platform in Southeast Asia show that 65% of local users have bandwidth between 20Mbps and 50Mbps, 70% have a default TCP (Transmission Control Protocol) window size of 65535 bytes, and 58% use TLS 1.2.

[0061] Computer equipment generates network characteristic parameters based on this statistical information. For bandwidth parameters, it may use a range generation method, selecting a value such as 35Mbps within the range of 20Mbps to 50Mbps. For TCP window size, a typical value of 65535 bytes is directly adopted based on weighted probability. For TLS (Transport Layer Security) fingerprint parameters, a fingerprint identifier that matches the statistical distribution of the region is generated by combining information such as version and cipher suite.

[0062] The computer equipment ultimately configures these parameters into the created virtual device. When the virtual device accesses the network through the platform, the established network connection will specifically manifest as a 35Mbps bandwidth channel, a 65535-byte TCP window size, and specific TLS handshake characteristics, thereby simulating the typical environment of a real user in the region at the network layer.

[0063] S4: Input the user behavior dataset of the cross-border platform, the hardware feature parameters, and the network feature parameters into a pre-trained large language model to generate user interaction sequences.

[0064] The process involves the computer device generating a user interaction sequence. This process first integrates three key input elements: the user behavior dataset of the cross-border platform, hardware characteristic parameters set for the virtual device, and network characteristic parameters. The user behavior dataset contains operating habits and browsing patterns recorded in historical user sessions on the platform.

[0065] The user behavior dataset is a legally compliant dataset generated based on real user actions. Computer devices use application programming interfaces (APIs) to call third-party data analysis services explicitly authorized by the user. These services can be derived from internal enterprise data or server logs, datasets from professional data providers like IPIDA (e.g., e-commerce platform product information, sales volume, user reviews), social media datasets, or publicly available data sources such as market research reports, industry statistical annual reports, and publicly available datasets on comprehensive data platforms. During processing, the computer device initiates an authentication request to the third-party service platform. After obtaining data access permissions, it retrieves aggregated and anonymized user behavior statistical reports or anonymized behavior sequences as needed, converts them to a format compatible with the platform, and finally outputs a supplementary user behavior dataset. All data in the user behavior dataset is anonymized to prevent identification of specific individuals and to ensure data integrity. For example, fields that directly identify individuals, such as name, ID number, mobile phone number, and precise address, are directly deleted or replaced. For identifiers that need to be retained for correlation analysis but cannot be in plaintext (such as user IDs), an irreversible encryption process can be performed using a hash function (such as SHA-256), converting it into a fixed string of random characters. This reduces the precision of the data to obscure individual identities. For example, precise ages can be converted into age ranges (such as "20-30 years old"), and specific geographical locations can be obscured to the city or region level.

[0066] The computer device feeds the integrated input elements into a pre-trained large language model. This large language model, trained on massive amounts of human behavior data, is capable of deeply understanding the user's operational context and simulating human decision-making logic. By analyzing the behavioral probability distribution inherent in the input dataset and comprehensively considering the device's operational characteristics defined by hardware feature parameters and the response latency influenced by network feature parameters, the model infers and generates a logical user interaction sequence that exhibits the randomness of human behavior.

[0067] For example, a computer device inputs a set of user datasets containing high-frequency product image browsing and price comparison behaviors, along with hardware feature parameters such as touch screen device metrics and network feature parameters such as network latency in the hundreds of milliseconds, into a large language model.

[0068] It should be noted that the training process of the large language model in this embodiment includes:

[0069] Construct the training dataset. The training data comes from anonymized user session records from multiple cross-border e-commerce platforms. These records contain complete user operation sequences and their corresponding environmental contexts. Each training sample consists of three parts: a sequence of user operation instructions, a description of the device hardware configuration at the time of the operation, and network environment parameters during the operation.

[0070] Data preprocessing is performed. Semantic annotation is applied to the original operation sequences, aggregating low-level user operations into high-level behavioral units with business meaning. Simultaneously, device hardware configuration and network environment parameters are standardized and encoded, transforming them into feature vectors that the model can understand.

[0071] Then comes the model pre-training phase. A multi-layer Transformer architecture is used as the model foundation, and representation learning is performed on massive training data through self-supervised learning. In this phase, the model learns to understand the logical relationships between user operations and establishes the implicit relationships between operational behaviors and device performance and network conditions.

[0072] Finally, model fine-tuning is performed. The pre-trained model is optimized using scenario-based data from a specific cross-border e-commerce platform, and platform-specific interaction rules are introduced through reinforcement learning. During fine-tuning, the focus is on whether the model's output operation sequence conforms to common user behavior patterns on the platform, while ensuring that the time interval parameters in the sequence match the natural rhythm of human operation.

[0073] After computation, the large language model generates a sequence of user interactions. This sequence might begin with a slow, random scrolling motion on a product list page, with the scrolling speed fluctuating to simulate human reading habits. The sequence then includes a mouse click on a specific product cover, followed by a focus-switching delay due to simulated network latency and human reaction time. The sequence then triggers a dragging action to add the product to the shopping cart, and intermittent pauses and cursor movements occur on the checkout page, ultimately completing the order submission. The action parameters throughout the interaction sequence, such as scrolling speed and focus delay, exhibit non-fixed, probability-distributed randomness, effectively simulating real user actions.

[0074] S5: Instruct the virtual device to interact with the cross-border platform using the user interaction sequence, and to collect target data from the webpage content returned by the cross-border platform in response to the user interaction sequence.

[0075] The computer device instructs the virtual device to perform automated interaction and data acquisition tasks. This process begins with the computer device loading a generated user interaction sequence into the virtual device's operation instruction queue. Based on the interaction logic and timing relationships defined by this sequence, the virtual device initiates continuous and human-behavioral interactive operations with the target cross-border platform through its simulated input / output interfaces.

[0076] During the interactive operation, the cross-border platform's server responds to the series of operations, returning the corresponding web page content data stream. The virtual device receives and caches this data stream in real time, and the computer device then initiates the data acquisition process. The computer device receives the web page content data stream returned by the platform in real time. This data stream contains a complete Hypertext Markup Language (HTML) document and its associated Cascading Style Sheets (CSS) files and script files. The computer device starts its content parsing engine to perform structured processing on the received web page content. This processing includes building a Document Object Model (DOM) tree, executing embedded scripts to reconstruct dynamically generated content, and extracting rendering style information for key visual elements. The computer device locates target data elements in the structured document according to preset data acquisition rules. This location process employs a multi-path matching strategy, including precise location based on DOM paths, region recognition based on visual features, and context extraction based on semantic understanding. For text data, its text content and associated attributes are directly extracted; for multimedia data, its storage address and metadata information are recorded. The computer device standardizes and encapsulates the extracted target data. This process includes unifying the data format, supplementing the acquisition timestamp and data source identifier, performing data validity verification, and persistently storing the final data in the target database.

[0077] In one possible embodiment, S1: Generating hardware feature parameters based on the hardware feature statistics includes:

[0078] S11: The hardware feature statistics are parsed to obtain multiple statistical feature items.

[0079] Specifically, the computer device performs a process of parsing hardware characteristic statistics. This process first reads the raw hardware characteristic statistics stored in the database. This information exists in the form of a structured dataset, recording the distribution of various hardware attributes within a specific geographical area.

[0080] Computer equipment deconstructs and classifies the raw statistical information input. The process includes identifying different hardware attribute categories in the dataset and breaking down the aggregated statistical data into independent statistical features that characterize individual hardware attributes. Each statistical feature includes its corresponding feature name and its specific distribution data within the target region.

[0081] After parsing, the computer outputs a set of multiple independent statistical features. Each feature in this set clearly points to a specific hardware characteristic and provides a clear data foundation for the subsequent generation of feature parameters.

[0082] For example, a computer device reads raw data containing statistical information about the hardware characteristics of devices in the North American market. This data might be a composite record containing multiple fields such as screen resolution, operating system version, and memory capacity.

[0083] After analysis, the computer device broke down the composite record. Screen resolution was identified as a separate statistical feature, with 45% of the data being 1920*1080 pixels and 30% being 2560*1440 pixels. Simultaneously, the operating system version was identified as another separate statistical feature, with 60% being Windows 11 and 35% being Windows 10. Similarly, memory capacity was also separated as a separate statistical feature, with 50% being 16GB and 30% being 32GB.

[0084] Finally, the computer device outputs a clear list containing multiple independent statistical features such as screen resolution, operating system version, and memory capacity. Each feature is accompanied by its specific statistical distribution data, which is used for the next step of parameter generation.

[0085] S12: If the statistical feature item is discrete data, a value is randomly selected from multiple hardware feature values ​​of the statistical feature item as a hardware feature parameter using random sampling.

[0086] When computer devices process discrete statistical features, they execute a parameter generation process based on probability distributions. The input data for this process is the parsed discrete statistical features, which contain a finite set of candidate values ​​and the frequency or statistical weight of each candidate value.

[0087] The computer device first establishes a probability distribution model for the feature, associating each candidate hardware feature value with its statistical weight. Then, the computer device invokes its random number generation unit to perform a weighted random sampling operation based on the probability distribution. This sampling process ensures that the probability of each candidate value being selected is proportional to its actual statistical distribution frequency in the target area.

[0088] Through this process, the computer device determines a specific value from multiple candidate hardware feature values ​​and outputs this value as the final hardware feature parameter for this discrete feature. This parameter reflects both the statistical regularity of regional hardware distribution and includes an appropriate degree of randomness.

[0089] For example, a computer device processes the operating system version, a discrete statistical feature. The input data shows that the candidate value set for this feature and its statistical weights are: Android 10 50%, Android 11 30%, and Android 12 20%.

[0090] The computer device constructs a probability model based on this and performs a weighted random sampling. In this sampling, Android 10 has the highest statistical weight and is therefore most likely to be selected. Assuming that the sampling result happens to select Android 10, the computer device outputs "Android 10" as the hardware characteristic parameter of the operating system version. This parameter will be used in the subsequent configuration of the virtual device to ensure that it is consistent with the mainstream device environment in the target region in terms of this characteristic.

[0091] S13: If the statistical feature is continuous data, determine the hardware value range of the statistical feature, use a pseudo-random generator to generate a random value that conforms to a normal distribution and is located within the hardware value range, and use this random value as a hardware feature parameter.

[0092] When processing continuous statistical features, the computer equipment executes a parameter generation process based on a normal distribution model. The input data for this process is the parsed continuous statistical features, which contain numerical interval information characterizing the central tendency and dispersion of hardware attributes in the target region.

[0093] The computer device first analyzes the statistical distribution data of the feature to determine its core hardware value range. During processing, the computer device uses this value range as a basic constraint and calculates the corresponding mean and standard deviation parameters based on the normal distribution characteristics. Subsequently, the computer device calls a pseudo-random number generator to generate random values ​​according to the established normal distribution model, and uses boundary checks to ensure that the output values ​​strictly fall within the aforementioned hardware value range.

[0094] Through the above processing, the computer device outputs a specific value that conforms to the statistical laws of the target area and has reasonable random fluctuations. This value is the continuous hardware characteristic parameter.

[0095] For example, the continuous statistical characteristic of computer equipment battery capacity. Input data shows that the battery capacity of mainstream mobile devices in the target region is concentrated between 3800 mAh and 4100 mAh.

[0096] After determining the hardware value range, the computer device calculates a mean of approximately 3950 mAh and sets an appropriate standard deviation. A pseudo-random number generator generates random values ​​based on this normal distribution model, producing a result of 3975 mAh in one execution. After verification that this value falls within the predetermined range, the computer device outputs 3975 mAh as a hardware characteristic parameter of the battery capacity. This parameter reflects both the central tendency of battery capacity in the region and preserves reasonable natural fluctuations.

[0097] S14. Summarize the hardware feature parameters corresponding to each statistical feature item to obtain the final hardware feature parameters; the hardware feature parameters include: screen resolution, operating system version, number of CPU (Central Processing Unit) cores, memory size, graphics card model, driver version, and one or more of the installed font list.

[0098] The final integration process for hardware characteristic parameters is performed by the computer device. This process uses the independent hardware characteristic parameters generated in the preceding steps as input data, and these parameters correspond to different hardware attributes of the device.

[0099] The computer equipment, through its data integration module, organizes and assembles the various independent input parameters according to a predefined device characteristic data model. During processing, the computer equipment establishes a unified structured data object, collecting all characteristic parameters such as screen resolution, operating system version, CPU core count, and memory size according to their technical categories, and verifying the logical consistency between the parameters.

[0100] After integration, the computer device outputs a complete set of final hardware characteristic parameters. This set constitutes a complete hardware identity description of the virtual device and can serve as a direct basis for constructing the virtual device.

[0101] For example, the computer device summarizes the various individual parameters that have been generated. These parameters include screen resolution of 1920×1080, operating system version Android 10, number of CPU cores (8 cores), memory size of 8GB, graphics card model Adreno 660, driver version V512, and font list Arial and Times New Roman, etc.

[0102] The computer device integrates these parameters into a structured dataset according to the device characteristic data model. The final output complete hardware characteristic parameter set contains all the above characteristic parameters, fully describing a virtual device with a specific hardware configuration, and providing a comprehensive parameter foundation for subsequent virtual device instantiation.

[0103] This embodiment achieves automation and intelligence in generating hardware feature parameters through the above method, ensuring that the generated virtual devices highly match the statistical characteristics of the real device group in the target area at the hardware level, thereby enhancing the success rate and efficiency of data collection operations.

[0104] In one possible embodiment, S4: The step of inputting the user behavior dataset of the cross-border platform, the hardware feature parameters, and the network feature parameters into a pre-trained large language model to generate a user interaction sequence includes:

[0105] S41: Classify the user behavior dataset of the cross-border platform according to the operation scenario, convert the hardware feature parameters into device performance constraints, and map the network feature parameters into operation latency impact factors.

[0106] Specifically, the computer equipment performs a structured preprocessing process on multi-source input data. This process first performs contextual analysis on the user behavior dataset of the cross-border platform. By parsing the target page, business process, and operation sequence pattern of user operations, the computer equipment categorizes the raw behavioral data into predefined operation scenario categories, forming scenario classification results with clear semantics.

[0107] Subsequently, the computer equipment converts hardware characteristic parameters into device performance constraints. This conversion process analyzes the actual impact of each hardware parameter on interactive operations, transforming static hardware specification data into a dynamic description of operational capabilities and establishing a correspondence between hardware configuration and operational execution capabilities.

[0108] Device performance constraints refer to the set of objective limitations imposed on user interaction methods and capabilities by hardware configuration. These constraints are a quantitative description of their functional impact during interaction. The process of converting hardware characteristic parameters into device performance constraints is as follows: Computer devices use predefined mapping rules to transform static hardware specifications into dynamic operational capability indicators. For example, screen resolution parameters are converted into constraints on the operable precision of interface elements and viewport scrolling speed; the number of CPU cores and memory size are combined into the maximum parallel operation load threshold under complex page rendering tasks; and the combination of device model and graphics card model is converted into trajectory smoothness control parameters for graphics-intensive interactions (such as slider verification).

[0109] Simultaneously, the computer equipment maps network characteristic parameters into operational latency influencing factors. This mapping process, through quantitative analysis of the impact mechanism of network environment characteristics on operational timing, transforms network parameters into specific time latency constraints, thus forming limiting conditions for the rhythm of interactive operations.

[0110] The operation delay impact factor is a quantitative indicator of the delay effect introduced by network environment characteristics on the timing logic of interactive operations. This factor characterizes the calculable impact of network conditions on operation rhythm, response waiting time, and session continuity. The process of mapping network characteristic parameters to the operation delay impact factor is as follows: Based on the network transmission model, the computer equipment converts the original network parameters into timing control variables in the interaction sequence. For example, the network round-trip time parameter is mapped to the basic waiting time that must be maintained after a single operation command is issued; the network bandwidth parameter is mapped to the expected additional delay during the page resource loading phase; and the transmission control protocol window size and packet loss rate are jointly mapped to the minimum safe interval between consecutive data request operations.

[0111] For example, when computer devices analyze user behavior datasets, they categorize operation sequences such as keyword input, filter setting, and sorting method selection into product search scenarios, and operation sequences such as viewing product details, browsing user reviews, and comparing specifications into product evaluation scenarios.

[0112] In terms of hardware parameter conversion, computer equipment converts the 1920×1080 screen resolution into performance constraints that support fine-grained interface element operations, and converts the 8-core CPU configuration into computing power constraints that can handle complex page rendering.

[0113] Regarding network parameter mapping, the computer device maps a network round-trip latency of 100 milliseconds as a factor affecting the basic waiting time required for interface response, and maps a bandwidth parameter of 50 Mbps as a factor affecting the resource loading capability that supports smooth loading of high-definition images.

[0114] S42: Generate a comprehensive context data packet based on the scene semantics, the device performance constraints, and the operation delay impact factor.

[0115] Specifically, the computer equipment performs multi-dimensional environmental information fusion processing to generate a comprehensive context data packet. This process integrates the classified operational scenario semantics, the converted device performance constraints, and the operational latency impact factors generated by the mapping.

[0116] Computer devices achieve information fusion by constructing structured data objects, taking scene semantics as the core of the task objective, device performance constraints as the operational capability boundary, and operational latency influencing factors as the timing control benchmark. These three types of information are organized within a unified descriptive framework, forming a machine-readable data set that fully defines the operating environment of the virtual device.

[0117] For example, a computer device integrates semantic information about the product search scenario, constraints on the fine-grained manipulation capabilities of interface elements derived from a 1920×1080 screen resolution, and a baseline latency factor determined by a 100-millisecond network delay. This information is encapsulated into a structured data packet that explicitly describes all environmental parameters required to perform a product search task under specific hardware performance and network conditions.

[0118] S43: Input the comprehensive context data package into the pre-trained large language model and output a user interaction sequence, which includes multiple interaction operations, operation parameters of each interaction operation, and start timestamp of each interaction operation.

[0119] Specifically, the computer device inputs the comprehensive context data packet into a pre-trained large language model. This large language model is trained based on massive amounts of human interaction behavior data and has the ability to deeply understand the semantics of the operation scenario, accurately parse device performance constraints, and reasonably assess the impact of network latency.

[0120] After receiving the comprehensive context data packet, the model performs inference calculations through its internal multi-layer neural network. This calculation process simulates human decision-making logic and behavioral patterns under specific environments and task objectives, generating a coherent sequence of operations that meets the requirements of the scenario. The model output is a complete user interaction sequence, which not only defines multiple interactive operations arranged in a logical order, but also precisely sets specific behavioral parameters for each operation. Based on the operation delay impact factor and device performance constraints, it assigns an appropriate start timestamp to each operation, thus forming a set of operation instructions that are precisely arranged in the time dimension and anthropomorphized in the behavioral dimension.

[0121] For example, a computer device inputs a comprehensive contextual data packet describing "searching for goods on a medium-performance mobile device and in a high-latency network environment" into a large language model. After inference, the model may output the following sequence: at the start timetamp T0, the user performs a "tap the search box" operation with the parameter "brief press"; at the start timetamp T1 (T0 + 1200 milliseconds, simulating network latency and human reaction time), the user performs a "keyword input" operation with the parameter "input speed with random fluctuations"; at the start timetamp T2, the user performs a "click the search button" operation with the parameter "press duration 150 milliseconds". The entire sequence accurately simulates the complete search behavior that a real user might take under given constraints.

[0122] This embodiment transforms discrete environmental data into a machine-understandable comprehensive context using the above method, and leverages the reasoning capabilities of a large language model to generate anthropomorphic interaction sequences that are highly adapted to scene semantics, device performance, and network environment, significantly improving the realism of virtual device behavior and the success rate of data acquisition tasks.

[0123] In one possible embodiment, it also includes:

[0124] S6: If the cross-border platform is detected to return a verification code for the current interaction operation in the user interaction sequence, the operation parameters and start timestamp of the interaction operations that have not been executed in the user interaction sequence are randomly perturbed.

[0125] Specifically, random perturbations include random perturbations of operating parameters and random perturbations of the start timestamp. Random perturbations of operating parameters further include perturbations of operating trajectory and motion precision.

[0126] The computer device does not generate a linear trajectory from the current cursor position to the target position. Instead, it employs an algorithm that simulates human arm movements (such as one based on Bézier curves or an adaptive step size model). This algorithm generates a slightly curved or undulating non-linear path. This path typically starts with a small acceleration, moves fastest in the middle, slows down near the target, and may include minor overshoots and bounces. A series of intermediate coordinate points are injected at irregular time intervals along the generated path, forming a coherent but non-mechanical movement trajectory. In other words, it outputs a mouse movement trajectory that conforms to human kinematics, consisting of a sequence of multiple (x, y, t) coordinate points, replacing the simple target point in the original parameters.

[0127] Furthermore, the computer device does not precisely click the absolute center of the target. Instead, it randomly generates an offset around the center point of the target element using a two-dimensional normal distribution (i.e., Gaussian distribution) model. The vast majority of clicks fall within the central region of the element, but there is a certain probability of clicking on the edge, simulating human hand-eye coordination errors. For text input, random fluctuations are introduced in the input speed (character interval time), and common typing errors (such as occasional backspace deletion corrections) are simulated before the correct characters are entered. Finally, the output is either the final click coordinates with random but reasonable offsets, or a sequence of text inputs with speed fluctuations and minor error correction.

[0128] In one possible embodiment, the random perturbation of the operation parameters and start timestamp of the interaction operations not executed in the user interaction sequence includes:

[0129] Get the timestamp T_last of the last interactive operation before the verification code was triggered;

[0130] The corresponding delay value is determined based on the type of the verification code, and the starting base time T_base is obtained by adding the delay value to the timestamp T_last;

[0131] Calculate the time interval between two adjacent interactive operations among all unexecuted interactive operations, randomly select a scaling factor from a preset numerical range, and multiply the scaling factor by the time interval to obtain the base time interval.

[0132] The starting reference time is added to the basic time interval to obtain the new timestamp of the first interactive operation in the two interactive operations.

[0133] Specifically, this involves the temporal reconstruction process of the user interaction sequence executed by the computer device. This process begins by obtaining the timestamp corresponding to the last successfully executed interaction operation before the verification code is triggered, denoted as T_last.

[0134] The computer device determines a baseline delay value from a predefined policy configuration based on the specific type of the identified CAPTCHA. This delay value simulates the typical time required for a human user to recognize and respond to the CAPTCHA. The delay value is related to the complexity of the CAPTCHA; for example, the delay value for an image CAPTCHA is 8 seconds, and the delay value for a character CAPTCHA is 5 seconds. The computer device arithmetically adds the timestamp T_last to this delay value to calculate a new sequence start baseline time T_base. This time point marks the start moment when the virtual device resumes interactive operation after the CAPTCHA interruption.

[0135] For all unexecuted interactive operations, the computer device sequentially calculates the original time interval between every two adjacent operations. For each calculated time interval, the computer device randomly selects a scaling factor from a preset numerical range, multiplies this scaling factor by the original time interval, and thus generates a new base time interval. This process alters the rhythmic distribution of the original operations.

[0136] The computer device uses the aforementioned initial base time T_base as an initial reference, adds the first newly generated base time interval to it, and obtains a new timestamp for the first unexecuted interactive operation. The timestamps of subsequent operations are then determined one by one based on the new timestamp of the previous operation plus the corresponding new base time interval, thereby completing the rearrangement of the timestamps of the entire sequence of unexecuted operations.

[0137] For example, a computer device detects that a verification code is triggered when the timestamp T_last is 15000 milliseconds. Based on the verification code type, it determines that a 5-second delay is needed, and the computer device calculates the starting base time T_base to be 20000 milliseconds.

[0138] Assume the interval between the first unexecuted operation and the previous operation in the original sequence is 2000 milliseconds. The computer randomly selects a scaling factor of 1.2 from a preset range of 0.8 to 1.5, calculating a new base time interval of 2400 milliseconds. Therefore, the new timestamp for the first unexecuted operation is determined to be 20000 milliseconds plus 2400 milliseconds, which is 22400 milliseconds.

[0139] The second operation is spaced 1500 milliseconds from the first operation. After selecting a scaling factor of 0.9, the new interval is 1350 milliseconds. The new timestamp is then 22400 milliseconds plus 1350 milliseconds, which is 23750 milliseconds. In this way, the timestamps of the entire unexecuted sequence are systematically recalculated, forming a new operation rhythm.

[0140] In one possible embodiment, it also includes:

[0141] Detect whether there is sensitive data in the target data;

[0142] If so, determine the corresponding anonymization strategy based on the geographical location of the cross-border platform;

[0143] The sensitive data is processed using the anonymization strategy described above.

[0144] Specifically, computer equipment executes a data compliance processing flow. This flow first performs content scanning and feature matching on the collected target data to detect whether it contains sensitive data protected by law or platform policies. The detection process uses a predefined sensitive data feature library for pattern recognition to identify whether there are fields or content fragments in the target data that match the characteristics of sensitive data.

[0145] When sensitive data is confirmed to be present, the computer equipment retrieves anonymization strategies that match the data protection regulations of the jurisdiction based on the geographic location information of the target cross-border platform. This strategy clearly defines the technical processing standards to be adopted for different types of sensitive data.

[0146] The computer equipment then invokes the data processing algorithm corresponding to the strategy to perform anonymization transformation on the identified sensitive data. This process aims to eliminate the personal identification of the data while maximizing its statistical utility and business value in the target business analysis scenario.

[0147] For example, when a computer device collects user order data, it detects a field containing a personal mobile phone number through regular expression matching, and this field is identified as sensitive data.

[0148] Because the target platform is located in the European Union, the computer equipment was determined to need to adopt an anonymization strategy compatible with the EU's General Data Protection Regulation (GDPR). According to this strategy, the computer equipment performs hash encryption on the phone number field, replacing the original number with a fixed-length, irreversible ciphertext string.

[0149] For users' precise geographic coordinates, the same strategy may require spatial blurring, such as converting the coordinates into random locations within a one-kilometer radius, to protect personal privacy while still supporting regional-level business analytics.

[0150] This embodiment ensures that the data collection process strictly complies with the laws and regulations of different regions through automated sensitive data detection and geolocation-adaptive anonymization, effectively protecting user privacy. At the same time, by retaining the statistical characteristics of the data, it provides a compliant and usable data foundation for subsequent data analysis.

[0151] Please see Figure 3 As shown, in one embodiment, a data acquisition device for an environmental platform is provided, the device comprising:

[0152] The generation module 301 is used to determine the corresponding hardware feature statistics based on the geographical location of the cross-border platform when the device fingerprint update conditions are met, and to generate hardware feature parameters based on the hardware feature statistics.

[0153] The creation module 302 is used to create a virtual device by calling hardware resources in the hardware resource library according to the hardware feature parameters;

[0154] The configuration module 303 is used to determine the corresponding network feature statistics based on the geographical location of the cross-border platform, generate network feature parameters based on the network feature statistics, and configure the network feature parameters for the virtual device.

[0155] The model processing module 304 is used to input the user behavior dataset of the cross-border platform, the hardware feature parameters, and the network feature parameters into a pre-trained large language model to generate user interaction sequences.

[0156] The data acquisition module 305 is used to instruct the virtual device to perform interactive operations on the cross-border platform using the user interaction sequence, and to collect target data from the web page content returned by the cross-border platform in response to the user interaction sequence.

[0157] In one or more possible embodiments, generating hardware feature parameters based on the hardware feature statistics includes:

[0158] The hardware feature statistics are parsed to obtain multiple statistical feature items;

[0159] If the statistical feature is discrete data, a value is randomly selected from multiple hardware feature values ​​of the statistical feature as a hardware feature parameter using a random sampling method.

[0160] If the statistical feature is continuous data, determine the hardware value range of the statistical feature, use a pseudo-random generator to generate a random value that conforms to a normal distribution and is located within the hardware value range, and use this random value as a hardware feature parameter.

[0161] The final hardware characteristic parameters are obtained by summarizing the hardware characteristic parameters corresponding to each statistical characteristic item; the hardware characteristic parameters include: screen resolution, operating system version, number of CPU cores, memory size, graphics card model, driver version, and one or more of the installed font list.

[0162] In one or more possible embodiments, the step of inputting the user behavior dataset of the cross-border platform, the hardware feature parameters, and the network feature parameters into a pre-trained large language model to generate a user interaction sequence includes:

[0163] The user behavior dataset of the cross-border platform is classified according to the operation scenario, the hardware feature parameters are converted into device performance constraints, and the network feature parameters are mapped into operation latency impact factors.

[0164] A comprehensive context data packet is generated based on the scene semantics, the device performance constraints, and the operation delay impact factor.

[0165] The comprehensive context data package is input into a pre-trained large language model, which outputs a user interaction sequence. The user interaction sequence includes multiple interaction operations, operation parameters of each interaction operation, and start timestamp of each interaction operation.

[0166] In one or more possible embodiments, it also includes:

[0167] The perturbation module is used to randomly perturb the operation parameters and start timestamp of the interaction operations that have not been executed in the user interaction sequence if the cross-border platform returns a verification code for the current interaction operation in the user interaction sequence.

[0168] In one or more possible embodiments, the random perturbation of the operation parameters and start timestamp of the interaction operations not performed in the user interaction sequence includes:

[0169] Get the timestamp of the last interactive operation before the verification code is triggered;

[0170] The corresponding delay value is determined based on the type of the verification code, and the starting base time is obtained by adding the delay value to the timestamp;

[0171] Calculate the time interval between two adjacent interactive operations among all unexecuted interactive operations, randomly select a scaling factor from a preset numerical range, and multiply the scaling factor by the time interval to obtain the base time interval.

[0172] The starting reference time is added to the basic time interval to obtain the new timestamp of the first interactive operation in the two interactive operations.

[0173] In one or more possible embodiments, the device fingerprint update condition includes: periodically updating the device fingerprint.

[0174] In one or more possible embodiments, it also includes:

[0175] An anonymization module is used to detect whether sensitive data exists in the target data;

[0176] If so, determine the corresponding anonymization strategy based on the geographical location of the cross-border platform;

[0177] The sensitive data is processed using the anonymization strategy described above.

[0178] It should be noted that the above-described embodiment of the device 3, when executing the data acquisition method of the environmental platform, is only illustrated by the division of the above-described functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the above functions. In addition, the data acquisition device and the data acquisition method embodiment of the environmental platform provided in the above-described embodiment belong to the same concept, and the implementation process is detailed in the method embodiment, which will not be repeated here.

[0179] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0180] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a cross-border platform data acquisition method on the client side.

[0181] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, performs the following steps:

[0182] When the device fingerprint update conditions are met, the corresponding hardware feature statistics are determined based on the geographical location of the cross-border platform, and hardware feature parameters are generated based on the hardware feature statistics.

[0183] A virtual device is created by calling hardware resources in the hardware resource library based on the hardware characteristic parameters;

[0184] The system determines the corresponding network feature statistics based on the geographical location of the cross-border platform, generates network feature parameters based on the network feature statistics, and configures the network feature parameters for the virtual device.

[0185] The user behavior dataset of the cross-border platform, the hardware feature parameters, and the network feature parameters are input into a pre-trained large language model to generate user interaction sequences.

[0186] The virtual device is instructed to interact with the cross-border platform using the user interaction sequence, and to collect target data from the webpage content returned by the cross-border platform in response to the user interaction sequence.

[0187] This application utilizes a dual approach of environmental simulation and behavioral anthropomorphism to ensure the continuity and stability of data acquisition tasks.

[0188] In one embodiment, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program that, when executed by a processor, performs the following steps:

[0189] When the device fingerprint update conditions are met, the corresponding hardware feature statistics are determined based on the geographical location of the cross-border platform, and hardware feature parameters are generated based on the hardware feature statistics.

[0190] A virtual device is created by calling hardware resources in the hardware resource library based on the hardware characteristic parameters;

[0191] The system determines the corresponding network feature statistics based on the geographical location of the cross-border platform, generates network feature parameters based on the network feature statistics, and configures the network feature parameters for the virtual device.

[0192] The user behavior dataset of the cross-border platform, the hardware feature parameters, and the network feature parameters are input into a pre-trained large language model to generate user interaction sequences.

[0193] The virtual device is instructed to interact with the cross-border platform using the user interaction sequence, and to collect target data from the webpage content returned by the cross-border platform in response to the user interaction sequence.

[0194] This application utilizes a dual approach of environmental simulation and behavioral anthropomorphism to ensure the continuity and stability of data acquisition tasks.

[0195] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0196] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0197] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0198] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A data collection method of a cross-border platform, characterized in that, The method comprises: When a device fingerprint update condition is met, determining corresponding hardware feature statistical information according to a geographical position of a cross-border platform, and generating a hardware feature parameter according to the hardware feature statistical information; the generation of the hardware feature parameter according to the hardware feature statistical information comprises: analyzing the hardware feature statistical information to obtain a plurality of statistical feature items; if the statistical feature item is discrete data, a random sampling method is used to extract a value from a plurality of hardware feature values of the statistical feature item as a hardware feature parameter; if the statistical feature item is continuous data, a hardware value interval of the statistical feature item is determined, a pseudo-random generator is used to generate a random value conforming to a normal distribution within the hardware value interval, and the random value is taken as a hardware feature parameter; and the hardware feature parameters corresponding to the statistical feature items are summarized to obtain a final hardware feature parameter; the hardware feature parameter comprises one or more of a screen resolution, an operating system version, a CPU core number, a memory size, a graphics card model, a driver version, and a font list installed; A virtual device is created by calling a hardware resource in a hardware resource library according to the hardware feature parameter; Network feature statistical information corresponding to the cross-border platform is determined according to a geographical position of the cross-border platform, network feature parameters are generated according to the network feature statistical information, and the network feature parameters are configured for the virtual device; A user interaction sequence is generated by inputting a user behavior data set of the cross-border platform, the hardware feature parameters, and the network feature parameters into a pre-trained large language model; the inputting of the user behavior data set of the cross-border platform, the hardware feature parameters, and the network feature parameters into the pre-trained large language model to generate the user interaction sequence comprises: classifying the user behavior data set of the cross-border platform according to an operation scenario, converting the hardware feature parameters into device performance constraint conditions, and mapping the network feature parameters into operation delay influence factors; a comprehensive context data package is generated according to scene semantic information, the device performance constraint conditions, and the operation delay influence factors; the comprehensive context data package is input into the pre-trained large language model, and a user interaction sequence is output; the user interaction sequence comprises a plurality of interaction operations, operation parameters of each interaction operation, and a start timestamp of each interaction operation; the scene semantic information represents semantic description of a specific scene; The virtual device is instructed to interact with the cross-border platform using the user interaction sequence, and target data is obtained by collecting webpage content returned by the cross-border platform in response to the user interaction sequence.

2. The data collection method of a cross-border platform according to claim 1, wherein, Further comprising: If it is monitored that the cross-border platform returns a verification code in response to a current interaction operation in the user interaction sequence, the operation parameters and the start timestamp of an unexecuted interaction operation in the user interaction sequence are randomly disturbed.

3. The data collection method of a cross-border platform according to claim 2, wherein, The random disturbance of the operation parameters and the start timestamp of the unexecuted interaction operation in the user interaction sequence comprises: A timestamp of a last interaction operation before triggering of the verification code is obtained. determining a corresponding delay value according to the type of the verification code, and adding the delay value to the timestamp to obtain a starting reference time; calculating a time interval between two adjacent interactive operations in all unexecuted interactive operations, randomly selecting a scaling factor in a preset numerical interval, and multiplying the scaling factor by the time interval to obtain a basic time interval; adding the starting reference time and the basic time interval to obtain a new timestamp of a previous interactive operation in the two interactive operations.

4. The data collection method of a cross-border platform according to claim 2 or 3, characterized in that, The device fingerprint updating condition comprises: periodically updating the device fingerprint.

5. The method of claim 4, wherein, Further comprising: detecting whether sensitive data exists in the target data; if yes, determining a corresponding anonymization processing strategy according to the geographical position of the cross-border platform; processing the sensitive data by using the anonymization processing strategy.

6. A data acquisition device for a cross-border platform, characterized in that, The device comprises: a generation module configured to, when a device fingerprint updating condition is met, determine corresponding hardware feature statistical information according to the geographical position of the cross-border platform, and generate hardware feature parameters according to the hardware feature statistical information; the generation of the hardware feature parameters according to the hardware feature statistical information comprises: analyzing the hardware feature statistical information to obtain a plurality of statistical feature items; if the statistical feature item is discrete data, a random sampling method is used to extract a value from a plurality of hardware feature values of the statistical feature item as a hardware feature parameter; if the statistical feature item is continuous data, a hardware value interval of the statistical feature item is determined, a pseudo-random generator is used to generate a random value conforming to a normal distribution in the hardware value interval, and the random value is taken as a hardware feature parameter; the hardware feature parameters corresponding to the statistical feature items are summarized to obtain final hardware feature parameters; the hardware feature parameters comprise one or more of a screen resolution, an operating system version, a CPU core number, a memory size, a graphics card model, a driver version, and a font list installed; a creation module configured to create a virtual device by calling hardware resources in a hardware resource library according to the hardware feature parameters; a configuration module configured to determine corresponding network feature statistical information according to the geographical position of the cross-border platform, generate network feature parameters according to the network feature statistical information, and configure the network feature parameters for the virtual device. The model processing module is configured to input the user behavior data set of the cross-border platform, the hardware feature parameter, and the network feature parameter into a pre-trained large language model to generate a user interaction sequence; the inputting the user behavior data set of the cross-border platform, the hardware feature parameter, and the network feature parameter into the pre-trained large language model to generate the user interaction sequence comprises: classifying the user behavior data set of the cross-border platform according to operation scenarios, converting the hardware feature parameter into a device performance constraint condition, and mapping the network feature parameter into an operation delay influence factor; generating a comprehensive context data package according to scene semantic information, the device performance constraint condition, and the operation delay influence factor; inputting the comprehensive context data package into the pre-trained large language model to output the user interaction sequence, the user interaction sequence comprising a plurality of interaction operations, operation parameters of each interaction operation, and a start timestamp of each interaction operation; and the scene semantic information representing semantic description of a specific scene. The acquisition module is configured to instruct the virtual device to perform interaction operations on the cross-border platform by using the user interaction sequence, and to acquire target data from web page content returned by the cross-border platform in response to the user interaction sequence.

7. A computer device, comprising: The computer device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the data acquisition method of the cross-border platform according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the data acquisition method of the cross-border platform according to any one of claims 1 to 5.

Images