Data collection method and device based on screening conditions

Abstract

The present application provides a data collection method and device based on screening conditions, which relates to the field of data processing technology and solves the technical problem of low efficiency in collecting website data. The method includes: determining a request loading mode of a data collection request; if the request loading mode is static loading, then determining a target expression from multiple expressions by specifying a static screening condition according to a target collection format, a target collection content type, and a target interface protocol, and selecting, extracting, and collecting data from a target website through the target expression to obtain a first data collection result of the target website; if the request loading mode is dynamic loading, then simulating the collection behavior operation using an automated tool by specifying a dynamic screening condition to obtain page elements of the target website; identifying page elements based on the page elements of the target website through a CSS selector and an element selector to obtain the text content corresponding to the page elements.

Description

Technical Field

[0001] The present application relates to the technical field of data processing, and in particular to a method and device for collecting data based on screening conditions. Background Art

[0002] At present, it is often necessary to collect data from multiple different websites. For example, the website administrator needs to save some content from the department websites corresponding to multiple departments of the company to the server of the company headquarters, or download the data of multiple websites of multiple subsidiaries of the company to the website of the head office, and then extract the relevant fields from the content and publish them to the website system of the headquarters. Sometimes it is necessary to save the files related to the web page locally, such as pictures, attachments, etc. However, in the process of website data collection, the collection efficiency of website data is often low due to the different loading methods of collection requests. Summary of the invention

[0003] The purpose of the present invention is to provide a data collection method and device based on screening conditions to solve the technical problem of low efficiency in collecting website data.

[0004] In a first aspect, an embodiment of the present application provides a data collection method based on screening conditions, the method comprising:

[0005] Obtaining a data collection request for a target website, and determining a request loading mode of the data collection request; wherein the request loading mode includes static loading and dynamic loading, the dynamic loading corresponds to a specified dynamic screening condition, the static loading corresponds to a specified static screening condition, the specified static screening condition has a plurality of expressions corresponding to interface protocols corresponding to different collection formats, different collection content types, and different collection regions, and the plurality of expressions includes a regular expression;

[0006] If the requested loading mode is the static loading, then determining the target acquisition format, target acquisition content type and target interface protocol corresponding to the target acquisition region for the target website, determining a target expression from the multiple expressions through the specified static screening condition according to the target acquisition format, the target acquisition content type and the target interface protocol, and selecting, extracting and acquiring data for the target website through the target expression to obtain a first data acquisition result for the target website;

[0007] If the request loading mode is the dynamic loading, simulating the collection behavior operation by using an automated tool through the specified dynamic screening condition, and obtaining the page elements of the target website through the collection behavior operation; wherein the collection behavior operation includes at least one of a click operation, a slide operation, and a long press operation;

[0008] Based on the page elements of the target website, the page elements are identified through CSS selectors and element selectors to obtain the text content corresponding to the page elements, and the text content is used as the second data collection result of the target website; wherein the text content includes at least one of a text title, a text picture and a text text.

[0009] In a possible implementation, the step of simulating the acquisition behavior operation by using an automated tool through the specified dynamic screening condition includes:

[0010] Obtain historical operation data for terminal devices;

[0011] The plurality of historical operation data are classified and stored according to the length of the historical operation cursor motion track in the historical operation data, so that the historical operation data whose lengths meet the same specified length interval are stored in the same type, and different trajectory storage types corresponding to the plurality of different specified length intervals are obtained; wherein the historical operation data includes any one or more of the following: the curvature of the historical operation cursor motion track, the shape of the historical operation cursor motion track, the length of the historical operation cursor motion track, the running time of the historical operation cursor motion track, and the running rhythm of the historical operation cursor motion track;

[0012] Using the specified dynamic screening condition, using an automated tool to obtain the total length of the cursor movement corresponding to the acquisition behavior operation to be simulated;

[0013] Based on the total length of the cursor movement to be completed, multiple target historical operation cursor movement trajectories are selected from the multiple trajectory storage types corresponding to the multiple specified length intervals, and the multiple target historical operation cursor movement trajectories are spliced ​​to obtain a historical operation cursor movement trajectory splicing result that meets the total length of the cursor movement to be completed, and the historical operation cursor movement trajectory splicing result is used to simulate the collection behavior operation.

[0014] In a possible implementation, the step of splicing the plurality of target historical operation cursor motion trajectories to obtain a splicing result of historical operation cursor motion trajectories that meets the total length of the cursor to be moved includes:

[0015] If the length of the multiple target historical operation cursor motion trajectories after splicing is greater than the total length of the cursor to be moved, the first historical operation cursor motion trajectory splicing result after splicing the multiple target historical operation cursor motion trajectories is cropped based on the total length of the cursor to be moved to obtain a second historical operation cursor motion trajectory splicing result that meets the total length of the cursor to be moved.

[0016] In a possible implementation, after the step of classifying and storing the plurality of historical operation data according to the length of the historical operation cursor motion track in the historical operation data, the method further includes:

[0017] When new historical operation data is collected, a target length of a new historical operation cursor motion track in the new historical operation data and a target designated length interval that the target length corresponds to among the multiple designated length intervals are determined, and the new historical operation data replaces the old historical operation data stored in the target track storage type corresponding to the target designated length interval, so that the number of tracks corresponding to the historical operation data in the target track storage type remains unchanged; or

[0018] Periodically extract a certain percentage of the historical operation data from the plurality of track storage types corresponding to the plurality of specified length intervals for deletion, and fill the track storage type with newly collected historical operation data to update the historical operation data in the track storage type.

[0019] In a possible implementation, a place space corresponding to a local organization includes a plurality of branch terminal devices belonging to the local organization; the step of obtaining historical operation data for the terminal devices includes:

[0020] The local historical operation data of the local system target website corresponding to the local organization in the branch terminal device is collected, and the local historical operation data is transmitted to the server corresponding to the local organization.

[0021] In a possible implementation, the step of acquiring the page elements of the target website through the acquisition behavior operation includes:

[0022] Analyzing the vividness of the page display of the current page in the target website by using the AI ​​module in the automation tool, wherein the vividness of the page display includes any one or more of the following: the number of page illustrations displayed, the density of text typesetting, the number of page colors displayed, the quality of page illustrations displayed, and the quality of page color display;

[0023] If the page display vividness is lower than a preset display vividness threshold, reducing the page opening time of the current page, and acquiring the page elements of the target website in the current page through the simulated acquisition behavior operation;

[0024] If the page display vividness is higher than or equal to the preset display vividness threshold, the page opening time of the current page is increased, and the page elements of the target website in the current page are obtained through the simulated acquisition behavior operation.

[0025] In a possible implementation, the step of acquiring the page elements of the target website through the acquisition behavior operation further includes:

[0026] Determine whether the displayed content of the current page of the target website includes a bibliography and / or a document abstract; if the displayed content includes the bibliography and / or the document abstract, increase the page opening time of the current page, and obtain the page elements of the target website in the current page through the simulated acquisition behavior operation; and / or,

[0027] Determine whether the displayed content of the current page in the target website includes a data table and / or an experimental value. If the displayed content includes the data table and / or the experimental value, reduce the page opening time of the current page, and obtain the page elements of the target website in the current page through the simulated collection behavior operation.

[0028] In a second aspect, a data collection device based on screening conditions is provided, comprising:

[0029] an acquisition module, used to acquire a data collection request for a target website, and determine a request loading mode of the data collection request; wherein the request loading mode includes static loading and dynamic loading, the dynamic loading corresponds to a specified dynamic screening condition, the static loading corresponds to a specified static screening condition, the specified static screening condition has a plurality of expressions corresponding to different collection formats, different collection content types and interface protocols corresponding to different collection regions, and the plurality of expressions includes a regular expression;

[0030] a determination module, for determining, if the requested loading mode is the static loading, a target acquisition format, a target acquisition content type, and a target interface protocol corresponding to a target acquisition region for the target website, determining a target expression from a plurality of the expressions through the specified static screening condition according to the target acquisition format, the target acquisition content type, and the target interface protocol, and selecting, extracting, and acquiring data for the target website through the target expression to obtain a first data acquisition result for the target website;

[0031] a simulation module, configured to simulate a collection behavior operation by using an automated tool through the specified dynamic screening condition if the request loading mode is the dynamic loading, and obtain the page elements of the target website through the collection behavior operation; wherein the collection behavior operation includes at least one of a click operation, a slide operation, and a long press operation;

[0032] An identification module is used to identify the page elements of the target website through CSS selectors and element selectors, obtain the text content corresponding to the page elements, and use the text content as the second data collection result of the target website; wherein the text content includes at least one of a text title, a text picture, and a text text.

[0033] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program that can be executed on the processor, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0034] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to execute the method described in the first aspect above.

[0035] The embodiments of the present application bring the following beneficial effects:

[0036] The present application provides a method and device for data collection based on screening conditions, which can obtain a data collection request for a target website and determine a request loading mode for the data collection request; wherein the request loading mode includes static loading and dynamic loading, the dynamic loading corresponds to a specified dynamic screening condition, the static loading corresponds to a specified static screening condition, the specified static screening condition has a plurality of expressions corresponding to different collection formats, different collection content types and interface protocols corresponding to different collection areas, and the plurality of expressions include regular expressions; if the request loading mode is the static loading, the target collection format, target collection content type and target interface protocol corresponding to the target website are determined, and the target collection format, the target collection content type and the target interface protocol are determined through the specified The static screening condition determines the target expression from the multiple expressions, and selects, extracts and collects data for the target website through the target expression to obtain the first data collection result of the target website; if the request loading mode is the dynamic loading, the collection behavior operation is simulated by an automation tool through the specified dynamic screening condition, and the page element of the target website is obtained through the collection behavior operation; wherein, the collection behavior operation includes at least one of a click operation, a slide operation and a long press operation; based on the page element of the target website, the page element is identified through a CSS selector and an element selector to obtain the text content corresponding to the page element, and the text content is used as the second data collection result of the target website; wherein, the text content includes at least one of a text title, a text picture and a text text. In this scheme, by determining whether the request loading mode of the data collection request is static loading or dynamic loading, it is possible to determine a target expression that matches the target collection format, target collection content type and target collection area corresponding to the target website by specifying static screening conditions when the request loading mode is static loading. The efficiency of data selection, data extraction and data collection for the target website is improved through this target expression. Moreover, when the request loading mode is dynamic loading, it is also possible to simulate the collection behavior operation by specifying dynamic screening conditions to obtain the page elements of the target website and identify them, so as to more efficiently obtain the text content of the target website. Therefore, it is achieved that more appropriate different website data collection methods are implemented corresponding to different request loading methods, thereby improving the collection efficiency of website data and solving the technical problem of low collection efficiency of website data.

[0037] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 A schematic diagram of a process for a data collection method based on screening conditions provided in an embodiment of the present application;

[0040] Figure 2 Another schematic diagram of a process for collecting data based on screening conditions provided in an embodiment of the present application;

[0041] Figure 3 A schematic diagram of the structure of a data acquisition device based on screening conditions provided in an embodiment of the present application;

[0042] Figure 4 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0044] The terms "including" and "having" and any variations thereof mentioned in the embodiments of the present application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0045] At present, in the process of website data collection, the situation of low website data collection efficiency often occurs due to different collection request methods. Based on this, the embodiment of the present application provides a data collection method and device based on screening conditions, which can solve the technical problem of low website data collection efficiency.

[0046] The embodiments of the present invention are further described below in conjunction with the accompanying drawings.

[0047] Figure 1A flow chart of a data collection method based on screening conditions provided in an embodiment of the present application. Figure 1 As shown, the method includes:

[0048] Step S110, obtaining a data collection request for a target website, and determining a request loading method for the data collection request.

[0049] Among them, the request loading method includes static loading and dynamic loading. Dynamic loading corresponds to specified dynamic filtering conditions, and static loading corresponds to specified static filtering conditions. The specified static filtering conditions correspond to multiple expressions according to different collection formats, different collection content types, and different collection areas corresponding to the interface protocol, and the multiple expressions include regular expressions.

[0050] In the embodiment of the present application, the specified dynamic screening conditions and the specified static screening conditions correspond to the screening conditions of various dimensions in the website data collection process, namely, the static loading dimension and the static loading dimension. Different loading dimensions according to the request loading method correspond to different screening conditions in the website data collection process.

[0051] As an optional implementation method, data collection for target websites usually involves several key steps, including determining data requirements, selecting appropriate tools and technologies, and implementing data crawling. First, determine the data requirements, and clarify what data to collect from the target website. For example, you may need to collect product information, user reviews, news articles, etc. Clarifying data requirements will help with subsequent steps. Then, analyze the target website, that is, analyze the structure and content of the target website to determine the specific location of the data. This step can be done by using developer tools (such as the "Inspect" function of the browser). Then choose the appropriate data collection tool, that is, choose the appropriate data collection tool based on the characteristics and technical background of the target website. Common tools include: Python libraries: such as requests, BeautifulSoup, Scrapy; JavaScript libraries: such as Puppeteer, Cheerio; Commercial tools: such as Octoparse, ParseHub.

[0052] Then, determine the request loading method. Different websites may use different loading methods, mainly the following: a. Static pages. Static pages refer to pages whose contents are directly sent to the client browser after being generated on the server side. Such pages can directly obtain the complete content through HTTP requests. b. Dynamic pages. Dynamic pages refer to pages whose partial or complete contents are dynamically loaded on the client side through JavaScript. In this case, a simple HTTP request cannot obtain the complete content, and browser behavior needs to be simulated. c. AJAX requests. Some websites load data asynchronously through AJAX requests. In this case, these AJAX requests need to be found and simulated.

[0053] After that, data crawling is implemented. According to the above analysis and selected tools, a specific crawling script is written. Make sure to handle abnormal situations and anti-crawler mechanisms (such as setting a reasonable request interval, using proxy IP, etc.). Finally, test and debug, run the crawling script, test whether it can correctly obtain the required data, and debug and optimize according to the test results.

[0054] Step S120, if the requested loading method is static loading, determine the target collection format, target collection content type and target interface protocol corresponding to the target collection area for the target website, determine the target expression from multiple expressions by specifying static screening conditions according to the target collection format, target collection content type and target interface protocol, and select, extract and collect data for the target website through the target expression to obtain the first data collection result of the target website.

[0055] In an optional implementation, different expressions may be used to collect data based on aspects such as the data collection format for the target website, the interface protocol corresponding to the collection area, and the content subject type collected. Specifically, a target expression that matches the above-mentioned collection format, interface protocol, and content subject type is selected from multiple expressions corresponding to the above-mentioned specified static screening conditions.

[0056] As an example, if the request loading mode is static loading, data is collected and extracted through regular expressions, for example, expressions are selected and data is collected through jsonpathg expressions, xpath expressions, etc.

[0057] As a possible implementation method, first determine the target acquisition format, which usually refers to the format in which the captured data is to be stored or processed. Common formats include: CSV: suitable for tabular data; JSON: suitable for structured data; XML: suitable for complex data structures; databases: such as MySQL, MongoDB, etc. Then, determine the target acquisition content type, which refers to the specific type of information to be captured from the website. For example, text: articles, comments, descriptions, etc.; pictures: product pictures, avatars, etc.; links: related links, download links, etc.; metadata: timestamps, author information, etc. Then determine the target acquisition region. If data for a specific region needs to be collected, it can be filtered by the regional identifier in the URL or the regional information on the page. For example, some websites may have different language versions or regional versions. Then determine the target interface protocol. For statically loaded websites, HTTP / HTTPS protocols are usually used for data requests. Make sure that the request headers are set correctly to simulate real user access. Finally, determine the target expression by specifying static filtering conditions. According to the target acquisition format, content type and region, select the appropriate expression for data selection, extraction and collection. Commonly used tools and techniques include BeautifulSoup and regular expressions.

[0058] Step S130, if the requested loading mode is dynamic loading, then the collection behavior operation is simulated by using an automation tool by specifying dynamic screening conditions, and the page elements of the target website are obtained through the collection behavior operation.

[0059] The acquisition behavior operation includes at least one of a click operation, a slide operation, and a long press operation. For example, if the request loading mode is dynamic loading, an automated tool is used to simulate human click operations, slide operations, long press operations, etc., so as to achieve more efficient acquisition of page elements of the target website.

[0060] As an example, when the request loading method is dynamic loading, the web page content is usually generated by client-side rendering through JavaScript. This means that the complete page content cannot be obtained directly using traditional HTTP requests (such as the requests library). At this time, you need to use an automated tool that can execute JavaScript to simulate user behavior and capture dynamically loaded content. Commonly used tools include Selenium, Puppeteer, etc. The following uses Selenium as an example to illustrate how to use automated tools for data collection by specifying dynamic filtering conditions: First, install the necessary libraries and ensure that Selenium and WebDriver are installed. Then, write a data collection script for dynamically loaded pages. For example, first import the required libraries, then set up WebDriver, then visit the target website, and wait for the page to load. Then, simulate scrolling or clicking operations. For example, some pages may require scrolling to the bottom or clicking the "Load More" button to load more content. After that, extract the page elements, save the results, and finally close WebDriver.

[0061] Step S140, based on the page elements of the target website, the page elements are identified by using CSS selectors and element selectors, the text content corresponding to the page elements is obtained, and the text content is used as the second data collection result of the target website.

[0062] The main content includes at least one of a main title, a main image, and a main text.

[0063] In actual applications, CSS selectors and / or element selectors can be used to identify the corresponding text content in the above-mentioned website page elements, for example, the corresponding text title, corresponding text image, etc. in the above-mentioned website page elements, so as to collect the text content, thereby realizing automatic collection of website data and improving collection efficiency.

[0064] As an optional implementation, in order to identify page elements based on the target website's page elements through CSS selectors and element selectors and obtain the body content, you can continue to use Selenium to complete this task. The following is a complete sample code that shows how to extract the body content of page elements through CSS selectors and element selectors, and use it as the second data collection result. First, import the required libraries, set up WebDriver, and then visit the target website, wait for the page to load, and then use CSS selectors and element selectors to extract the body content. Suppose you want to extract the title, author, publication date, and body content of the article to locate these elements through CSS selectors and element selectors. Finally, save the extracted content in JSON format and close WebDriver.

[0065] In an embodiment of the present application, by determining whether the request loading mode of the data acquisition request is static loading or dynamic loading, it is possible to determine a target expression that matches the target acquisition format, target acquisition content type, and target acquisition area of ​​the target website by specifying static screening conditions when the request loading mode is static loading. The efficiency of data selection, data extraction, and data acquisition for the target website is improved through the target expression. Moreover, when the request loading mode is dynamic loading, it is also possible to simulate the acquisition behavior operation by specifying dynamic screening conditions to obtain the page elements of the target website and identify them, thereby more efficiently obtaining the text content of the target website. Therefore, it is achieved that different more appropriate website data acquisition methods are implemented corresponding to different request loading modes, thereby improving the efficiency of collecting website data.

[0066] The above steps are described in detail below.

[0067] In some embodiments, in the above step S130, the process of simulating the collection behavior operation by using an automated tool by specifying dynamic screening conditions is as follows: Figure 2 As shown, the following steps may be specifically included:

[0068] Step S210, obtaining historical operation data for the terminal device;

[0069] Step S220, classifying and storing the plurality of historical operation data according to the length of the historical operation cursor motion track in the historical operation data, so as to store the historical operation data whose lengths meet the same specified length interval into the same type, and obtain different track storage types corresponding to the plurality of different specified length intervals; wherein the historical operation data includes any one or more of the following: the curvature of the historical operation cursor motion track, the shape of the historical operation cursor motion track, the length of the historical operation cursor motion track, the running time of the historical operation cursor motion track, and the running rhythm of the historical operation cursor motion track;

[0070] Step S230, obtaining the total length of the cursor movement corresponding to the acquisition behavior operation to be simulated by using an automated tool by specifying a dynamic screening condition;

[0071] Step S240, based on the total length of the cursor movement to be waited, select multiple target historical operation cursor movement trajectories from multiple trajectory storage types corresponding to multiple specified length intervals, and splice the multiple target historical operation cursor movement trajectories to obtain the historical operation cursor movement trajectory splicing result that meets the total length of the cursor movement to be waited, and use the historical operation cursor movement trajectory splicing result to simulate the collection behavior operation.

[0072] As an example, the historical data of various historical operations of the computer can be recorded first, and the historical data includes: the curvature of the cursor's motion trajectory, the shape of the cursor's motion trajectory, the length of the cursor's motion trajectory, the running time of the motion trajectory, the running rhythm of the motion trajectory, etc. Then, all motion trajectories are stored in a database according to the length of the trajectory, forming multiple cursor motion trajectory schemes that meet the real human effect. For example, the trajectory with a curve length of 5-10 cm is the first-grade motion trajectory, and the trajectory with a curve length of 10cm-15cm is the second-grade motion trajectory.

[0073] For example, when performing a formal data acquisition operation, at least one motion track of the grade is selected and called from the motion tracks stored in the grades according to the total length of the cursor to be moved, and the total length of the cursor to be moved is formed by calling the at least one selected motion track and splicing the tracks. For example, if the total length of the cursor to be moved is 55 cm, then 5 10 cm motion tracks are selected from the second grade motion track (curve length is 10 cm-15 cm), and then 1 5 cm motion track is selected from the first grade motion track (curve length is 5-10 cm), and they are spliced ​​together to form a motion track with a curve length of 55 cm.

[0074] By simulating various human operations, it is possible to avoid the system-controlled cursor movement trajectory being horizontal and vertical, which would result in the inability to effectively simulate real human operations, thereby avoiding the website's corresponding server from determining that the current reading is machine reading rather than real human reading, which would affect the machine's collection of website data.

[0075] In some embodiments, the process of splicing a plurality of target historical operation cursor motion trajectories to obtain a splicing result of historical operation cursor motion trajectories that meets the total length of the target cursor motion may specifically include the following steps:

[0076] If the length of multiple target historical operation cursor motion trajectories after splicing is greater than the total length of the cursor to be moved, the first historical operation cursor motion trajectory splicing result after splicing the multiple target historical operation cursor motion trajectories is cropped based on the total length of the cursor to be moved to obtain a second historical operation cursor motion trajectory splicing result that meets the total length of the cursor to be moved.

[0077] In practical applications, if the length of the trajectory curve formed by splicing and combining together (such as 60 cm) is longer than the total length of the cursor to be moved (such as 55 cm), the trajectory formed by splicing and combining together can be cut.

[0078] For example, if the total length of the cursor to be moved is 52 cm, then five 10 cm motion trajectories are selected from the second-grade motion trajectory (curve length is 10 cm-15 cm), and then one 5 cm motion trajectory is selected from the first-grade motion trajectory (curve length is 5-10 cm), and they are spliced ​​together to form a motion trajectory with a curve length of 55 cm, and then the motion trajectory with a curve length of 55 cm is trimmed by 3 cm to form a motion trajectory with a curve length of 52 cm.

[0079] By calling at least one selected motion trajectory and combining the trajectory splicing process with the trajectory cutting process, a cursor motion trajectory splicing result with a more accurate value that meets the total length of the cursor to be moved can be formed.

[0080] In some embodiments, after the above step of classifying and storing a plurality of historical operation data according to the length of the historical operation cursor motion track in the historical operation data, the method may further include any one of the following two steps:

[0081] When new historical operation data is collected, a target length of a new historical operation cursor motion track in the new historical operation data and a target specified length interval that the target length meets among multiple specified length intervals are determined, and the new historical operation data replaces the old historical operation data stored in the target track storage type corresponding to the target specified length interval, so that the number of tracks corresponding to the historical operation data in the target track storage type remains unchanged;

[0082] Periodically extract a certain percentage of historical operation data from multiple track storage types corresponding to multiple specified length intervals for deletion, and fill the track storage type with newly collected historical operation data to update the historical operation data in the track storage type.

[0083] As an example, a certain percentage of historical operation data can be randomly extracted from the above database for deletion at regular intervals, that is, old historical operation data that has been stored for a long time can be eliminated, and then the newly collected historical operation data can be updated and stored to ensure the fluidity and flexibility of the historical operation data in the database.

[0084] As another example, when new historical operation data is collected, it replaces the old historical operation data in the storage file of the corresponding length, so that each storage file is kept with a sufficient amount of the latest collected historical operation data.

[0085] In the embodiment of the present application, by timely updating the historical operation data in the trajectory storage type, the liquidity and flexibility of the historical operation data in the trajectory storage type can be improved, and the same historical operation trajectory can be avoided from being reused multiple times, which affects the real simulation effect of human operation.

[0086] In some embodiments, the venue space corresponding to the local organization includes a plurality of branch terminal devices belonging to the local organization; the above process of obtaining the historical operation data for the terminal device may specifically include the following steps:

[0087] The local historical operation data of the local system target website corresponding to the local organization in the branch terminal device is collected, and the local historical operation data is transmitted to the server corresponding to the local organization.

[0088] For example, in the process of recording various historical operations of a computer, the historical data of the historical operations can be the historical operation data on the company's system website, that is, the historical operation data collected by the company's system on the local website, which are uploaded to the company's server for aggregation to avoid the historical operation data obtained by the company's system involving personal operation data, thereby protecting the data privacy of personal operation data.

[0089] In some embodiments, the process of acquiring page elements of the target website through the collection behavior operation in the above step S130 may specifically include the following steps:

[0090] Analyze the vividness of the page display of the current page in the target website by using the AI ​​module in the automation tool, wherein the vividness of the page display includes any one or more of the following: the number of page illustrations, the density of text layout, the number of page colors, the quality of page illustrations, and the quality of page colors;

[0091] If the page display vividness is lower than the preset display vividness threshold, the page opening time of the current page is reduced, and the page elements of the target website in the current page are obtained through simulated collection behavior operations;

[0092] If the page display vividness is higher than or equal to the preset display vividness threshold, the page opening time of the current page is increased, and the page elements of the target website in the current page are obtained through simulated collection behavior operations.

[0093] As an optional implementation, AI is used to judge the vividness of the article. For example, when there are no pictures or few pictures on the current page, the text layout on the current page is too dense, the number of colors on the current page is small, the image quality is low, etc., it is determined that the article is less vivid, that is, the article is more boring. For another example, when there are many pictures on the current page, the text layout on the current page is not dense, the number of colors on the current page is rich, the image quality is high, etc., it can be determined that the article is more vivid, that is, the article is less boring.

[0094] When the above-mentioned article is less vivid (i.e., more boring), the reading time of the current page, i.e., the display time of the current page, can be appropriately reduced to simulate people's reading habits, that is, when faced with an article with less vividness, people ignore a lot of boring content and read it at a faster speed.

[0095] When the above-mentioned article is more vivid (that is, the article is less boring), the reading time of the current page, that is, the display time of the current page, can be appropriately increased to simulate people's reading habits, that is, when faced with an article with a higher degree of vividness, people will spend a longer time reading and staying on the current page.

[0096] In the process of simulating human reading habits, the shortening or prolonging of the page dwell time mentioned above can avoid the website's corresponding server from determining that the current reading is machine reading rather than real human reading, thereby affecting the machine's collection of website data.

[0097] In some embodiments, the process of acquiring page elements of the target website through the collection behavior operation in the above step S130 may also include at least one of the following steps:

[0098] Determine whether the displayed content of the current page of the target website includes a bibliography and / or a document abstract. If the displayed content includes a bibliography and / or a document abstract, increase the page opening time of the current page, and obtain the page elements of the target website in the current page through a simulated acquisition behavior operation;

[0099] Determine whether the displayed content of the current page in the target website includes data tables and / or experimental values. If the displayed content includes data tables and / or experimental values, reduce the page opening time of the current page, and obtain the page elements of the target website in the current page through simulated collection behavior operations.

[0100] For example, in the process of simulating people's reading habits, the dwell time of pages such as bibliography and abstract can be appropriately extended; the dwell time of pages with many data tables can be appropriately shortened.

[0101] In the embodiment of the present application, the above-mentioned processing method of shortening or extending the page dwell time can avoid the same opening time between each page, so as to simulate human reading habits and avoid the website corresponding server determining that the current reading is machine reading rather than real human reading, which affects the machine's collection of website data.

[0102] Figure 3 A structural schematic diagram of a data acquisition device based on screening conditions is provided. Figure 3 As shown, the data collection device 300 based on the screening condition includes:

[0103] The acquisition module 301 is used to acquire a data collection request for a target website and determine a request loading mode of the data collection request; wherein the request loading mode includes static loading and dynamic loading, the dynamic loading corresponds to a specified dynamic screening condition, the static loading corresponds to a specified static screening condition, the specified static screening condition has a plurality of expressions corresponding to different collection formats, different collection content types and interface protocols corresponding to different collection regions, and the plurality of expressions includes a regular expression;

[0104] The determination module 302 is used to determine the target acquisition format, target acquisition content type and target interface protocol corresponding to the target acquisition region for the target website if the requested loading mode is the static loading, determine the target expression from the multiple expressions according to the target acquisition format, the target acquisition content type and the target interface protocol through the specified static screening condition, and select, extract and acquire data for the target website through the target expression to obtain the first data acquisition result of the target website;

[0105] The simulation module 303 is used to simulate the collection behavior operation by using an automation tool according to the specified dynamic screening condition if the request loading mode is the dynamic loading, and obtain the page elements of the target website through the collection behavior operation; wherein the collection behavior operation includes at least one of a click operation, a slide operation and a long press operation;

[0106] The identification module 304 is used to identify the page elements of the target website through CSS selectors and element selectors, obtain the text content corresponding to the page elements, and use the text content as the second data collection result of the target website; wherein the text content includes at least one of a text title, a text picture and a text text.

[0107] In some embodiments, the simulation module is specifically used to:

[0108] Obtain historical operation data for terminal devices;

[0109] The plurality of historical operation data are classified and stored according to the length of the historical operation cursor motion track in the historical operation data, so that the historical operation data whose lengths meet the same specified length interval are stored in the same type, and different trajectory storage types corresponding to the plurality of different specified length intervals are obtained; wherein the historical operation data includes any one or more of the following: the curvature of the historical operation cursor motion track, the shape of the historical operation cursor motion track, the length of the historical operation cursor motion track, the running time of the historical operation cursor motion track, and the running rhythm of the historical operation cursor motion track;

[0110] Using the specified dynamic screening condition, using an automated tool to obtain the total length of the cursor movement corresponding to the acquisition behavior operation to be simulated;

[0111] Based on the total length of the cursor movement to be completed, multiple target historical operation cursor movement trajectories are selected from the multiple trajectory storage types corresponding to the multiple specified length intervals, and the multiple target historical operation cursor movement trajectories are spliced ​​to obtain a historical operation cursor movement trajectory splicing result that meets the total length of the cursor movement to be completed, and the historical operation cursor movement trajectory splicing result is used to simulate the collection behavior operation.

[0112] In some embodiments, the simulation module is further configured to:

[0113] If the length of the multiple target historical operation cursor motion trajectories after splicing is greater than the total length of the cursor to be moved, the first historical operation cursor motion trajectory splicing result after splicing the multiple target historical operation cursor motion trajectories is cropped based on the total length of the cursor to be moved to obtain a second historical operation cursor motion trajectory splicing result that meets the total length of the cursor to be moved.

[0114] In some embodiments, the simulation module is further configured to:

[0115] After the plurality of historical operation data are classified and stored according to the lengths of the historical operation cursor motion trajectories in the historical operation data, when new historical operation data is collected, a target length of a new historical operation cursor motion trajectory in the new historical operation data and a target designated length interval that the target length meets in the plurality of designated length intervals are determined, and the new historical operation data replaces the old historical operation data stored in the target trajectory storage type corresponding to the target designated length interval, so that the number of trajectories corresponding to the historical operation data in the target trajectory storage type remains unchanged; or, a certain percentage of the historical operation data is periodically extracted from the plurality of trajectory storage types corresponding to the plurality of designated length intervals for deletion, and the newly collected historical operation data is filled in the trajectory storage type to update the historical operation data in the trajectory storage type.

[0116] In some embodiments, the venue space corresponding to the local organization includes a plurality of branch terminal devices belonging to the local organization; the simulation module is specifically used for:

[0117] The local historical operation data of the local system target website corresponding to the local organization in the branch terminal device is collected, and the local historical operation data is transmitted to the server corresponding to the local organization.

[0118] In some embodiments, the simulation module is specifically used to:

[0119] Analyzing the vividness of the page display of the current page in the target website by using the AI ​​module in the automation tool, wherein the vividness of the page display includes any one or more of the following: the number of page illustrations displayed, the density of text typesetting, the number of page colors displayed, the quality of page illustrations displayed, and the quality of page color display;

[0120] If the page display vividness is lower than a preset display vividness threshold, reducing the page opening time of the current page, and acquiring the page elements of the target website in the current page through the simulated acquisition behavior operation;

[0121] If the page display vividness is higher than or equal to the preset display vividness threshold, the page opening time of the current page is increased, and the page elements of the target website in the current page are obtained through the simulated acquisition behavior operation.

[0122] In some embodiments, the simulation module is further configured to:

[0123] Determine whether the displayed content of the current page of the target website includes a bibliography and / or a document abstract; if the displayed content includes the bibliography and / or the document abstract, increase the page opening time of the current page, and obtain the page elements of the target website in the current page through the simulated acquisition behavior operation; and / or,

[0124] Determine whether the displayed content of the current page in the target website includes a data table and / or an experimental value. If the displayed content includes the data table and / or the experimental value, reduce the page opening time of the current page, and obtain the page elements of the target website in the current page through the simulated collection behavior operation.

[0125] The data collection device based on screening conditions provided in the embodiment of the present application has the same technical features as the data collection method based on screening conditions provided in the above-mentioned embodiment, so it can also solve the same technical problems and achieve the same technical effects.

[0126] An electronic device provided in an embodiment of the present application is Figure 4 As shown, the electronic device 400 includes a processor 402 and a memory 401, wherein the memory stores a computer program that can be run on the processor, and the processor implements the steps of the method provided in the above embodiment when executing the computer program.

[0127] See also Figure 4The electronic device further includes: a bus 403 and a communication interface 404, a processor 402, a communication interface 404 and a memory 401 are connected via the bus 403; the processor 402 is used to execute an executable module stored in the memory 401, such as a computer program.

[0128] The memory 401 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 404 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0129] The bus 403 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0130] Among them, the memory 401 is used to store programs, and the processor 402 executes the program after receiving the execution instruction. The method executed by the device defined by the process disclosed in any embodiment of the present application can be applied to the processor 402 or implemented by the processor 402.

[0131] The processor 402 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 402. The above processor 402 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor are combined to execute. The software module may be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 401, and the processor 402 reads the information in the memory 401 and completes the steps of the above method in combination with its hardware.

[0132] Corresponding to the above-mentioned data collection method based on screening conditions, an embodiment of the present application also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to execute the steps of the above-mentioned data collection method based on screening conditions.

[0133] The data acquisition device based on the screening condition provided in the embodiment of the present application can be specific hardware on the device or software or firmware installed on the device. The device provided in the embodiment of the present application, its implementation principle and the technical effect produced are the same as those in the aforementioned method embodiment. For the sake of brief description, the parts not mentioned in the device embodiment can refer to the corresponding contents in the aforementioned method embodiment. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

[0134] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0135] For another example, the flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the device, method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0136] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0137] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0138] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a terminal device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the data acquisition method based on screening conditions described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0139] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0140] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solution of the present application, rather than to limit it. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the aforementioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solution recorded in the aforementioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solution deviate from the scope of the technical solution of the embodiment of the present application. They should all be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A data collection method based on screening conditions, characterized in that: The method comprises: Obtaining a data collection request for a target website, and determining a request loading mode of the data collection request; wherein the request loading mode includes static loading and dynamic loading, the dynamic loading corresponds to a specified dynamic screening condition, the static loading corresponds to a specified static screening condition, the specified static screening condition has a plurality of expressions corresponding to interface protocols corresponding to different collection formats, different collection content types, and different collection regions, and the plurality of expressions includes a regular expression; If the requested loading mode is the static loading, then determining the target acquisition format, target acquisition content type and target interface protocol corresponding to the target acquisition region for the target website, determining a target expression from the multiple expressions through the specified static screening condition according to the target acquisition format, the target acquisition content type and the target interface protocol, and selecting, extracting and acquiring data for the target website through the target expression to obtain a first data acquisition result for the target website; If the request loading mode is the dynamic loading, simulating the collection behavior operation by using an automated tool through the specified dynamic screening condition, and obtaining the page elements of the target website through the collection behavior operation; wherein the collection behavior operation includes at least one of a click operation, a slide operation, and a long press operation; Based on the page elements of the target website, the page elements are identified by using CSS selectors and element selectors to obtain the text content corresponding to the page elements, and the text content is used as the second data collection result of the target website; wherein the text content includes at least one of a text title, a text picture, and a text text; The step of simulating the acquisition behavior operation by using an automated tool through the specified dynamic screening condition comprises: Obtain historical operation data for terminal devices; The plurality of historical operation data are classified and stored according to the length of the historical operation cursor motion track in the historical operation data, so that the historical operation data whose lengths meet the same specified length interval are stored in the same type, and different trajectory storage types corresponding to the plurality of different specified length intervals are obtained; wherein the historical operation data includes any one or more of the following: the curvature of the historical operation cursor motion track, the shape of the historical operation cursor motion track, the length of the historical operation cursor motion track, the running time of the historical operation cursor motion track, and the running rhythm of the historical operation cursor motion track; Using the specified dynamic screening condition, using an automated tool to obtain the total length of the cursor movement corresponding to the acquisition behavior operation to be simulated; Based on the total length of the cursor movement to be completed, multiple target historical operation cursor movement trajectories are selected from the multiple trajectory storage types corresponding to the multiple specified length intervals, and the multiple target historical operation cursor movement trajectories are spliced ​​to obtain a historical operation cursor movement trajectory splicing result that meets the total length of the cursor movement to be completed, and the historical operation cursor movement trajectory splicing result is used to simulate the collection behavior operation.

2. The data collection method based on screening conditions according to claim 1, characterized in that: The step of splicing the plurality of target historical operation cursor motion trajectories to obtain a splicing result of historical operation cursor motion trajectories that meets the total length of the cursor to be moved includes: If the length of the multiple target historical operation cursor motion trajectories after splicing is greater than the total length of the cursor to be moved, the first historical operation cursor motion trajectory splicing result after splicing the multiple target historical operation cursor motion trajectories is cropped based on the total length of the cursor to be moved to obtain a second historical operation cursor motion trajectory splicing result that meets the total length of the cursor to be moved.

3. The data collection method based on screening conditions according to claim 1, characterized in that: After the step of classifying and storing the plurality of historical operation data according to the length of the historical operation cursor motion track in the historical operation data, the method further includes: When new historical operation data is collected, a target length of a new historical operation cursor motion track in the new historical operation data and a target designated length interval that the target length corresponds to among the multiple designated length intervals are determined, and the new historical operation data replaces the old historical operation data stored in the target track storage type corresponding to the target designated length interval, so that the number of tracks corresponding to the historical operation data in the target track storage type remains unchanged; or Periodically extract a certain percentage of the historical operation data from the plurality of track storage types corresponding to the plurality of specified length intervals for deletion, and fill the track storage type with newly collected historical operation data to update the historical operation data in the track storage type.

4. The data collection method based on screening conditions according to claim 1, characterized in that: The place space corresponding to the local organization includes a plurality of branch terminal devices belonging to the local organization; The step of obtaining historical operation data for the terminal device includes: The local historical operation data of the local system target website corresponding to the local organization in the branch terminal device is collected, and the local historical operation data is transmitted to the server corresponding to the local organization.

5. The data collection method based on screening conditions according to claim 1, characterized in that: The step of acquiring the page elements of the target website through the acquisition behavior operation includes: Analyzing the vividness of the page display of the current page in the target website by using the AI ​​module in the automation tool, wherein the vividness of the page display includes any one or more of the following: the number of page illustrations displayed, the density of text typesetting, the number of page colors displayed, the quality of page illustrations displayed, and the quality of page color display; If the page display vividness is lower than a preset display vividness threshold, reducing the page opening time of the current page, and acquiring the page elements of the target website in the current page through the simulated acquisition behavior operation; If the page display vividness is higher than or equal to the preset display vividness threshold, the page opening time of the current page is increased, and the page elements of the target website in the current page are obtained through the simulated acquisition behavior operation.

6. The data collection method based on screening conditions according to claim 5, characterized in that: The step of acquiring the page elements of the target website through the acquisition behavior operation also includes: Determine whether the displayed content of the current page of the target website includes a bibliography and / or a document abstract; if the displayed content includes the bibliography and / or the document abstract, increase the page opening time of the current page, and obtain the page elements of the target website in the current page through the simulated acquisition behavior operation; and / or, Determine whether the displayed content of the current page in the target website includes a data table and / or an experimental value. If the displayed content includes the data table and / or the experimental value, reduce the page opening time of the current page, and obtain the page elements of the target website in the current page through the simulated collection behavior operation.

7. A data collection device based on screening conditions, characterized in that: include: an acquisition module, used to acquire a data collection request for a target website, and determine a request loading mode of the data collection request; wherein the request loading mode includes static loading and dynamic loading, the dynamic loading corresponds to a specified dynamic screening condition, the static loading corresponds to a specified static screening condition, the specified static screening condition has a plurality of expressions corresponding to different collection formats, different collection content types and interface protocols corresponding to different collection regions, and the plurality of expressions includes a regular expression; a determination module, for determining, if the requested loading mode is the static loading, a target acquisition format, a target acquisition content type, and a target interface protocol corresponding to a target acquisition region for the target website, determining a target expression from a plurality of the expressions through the specified static screening condition according to the target acquisition format, the target acquisition content type, and the target interface protocol, and selecting, extracting, and acquiring data for the target website through the target expression to obtain a first data acquisition result for the target website; a simulation module, configured to simulate a collection behavior operation by using an automated tool through the specified dynamic screening condition if the request loading mode is the dynamic loading, and obtain the page elements of the target website through the collection behavior operation; wherein the collection behavior operation includes at least one of a click operation, a slide operation, and a long press operation; an identification module, configured to identify the page elements of the target website through a CSS selector and an element selector, obtain the text content corresponding to the page elements, and use the text content as the second data collection result of the target website; wherein the text content includes at least one of a text title, a text picture, and a text text; The simulation module is specifically used to: obtain historical operation data for the terminal device; classify and store multiple historical operation data according to the length of the historical operation cursor movement trajectory in the historical operation data, so as to store the historical operation data whose length meets the same specified length interval into the same type, and obtain different trajectory storage types corresponding to multiple different specified length intervals; wherein the historical operation data includes any one or more of the following: the curvature of the historical operation cursor movement trajectory, the shape of the historical operation cursor movement trajectory, the length of the historical operation cursor movement trajectory, the running time of the historical operation cursor movement trajectory, and the running rhythm of the historical operation cursor movement trajectory; obtain the total length of the cursor movement to be simulated corresponding to the acquisition behavior operation by using the specified dynamic screening condition using an automated tool; based on the total length of the cursor movement to be simulated, select multiple target historical operation cursor movement trajectories from multiple trajectory storage types corresponding to multiple specified length intervals, and splice the multiple target historical operation cursor movement trajectories to obtain a splicing result of the historical operation cursor movement trajectory that meets the total length of the cursor movement to be simulated, and use the splicing result of the historical operation cursor movement trajectory to simulate the acquisition behavior operation.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the steps of the data collection method based on screening conditions described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are called and executed by the processor, the computer-executable instructions prompt the processor to execute the data collection method based on screening conditions as described in any one of claims 1 to 6.

Images