Information collection method, device, computer equipment and storage medium

Abstract

The present invention relates to the field of big data technology, and discloses an information collection method, device, computer device and storage medium. When responding to a user's first operation behavior, a target account is selected from a preset account information list for login, and an information login page is displayed; when responding to a user's second operation behavior on the information login page, an operation instruction is generated; if the operation instruction is a first operation instruction, a first instance page is displayed, each candidate instance data in the first instance page is traversed, each candidate instance data is identified, and first target instance data that meets preset requirements is determined, the first target instance data includes a first target instance ID; a corresponding first instance address link is constructed based on the first target instance ID; after a collection trigger instruction is received, each first instance address link is screened, and first target instance information corresponding to the first target instance link that meets a preset policy is collected; thereby improving the efficiency of information collection.

Description

Technical Field

[0001] The present invention relates to the field of big data processing, and in particular to an information collection method, device, computer device and storage medium. Background Art

[0002] With the wide application of big data technology, information collection technology has been paid more and more attention. Automated information collection is an effective technical means to obtain site information, and has extensive applications in data collection scenarios such as news report collection and public opinion monitoring. At present, most traditional information collection methods are to crawl the information of the target site by customizing the request header, or to simulate a real browser through a class library to collect information. However, these two information collection methods are often unable to cope when facing sites with strong verification, and at the same time, when the number of sites is large, the type of browser used by the user will be restricted, resulting in low efficiency of information collection and great inconvenience to the user. Summary of the Invention

[0003] Embodiments of the present invention provide an information collection method, device, computer device and storage medium to solve the problem of low efficiency of information collection.

[0004] An information collection method includes:

[0005] Responding to a first operation behavior of a user, selecting a target account from a preset account information list for login, and displaying an information login page;

[0006] Responding to a second operation behavior of the user on the information login page, generating an operation instruction;

[0007] If the operation instruction is a first operation instruction, displaying a first instance page, traversing each candidate instance data in the first instance page, identifying each candidate instance data, and determining first target instance data that meets preset requirements, the first target instance data includes a first target instance ID, and the first operation instruction instructs the user to select a first data collection method;

[0008] Constructing a corresponding first instance address link based on the first target instance ID;

[0009] After receiving a collection trigger instruction, screening each of the first instance address links, and collecting first target instance information corresponding to a first target instance link that conforms to a preset policy.

[0010] An information collection device includes:

[0011] A first response module, configured to respond to a first operation behavior of a user, select a target account from a preset account information list for login, and display an information login page;

[0012] A second response module, configured to generate an operation instruction in response to a second operation behavior of a user on the information login page;

[0013] A first display module, configured to, when the operation instruction is a first operation instruction, display a first instance page, traverse each piece of candidate instance data in the first instance page, identify each piece of the candidate instance data, and determine first target instance data that meets a preset requirement, where the first target instance data includes a first target instance ID, and the first operation instruction instructs the user to select a first data collection method;

[0014] A first construction module, configured to construct a corresponding first instance address link based on the first target instance ID;

[0015] A first collection module, configured to, after receiving a collection trigger instruction, screen each of the first instance address links, and collect first target instance information corresponding to a first target instance link that conforms to a preset policy.

[0016] A computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where when the processor executes the computer program, the above information collection method is implemented.

[0017] A computer-readable storage medium storing a computer program, where when the computer program is executed by a processor, the above information collection method is implemented.

[0018] The above information collection method, device, computer equipment and storage medium respond to the user's first operation behavior, select a target account from a preset account information list for login, and display an information login page; respond to the user's second operation behavior on the information login page to generate an operation instruction; if the operation instruction is a first operation instruction, display a first instance page, traverse each candidate instance data in the first instance page, identify each candidate instance data, and determine first target instance data that meets preset requirements. The first target instance data includes a first target instance ID. The first operation instruction instructs the user to select a first data collection method; construct a corresponding first instance address link based on the first target instance ID; after receiving a collection trigger instruction, screen each first instance address link, and collect first target instance information corresponding to the first target instance link that meets a preset policy. This solution introduces automated testing technology into automated information collection applications, which not only facilitates users to log in and out of the system, but also traverses all first target instance data, extracts instance IDs that meet the requirements for preservation, and then constructs links in a loop through the instance IDs and accesses each first instance address link to obtain instance information. This solution can be used in scenarios that require repeated manual operations. During the entire information collection process, only one Chrome instance will be maintained in memory so that it can quickly respond when navigation and analysis tasks change; thereby improving the efficiency of information collection. Brief Description of the Drawings

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments of the present invention. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0020] Figure 1 It is a schematic diagram of an application environment of the information collection method in an embodiment of the present invention;

[0021] Figure 2 It is an example diagram of the information collection method in an embodiment of the present invention;

[0022] Figure 3 It is another example diagram of the information collection method in an embodiment of the present invention;

[0023] Figure 4 It is another example diagram of the information collection method in an embodiment of the present invention;

[0024] Figure 5 It is another example diagram of the information collection method in an embodiment of the present invention;

[0025] Figure 6It is another exemplary diagram of the information collection method in an embodiment of the present invention;

[0026] Figure 7 It is a principle block diagram of the information collection device in an embodiment of the present invention;

[0027] Figure 8 It is another principle block diagram of the information collection device in an embodiment of the present invention;

[0028] Figure 9 It is another principle block diagram of the information collection device in an embodiment of the present invention;

[0029] Figure 10 It is a schematic diagram of a computer device in an embodiment of the present invention. Detailed implementation manners

[0030] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0031] The information collection method provided by the embodiments of the present invention can be applied in the Figure 1 shown application environment. Specifically, the information collection method is applied in an information collection system, and the information collection system includes the Figure 1 shown client and server. The client communicates with the server through a network to solve the problem of low efficiency of information collection. Among them, the client, also known as the user side, refers to a program that provides local services for the client corresponding to the server side. The client can be installed on but not limited to various personal computers, laptop computers, smart phones, tablet computers, and portable wearable devices. The server side can be implemented by an independent server or a server cluster composed of multiple servers.

[0032] In one embodiment, as Figure 2 shown, a kind of information collection method is provided. Taking the method applied in the Figure 1 server side as an example for illustration, it includes the following steps:

[0033] S10: In response to the user's first operation behavior, select a target account from the preset account information list for login, and display the information login page.

[0034] Among them, the first operation behavior refers to the behavior of the user triggering account login in the client. Specifically, when the user performs the first operation behavior in the client, the server will respond to the first operation behavior and select a target account from the preset account information list for login. That is, the first operation behavior is a triggering behavior. After the user performs the first operation behavior in the client, the first operation behavior triggers the server to select a target account from the preset account information list for login, automatically completes the login operation of entering the account password, and displays the information login page on the client. Preferably, in this embodiment, the information login page is displayed in the browser instance of the client. Among them, the target account refers to the account selected from the account information list for login. The server selects a target account from the preset account information list for login, and the automatic completion of the login operation of entering the account password can replace the user's manual operation of entering the account password. The information login page refers to the page generated when accessing the main page of information collection after account login.

[0035] In a specific embodiment, a number of candidate accounts are pre-stored in the account information list. After the user performs the first operation behavior, first start a browser instance through the Driver of Selenium to navigate to the cloud login interface, and then automatically select any one of the candidate accounts from the account information list as the target account for login, and display the information login page in the browser instance of the client. In a specific embodiment, the candidate accounts included in the account information list include the main account and the sub-accounts. Preferably, the server selects the main account from the account information list as the target account for login. Among them, Selenium is a tool for testing Web applications.

[0036] S20: Respond to the second operation behavior of the user on the information login page and generate an operation instruction.

[0037] Among them, the second operation behavior is the behavior of selecting the information collection method. The second operation behavior includes, but is not limited to, the user's swiping, clicking, or long-pressing on the information login page, etc. The operation instruction refers to the instruction generated after selecting the information collection method. In a specific application scenario, there are multiple different options for selecting the information collection method on the information login page. For example, the information collection methods include the first information collection method, the second information collection method, and the third information collection method, etc. Specifically, when the user performs the second operation behavior on the option of the information collection method on the information login page, the server will respond to the user's second operation behavior and trigger the generation of the corresponding operation instruction. The second operation behavior is actually a triggering behavior. When the user performs the second operation behavior of selecting the information collection method on the information login page, this second operation behavior will trigger the server to generate the corresponding operation instruction. It can be understood that if the information collection methods selected by the user are different, the corresponding generated operation instructions will also be different. For example, if the user selects the "first information collection method", the first operation instruction will be generated; if the "second information collection method" is selected, the second operation instruction will be generated.

[0038] S30: If the operation instruction is the first operation instruction, display the first instance page, traverse each candidate instance data in the first instance page, identify each candidate instance data, and determine the first target instance data that meets the preset requirements. The first target instance data includes the first target instance ID, and the first operation instruction instructs the user to select the first data collection method.

[0039] Among them, the first operation instruction instructs the user to select the first data collection method. The first data collection method is preferably the ECS crawling method. Specifically, if the operation instruction is the first operation instruction, the page is navigated to the first instance page through the Driver (driver program). After the page DOM of the first instance page is loaded, the first instance page is displayed in the browser instance on the client side. In a specific embodiment, several candidate instance data are distributed on the first instance page. Specifically, each candidate instance data includes instance ID, IP address, availability zone, status, image, creator, creation time, etc. Further, each candidate instance data in the first instance page is traversed, and each candidate instance data is identified. In this embodiment, each candidate instance data is identified mainly by using XPATH9 (XML Path Language) or CssSelector (CSS selector) to locate the instance ID in each candidate instance data, and then the instance ID in each candidate instance data is identified. The candidate instance data whose instance ID meets the preset requirements is determined as the first target instance data. Among them, the instance ID is the unique number of each candidate instance data, mainly composed of numbers and letters. The preset requirement refers to the strategy preset for evaluating whether the candidate instance data meets the requirements. Specifically, the preset requirement is that there is a unified preset prefix in the instance ID. The preset prefix is the preset ID prefix information. That is, the candidate instance data corresponding to the instance ID with the unified preset prefix is determined as the first target instance data. For example: if the preset prefix is LNP, the candidate instance data corresponding to the instance ID with the prefix LNP is determined as the first target instance data. The first target instance data includes the first target instance ID. The first target instance ID is the instance ID corresponding to the first target instance data.

[0040] Preferably, before traversing each candidate instance data in the first instance page, in order to facilitate subsequent connection for information collection, the server will automatically enumerate all debugging ports of Chrome in the background. Since Chrome is multi-process, that is, there may be a situation where each time a Chrome tab is opened, a new chrome.exe process is added to the system. Since accessing process information through system functions, that is, the Process class, filters by process name, there may be multiple processes in the obtained process list. Therefore, in this embodiment, the Chrome debugging ports are filtered, and the Chrome debugging ports that meet the conditions are returned. Among them, meeting the conditions means that the startup parameter of the Chrome debugging port process contains the --remote-debugging-port option and the port number is greater than 12151.

[0041] S40: Construct a corresponding first instance address link based on the first target instance ID.

[0042] Since all ECS (Elastic Compute Service) resources have a common address prefix, after determining the first target instance ID, the corresponding first instance address link can be constructed based on the first target instance ID. Specifically, based on the first target instance ID, a constructor can be used to construct the first instance address link. For example, if the first target instance ID is LNPyoV7twx, the first instance address link constructed using the constructor is https: / / yun.city.pingan.com / console / ecs / instance / Instance-LNPyoV7twx. Further, after constructing and generating the first instance address link, each first instance address link will be stored in the first link list. The first link list is a pre-set list for storing the first instance address link.

[0043] S50: After receiving the collection trigger instruction, filter each first instance address link, and collect the first target instance information corresponding to the first target instance link that meets the preset policy.

[0044] The collection trigger instruction refers to an instruction that triggers the collection of instance information. The first target instance information refers to the page information contained in the first target instance link. Specifically, when the information collection event is triggered, that is, when the server receives the trigger instruction for collecting instance information, the first target instance link will be taken out from the first link list in sequence for access, and each first instance address link will be filtered to obtain the first target instance information corresponding to the first instance address link that meets the preset policy. The preset policy refers to a pre-set policy for evaluating whether the first instance address link can be used for information collection. In this embodiment, the preset policy is to determine the first instance address link that meets the preset policy by judging whether the cloud host type corresponding to the first instance address link meets the requirements. Further, judging whether the cloud host type corresponding to the first instance address link meets the requirements can be achieved by comparing the cloud host type corresponding to the first instance address link with the pre-set cloud host types that meet the requirements. If the cloud host type corresponding to the first instance address link is among the cloud host types that meet the requirements, it is judged that the terminal type corresponding to the first instance address link meets the requirements; if the cloud host type corresponding to the first instance address link is not among the terminal types that meet the requirements, it is judged that the terminal type corresponding to the first instance address link does not meet the requirements. The cloud host type refers to the host type of the ECS (Elastic Compute Service) corresponding to the first instance address link.

[0045] In this embodiment, in response to the user's first operation behavior, a target account is selected from a preset list of account information for login, and an information login page is displayed; in response to the user's second operation behavior on the information login page, an operation instruction is generated; if the operation instruction is a first operation instruction, a first instance page is displayed, each candidate instance data in the first instance page is traversed, each candidate instance data is identified, and first target instance data that meets the preset requirements is determined. The first target instance data includes a first target instance ID, and the first operation instruction instructs the user to select a first data collection method; a corresponding first instance address link is constructed based on the first target instance ID; after a collection trigger instruction is received, each first instance address link is screened, and first target instance information corresponding to the first target instance link that meets the preset policy is collected; by introducing automated testing technology into the automated information collection application, this solution not only facilitates the user to log in and out of the system, but also traverses all first target instance data, extracts the instance IDs that meet the requirements for storage, and then constructs links in a loop through the instance IDs and accesses each first instance address link to obtain instance information. This solution can be used in scenarios that require repeated manual operations. During the entire information collection process, only one Chrome instance will be maintained in memory so that it can quickly respond when navigation and analysis tasks change; thus, the efficiency of information collection is improved.

[0046] In one embodiment, as Figure 4 shown, after generating an operation instruction in response to the user's second operation behavior on the information login page, the information collection method further specifically includes the following steps:

[0047] S21: If the operation instruction is a second operation instruction, a second instance page is displayed, the candidate relational database set supported by the second instance page is detected, and a target relational database is arbitrarily selected from the candidate relational database set, where the second operation instruction instructs the user to select a second data collection method.

[0048] Among them, the second operation instruction instructs the user to select a second data collection method. The second data collection method is preferably the DB crawling method. Specifically, if the operation instruction is the second operation instruction, the page is navigated to the second instance page through the Driver, and the second instance page is displayed on the client interface. The second instance page is preferably the DB instance list page. In a specific embodiment, there is a "Get Available DB" button on the second instance page. After the user clicks the "Get Available DB" button, it is possible to detect the candidate relational database set supported by the second instance page by determining whether the page element contains a div tag with the title RDS-PostgreSQL and a tag with the title RDS-MySQL. Since there are various types of relational databases in the candidate relational database set. For example, relational databases include multiple types such as MySQL, PostgreSQL, and Microsoft Access. Therefore, in this step, before information collection, an arbitrary target relational database is selected from the candidate relational database set to perform information collection.

[0049] S22: In response to the data acquisition request, obtain second target instance data from the target relational database, and each second target instance data includes a second target instance ID.

[0050] Among them, the data acquisition request refers to a request that triggers the acquisition of instance data. Optionally, the data acquisition request can be triggered by the user performing a click operation on the "Data Acquisition" button on the second instance page. Or it can be triggered and generated by entering the corresponding control instruction in the command line. Specifically, after arbitrarily selecting a target relational database from the candidate relational database set, the Driver on the server side obtains the data acquisition request sent by the client through the debugging port, and obtains the second target instance data from the target relational database. Each second target instance data includes a second target instance ID. It can be understood that the types of the second target instance data stored in different types of target relational databases are also different. For example, the second target instance data stored in the MySQL database is MySQL instance data. Specifically, the second target instance data includes: second target instance ID, IP address, available zone, status, mirror image, creator, and creation time, etc. Among them. The second target instance ID is the unique number of each candidate instance data, mainly composed of numbers and letters.

[0051] S23: Construct a corresponding second instance address link based on the second target instance ID.

[0052] Since the DB instance links all have a common address prefix, after the second target instance ID is determined, the corresponding second instance address link can be constructed according to the second target instance ID. Specifically, based on the second target instance ID, a constructor can be used to construct the second instance address link. For example, if the second target instance ID is bxwK6mab3m, the second instance address link constructed using the constructor is https: / / yun.city.pingan.com / console / rds / mysql / 33081 / MySQLInstance-bxwK6mab3m / normal. Further, after the second instance address link is constructed and generated, each second instance address link will be stored in the second link list. The second link list is a pre-set list for storing second instance address links.

[0053] S24: After receiving the information collection instruction, collect the second target instance information corresponding to each second target instance link.

[0054] The information collection instruction refers to an instruction that triggers the collection of instance information. The second target instance information refers to the page information included in the second target instance link. Specifically, when the information collection event is triggered, that is, when the server receives the trigger instruction for collecting instance information, the second target instance links will be taken out from the second link list in sequence for access, and the second target instance information corresponding to each second instance address link will be collected.

[0055] In this embodiment, if the operation instruction is the second operation instruction, the second instance page is displayed, the candidate relational database set supported by the second instance page is detected, and a target relational database is arbitrarily selected from the candidate relational database sets. The second operation instruction instructs the user to select the second data collection method; in response to the data acquisition request, the second target instance data is acquired from the target relational database, and each second target instance data includes the second target instance ID; the corresponding second instance address link is constructed based on the second target instance ID; after receiving the information collection instruction, the second target instance information corresponding to each second target instance link is collected; different information collection methods can be adopted in combination with the actual scenario in this solution. In this embodiment, by traversing all the second target instance data, the instance IDs that meet the requirements are extracted and saved, and then the second instance address links are constructed in a loop through the instance IDs, and each second instance address link is accessed to obtain the instance information. During the entire data collection process, only one Chrome instance will be maintained in the memory so that it can respond quickly when the navigation and analysis tasks change; thereby further improving the efficiency of information collection.

[0056] In one embodiment, as Figure 4As shown, before responding to the user's first operation behavior, the information collection method further specifically includes the following steps:

[0057] S01: Obtain a system authentication request, where the system authentication request includes account information.

[0058] Among them, the system authentication request refers to a request used to trigger the entry of an account password. Specifically, the system authentication request can be triggered and generated after the user enters the account name and password in the "account password input box" on the client interface and clicks the "confirm" button. The system authentication request includes account information. Specifically, the account information includes the account type (main account or sub-account), account name, and password. Additionally, the server can also directly select the account information from the account password manager. Optionally. In this embodiment, the entry of sub-accounts is supported, and multiple account information can also be entered simultaneously. In the case where multiple account information is entered simultaneously, it is possible to custom-set which account information is the main account information or sub-account information.

[0059] S02: Load the account information to generate a target text.

[0060] Specifically, since the obtained account information is stored in memory variables, in this embodiment, the account information is further processed and exported to a text in a specified format. In this embodiment, the specified text format can be either the TXT or XLSX format. Specifically, according to the preset texts in the provided TXT and XLSX formats, after obtaining the account information carried by the information entry request, the account information is loaded, and the account name and the corresponding password in the account information are imported into the pre-text in the form of key-value pairs, thereby generating the target text. Among them, the TXT format needs to be written line by line in the format of account name and password, and the XLSX format generates a table with the help of relevant libraries. Additionally, in a specific embodiment, when the server obtains the account information, it also obtains the IP and host name of the ECS (cloud server), and the IP and host name of the ECS (cloud server) are also stored in memory variables. Therefore, when loading the account information to process and export the account name and the corresponding password in the account information to a text in a specified format, it is also necessary to load the corresponding IP and host name of the ECS (cloud server) and export them to a text in a specified format.

[0061] Furthermore, after importing the account information into the preset text to generate the target text, in order to ensure the real-time update of the account information in the target text, it is also necessary to refresh the target text in a timely manner, thereby improving the accuracy of the account information stored in the target text.

[0062] S03: Parse the target text to generate an account information list for display.

[0063] Among them, the account information list refers to a list used to display account information. Specifically, in order to facilitate users to intuitively and accurately know the input account information, after generating the target text, it is also necessary to use the JSON format parsing function to parse the account information in the target text, generate an account information list, and display the account information list on the client interface. Preferably, in this embodiment, after parsing the target text to generate an account information list, the account information list is displayed in columns in the account password manager on the client interface. When the account password manager is opened, these accounts can be seen in a columnar form and it can be determined whether they are main accounts or sub-accounts. At the same time, since the login pages of the main account and sub-accounts in the account information list are located at different addresses, the accounts stored in the account information list also distinguish between the main account and sub-accounts. The specific differentiation method is based on the flag bits related to the accounts stored in the JSON.

[0064] In this embodiment, system authentication information is obtained. The system authentication information includes account password information; the account password information is loaded to generate a target text; the target text is parsed to generate an account information list for display; this embodiment allows users to input custom account passwords and supports the entry of multiple different sub-accounts. In addition, all account passwords are managed through the use of an account password manager; thereby improving the convenience of users logging in and out of the system.

[0065] In one embodiment, as Figure 5 shown, after selecting a target account from a preset account information list for login and displaying the information login page, and before responding to the user's second operation behavior on the information login page, this information collection method further specifically includes the following steps:

[0066] S11: If there is a slider verification code on the information login page, obtain the position information of the sliding button in the slider verification code and the position information of the verification code area. The position information of the verification code area includes the position information of the slider and the position information of the slider gap.

[0067] Among them, the verification code is a public fully automatic program that distinguishes whether the user is a computer or a human. The slider verification code is one type of verification code and is widely adopted due to its simple operation, good user experience, strong security, etc. The sliding button and the verification code area are both parts of the sliding verification code. Among them, the sliding button refers to the part used for sliding in the slider verification code. The verification code area refers to the position where the image for verifying the answer in the verification code is located. The verification code area includes the slider and the slider gap. The slider corresponds to the sliding button, and the shapes of the slider and the slider gap are generally the same. The slider can be moved by sliding the sliding button, and moving the slider to the position of the slider gap can complete the verification of the slider verification code.

[0068] Specifically, the position information of the sliding button and the position information of the verification code area in the slider verification code can be obtained through a slider verification code analyzer. In the implementation of the slider verification code analyzer, a grayscale progressive method based on image processing is mainly used to identify specific positions in the page image, so as to extract the position information of the sliding button and the position information of the verification code area in the page image. In one embodiment, a corresponding coordinate system can be established on the page image, and then the position of the sliding button and the position of the verification code area are recorded through a four-dimensional vector (x min , y min , x max , y max ). Where x min represents the minimum abscissa of the sliding button or the verification code area in the coordinate system, y min represents the minimum ordinate of the sliding button or the verification code area in the coordinate system, x max represents the maximum abscissa of the sliding button or the verification code area in the coordinate system, and y max represents the maximum ordinate of the sliding button or the verification code area in the coordinate system. After obtaining this value, the optimal displacement required to move the slider to satisfy the slider falling on the shadow can be determined.

[0069] S12: Determine the slider offset according to the position information of the slider and the position information of the slider notch.

[0070] Among them, the slider offset refers to the distance that the slider moves, that is, the pixel distance between the slider and the slider notch. The sliding verification can be performed by moving the slider in the sliding bar by a distance corresponding to the slider offset. Specifically, a coordinate system can be established, and then the position information of the center of the slider and the position information of the center of the slider notch in the coordinate system can be used to determine the distance that the slider needs to slide.

[0071] In addition, for a slider verification code that only contains a background and a shadow, the background part, the slider, and the slider notch of the verification code area can also be color-picked, and the color-picking result is an RGB value. Then, through a digital method, the offset value that meets the conditions is calculated using the Red component of the RGB. This offset value is the slider offset required for the slider to enter the slider notch. For example: the coordinates of the slider notch are (220, 15) -> color-picking (255, 205, 204, 203), the coordinates of the verification code area are (47, 15) -> color-picking (255, 245, 245, 245), then when the Red component is less than or equal to 205, it can be considered that the slider is located in the corresponding slider notch position.

[0072] S13: Control the sliding of the sliding button according to the sliding offset to perform sliding verification.

[0073] Specifically, the sliding button on the login information page can be controlled to move according to the calculated sliding offset, so that the slider corresponding to the sliding button can be moved to the position corresponding to the slider notch for sliding verification.

[0074] In this embodiment, if there is a slider verification code on the information login page, the position information of the sliding button and the position information of the verification code area in the slider verification code are obtained. The position information of the verification code area includes the position information of the slider and the position information of the slider notch. The slider offset is determined according to the position information of the slider and the position information of the slider notch. The sliding button is controlled to slide according to the sliding offset for sliding verification. This embodiment mainly uses a slider verification code analyzer to verify the slider verification code, solves the problem that slider verification in the verification process needs to be manually processed, and better helps the tool to achieve true automated information collection.

[0075] In one embodiment, as Figure 6 shown, each first instance address link is screened, and the first target instance information corresponding to the first target instance link that meets the preset policy is obtained. The specific steps are as follows:

[0076] S501: Traverse each first instance address link and determine the type of cloud host corresponding to each first instance address link.

[0077] S502: If the type of cloud host corresponding to the first instance address link is the first type, the first instance address link is determined as the first target instance link.

[0078] Specifically, after receiving the trigger instruction for information collection, each first target instance link is traversed, and the type of cloud host corresponding to each first target instance link is determined. Among them, the cloud host type refers to the host type of the ECS (Elastic Compute Service) corresponding to the first instance address link. Specifically, if the type of cloud host corresponding to the first instance address link is the first type, the first instance address link is determined as the first target instance link. Among them, the first type refers to the host type of the ECS (Elastic Compute Service) that can perform information collection. In a specific embodiment, the first type that can perform information collection has been preset in advance. For example: the first type can be public cloud or hybrid cloud, etc. The user can customize the first type according to the actual situation. Specifically, after obtaining the type of cloud host corresponding to the first target instance link, the type of cloud host corresponding to the first target instance link can be compared with the previously collected first type. If the type of cloud host corresponding to the first target instance link is the same as the previously collected first type, the type of cloud host corresponding to the first instance address link is determined as the first type, and the first instance address link is determined as the first target instance link.

[0079] Preferably, after it is determined that the cloud host type corresponding to the first target instance link is the first type, it is further determined whether there is a classic password button on the access page where the first target instance link is located, where the classic password button is a "ON / OFF switch type" button; it is usually "located in the label tag. If there is a classic password button on the access page, click to display the password and then collect the first target instance information corresponding to the first target instance link.

[0080] S503: Collect the first target instance information corresponding to the first target instance link and store the first target instance information in a preset text.

[0081] Specifically, the preset text refers to the text used to store the first target instance information. The preset text can be a TXT plain text or an Excel table. Specifically, after the first target instance link is determined, the first target instance information corresponding to the first target instance link is collected. Further, after the first target instance information is obtained, the first target instance information is stored in the preset text for importing to other platforms.

[0082] In this embodiment, each first instance address link is traversed to determine the cloud host type corresponding to each first instance address link; if the cloud host type corresponding to the first instance address link is the first type, the first instance address link is determined as the first target instance link; collect the first target instance information corresponding to the first target instance link and store the first target instance information in a preset text; in the formal information collection stage, by screening the first target instance link, the accuracy of the collected information is ensured.

[0083] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0084] In one embodiment, an information collection device is provided, and the information collection device corresponds one-to-one with the information collection method in the above embodiment. As Figure 7 shown, the information collection device includes a first response module 10, a second response module 20, a first display module 30, a first construction module 40, and a first collection module 50. The detailed description of each functional module is as follows:

[0085] The first response module 10 is configured to respond to the user's first operation behavior, select a target account from a preset account information list for login, and display an information login page;

[0086] The second response module 20 is configured to respond to the user's second operation behavior on the information login page and generate an operation instruction;

[0087] The first display module 30 is configured to display a first instance page when the operation instruction is a first operation instruction, traverse each candidate instance data in the first instance page, identify each candidate instance data, and determine first target instance data that meets a preset requirement. The first target instance data includes a first target instance ID. The first operation instruction instructs the user to select a first data collection method;

[0088] The first construction module 40 is configured to construct a corresponding first instance address link based on the first target instance ID;

[0089] The first collection module 50 is configured to, after receiving a collection trigger instruction, screen each first instance address link and collect first target instance information corresponding to the first target instance link that conforms to a preset policy.

[0090] Preferably, as Figure 8 shown, the information collection device further includes:

[0091] The second display module 21 is configured to display a second instance page when the operation instruction is a second operation instruction, detect a candidate relational database set supported by the second instance page, and arbitrarily select a target relational database from the candidate relational database set. The second operation instruction instructs the user to select a second data collection method;

[0092] The acquisition module 22 is configured to, in response to a data acquisition request, acquire second target instance data from the target relational database. Each second target instance data includes a second target instance ID;

[0093] The second construction module 23 is configured to construct a corresponding second instance address link based on the second target instance ID;

[0094] The second collection module 24 is configured to, after receiving an information collection instruction, collect second target instance information corresponding to each second target instance link.

[0095] Preferably, as Figure 9 shown, the information collection device further includes:

[0096] The second acquisition module 01 is configured to acquire a system authentication request, and the system authentication request includes account information;

[0097] The loading module 02 is configured to load the account information to generate a target text;

[0098] The parsing module 03 is configured to parse the target text to generate an account information list for display.

[0099] Preferably, the information collection device further includes:

[0100] A third acquisition module, configured to obtain the position information of the sliding button in the slider verification code and the position information of the verification code area when there is a slider verification code on the information login page, where the position information of the verification code area includes the position information of the slider and the position information of the slider notch;

[0101] A determination module, configured to determine the slider offset according to the position information of the slider and the position information of the slider notch;

[0102] A sliding verification module, configured to control the sliding of the sliding button according to the sliding offset for sliding verification.

[0103] Preferably, the first acquisition module 50 includes:

[0104] A judgment unit, configured to traverse each first instance address link and judge the cloud host type corresponding to each first instance address link;

[0105] A determination unit, configured to determine the first instance address link as the first target instance link when the cloud host type corresponding to the first instance address link is the first type;

[0106] An acquisition unit, configured to acquire the first target instance information corresponding to the first target instance link and store the first target instance information in a preset text.

[0107] For the specific limitations of the information acquisition device, reference may be made to the limitations on the information acquisition method in the foregoing text, which will not be elaborated here. Each module in the above information acquisition device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in the form of hardware or be independent of it, or can be stored in the memory of the computer device in the form of software, so as to facilitate the processor to call and execute the operations corresponding to the above modules.

[0108] In one embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 10 shown. The computer device includes a processor, a memory, a network interface, and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the data used in the information acquisition method in the above embodiment. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements an information acquisition method.

[0109] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the information collection method in the above embodiment is implemented.

[0110] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the information collection method in the above embodiment is implemented.

[0111] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0112] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the above-mentioned division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0113] The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. An information collection method, characterized in that, Including: In response to the user's first operation behavior, select a target account from a preset list of account information for login and display an information login page; In response to the user's second operation behavior on the information login page, generate an operation instruction; If the operation instruction is a first operation instruction, display a first instance page, traverse each candidate instance data in the first instance page, identify each candidate instance data, and determine first target instance data that meets preset requirements. The first target instance data includes a first target instance ID. The first operation instruction instructs the user to select a first data collection method; the first data collection method is the ECS crawling method; Construct a corresponding first instance address link based on the first target instance ID; After receiving a collection trigger instruction, filter each first instance address link and collect first target instance information corresponding to the first target instance link that meets the preset policy; After generating the operation instruction in response to the user's second operation behavior on the information login page, the information collection method further includes: If the operation instruction is a second operation instruction, display a second instance page, detect a candidate relational database set supported by the second instance page, and arbitrarily select a target relational database from the candidate relational database set. The second operation instruction instructs the user to select a second data collection method; the second data collection method is the DB crawling method; In response to a data acquisition request, obtain second target instance data from the target relational database. Each second target instance data includes a second target instance ID; Construct a corresponding second instance address link based on the second target instance ID; After receiving an information collection instruction, collect second target instance information corresponding to each second target instance link.

2. The information acquisition method according to claim 1, wherein Before the information collection method responds to the user's first operation behavior, the information collection method further includes: Obtain a system authentication request, where the system authentication request includes account information; Load the account information to generate a target text; Parse the target text to generate a list of account information for display.

3. The information collection method according to claim 1, characterized in that After selecting a target account from the preset list of account information for login and displaying the information login page, and before responding to the user's second operation behavior on the information login page, the information collection method further includes: If there is a slider verification code on the information login page, obtain the position information of the sliding button in the slider verification code and the position information of the verification code area. The position information of the verification code area includes the position information of the slider and the position information of the slider notch; Determine the slider offset according to the position information of the slider and the position information of the slider notch; Control the sliding button to slide according to the sliding offset for sliding verification.

4. The information collection method according to claim 1, wherein The filtering of each first instance address link to obtain first target instance information corresponding to the first target instance link that meets the preset policy includes: Traverse each first instance address link and determine the cloud host type corresponding to each first instance address link; If the cloud host type corresponding to the first instance address link is the first type, determine the first instance address link as the first target instance link; Collect the first target instance information corresponding to the first target instance link, and store the first target instance information in a preset text.

5. An information collection device, characterized in that, Including: A first response module, configured to respond to a first operation behavior of a user, select a target account from a preset list of account information for login, and display an information login page; A second response module, configured to generate an operation instruction in response to a second operation behavior of the user on the information login page; A first display module, configured to, when the operation instruction is a first operation instruction, display a first instance page, traverse each candidate instance data in the first instance page, identify each candidate instance data, and determine first target instance data that meets a preset requirement. The first target instance data includes a first target instance ID. The first operation instruction instructs the user to select a first data collection method; the first data collection method is an ECS crawling method; A first construction module, configured to construct a corresponding first instance address link based on the first target instance ID; A first collection module, configured to, after receiving a collection trigger instruction, screen each first instance address link, and collect first target instance information corresponding to a first target instance link that meets a preset policy; The information collection device further includes: A second display module, configured to, when the operation instruction is a second operation instruction, display a second instance page, detect a candidate relational database set supported by the second instance page, and arbitrarily select a target relational database from the candidate relational database set. The second operation instruction instructs the user to select a second data collection method; the second data collection method is a DB crawling method; An acquisition module, configured to respond to a data acquisition request, and acquire second target instance data from the target relational database. Each second target instance data includes a second target instance ID; A second construction module, configured to construct a corresponding second instance address link based on the second target instance ID; A second collection module, configured to, after receiving an information collection instruction, collect second target instance information corresponding to each second target instance link.

6. The information acquisition device according to claim 5, characterized in that, The information collection device further includes: A second acquisition module, configured to acquire a system authentication request, where the system authentication request includes account information; A loading module, configured to load the account information to generate a target text; An analysis module, configured to analyze the target text to generate a list of account information for display.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the information collection method according to any one of claims 1 to 4.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the information collection method according to any one of claims 1 to 4.

Images