Application data acquisition method, device, equipment and computer storage medium
Receive, modify and send data requests through the intermediary agent service tool, analyze response data, solve the problem of obtaining data displayed on the mobile terminal application page, realize fast and simple data acquisition, and avoid high pressure and legal risks on corporate servers. .
Patent Information
- Application Number
- CN202111232497.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-22
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-10-22
AI Technical Summary
The prior art is difficult to easily and quickly obtain page display data of mobile terminal applications, and cracking encryption request methods is time-consuming and labor-intensive, and there are legal risks and are not universal.
The intermediary agent service tool is used to communicate with the mobile terminal application and the server, receive and modify data requests, and send them to the server to obtain page display data, and parse response data to realize data acquisition.
No need to crack encryption requests, it can easily and quickly obtain page display data of mobile terminal applications, reduce pressure on enterprise servers and avoid legal risks, and is universal.
Smart Images

Figure CN113934498B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of Internet technology, and in particular to a method, device, electronic device, and computer storage medium for acquiring data from an application. Background Art
[0002] With the advent of the era of information explosion, information acquisition, as the basic support for later analysis, has become increasingly important. With the rise of mobile terminals, more and more companies have mobile terminal applications, displaying product information on mobile terminal pages and applications. Some companies have even abandoned the mobile terminal page and focused on creating product information display in mobile terminal applications. However, most of the existing information acquisition technologies are still focused on how to obtain page display data on the mobile terminal page. Therefore, how to quickly obtain page display data on mobile terminal applications is becoming increasingly important.
[0003] At present, the known method of obtaining the page display data of the mobile terminal application is to crack the request encryption parameters of the application of each enterprise's mobile terminal, and then forge the request and send it to the enterprise server to obtain the data carried in the response returned by the enterprise server. However, it is difficult to forge the request and send it to the enterprise server by cracking the request encryption parameters of the application of each enterprise's mobile terminal. Especially now that more and more large Internet companies are selling professional mobile terminal application packaged shell products, the request parameters are protected, so that data crawlers are often helpless, or need to invest a lot of energy and time in the early stage. Although this method can frequently access the enterprise server to obtain data after cracking the parameters in the later stage, it puts a lot of pressure on the enterprise server and has the hidden danger of taking legal risks. In addition, the encryption method of the application of the mobile terminal of each enterprise may be different, and this method is not universal.
[0004] Therefore, how to easily and quickly obtain the page display data of the application program of the mobile terminal has become a technical problem that needs to be solved urgently. Summary of the invention
[0005] In view of this, an embodiment of the present invention provides an application data acquisition method, device, electronic device and computer storage medium to solve the technical problem existing in the prior art of how to easily and quickly acquire page display data of an application of a mobile terminal.
[0006] According to a first aspect of an embodiment of the present invention, there is provided a method for acquiring data of an application, which is applied to a middleman proxy service tool, wherein the middleman proxy service tool is respectively communicated with an application of a mobile terminal and a server of the application, and the method comprises: receiving a data request generated based on an automated control operation and sent by the application, wherein the data request is used to request page display data of the application; performing a modification operation on the data request to obtain a modified data request, wherein the modified data request is used to request page display data of the application to be acquired; sending the modified data request to the server, so that the server generates a data response corresponding to the modified data request, wherein the data response carries page display data of the application to be acquired; receiving the data response returned by the server, and parsing the data response to acquire page display data of the application.
[0007] According to a second aspect of an embodiment of the present invention, there is provided a device for acquiring data of an application, which is applied to a middleman proxy service tool, wherein the middleman proxy service tool is respectively communicated with an application of a mobile terminal and a server of the application, and the device comprises: a receiving module, which is used to receive a data request generated based on an automated control operation and sent by the application, wherein the data request is used to request page display data of the application; a modifying module, which is used to perform a modification operation on the data request to obtain a modified data request, wherein the modified data request is used to request page display data of the application to be acquired; a sending module, which is used to send the modified data request to the server, so that the server generates a data response corresponding to the modified data request, wherein the data response carries page display data of the application to be acquired; and a parsing module, which is used to receive the data response returned by the server, and parse the data response to acquire page display data of the application.
[0008] According to a third aspect of an embodiment of the present invention, there is provided an electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the data acquisition method of the application as described in the first aspect.
[0009] According to a fourth aspect of an embodiment of the present invention, there is provided a computer storage medium on which a computer program is stored. When the program is executed by a processor, the data acquisition method for the application program as described in the first aspect is implemented.
[0010] The data acquisition scheme for an application provided in an embodiment of the present invention is applied to a middleman proxy service tool that is respectively connected to the application of a mobile terminal and the server of the application. The middleman proxy service tool performs a modification operation on a data request generated by the received application based on an automated control operation, obtains a request for requesting the page display data of the application to be obtained, and sends the request to the server, so that the server generates a data response corresponding to the request, and then parses the data response returned by the server to obtain the page display data of the application carried in the data response. There is no need to invest a lot of time and effort in the early stage to complete the cracking of the encryption request, and the page display data of the application of the mobile terminal can be easily and quickly acquired. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.
[0012] Figure 1A This is a flowchart of the steps of the data acquisition method of the application in the first embodiment of the present application;
[0013] Figure 1B This is a schematic diagram of the interaction of the middleman proxy service tool provided according to the first embodiment of the present application;
[0014] Figure 2 This is a schematic diagram of the structure of a data acquisition device for an application in Embodiment 2 of the present application;
[0015] Figure 3 This is a schematic diagram of the structure of the electronic device in Example 3 of the present application. DETAILED DESCRIPTION
[0016] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in the field based on the embodiments in the embodiments of the present invention should fall within the scope of protection of the embodiments of the present invention.
[0017] The specific implementation of the embodiment of the present invention is further described below in conjunction with the accompanying drawings of the embodiment of the present invention.
[0018] Reference Figure 1A, showing a step flow chart of the data acquisition method of the application in the first embodiment of the present application.
[0019] This embodiment describes the data acquisition method of the application provided by the embodiment of the present invention from the perspective of the middleman proxy service tool. The middleman proxy service tool is respectively connected to the application of the mobile terminal and the server of the application. Specifically, the data acquisition method of the application provided by this embodiment includes the following steps:
[0020] In step S101, a data request generated based on an automated control operation and sent by the application is received.
[0021] In this embodiment, the data request is used to request page display data of the application, for example, data related to enterprise introduction, data related to enterprise products, etc. It can be understood that the above description is only exemplary, and this embodiment does not make any limitation to this.
[0022] In a specific example, the automation control plug-in of the mobile terminal performs an automation control operation on the control of the page of the application according to the automation control instruction, the application receives the automation control operation on the control of the page of the application, generates the data request based on the automation control operation, and then sends the generated data request to the middleman proxy service tool. It can be understood that the above description is only exemplary and this embodiment does not impose any limitation on this.
[0023] In some optional embodiments, before receiving the data request generated based on the automated control operation sent by the application, the method further includes: calling the automated control interface of the application, sending the automated control instruction of the application to the automated control plug-in of the mobile terminal, so that the automated control plug-in performs automated control operations on the controls of the page of the application according to the automated control instructions. In this way, the automated control operations can be effectively performed on the controls of the page of the application. It can be understood that the above description is only exemplary and this embodiment does not impose any limitation on this.
[0024] In a specific example, the automation control interface can be understood as an application program interface with the function of automatically controlling the application program, and the automation control interface can be written in programming languages such as Java, C, C++, C#, PHP, Python, and VB. The automation control plug-in can be understood as a plug-in with the function of automatically controlling the application program of the mobile terminal. Specifically, uiautomator2 is used to obtain the control properties of the application program on the screen of the mobile terminal, and based on the control properties of the application program on the screen of the mobile terminal, the application program interface of uiautomator2 is called to write the automation control plug-in. Among them, uiautomator2 is a library that can use Python to perform UI automation on Android devices. Its bottom layer is based on Google uiautomator. The uiautomator library provided by Google can obtain any control property of any application program on the screen of the mobile terminal and perform any operation on it. In addition to using uiautomator2 to realize the automatic control of the application program of the mobile terminal, mobile terminal automation testing tools such as appnium, monkey, and Instrumentation can also be used to realize automatic operation of the mobile terminal. The mobile terminal may include a mobile phone, a tablet computer, etc. It can be understood that the above description is only exemplary, and this embodiment does not make any limitation to this.
[0025] In a specific example, before calling the automation control interface of the application, the mobile terminal is configured accordingly. Specifically, after downloading an application with data acquisition requirements in a mobile phone connected to the Internet, the mobile phone can be connected via USB, or the same local area network can be used together to bind the mobile phone to the device where the middleman proxy service tool is located. After binding the mobile phone to the device where the middleman proxy service tool is located, the middleman proxy service tool can be implemented through the open source framework Mitmproxy based on Python middleman proxy, and the middleman proxy service tool can also be implemented through the middleman proxy framework Anyproxy developed based on nodejs. After implementing the middleman proxy service tool, the IP address and port of the device where the middleman proxy service tool is located are filled in the mobile phone, and the mobile phone requests the middleman proxy framework website to install the certificate, and then the certificate is set to trust in the mobile phone. In this way, the request of the mobile phone and the returned response will pass through the middleman proxy service tool. It can be understood that the above description is only exemplary, and this embodiment does not make any limitation to this.
[0026] In step S102, a modification operation is performed on the data request to obtain a modified data request.
[0027] In this embodiment, the modified data request is used to request the page display data of the application to be obtained, such as the classification of enterprise products, enterprise contact information, etc. It can be understood that the above description is only exemplary and this embodiment does not make any limitation to this.
[0028] In some optional embodiments, when performing a modification operation on the data request, a modification operation is performed on the request content parameter of the request content field in the data request to obtain the modified data request. Thus, by performing a modification operation on the request content parameter of the request content field in the data request, the page display data of the application to be obtained can be accurately requested. It can be understood that the above description is only exemplary and this embodiment does not impose any limitation on this.
[0029] In a specific example, if the request content parameter of the request content field in the data request before modification is the product introduction of the enterprise, the request content parameter of the request content field in the data request after modification may be the product classification of the enterprise. If the request content parameter of the request content field in the data request before modification is the product classification of the enterprise, the request content parameter of the request content field in the data request after modification may be the contact information of the enterprise. It can be understood that the above description is only exemplary and this embodiment does not impose any limitation on this.
[0030] In step S103, the modified data request is sent to the server, so that the server generates a data response corresponding to the modified data request.
[0031] In this embodiment, the data response carries the page display data of the application to be acquired, such as the classification of enterprise products, enterprise contact information, etc. It can be understood that the above description is only exemplary, and this embodiment does not make any limitation to this.
[0032] In step S104, the data response returned by the server is received, and the data response is parsed to obtain the page display data of the application.
[0033] In some optional embodiments, the method further includes: if it is detected that there is a field for encrypting parameters in the data request, then obtaining the encrypted parameters from the field in the data request and saving the encrypted parameters; within the validity period of the encrypted parameters, constructing multiple requests for requesting the page display data of the application to be obtained according to the encrypted parameters. Among them, the field for encrypting parameters includes a cookie field and / or a token field in the data request. Thus, by reusing the encrypted parameters, multiple requests for requesting the page display data of the application to be obtained can be constructed, thereby effectively improving the efficiency of obtaining the page display data of the application. It can be understood that the above description is only exemplary, and this embodiment does not impose any limitation on this.
[0034] In a specific example, if the data request encrypts the cookie field and / or token field, the encrypted parameters in the cookie field and / or token field can be obtained, and the encrypted parameters in the cookie field and / or token field can be saved. Generally, the encrypted parameters in the cookie field and / or token field have a validity period. Within this validity period, the middleman proxy service tool calls the request construction interface, and constructs multiple requests for requesting the page display data of the application to be obtained according to the encrypted parameters. Among them, the request construction interface can be understood as an application program interface with a request construction function, and the request construction interface can be written in programming languages such as Java, C, C++, C#, PHP, Python, VB, etc. It can be understood that the above description is only exemplary, and this embodiment does not impose any limitation on this.
[0035] In some optional embodiments, the method further includes: when the middleman proxy service tool receives multiple data responses returned by the server at the same time point, calling a data response filtering interface, performing a filtering operation on the multiple data responses received by the middleman proxy service tool at the same time point to obtain a target data response, and parsing the target data response to obtain the page display data of the application. Thus, by calling the data response filtering interface and performing a filtering operation on the multiple data responses received by the middleman proxy service tool at the same time point, the target data response can be accurately obtained, and then the page display data of the application can be accurately obtained. It can be understood that the above description is only exemplary, and this embodiment does not impose any limitation on this.
[0036] In a specific example, the data response screening interface can be understood as an application program interface with a data response screening function, and the data response screening interface can be written in Java, C, C++, C#, PHP, Python, VB, etc. It can be understood that the above description is only exemplary and this embodiment does not impose any limitation on this.
[0037] In some optional embodiments, when the data response filtering interface is called to perform a filtering operation on the multiple data responses received by the middleman proxy service tool at the same time point, the data response filtering interface is called to perform a filtering operation on the multiple data responses according to the destination addresses of the multiple data responses to obtain the target data response. Thus, through the data response filtering interface, the multiple data responses are filtered according to the destination addresses of the multiple data responses, and the target data response can be accurately obtained. It can be understood that the above description is only exemplary and this embodiment does not make any limitation to this.
[0038] In a specific example, through the data response screening interface, the address corresponding to the page display data of the application to be obtained in the application of the mobile terminal is determined, and the address is compared with the destination addresses of the multiple data responses respectively. If the destination addresses of some data responses in the multiple data responses are the same as the address, the partial data responses are determined to be target data responses. It can be understood that the above description is only exemplary and this embodiment does not impose any limitation on this.
[0039] In some optional embodiments, the number of the mobile terminals is multiple, and the multiple mobile terminals are respectively connected to the device where the middleman proxy service tool is located through the interface of the hub. In this way, the page display data of the application programs of the multiple mobile terminals can be obtained simply and quickly. It can be understood that the above description is only exemplary and this embodiment does not make any limitation to this.
[0040] In a specific example, the middleman proxy service tool calls the automation control interface, and sends automation control instructions of different applications of different mobile terminals to the automation control plug-ins of different mobile terminals, so that the automation control plug-ins of different mobile terminals perform automation control operations on the controls of the pages of different applications of the different mobile terminals according to the automation control instructions, and perform modification operations on the data requests generated based on the automation control operations and sent by the different applications of the different mobile terminals, obtain a request for requesting the page display data of the application to be obtained, and forward the request to the server, so that the server generates a data response corresponding to the request, and then parses the data response returned by the server to obtain the page display data of the application carried in the data response. It can be understood that the above description is only exemplary, and this embodiment does not impose any limitation on this.
[0041] In a specific example, Figure 1B As shown, the middleman proxy service tool sends the automation control instruction of the application to the automation control plug-in of the mobile phone by using the automation control interface compiled by the uiautomator library, so that the automation control plug-in performs the automation control operation on the control of the page of the application according to the automation control instruction, and the application receives the automation control operation for the control of the page of the application, and generates the http / https request based on the automation control operation, and then sends the generated http / https request to the middleman proxy service tool implemented by the open source framework Mitmproxy using the middleman proxy, the middleman proxy service tool performs the modification operation on the http / https request, and sends the modified http / https request to the server corresponding to the application, the server generates the data response corresponding to the modified http / https request, and returns the data response to the middleman proxy service tool, the middleman proxy service tool screens the data response, and retains the page display data of the application to be obtained carried in the data response, and then returns the data response to the application of the mobile phone. In addition, if it is detected that there is a field for encrypting parameters in the http / https request before modification, the encrypted parameters are obtained from the field and the encrypted parameters are saved. Within the validity period of the encrypted parameters, multiple http / https requests for requesting the page display data of the application to be obtained are constructed according to the encrypted parameters. It can be understood that the above description is only exemplary and this embodiment does not make any limitation to this.
[0042] The data acquisition method for an application provided by an embodiment of the present invention is applied to a middleman proxy service tool that is respectively connected to the application of a mobile terminal and the server of the application. The middleman proxy service tool performs a modification operation on a data request generated by the received application based on an automated control operation, obtains a request for requesting page display data of the application to be obtained, and sends the request to the server, so that the server generates a data response corresponding to the request, and then parses the data response returned by the server to obtain the page display data of the application carried in the data response. There is no need to invest a lot of time and effort in the early stage to complete the cracking of the encryption request, and the page display data of the application of the mobile terminal can be easily and quickly obtained.
[0043] The data acquisition method of the application provided in this embodiment can be executed by any appropriate device with data processing capabilities, including but not limited to: cameras, terminals, mobile terminals, PCs, servers, vehicle-mounted devices, entertainment equipment, advertising equipment, personal digital assistants (PDAs), tablet computers, laptop computers, handheld game consoles, smart glasses, smart watches, wearable devices, virtual display devices or display enhancement devices, etc.
[0044] Reference Figure 2 , showing a schematic diagram of the structure of the data acquisition device of the application in the second embodiment of the present application.
[0045] The data acquisition device for the application provided in this embodiment is applied to a middleman proxy service tool, which is respectively communicated with the application of the mobile terminal and the server of the application, and the device includes: a receiving module 201, which is used to receive a data request generated based on an automated control operation and sent by the application, and the data request is used to request page display data of the application; a modification module 202, which is used to perform a modification operation on the data request to obtain a modified data request, and the modified data request is used to request page display data of the application to be obtained; a sending module 203, which is used to send the modified data request to the server, so that the server generates a data response corresponding to the modified data request, and the data response carries the page display data of the application to be obtained; a parsing module 204, which is used to receive the data response returned by the server, and parse the data response to obtain page display data of the application.
[0046] Optionally, before the receiving module 201, the device also includes: a calling module, used to call the automation control interface of the application, and send the automation control instructions of the application to the automation control plug-in of the mobile terminal, so that the automation control plug-in performs automation control operations on the controls of the page of the application according to the automation control instructions.
[0047] Optionally, the modification module 202 is specifically configured to: perform a modification operation on a request content parameter of a request content field in the data request to obtain the modified data request.
[0048] Optionally, the device also includes: an acquisition module, which is used to obtain the encrypted parameters from the field in the data request and save the encrypted parameters if it is detected that there is a field for encrypting parameters in the data request; a construction module, which is used to construct multiple requests for requesting page display data of the application to be obtained according to the encrypted parameters within the validity period of the encrypted parameters.
[0049] Optionally, the device also includes: an execution module, which is used to call a data response filtering interface when the middleman proxy service tool receives multiple data responses returned by the server at the same time point, and perform a filtering operation on the multiple data responses received by the middleman proxy service tool at the same time point to obtain a target data response, and parse the target data response to obtain the page display data of the application.
[0050] Optionally, the execution module is specifically used to: call a data response screening interface, and perform a screening operation on the multiple data responses according to destination addresses of the multiple data responses to obtain the target data response.
[0051] Optionally, there are multiple mobile terminals, and the multiple mobile terminals are respectively connected to the device where the middleman proxy service tool is located through the interface of the hub.
[0052] The application data acquisition device provided in this embodiment is used to implement the corresponding application data acquisition methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.
[0053] Reference Figure 3 , shows a schematic diagram of the structure of an electronic device according to Embodiment 3 of the present invention. The specific embodiment of the present invention does not limit the specific implementation of the electronic device.
[0054] like Figure 3As shown, the electronic device may include: a processor (processor) 302 , a communication interface (Communications Interface) 304 , a memory (memory) 306 , and a communication bus 308 .
[0055] in:
[0056] The processor 302 , the communication interface 304 , and the memory 306 communicate with each other via the communication bus 308 .
[0057] The communication interface 304 is used to communicate with other electronic devices or servers.
[0058] The processor 302 is used to execute the program 310, and specifically can execute the relevant steps in the embodiment of the data acquisition method of the above application program.
[0059] Specifically, the program 310 may include program codes, which include computer operation instructions.
[0060] The processor 302 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.
[0061] The memory 306 is used to store the program 310. The memory 306 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.
[0062] Program 310 can be specifically used to enable the processor 302 to perform the following operations: receive a data request generated based on an automated control operation and sent by the application, the data request being used to request page display data of the application; perform a modification operation on the data request to obtain a modified data request, the modified data request being used to request page display data of the application to be obtained; send the modified data request to the server, so that the server generates a data response corresponding to the modified data request, the data response carrying page display data of the application to be obtained; receive the data response returned by the server, and parse the data response to obtain page display data of the application.
[0063] In an optional embodiment, the program 310 is also used to enable the processor 302 to call the automation control interface of the application before receiving the data request generated based on the automation control operation sent by the application, and send the automation control instructions of the application to the automation control plug-in of the mobile terminal, so that the automation control plug-in performs automation control operations on the controls of the page of the application according to the automation control instructions.
[0064] In an optional embodiment, the program 310 is also used to enable the processor 302 to perform a modification operation on the request content parameter of the request content field in the data request to obtain the modified data request when performing a modification operation on the data request to obtain the modified data request.
[0065] In an optional embodiment, the program 310 is also used to enable the processor 302 to obtain the encrypted parameters from the field in the data request and save the encrypted parameters if it detects that there is a field for encrypting parameters in the data request; within the validity period of the encrypted parameters, construct multiple requests for requesting page display data of the application to be obtained based on the encrypted parameters.
[0066] In an optional embodiment, program 310 is also used to enable processor 302 to call a data response filtering interface when the middleman proxy service tool receives multiple data responses returned by the server at the same time point, perform a filtering operation on the multiple data responses received by the middleman proxy service tool at the same time point to obtain a target data response, and parse the target data response to obtain the page display data of the application.
[0067] In an optional embodiment, the program 310 is also used to enable the processor 302 to call the data response filtering interface and perform a filtering operation on the multiple data responses received by the middleman proxy service tool at the same time point, and to perform a filtering operation on the multiple data responses according to the destination addresses of the multiple data responses to obtain the target data response.
[0068] In an optional implementation, there are multiple mobile terminals, and the multiple mobile terminals are respectively connected to the device where the middleman proxy service tool is located through an interface of a hub.
[0069] The specific implementation of each step in program 310 can refer to the corresponding description of the corresponding steps and units in the data acquisition method embodiment of the above application program, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the above-mentioned method embodiment, which will not be repeated here.
[0070] Through the electronic device of this embodiment, the middleman proxy service tool performs a modification operation on the data request generated by the received application based on the automated control operation, obtains a request for requesting the page display data of the application to be obtained, and sends the request to the server, so that the server generates a data response corresponding to the request, and then parses the data response returned by the server to obtain the page display data of the application carried in the data response. There is no need to invest a lot of time and energy in the early stage to complete the cracking of the encryption request, and the page display data of the application of the mobile terminal can be obtained simply and quickly.
[0071] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present invention can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present invention.
[0072] The above-described method according to an embodiment of the present invention may be implemented in hardware, firmware, or as software or computer code that may be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code that is originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded through a network and will be stored in a local recording medium, so that the method described herein may be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, a processor, or hardware, the data acquisition method for the application described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the data acquisition method for the application shown herein, the execution of the code converts the general-purpose computer into a dedicated computer for executing the data acquisition method for the application shown herein.
[0073] Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present invention.
[0074] The above implementation methods are only used to illustrate the embodiments of the present invention, and are not limitations of the embodiments of the present invention. Ordinary technicians in the relevant technical field may make various changes and modifications without departing from the spirit and scope of the embodiments of the present invention. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present invention. The patent protection scope of the embodiments of the present invention should be defined by the claims.
Claims
1. A method for acquiring data of an application, characterized in that: Applied to a middleman proxy service tool, the middleman proxy service tool is respectively connected to an application of a mobile terminal and a server of the application, the method comprising: receiving a data request generated based on an automated control operation and sent by the application, wherein the data request is used to request page display data of the application; Performing a modification operation on the data request to obtain a modified data request, wherein the modified data request is used to request page display data of the application to be obtained; Sending the modified data request to the server, so that the server generates a data response corresponding to the modified data request, wherein the data response carries the page display data of the application to be obtained; Receiving the data response returned by the server, and parsing the data response to obtain page display data of the application; The performing of a modification operation on the data request to obtain a modified data request includes: Performing a modification operation on the request content parameter of the request content field in the data request to obtain the modified data request; if the request content parameter of the request content field in the data request before modification is the product introduction of the enterprise, the request content parameter of the request content field in the modified data request is the product classification of the enterprise; if the request content parameter of the request content field in the data request before modification is the product classification of the enterprise, the request content parameter of the request content field in the modified data request is the contact information of the enterprise; Wherein, the method further comprises: If it is detected that there is a field for encrypting parameters in the data request, obtaining the encrypted parameters from the field in the data request, and saving the encrypted parameters; Within the validity period of the encrypted parameters, multiple requests for requesting the page display data of the application to be obtained are constructed according to the encrypted parameters, wherein the field for encrypting the parameters includes the cookie field and / or the token field in the data request, and multiple requests for requesting the page display data of the application to be obtained are constructed by reusing the encrypted parameters, and if the data request encrypts the cookie field and / or the token field, the encrypted parameters in the cookie field and / or the token field are obtained, and the encrypted parameters in the cookie field and / or the token field are saved, and the encrypted parameters in the cookie field and / or the token field have a validity period. Within this validity period, the middleman proxy service tool calls the request construction interface, and constructs multiple requests for requesting the page display data of the application to be obtained according to the encrypted parameters; Wherein, the method further comprises: When the middleman proxy service tool receives multiple data responses returned by the server at the same time point, the data response screening interface is called to perform screening operations on the multiple data responses received by the middleman proxy service tool at the same time point to obtain target data responses, and the target data responses are parsed to obtain page display data of the application; The calling of the data response screening interface to perform screening operations on the multiple data responses received by the middleman proxy service tool at the same time point includes: A data response screening interface is called, and a screening operation is performed on the multiple data responses according to the destination addresses of the multiple data responses to obtain the target data response.
2. The method for acquiring data from an application according to claim 1, characterized in that: Before receiving the data request generated based on the automated control operation sent by the application, the method further includes: The automation control interface of the application is called, and the automation control instruction of the application is sent to the automation control plug-in of the mobile terminal, so that the automation control plug-in performs an automation control operation on the control of the page of the application according to the automation control instruction.
3. The method for acquiring data of an application according to claim 1, characterized in that: There are multiple mobile terminals, and the multiple mobile terminals are respectively connected to the device where the middleman proxy service tool is located through the interface of the hub.
4. A data acquisition device for an application, characterized in that: Applied to a middleman proxy service tool, the middleman proxy service tool is respectively connected to the application of the mobile terminal and the server of the application, and the device includes: A receiving module, used for receiving a data request generated based on an automated control operation and sent by the application, wherein the data request is used for requesting page display data of the application; A modification module, used for performing a modification operation on the data request to obtain a modified data request, wherein the modified data request is used for requesting page display data of the application to be obtained; A sending module, used for sending the modified data request to the server, so that the server generates a data response corresponding to the modified data request, wherein the data response carries the page display data of the application to be obtained; A parsing module, used for receiving the data response returned by the server, and parsing the data response to obtain the page display data of the application; Wherein, the modification mold body performs a modification operation on the request content parameter of the request content field in the data request to obtain the modified data request; if the request content parameter of the request content field in the data request before modification is the product introduction of the enterprise, the request content parameter of the request content field in the modified data request is the product classification of the enterprise; if the request content parameter of the request content field in the data request before modification is the product classification of the enterprise, the request content parameter of the request content field in the modified data request is the contact information of the enterprise; The device further includes: an acquisition module, which is used to acquire the encrypted parameters from the field in the data request and save the encrypted parameters if it is detected that there is a field for encrypting parameters in the data request; a construction module, which is used to construct multiple requests for requesting page display data of the application to be acquired according to the encrypted parameters within the validity period of the encrypted parameters, wherein the field for encrypting parameters includes a cookie field and / or a token field in the data request, and multiple requests for requesting page display data of the application to be acquired are constructed by reusing the encrypted parameters, and if the data request encrypts the cookie field and / or the token field, the encrypted parameters in the cookie field and / or the token field are acquired and saved, and the encrypted parameters in the cookie field and / or the token field have a validity period, and within this validity period, the middleman proxy service tool calls the request construction interface, and constructs multiple requests for requesting page display data of the application to be acquired according to the encrypted parameters; The device further includes: an execution module, which is used to call a data response screening interface when the middleman proxy service tool receives multiple data responses returned by the server at the same time point, perform a screening operation on the multiple data responses received by the middleman proxy service tool at the same time point to obtain a target data response, and parse the target data response to obtain the page display data of the application; The execution module is specifically used to call a data response screening interface, and perform a screening operation on the multiple data responses according to the destination addresses of the multiple data responses to obtain the target data response.
5. An electronic device, characterized in that: The device comprises: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the data acquisition method of the application program according to any one of claims 1 to 3.
6. A computer storage medium, characterized in that: A computer program is stored thereon, and when the program is executed by a processor, the data acquisition method of the application program as claimed in any one of claims 1 to 3 is implemented.
Citation Information
Patent Citations
Data acquisition method, device and system
CN110347902A
Token acquisition method, data acquisition system, proxy server and storage medium
CN110620670A
Interface parameter and response data modification method and device
CN110851208A
Function test method and system for application program, proxy server and storage medium
CN111104323A
Data processing method and device, computer readable storage medium and electronic equipment
CN111104577A
