Data acquisition method, apparatus, device, and medium

Abstract

The application provides a data collection method, relates to the technical field of data processing, is applied to a web end, can listen to API request sent by a user when the user accesses a browser of the web end, and records response body content before the API request is sent by modifying a built-in function responding to the API request and injecting a first event into the built-in function, so that the situation that only response related information of the API request can be collected is avoided, and by acquiring the response body information and the response related information, a developer of the web browser product can understand the API request situation without invading the user business, optimize performance, solve problems, and provide better user experience.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data acquisition method, apparatus, device and medium. Background Technology

[0002] The usage of web products by browsers is influenced by many factors, such as server configuration, user client configuration, network conditions, user traffic, program coding, and architectural design. After most web products are launched, it's impossible to collect client-side resource request data in a timely and reasonable manner. If clients experience slow requests, slow rendering, slow responses, or abnormal responses, the product service provider cannot proactively detect it. Even if problems are identified through other feedback methods, the lack of timely recording and storage of resource request results makes it difficult to track and resolve them.

[0003] Furthermore, most web products do not collect web request data after going live, which makes it impossible for product developers to perceive problems in different scenarios for different users. Even if feedback is received offline, the lack of relevant data makes it impossible to conduct relevant analysis. Summary of the Invention

[0004] In view of the above, embodiments of this application provide a data acquisition method, apparatus, device, and medium to overcome or at least partially solve the above problems.

[0005] The first aspect of this application provides a data acquisition method applied to a web application, the method comprising:

[0006] Listen for various types of user requests on the web platform, including at least API requests;

[0007] In the built-in function that responds to the API request, a first event is injected, which is used to obtain the response body content of the API request;

[0008] Based on the API request, obtain the first data of the API request, the first data including response-related information and the response body content;

[0009] Report the first data to the server.

[0010] Optionally, the multiple types of requests also include static resource requests, which include a first static resource request and a second static resource request, and the method further includes;

[0011] Before the first collection script takes effect, the second data of the first static resource request is obtained through GetEntries;

[0012] After the first collection script takes effect, the second static resource request is obtained through PerformanceObserver, and the second data of the second static resource request is obtained;

[0013] The static resource request is obtained by filtering various types of requests from users on the web client according to a first preset filtering strategy. The second data includes second URL information, second time information, and second response size information.

[0014] The second data is reported to the server.

[0015] Optionally, in the built-in function responding to the API request, a first event is injected, including:

[0016] Define a primitive function and assign it the value of a built-in function.

[0017] Define a new function, assign the value of the built-in function to the new function, and add the first event to the new function to obtain the updated built-in function;

[0018] The step of obtaining the first data of the API request includes:

[0019] According to the second preset filtering strategy, various types of requests from users on the web terminal are filtered to obtain the API request;

[0020] The response-related information of the API request is stored in a predefined first local cache object. The response-related information includes at least the first request URL, first time information, and first response body size information of the API request.

[0021] The callback function of the first event injected in the update built-in function is used to determine whether the response body of the API request indicates that the reception is complete.

[0022] Once the response body of the API request is received, the response body is stored in a predefined second local cache object. The response body includes at least the first URL information, the HTTP status code, the business error code, and the business error information.

[0023] Optionally, before reporting the first data to the server, the method further includes:

[0024] Iterate through multiple response-related information entries in the first local cache object, and obtain multiple first request URLs of the API request from the response-related information entries;

[0025] Based on the first request URL, retrieve multiple response body contents that match the multiple first request URLs from the second local cache object;

[0026] The multiple response body contents and the response-related information are merged according to the matching relationship between the first request URL and stored in the first local cache object to obtain the third data of the API request;

[0027] The step of reporting the first data to the server includes reporting the third data in the first local cache object.

[0028] Optionally, the method further includes:

[0029] Store the second data of the first static resource request into a predefined third local cache object;

[0030] The second data of the second static resource request is stored in a predefined fourth local cache object, wherein the second data includes at least: the second request URL of the static resource request, the second time information, and the second response body size information;

[0031] Before storing the first second data of the second static resource request into a predefined fourth local cache object, the index of the first static resource request that matches the second request URL in the first second data is determined in the third local cache object based on the second request URL in the first second data.

[0032] Then, extract the second data of the first static resource request before the index from the third local cache object, and add the second data of the first static resource request before the index to the fourth local cache object.

[0033] Optionally, reporting the second data to the server includes:

[0034] Detect whether the web client triggers any of the following preset events, which include at least one of the following: uninstallation, closing, or view change;

[0035] When the preset event is triggered, the second data is reported.

[0036] Optionally, reporting the second data to the server includes:

[0037] When the priority of the current task on the web client is lower than the priority of the data reporting task that reports the second data to the server, the first function is called to report the second data.

[0038] A second aspect of this application provides a data acquisition device applied to a web interface, the device comprising:

[0039] The listening module is used to listen for various types of requests from users on the web platform, including at least API requests.

[0040] The response body content acquisition module is used to inject a first event into the built-in function that responds to the API request, and the first event is used to acquire the response body content of the API request.

[0041] The first data acquisition module is used to acquire first data of the API request based on the API request, wherein the first data includes response-related information and the response body content;

[0042] The data reporting module is used to report the first data to the server.

[0043] A third aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the data acquisition method as described in the first aspect.

[0044] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the data acquisition method as described in this aspect.

[0045] The beneficial effects of this application are:

[0046] This application provides a data acquisition method, apparatus, device, and medium applied to a web application. The method includes: monitoring various types of requests from users on the web application, including at least API requests; injecting a first event into a built-in function responding to the API request, the first event being used to obtain the response body content of the API request; acquiring first data of the API request based on the API request, the first data including response-related information and the response body content; and reporting the first data to a server.

[0047] It can listen for API requests from users when they access web browsers, and by modifying the built-in function that responds to API requests, it injects a first event into the built-in function. Through the first event, the response body content is recorded before the API request is sent. This avoids the situation where only the response-related information of the API request can be collected. By obtaining the response body information and response-related information, it can help web browser product developers understand the status of API requests, optimize performance, solve problems, and provide a better user experience without intruding on the user's business. Attached Figure Description

[0048] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an undue limitation of this application.

[0049] To more clearly illustrate the technical solution of this application, the drawings used in the description of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a schematic flowchart illustrating a data acquisition method according to an embodiment of this application;

[0051] Figure 2 This is a schematic diagram illustrating the data collection of static resource requests in a data collection method provided in an embodiment of this application;

[0052] Figure 3 This is a schematic diagram illustrating the process of obtaining the response body content of an API request in a data acquisition method provided in an embodiment of this application;

[0053] Figure 4 This is a schematic diagram illustrating the acquisition of response-related information in a data acquisition method according to an embodiment of this application;

[0054] Figure 5 This is a schematic diagram illustrating the acquisition of third data in a data acquisition method provided in an embodiment of this application;

[0055] Figure 6 This is a schematic diagram of the framework of a data acquisition device shown in one embodiment of this application. Detailed Implementation

[0056] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0057] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0058] refer to Figure 1 , Figure 1 This is a flowchart illustrating a data acquisition method according to an embodiment of this application. Figure 1 As shown in the figure, this application provides a data acquisition method applied to a web platform, comprising the following steps:

[0059] In step S101, various types of requests from users on the web client are monitored, including at least API requests.

[0060] In step S102, a first event is injected into the built-in function that responds to the API request. The first event is used to obtain the response body content of the API request.

[0061] In step S103, based on the API request, the first data of the API request is obtained, the first data including response-related information and the response body content;

[0062] In step S104, the first data is reported to the server.

[0063] First, in step S101, when a user starts accessing a web browser, the PerformanceObserver provided by Performance is invoked to listen for various types of requests, including listening for and collecting API requests.

[0064] PerformanceObserver is part of the Web API. It listens for various types of requests made by users in their web browsers. PerformanceObserver is a JavaScript API used to monitor browser performance metrics. It collects and observes performance-related information asynchronously, allowing web browser developers to better understand and optimize their applications' performance.

[0065] These various types of requests include API (Application Programming Interface) requests, static resource requests, file upload requests, AJAX (Asynchronous JavaScript and XML) requests, JSONP (JSON with Padding) requests, WebSocket (a full-duplex communication protocol) requests, and OPTIONS requests.

[0066] Furthermore, in step S102, before the API request is sent, a first event is injected into the built-in function (i.e., the XMLHttpRequest object) that responds to the API request sent by the user. The XMLHttpRequest object is a technology for initiating HTTP requests in a browser to interact with the server. The first event is used to obtain the response body content of the API request, such as the response code, business information, etc.

[0067] As an example, the first event is the readystatechange event, which is triggered when the readyState property of the XMLHttpRequest object changes.

[0068] Furthermore, in step S103, for multiple API requests issued by the user, the first data of each API request is obtained. The first data of each API request includes two parts: response-related information and response body content. The response body content can be obtained through a first event (such as the readystatechange event), and the response-related information can be obtained through PerformanceObserver.

[0069] Finally, in step S104, after obtaining the first data containing response-related information and response body content from multiple API requests, the first data is sent to the server corresponding to the web browser developer, so that the developer can use it for subsequent performance problem analysis and tracking of the web browser product.

[0070] Through the above embodiments, it is possible to listen for API requests issued by users when they access web browsers. By modifying the built-in function that responds to API requests and injecting a first event into the built-in function, the response body content is recorded before sending the API request. This avoids the situation where only response-related information of the API request can be collected. By obtaining the response body information and response-related information, web browser product developers can understand the status of API requests, optimize performance, solve problems, and provide a better user experience without intruding on user business.

[0071] Optionally, the multiple types of requests also include static resource requests, which include a first static resource request and a second static resource request, and the method further includes;

[0072] Before the first collection script takes effect, the second data of the first static resource request is obtained through GetEntries;

[0073] After the first collection script takes effect, the second static resource request is obtained through PerformanceObserver, and the second data of the second static resource request is obtained;

[0074] The static resource request is obtained by filtering various types of requests from users on the web client according to a first preset filtering strategy. The second data includes second URL information, second time information, and second response size information.

[0075] The second data is reported to the server.

[0076] Figure 2 This is a schematic diagram illustrating the data collection of static resource requests in a data collection method provided in an embodiment of this application.

[0077] Specifically, in one embodiment, such as Figure 2 As shown, when a user starts accessing a web browser, the PerformanceObserver provided by Performance is called to listen for various types of requests, including static resource requests. The static resource requests include a first static resource request and a second static resource request. The static resource requests are obtained by filtering the various types of user requests obtained by the callback function of PerformanceObserver according to a first preset filtering strategy.

[0078] The first preset filtering strategy is pre-set according to the user's business scenario. For example, when it is necessary to filter JS requests in static resource requests, the regular expression / (\.js)(\?.*)? $ / can be used to filter out requests that match .js and .js? in the URLs of all kinds of requests, thus obtaining JS requests.

[0079] When a user accesses a web service through a web browser, a first collection script is loaded on the web to perform the data collection task corresponding to the static resource request. Since it cannot be guaranteed that the first collection script is the first static resource loaded, there is a possibility that some static resources may be loaded before the first collection script takes effect. Therefore, this application embodiment uses a first static resource request and a second static resource request to distinguish static resource requests. The first static resource request is the static resource request before the first collection script takes effect, and the second static resource request is the static resource request after the first collection script takes effect.

[0080] Before the first collection script takes effect, the second data of the first static resource request is obtained using GetEntries provided by Performance. Since the GetEntries method has a limit on the number of resources of type `resources` that can be counted (maximum 245), after the first collection script takes effect, the second static resource request needs to be obtained using PerformanceObserver provided by Performance. The second data of the second static resource request is then obtained; this second data includes the second URL information, second time information, and second response size information of the static resource request, such as:

[0081] The second URL information is: the URL for static resource requests;

[0082] The second time information includes:

[0083] fetchStart: The time when the web browser prepares to initiate a static resource request;

[0084] domainLookupStart: Returns the time when the DNS domain lookup started;

[0085] domainLookupEnd: Returns the time when the DNS domain lookup ended;

[0086] connectStart: The time it takes for the web browser to establish a TCP connection;

[0087] connectEnd: The time it took for the web browser to successfully establish a TCP connection;

[0088] requestStart: The time when the web browser sends an HTTP request to the server;

[0089] responseStart: The time when the web browser receives the first byte from the server;

[0090] responseEnd: The time when the web browser receives the last byte from the server;

[0091] The second response size information is:

[0092] decodeBodySize: The size of the response body for static resource requests.

[0093] After obtaining the second data, it reports the second data to the server.

[0094] Through the above embodiments, it is possible to listen for and collect static resource requests using GetEntries and PerformanceObserver provided by Performance, and obtain the second data of all static resource requests through a custom first preset filtering strategy.

[0095] Optionally, the method further includes:

[0096] Store the second data of the first static resource request into a predefined third local cache object;

[0097] The second data of the second static resource request is stored in a predefined fourth local cache object, wherein the second data includes at least: the second request URL of the static resource request, the second time information, and the second response body size information;

[0098] Before storing the first second data of the second static resource request into a predefined fourth local cache object, the index of the first static resource request that matches the second request URL in the first second data is determined in the third local cache object based on the second request URL in the first second data.

[0099] Then, extract the second data of the first static resource request before the index from the third local cache object, and add the second data of the first static resource request before the index to the fourth local cache object.

[0100] Specifically, such as Figure 2 As shown, after obtaining the second data of the first static resource request, the second data of the first static resource request needs to be stored in a predefined third local cache object. The second data of the first static resource request is stored in a predefined fourth local cache object. Both the third local cache object and the fourth local cache object are the user's local cache.

[0101] Furthermore, before storing the first second data of the second static resource request into the predefined fourth local cache object, it is necessary to determine the second request URL in the first second data and identify the index of the first static resource request that matches the second request URL in the first second data in the third local cache object.

[0102] Then, in the third local cache object, the second data of the first static resource request before this index is extracted, and the second data of the first static resource request before this index is added to the fourth local cache object.

[0103] Through the above embodiments, after the first collection script takes effect, before storing the second data of the second static resource request in the fourth local cache object, the second data of the first static resource request obtained in the third local cache object is transferred to the fourth local cache object.

[0104] Figure 3 This is a schematic diagram illustrating the process of obtaining the response body content of an API request in a data acquisition method provided in an embodiment of this application, as shown below. Figure 3 As shown.

[0105] Optionally, step S102 includes:

[0106] Define a primitive function and assign it the value of a built-in function.

[0107] Define a new function, assign the value of the built-in function to the new function, and add the first event to the new function to obtain the updated built-in function;

[0108] The step of obtaining the first data of the API request includes:

[0109] According to the second preset filtering strategy, various types of requests from users on the web terminal are filtered to obtain the API request;

[0110] The response-related information of the API request is stored in a predefined first local cache object. The response-related information includes at least the first request URL, first time information, and first response body size information of the API request.

[0111] The callback function of the first event injected in the update built-in function is used to determine whether the response body of the API request indicates that the reception is complete.

[0112] Once the response body of the API request is received, the response body is stored in a predefined second local cache object. The response body includes at least the first URL information, the HTTP status code, the business error code, and the business error information.

[0113] Specifically, in one embodiment, a primitive function named originOpen is first defined, and the value of the built-in function (the value of the built-in function is XMLHttpRequest.prototype.open) is assigned to the primitive function named originOpen, thereby saving the Open function of the XMLHttpRequest object. Further, a new function named newOriginOpen is defined, and the value of the built-in function is also assigned to the new function named newOriginOpen. In the XMLHttpRequest instance that responds to the current API request, a listener for the readystatechange event (i.e., the first event) is added. In the callback function of the readystatechange event, it is possible to check whether the status of the XMLHttpRequest instance is "received successfully".

[0114] The first event (readystatechange event) is triggered in the following stages:

[0115] Status code 0 (UNSENT): The API request has not been initialized and the open method has not been called.

[0116] Status code 1 (OPENED): The API request has been initialized, but the send method has not yet been called.

[0117] Status code 2 (HEADERS_RECEIVED): The send method has been called and the headers and status are available.

[0118] Status code 3 (LOADING): Receiving response data.

[0119] Status code 4 (DONE): The response data has been received and can be processed further.

[0120] Therefore, when the status code is 4, it indicates that the reception is complete. At this time, the response body content can be obtained, and the original function can be called and executed, namely the original function named originOpen.

[0121] Figure 4 This is a schematic diagram illustrating the acquisition of response-related information in a data acquisition method according to an embodiment of this application, as shown below. Figure 4 As shown.

[0122] Furthermore, according to the second preset filtering strategy, various types of requests from web users are filtered to obtain multiple API requests. The response information of the API requests is stored in a predefined first local cache object, wherein the response information includes at least the first request URL, the first time information, and the first response body size information of the API request.

[0123] The response-related information includes the first URL information, the first time information, and the first response body size information, such as:

[0124] The first URL information is: the URL of the API request;

[0125] First-hand information includes:

[0126] fetchStart: The time when the web browser prepares to initiate an API request;

[0127] domainLookupStart: Returns the time when the DNS domain lookup started;

[0128] domainLookupEnd: Returns the time when the DNS domain lookup ended;

[0129] connectStart: The time it takes for the web browser to establish a TCP connection;

[0130] connectEnd: The time it took for the web browser to successfully establish a TCP connection;

[0131] requestStart: The time when the web browser sends an HTTP request to the server;

[0132] responseStart: The time when the web browser receives the first byte from the server;

[0133] responseEnd: The time when the web browser receives the last byte from the server;

[0134] The size information of the first response is:

[0135] decodeBodySize: The size of the response body for the API request.

[0136] Furthermore, by updating the callback function of the first event injected in the built-in function, it is determined whether the response body of the API request has been received successfully. When the response body of the API request has been received successfully, the response body is stored in a predefined second local cache object. The response body includes at least the first URL information, the HTTP status code, the business error code, and the business error information. The first local cache object and the second local cache object are the user's local cache.

[0137] Through the above embodiments, all API requests can be filtered out from the pre-set second preset filtering strategy, and the first data of the API requests can be cached locally.

[0138] Optionally, before step S104, the method further includes:

[0139] Iterate through multiple response-related information entries in the first local cache object, and obtain multiple first request URLs of the API request from the response-related information entries;

[0140] Based on the first request URL, retrieve multiple response body contents that match the multiple first request URLs from the second local cache object;

[0141] The multiple response body contents and the response-related information are merged according to the matching relationship between the first request URL and stored in the first local cache object to obtain the third data of the API request;

[0142] The step of reporting the first data to the server includes reporting the third data in the first local cache object.

[0143] Figure 5 This is a schematic diagram illustrating the acquisition of third data in a data acquisition method according to an embodiment of this application, as shown below. Figure 5 As shown.

[0144] Specifically, in one embodiment, since the first data to be collected for an API request includes two parts: response-related information and response body content, which are stored in a first local cache object and a second local cache object respectively, the two parts need to be merged into the same local cache before the first data is reported to the server.

[0145] Therefore, the process first iterates through the multiple response-related information entries that need to be reported in the first local cache object, and determines the first URL information in each of these entries, resulting in multiple first URL entries. Based on these first URL entries, the response body content corresponding to each first URL entry in the second local cache object is found. The response body content and response-related information corresponding to the same first URL entry are then merged, i.e., the response body content is added to the corresponding response-related information and stored in the first local cache object, resulting in third data. The third data consists of multiple data entries that have combined response-related information and response body content.

[0146] Finally, the third-party data is uploaded to the server.

[0147] Through the above embodiments, before uploading the first data to the server, the response-related information and response body content in the first data that the API request needs to collect are merged according to the first URL information to obtain the third data, and then the third data is uploaded to the server.

[0148] Optionally, the method further includes:

[0149] To prevent users from caching too much data locally, the length of the first data of the third data in the first local cache object and the length of the first data of the second data in the fourth local cache object can be monitored in real time. When the length of the first data or the length of the second data exceeds a preset data length threshold, a preset proportion of the second data or the third data is taken from the first local cache object or the fourth local cache object and uploaded to the server.

[0150] As an example, the preset data length threshold is 100. When the first data length or the second data length reaches 100, the data with index 0 to 50 is retrieved from the first local cache object or the fourth local cache object and reported to the server. The second or third data that has been uploaded to the server from the first local cache object or the fourth local cache object is deleted periodically.

[0151] Through the above embodiments, data can be uploaded through a quantitative upload mechanism without affecting the performance of the user's web browser. Furthermore, the local cache is cleared after the upload to avoid excessive use of the user's local cache space.

[0152] Optionally, reporting the second data to the server includes:

[0153] Detect whether the web client triggers any of the following preset events, which include at least one of the following: uninstallation, closing, or view change;

[0154] When the preset event is triggered, the second data is reported.

[0155] Specifically, the data reporting process involves several scenarios, each requiring a different reporting method. When a user leaves the web browser page they are currently accessing, such as when the user minimizes or switches views, making the website invisible on their screen, the visibility event can be used to trigger the reporting of the second and third data. When a user closes the current browser tab or closes the entire browser, the unload event can be used to monitor and report the second and third data.

[0156] The above embodiments ensure that data is promptly reported to the server when a user leaves or closes the page, avoiding data loss and delayed reporting.

[0157] Optionally, reporting the second data to the server includes:

[0158] When the priority of the current task on the web client is lower than the priority of the data reporting task that reports the second data to the server, the first function is called to report the second data.

[0159] Specifically, in one embodiment, when the priority of the current task on the web client is lower than the priority of the data reporting task that reports the second data to the server, it is determined that the browser on the web client is in an idle state. At this time, the first function (such as the requestIdlecallback function) can be called to report the second or third data.

[0160] Furthermore, when XMLHttpRequest becomes unavailable, the browser's built-in function (Navigator.sendBeacon function) can be called as an independent process task to complete the data reporting, thus ensuring that the second or third data can be successfully reported to the server without delaying the browser's normal closing behavior.

[0161] Through the above embodiments, it is possible to perform the data reporting process in the idle state, and to complete the data reporting by calling the browser's built-in function (Navigator.sendBeacon function) when XMLHttpRequest is unavailable.

[0162] refer to Figure 6 , Figure 6 This is a schematic diagram of the framework of a data acquisition device according to an embodiment of this application, as shown below. Figure 6As shown in the figure, this application provides a data acquisition device applied to a web interface, the device comprising:

[0163] The listening module 11 is used to listen for various types of requests from users on the web client, including at least API requests.

[0164] The response body content acquisition module 12 is used to inject a first event into the built-in function that responds to the API request, and the first event is used to acquire the response body content of the API request.

[0165] The first data acquisition module 13 is used to acquire first data of the API request based on the API request, wherein the first data includes response-related information and the response body content;

[0166] The data reporting module 14 is used to report the first data to the server.

[0167] Optionally, the multiple types of requests also include static resource requests, which include a first static resource request and a second static resource request, and the apparatus further includes;

[0168] The second data acquisition module for the first static resource request is used to acquire the second data of the first static resource request through GetEntries before the first acquisition script takes effect.

[0169] The second data acquisition module for the second static resource request is used to acquire the second static resource request through PerformanceObserver and acquire the second data of the second static resource request after the first acquisition script takes effect.

[0170] The static resource request is obtained by filtering various types of requests from users on the web client according to a first preset filtering strategy. The second data includes second URL information, second time information, and second response size information.

[0171] The second data reporting module is used to report the second data to the server.

[0172] Optionally, the response body content retrieval module includes:

[0173] The first definition module is used to define primitive functions and assign the values ​​of built-in functions to the primitive functions.

[0174] The second definition module is used to define a new function, assign the new function to the value of the built-in function, and add the first event to the new function to obtain the updated built-in function;

[0175] The first data acquisition module 13 includes:

[0176] The first filtering module is used to filter various types of requests from users on the web terminal according to a second preset filtering strategy to obtain the API request.

[0177] A response-related information storage module is used to store the response-related information of the API request in a predefined first local cache object. The response-related information includes at least the first request URL, first time information, and first response body size information of the API request.

[0178] The determination module is used to determine whether the response body content of the API request is "received successfully" by using the callback function of the first event injected in the update built-in function;

[0179] The response body content storage module is used to store the response body content in a predefined second local cache object after the API request response body content has been received. The response body content includes at least a first URL information, an HTTP status code, a business error code, and business error information.

[0180] Optionally, the device further includes:

[0181] The traversal module is used to traverse multiple response-related information entries in the first local cache object before reporting the first data to the server, and to obtain multiple first request URLs of the API request from the response-related information entries;

[0182] The retrieval module is used to retrieve multiple response body contents that match the multiple first request URLs from the second local cache object based on the first request URL;

[0183] The merging module is used to merge the multiple response body contents and the response-related information according to the matching relationship between the first request URL, and store them in the first local cache object to obtain the third data of the API request;

[0184] The first data reporting module is used to report the first data to the server, including: reporting the third data in the first local cache object.

[0185] Optionally, the device further includes:

[0186] The second data storage module for the first static resource request is used to store the second data of the first static resource request into a predefined third local cache object;

[0187] The second data storage module for the second static resource request is used to store the second data of the second static resource request into a predefined fourth local cache object, wherein the second data includes at least: the second request URL of the static resource request, the second time information, and the second response body size information;

[0188] The matching module is used to determine, before storing the first second data of the second static resource request into a predefined fourth local cache object, the index of the first static resource request that matches the second request URL in the first second data in the third local cache object;

[0189] The interception module is used to intercept the second data of the first static resource request before the index in the third local cache object, and add the second data of the first static resource request before the index to the fourth local cache object.

[0190] Optionally, the second data reporting module includes:

[0191] The detection module is used to detect whether the web client triggers the following preset events, which include at least one of the following: uninstallation, closing, and view change;

[0192] The trigger reporting module is used to report the second data when the preset event is triggered.

[0193] Optionally, the second data reporting module includes:

[0194] The second data reporting unit is used to call the first function and report the second data when the priority of the current task on the web client is lower than the priority of the data reporting task that reports the second data to the server.

[0195] Based on the same inventive concept, another embodiment of this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the data acquisition method as described in any of the above embodiments.

[0196] Based on the same inventive concept, another embodiment of this application also provides a computer program product, including a computer program that is executed by a processor as the data acquisition method described in any of the above embodiments.

[0197] Based on the same inventive concept, another embodiment of this application provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the data acquisition method as described in any of the above embodiments.

[0198] As the apparatus is basically similar to the method embodiment, it is described in a relatively simple way. For relevant details, please refer to the description of the method embodiment.

[0199] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0200] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0201] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0202] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0203] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0204] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0205] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes the element.

[0206] The above provides a detailed description of a data acquisition method, apparatus, device, and medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A data acquisition method, characterized in that, When applied to the web, the method includes: Listen for various types of user requests on the web platform, including at least API requests; In the built-in function that responds to the API request, a first event is injected, which is used to obtain the response body content of the API request; Based on the API request, obtain the first data of the API request, the first data including response-related information and the response body content; Report the first data to the server; The various types of requests also include static resource requests, which include a first static resource request and a second static resource request; the method further includes: Before the first collection script takes effect, the second data of the first static resource request is obtained through GetEntries; After the first collection script takes effect, the second static resource request is obtained through PerformanceObserver, and the second data of the second static resource request is obtained; The static resource request is obtained by filtering various types of requests from users on the web client according to a first preset filtering strategy. Report the second data to the server; The method further includes: Store the second data of the first static resource request into a predefined third local cache object; The second data of the second static resource request is stored in a predefined fourth local cache object, wherein the second data includes at least: the second request URL of the static resource request, the second time information, and the second response body size information; Before storing the first second data of the second static resource request into a predefined fourth local cache object, the index of the first static resource request that matches the second request URL in the first second data is determined in the third local cache object based on the second request URL in the first second data. Then, extract the second data of the first static resource request before the index from the third local cache object, and add the second data of the first static resource request before the index to the fourth local cache object.

2. The data acquisition method according to claim 1, characterized in that, In the built-in function responding to the API request, a first event is injected, including: Define a primitive function and assign it the value of a built-in function. Define a new function, assign the value of the built-in function to the new function, and add the first event to the new function to obtain the updated built-in function; The step of obtaining the first data of the API request includes: According to the second preset filtering strategy, various types of requests from users on the web terminal are filtered to obtain the API request; The response-related information of the API request is stored in a predefined first local cache object. The response-related information includes at least the first request URL, first time information, and first response body size information of the API request. The callback function of the first event injected in the update built-in function is used to determine whether the response body of the API request indicates that the reception is complete. Once the response body of the API request is received, the response body is stored in a predefined second local cache object. The response body includes at least the first URL information, the HTTP status code, the business error code, and the business error information.

3. The data acquisition method according to claim 2, characterized in that, Before reporting the first data to the server, the method further includes: Iterate through multiple response-related information entries in the first local cache object, and obtain multiple first request URLs of the API request from the response-related information entries; Based on the first request URL, retrieve multiple response body contents that match the multiple first request URLs from the second local cache object; The multiple response body contents and the response-related information are merged according to the matching relationship between the first request URL and stored in the first local cache object to obtain the third data of the API request; The step of reporting the first data to the server includes reporting the third data in the first local cache object.

4. The data acquisition method according to claim 1, characterized in that, The step of reporting the second data to the server includes: Detect whether the web client triggers any of the following preset events, which include at least one of the following: uninstallation, closing, or view change; When the preset event is triggered, the second data is reported.

5. The data acquisition method according to claim 1, characterized in that, The step of reporting the second data to the server includes: When the priority of the current task on the web client is lower than the priority of the data reporting task that reports the second data to the server, the first function is called to report the second data.

6. A data acquisition device, characterized in that, The device, applied to the web, includes: The listening module is used to listen for various types of requests from users on the web platform, including at least API requests. The response body content acquisition module is used to inject a first event into the built-in function that responds to the API request, and the first event is used to acquire the response body content of the API request. The first data acquisition module is used to acquire first data of the API request based on the API request, wherein the first data includes response-related information and the response body content; The data reporting module is used to report the first data to the server; The various types of requests also include static resource requests, which include a first static resource request and a second static resource request; the device also includes: The second data acquisition module for the first static resource request is used to acquire the second data of the first static resource request through GetEntries before the first acquisition script takes effect. The second data acquisition module for the second static resource request is used to acquire the second static resource request through PerformanceObserver and acquire the second data of the second static resource request after the first acquisition script takes effect. The static resource request is obtained by filtering various types of requests from users on the web client according to a first preset filtering strategy. The second data includes second URL information, second time information, and second response size information. The second data reporting module is used to report the second data to the server; The device further includes: The second data storage module for the first static resource request is used to store the second data of the first static resource request into a predefined third local cache object; The second data storage module for the second static resource request is used to store the second data of the second static resource request into a predefined fourth local cache object, wherein the second data includes at least: the second request URL of the static resource request, the second time information, and the second response body size information; The matching module is used to determine, before storing the first second data of the second static resource request into a predefined fourth local cache object, the index of the first static resource request that matches the second request URL in the first second data in the third local cache object, based on the second request URL in the first second data. The interception module is used to intercept the second data of the first static resource request before the index in the third local cache object, and add the second data of the first static resource request before the index to the fourth local cache object.

7. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the data acquisition method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, It stores a computer program, wherein the computer program, when executed by a processor, implements the data acquisition method as described in any one of claims 1-5.

Images