Resource request processing method, device, electronic device and storage medium
By probing the resource status information of the data source station in parallel, the problem of excessively long user waiting time caused by polling origin requests in the CDN system is solved, and the effect of quickly responding to resource request errors is achieved.
Patent Information
- Application Number
- CN202110752689.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-07-02
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2041-07-02
AI Technical Summary
Existing CDN technology, when resources are not cached on the origin server, uses a round-robin approach to request resources from the origin server, combined with timeout settings, which results in excessively long waiting times for users and an inability to quickly respond to resource request errors.
By probing the resource status information of the data source station in real time or in advance in a parallel manner, it can quickly determine whether the target resource exists in the source station, and quickly feed back an error message if it does not exist, thus avoiding the time-consuming polling-style source request.
It reduces user waiting time, improves CDN system response speed, reduces the time spent determining resource availability, and ensures rapid feedback of resource request errors.
Smart Images

Figure CN113453047B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of computer networks, and in particular, relates to a method, device, electronic device and storage medium for processing resource requests. Background Art
[0002] CDN (Content Delivery Network) is an intelligent virtual network built on the Internet. It relies on servers deployed in various places (such as edge servers) and uses the load balancing, content distribution, scheduling and other functions of the central platform to enable users to obtain the required content nearby, thereby reducing network congestion and improving user access response speed and hit rate.
[0003] When the server in the CDN does not cache the resources requested by the user, it pulls the resources indicated by the resource request by performing a back-to-source operation on the source station. If the resource indicated by the resource request does not exist in the source station, the server responds to the client user with information indicating the resource request error (such as a resource request error status code) to disconnect the user request.
[0004] The current CDN technology initiates streaming media requests (back-to-source requests) to each source station in a polling manner and determines whether the required resources exist in the source station based on the set timeout period. This takes a long time. When the source station does not have the resources the user needs, it cannot quickly respond to the user with information indicating a resource request error, which prolongs the user's waiting time. Summary of the invention
[0005] In view of this, the present application provides a resource request processing method, device, electronic device and storage medium for quickly determining whether the required resources exist in the source station, so as to quickly respond to the user with information indicating a resource request error, at least when the source station does not have the data resources indicated by the user request, thereby reducing the user's waiting time.
[0006] The specific technical solutions are as follows:
[0007] A method for processing a resource request, comprising:
[0008] Get the client's resource request;
[0009] If the target resource indicated by the resource request is not cached locally, obtaining the resource status information of the target resource detected in advance or in real time; in the case where there are multiple corresponding data source stations, the real-time detection includes: detecting in real time in parallel whether the target resource exists in each data source station, so as to obtain the resource status information of the target resource;
[0010] Determining whether the target resource exists in the data source station according to the resource status information;
[0011] If not, feedback response information indicating a resource request error to the client;
[0012] In the case that there are multiple corresponding data source stations, the target resource does not exist in the data source station means: the target resource does not exist in any of the multiple data source stations.
[0013] Optionally, detecting the resource status information of the target resource in real time includes:
[0014] Obtaining the source station address of the data source station corresponding to the target resource in real time;
[0015] In the case where the target resource corresponds to multiple data source stations, sending a detection request carrying a resource identifier of the target resource to each data source station in parallel according to the source station address of each data source station;
[0016] Acquire the detection results fed back by each data source station, where the detection results fed back by the data source station are used to indicate whether the target resource exists in the data source station;
[0017] Determine resource status information of the target resource according to the detection results fed back by each data source station;
[0018] If all detection results corresponding to the target resource indicate that the target resource does not exist, the resource status information indicates that the target resource does not exist in the data source station.
[0019] Optionally, pre-detecting resource status information of the target resource includes:
[0020] When the detection condition is met, a detection request carrying a resource identifier of the data resource is sent to each data source station corresponding to the data resource; the data resource is at least one data resource that has been cached locally;
[0021] Acquire detection results fed back by each data source station corresponding to the data resource, where the detection results fed back by the data source station are used to indicate whether the data resource exists in the data source station;
[0022] Determine the resource status information of the data resource according to the obtained detection results; wherein, if all detection results corresponding to the data resource indicate that the data resource does not exist, then the resource status information of the data resource indicates that the data resource does not exist in the data source station;
[0023] A corresponding relationship between the metadata of the data resource and the resource status information is constructed and recorded; the metadata of the data resource at least includes a resource identifier of the data resource.
[0024] Optionally, pre-detecting resource status information of the target resource includes:
[0025] Obtaining a status notification of the data resource sent by each data source station corresponding to the data resource, wherein the status notification is used to indicate that the corresponding data resource is online or offline from the data source station;
[0026] Determine resource status information of the data resource according to status notifications sent by each data source station corresponding to the data resource; wherein, if all data source stations corresponding to the data resource send status notifications indicating that the data resource has gone offline, then the resource status information of the data resource indicates that the data resource does not exist in the data source station;
[0027] A corresponding relationship between the metadata of the data resource and the resource status information is constructed and recorded; the metadata of the data resource at least includes a resource identifier of the data resource.
[0028] Optionally, obtaining the resource status information of the target resource detected in advance includes:
[0029] From the corresponding relationship set between the metadata of the data resource and the resource status information, the resource status information matching the resource identifier of the target resource is searched to obtain the resource status information of the target resource.
[0030] Optionally, the detection request is a Hypertext Transfer Protocol (HTTP) request or a Hypertext Transfer Security (HTTPS) request generated based on a predetermined scripting language.
[0031] Optionally, the target resource is a live streaming media resource or a non-live data resource;
[0032] In the case where the target resource is a live streaming media resource, detecting resource status information of the target resource in real time;
[0033] In the case that the target resource is a non-live data resource, the resource status information of the target resource is detected in advance or in real time.
[0034] Optionally, the above method further includes:
[0035] If the target resource exists in the data source station, determining at least one data source station where the target resource exists according to the resource status information;
[0036] In the case that there are multiple data source stations where the target resource exists, determining a target source station that meets the source station condition from the multiple data source stations where the target resource exists;
[0037] A source request is sent back to the target source station to obtain the target resource from the target source station, and response information including at least part of the target resource is fed back to the client.
[0038] Optionally, the above method further includes:
[0039] In the process of acquiring the target resource from the target source station to implement back-to-source processing, if a resource request for requesting the target resource is obtained again, determining whether the back-to-source processing has timed out;
[0040] If the timeout occurs, the resource status information of the target resource is detected in real time;
[0041] If it has not timed out, continue to obtain the target resource from the target source station.
[0042] A resource request processing device, comprising:
[0043] A first acquisition module, used to acquire a resource request from a client;
[0044] The second acquisition module is used to acquire the resource status information of the target resource detected in advance or in real time when the target resource indicated by the resource request is not cached locally; when there are multiple corresponding data source stations, the real-time detection includes: detecting in real time in parallel whether the target resource exists in each data source station to obtain the resource status information of the target resource;
[0045] A determination module, used to determine whether the target resource exists in the data source station according to the resource status information;
[0046] A feedback module, configured to feed back response information representing a resource request error to the client when the determination result of the determination module is that the resource request does not exist;
[0047] In the case that there are multiple corresponding data source stations, the target resource does not exist in the data source station means: the target resource does not exist in any of the multiple data source stations.
[0048] An electronic device, comprising:
[0049] A memory for storing at least one set of instruction sets;
[0050] The processor is used to implement the resource request processing method as described in any one of the above items by executing the instruction set stored in the memory.
[0051] A computer-readable storage medium stores an instruction set, wherein the instruction set, when executed by a processor, implements the resource request processing method as described in any one of the above.
[0052] According to the above scheme, the resource request processing method, device, electronic device and storage medium provided by the present application, for the obtained resource request, when the target resource indicated by the request is not cached locally, obtains the resource status information of the target resource detected in advance or in real time, determines whether the target resource exists in the data source station based on the detected resource status information, and feeds back response information representing the resource request error when it does not exist. When the present application determines whether the required resources exist in the source station, the resource status information used as the basis for determination is obtained by pre-detection, or is detected in real time in parallel for each source station. Compared with the traditional method of initiating streaming media requests to each source station in a polling manner and making determinations in combination with a set timeout period, the time required for determination is reduced. When the required resources do not exist in the source station, information representing the resource request error can be quickly responded to the user, reducing the user's waiting time. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0054] Figure 1 This is a schematic diagram of the composition architecture of a typical application scenario provided by this application;
[0055] Figure 2 It is a flowchart of a method for processing resource requests provided by this application;
[0056] Figure 3 It is a process diagram of real-time detection of resource status information of target resources provided by the present application;
[0057] Figure 4 It is a schematic diagram of a server provided by the present application initiating detection requests to multiple source stations in parallel through multi-threading;
[0058] Figure 5 It is a process diagram of pre-detecting resource status information of target resources based on active detection method provided by the present application;
[0059] Figure 6 It is a process diagram of pre-detecting resource status information of target resources based on a passive detection method provided by the present application;
[0060] Figure 7 It is another flowchart of the method for processing resource requests provided by the present application;
[0061] Figure 8It is another flowchart of the method for processing resource requests provided by this application;
[0062] Fig. 9 It is a structural diagram of a resource request processing device provided by the present application;
[0063] Fig.10 It is a schematic diagram of the structure of the electronic device provided by this application. DETAILED DESCRIPTION
[0064] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0065] The current CDN technology initiates streaming media requests (back-to-source requests) to each source station in a polling manner and determines whether the required resources exist in the source station in combination with a set timeout period. Specifically, the server in the CDN initiates streaming media requests to the corresponding source stations in a polling manner to pull the resources indicated by the user request from the source station. If the pull fails until the polling ends or the set timeout period is reached, it is determined that the required resources do not exist in the source station, and a status code indicating a resource request error is responded to the client. This method takes a long time because it needs to wait until the polling of each source station ends or the set timeout period is reached before it can be determined that the required resources do not exist in the source station. In the case that the source station does not have the data resources required by the user, it cannot quickly respond to the user with information indicating a resource request error, which prolongs the user's waiting time.
[0066] In order to at least solve the above technical problems, the present application discloses a method, device, electronic device and storage medium for processing resource requests.
[0067] See also Figure 1, provides a composition architecture of a typical application scenario of the present application, including a client 10, a server 20 and an origin station 30, wherein the number of the client 10, the server 20 and the origin station 30 is not limited, and can be one or more respectively, and the server 20 can be but not limited to a server in a CDN, such as an edge server or a central server in a CDN. The client user sends a resource request to a server (such as an edge server) in a CDN, and obtains the required data resources based on the response of the server. After the server obtains the resource request of the client user, it first confirms whether the resource indicated by the resource request is cached locally. If it is not cached locally, it further pulls the required resource by performing a back-to-source operation on the origin station. If the required resource also does not exist in the origin station, the server responds to the client with information representing a resource request error (such as a resource request error status code) to disconnect the client request.
[0068] See also Figure 2 , Figure 2 A flowchart of a method for processing resource requests disclosed in the present application is shown, which can be applied to Figure 1 In the server of the architecture shown, but not limited to this, in other implementations, the method of the present application can also be applied to a management device / management platform independent of the server in the CDN. When the server does not have the target resource indicated by the resource request, the response processing based on resource status detection provided by the method of the present application is implemented with the help of the management device / management platform, and the method of the present application can be implemented in the form of but not limited to an embedded code module, or an independent service / plug-in / middleware.
[0069] The following will mainly apply this application method to Figure 1 The solution is described using the server in the shown architecture as an example.
[0070] like Figure 2 The method of the present application may include the following processing steps:
[0071] Step 201: Obtain the resource request of the client.
[0072] The client user can initiate a resource request to the server for obtaining the target resource by performing corresponding operations on the client. For example, the user can initiate a resource request to the server for obtaining the streaming media data by performing a playback operation for live or on-demand streaming media data on a browser page or an App (Application) page, or initiate a resource request to the server for obtaining the streaming media data by performing a download operation for a data file.
[0073] The server receives resource requests from clients through the network.
[0074] The resource request indicates the target resource to be obtained, which may be a live streaming media resource, or a data resource in any format in a non-live form, such as a non-live data resource in a format of text, picture, video, etc.
[0075] Step 202: If the target resource indicated by the resource request is not cached locally, obtain resource status information of the target resource detected in advance or in real time.
[0076] Wherein, when there are multiple data source stations corresponding to the target data, the above-mentioned real-time detection includes: detecting in real time in parallel whether there is a target resource indicated by the resource request in each data source station to obtain resource status information of the target resource.
[0077] The above-mentioned local refers to the local corresponding to the execution subject of the method of the present application, such as the local end corresponding to the server in the CDN, etc., which is relative to the data source station that is the source end of the data resource.
[0078] CDN improves the user access response speed by caching the data resources pulled from the source site in servers deployed in various places (such as edge servers) and using the load balancing, content distribution, scheduling and other functions of the central platform. After receiving the resource request from the client, the server in the CDN first confirms whether the target resource indicated by the resource request exists in the local cache. If so, it directly responds to the client based on the target resource in the local cache. If not, that is, the target resource indicated by the resource request is not cached locally, it is necessary to further pull the required target resource from the source site.
[0079] In order to avoid the disadvantage that it takes a long time to feed back information indicating a resource request error to the client when the required resources do not exist at the source station, the embodiments of the present application quickly determine whether there is a target resource indicated by the resource request in the data source station by pre-detecting or real-time detecting the resource status information of the target resource at the data source station.
[0080] See also Figure 3 As shown, the process of real-time detection of resource status information of target resources includes:
[0081] Step 301: Obtain the source station address of the data source station corresponding to the target resource in real time.
[0082] When real-time detection is performed on whether the target resource indicated by the resource request exists in the data source station, the source station address of the data source station corresponding to the target resource is first obtained, such as obtaining the IP (Internet Protocol) address of the data source station.
[0083] One data resource usually corresponds to multiple data source stations. Multiple data source stations can back up each other by storing the same data resource separately, so as to support switching to a backup non-faulty source station when a source station fails. In specific application scenarios, data resources (such as live or on-demand streaming media data) can be uploaded to one or more data source stations, or deleted / offlined from one or more source stations according to actual needs.
[0084] In the case that the target resource corresponds to multiple data source stations, this step 301 correspondingly obtains multiple source station addresses of the multiple data source stations corresponding to the target resource, such as multiple IP addresses.
[0085] Step 302: In the case where the target resource corresponds to multiple data source stations, a detection request carrying a resource identifier of the target resource is sent to each data source station in parallel according to the source station address of each data source station.
[0086] When the target resource corresponds to multiple data source stations, this embodiment sends a detection request carrying the resource identifier of the target resource to each data source station in parallel according to the source station address of each data source station, so as to quickly determine whether the target resource indicated by the resource request exists in the data source station.
[0087] Specifically, combined with reference Figure 4 The server creates a corresponding number of threads according to the number of source station addresses. Each thread corresponds to each source station one by one. In each thread, a probe request is initiated to the corresponding data source station according to the corresponding source station address. The information carried in the probe request includes but is not limited to the resource identifier of the target resource and the source station address. The resource identifier of the target resource can be but is not limited to any one or more of a resource ID (Identity document), a resource name, and a resource number.
[0088] After receiving the detection request, each data source station confirms whether it has the target resource indicated by the detection request, such as whether the target resource has not been deleted after going online, the target resource exists in the data source station, or the target resource has been offline from the data source station and the target resource does not exist accordingly. On this basis, the source station further feeds back the corresponding detection result to the server. The detection result can be implemented in the form of a status code or a notification message. Different status codes or different notification messages are used to indicate different states of whether the target resource exists or not in the data source station.
[0089] Optionally, the server can quickly request all source stations corresponding to the target resource through the Lua detection method, and obtain the status code returned by all source stations. In the Lua detection method, the detection request initiated by the server to each data source station can be an HTTP (Hypertext Transfer Protocol) request or HTTPS (Hyper Text Transfer Protocol over SecureSocket Layer) request generated based on Lua (a lightweight and compact scripting language). However, it is not limited to this. In actual applications, corresponding detection requests can also be generated based on other detection methods and other scripting languages.
[0090] The status code returned by the data source station may be, but is not limited to, an HTTP status code, such as HTTP status code 200 indicating that data resources exist in the source station.
[0091] Step 303: Acquire the detection results fed back by each data source station. The detection results fed back by the data source station are used to indicate whether the target resource indicated by the resource request exists in the data source station.
[0092] Step 304: Determine the resource status information of the target resource according to the detection results fed back by each data source station.
[0093] The detection results fed back by different data source stations may be the same or different, depending on the actual offline status of the target data at different data source stations (for example, after the target data is online to three source stations, the target data is offline from one source station based on actual needs).
[0094] The server obtains the detection results fed back by each data source station, and summarizes and integrates them to determine the resource status information of the target resource. If all detection results corresponding to the target resource indicate that the target resource does not exist, such as all detection results are non-200 status codes (all data sources have deleted / downlined the target resource), it means that the target resource indicated by the resource request does not exist in the data source station, and the resource status information that can characterize the non-existence of the target resource in the data source station is determined for the target resource, that is, the resource status information of the target resource.
[0095] On the contrary, if at least one detection result indicates the existence of the target resource, such as the status code 200 exists in each detection result, it means that the target data indicated by the resource request exists in the data source station, and resource status information that can characterize the existence of the target resource in the data source station is determined accordingly.
[0096] The current CDN technology initiates streaming media requests (back-to-source requests) to each source station in a polling manner and determines whether the required resources exist in the source station in combination with a set timeout period. The applicant has found through research that the polling method is used to initiate streaming media requests to different source stations in order to avoid repeatedly pulling the same resource from multiple source stations. Therefore, based on this feature, it is difficult to simply adjust the polling method of initiating requests to initiate requests in parallel to solve the defect of long response time when responding to users with information representing resource request errors. This will lead to other problems (such as causing the same resource to be repeatedly pulled from multiple source stations). In this regard, the present application proposes that before initiating a back-to-source request to the source station, a detection request that does not involve resource pulling / back-to-source is first initiated, and a detection request is initiated to each source station in parallel to effectively solve the above-mentioned problem of long response time and reduce user waiting time.
[0097] There are two ways to detect the resource status information of the target resource in advance: active detection and passive detection.
[0098] Among them, based on the active detection method, the process of pre-detecting the resource status information of the target resource can be implemented as follows: Figure 5 The process shown:
[0099] Step 501: When a detection condition is met, a detection request carrying a resource identifier of a data resource is sent to each data source station corresponding to the data resource; the data resource is at least one data resource that has been cached locally.
[0100] In this implementation, matching detection conditions are set on the server side in advance based on a certain detection strategy. The detection strategy may be, but is not limited to: for all data resources cached on the server side, if the time condition is met, go to the source station to detect whether these data resources have been offline, or combine the attribute characteristics of the data resources to only go to the source station to detect whether the data resources that meet the attribute conditions have been offline from the source station, etc.
[0101] Matchingly, the above detection conditions may specifically include time conditions and / or resource attribute conditions.
[0102] The time condition may refer to reaching a certain fixed time or reaching a set periodic time node.
[0103] Attribute conditions may include, but are not limited to: the type of data resources cached on the server side is a predetermined type, or the data resources include sensitive words, or the associated objects of the data resources are identified as sensitive objects, etc. For example, real-time news is more real-time and is frequently online and offline at the source site; if resources such as articles / videos include sensitive words, or the publisher of the articles / videos is identified as a sensitive person, or the content of the articles / videos involves the introduction of a sensitive person, etc., it can be predicted in advance that such articles / videos may be offline at the source site. For the above situations, matching attribute conditions can be set on the server side, so that the server can filter out data resources that meet the attribute conditions, and go to the source site to detect whether these data resources have been offline, so as to record their resource status information in advance, so as to facilitate direct and rapid response based on the records when receiving client requests for these resources.
[0104] When the detection conditions are met, the server sends a detection request carrying the resource identifier of the data resource to each data source station corresponding to the data resource. The detection request can be initiated to multiple data source stations in parallel or in polling mode (serial mode). Since this implementation method pre-detects the status of the data resource before obtaining the client's resource request, the polling method of initiating the detection request will not affect the subsequent response rate to the client request.
[0105] Step 502: Acquire the detection results fed back by each data source station corresponding to the data resource. The detection results fed back by the data source station are used to indicate whether the data resource indicated by the detection request exists in the data source station.
[0106] The detection result here can also be in the form of, but not limited to, status codes or notification messages. Different status codes or different notification messages are used to indicate different states of whether the target resource exists at the data source station.
[0107] Step 503: Determine resource status information of the data resource according to the detection results fed back by each data source station corresponding to the data resource.
[0108] The server obtains the detection results fed back by each data source station, summarizes and integrates them to determine the resource status information of the data resources.
[0109] Similarly, if all detection results corresponding to a data resource indicate that the data resource does not exist, for example, all detection results are status codes other than 200 (all data source stations have deleted / taken offline the target resource), then it means that the data target resource does not exist in the data source station, and accordingly, resource status information is determined for the data resource that can characterize the non-existence of the data resource in the data source station, that is, the resource status information of the data resource.
[0110] On the contrary, if at least one detection result corresponding to the data resource indicates the existence of the data resource, such as the status code 200 exists in each detection result, it means that the data resource exists in the data source station, and corresponding resource status information that can characterize the existence of the data resource in the data source station is determined for the data resource.
[0111] Step 504: construct and record the corresponding relationship between the metadata of the data resource and the resource status information; the metadata of the data resource at least includes the resource identifier of the data resource.
[0112] On the basis of obtaining the resource status information of each data resource, a correspondence between the metadata of each data resource and its resource status information is further constructed to form a correspondence set between the metadata of the data resource and its resource status information. The correspondence set can be, but is not limited to, represented in the form of a resource status information table.
[0113] The server predetermines and records metadata of each data resource it has cached.
[0114] The metadata of a data resource includes, but is not limited to, at least part of the name of the data resource (such as a file name), an information summary, and information of each corresponding source station (such as a source station address, a source station identifier).
[0115] Based on the correspondence between the metadata and resource status information of a data resource, the correspondence can be synchronously adjusted according to the results of a new round of detection. For example, if a data resource has been offline from all its corresponding source stations, the server can synchronously adjust the correspondence information of the data according to the detection results fed back by each data source station to indicate that the data resource does not exist in the data source station.
[0116] See also Figure 6 In one embodiment, based on the passive detection method, the process of pre-detecting the resource status information of the target resource can be implemented as follows:
[0117] Step 601: Obtain a data resource status notification sent by each data source station corresponding to the data resource, where the status notification is used to indicate that the corresponding data resource is online or offline at the data source station.
[0118] Step 602: Determine resource status information of the data resource according to status notifications sent by each data source station corresponding to the data resource.
[0119] In this implementation, an interaction protocol between the server and the source station is pre-established. Based on the protocol, when a data resource is newly online in the data source station, or a data resource is offline / deleted from the source station, the source station sends a status notification (or callback notification) to the server indicating that a data resource is online or offline / deleted. Subsequently, when the offline / deleted data resource comes back online, the source station sends a status notification again indicating that the data resource is back online.
[0120] The server receives notification information sent by each data source station, summarizes and integrates the notification information, and determines the resource status information of the data resource by summarizing and integrating the notifications sent by each data source station corresponding to the same data resource. If all source stations corresponding to a data resource feedback a status notification indicating that the data resource has been offline, resource status information that can represent that the data resource does not exist in the data source station is determined for the data resource; otherwise, resource status information that can represent that the data resource exists in the data source station is determined for the data resource.
[0121] Step 603: construct and record the corresponding relationship between the metadata of the data resource and the resource status information; the metadata of the data resource at least includes the resource identifier of the data resource.
[0122] Similar to the above-mentioned pre-active detection method, on the basis of obtaining the resource status information of each data resource, a correspondence between the metadata of each data resource and its resource status information is further constructed to form a correspondence set between the metadata of the data resource and its resource status information. The correspondence set can be but is not limited to being expressed in the form of a resource status information table.
[0123] Based on the correspondence between the metadata and resource status information of a data resource, the correspondence can be synchronously adjusted according to the status notification feedback from the data source station. For example, if a data resource has been offline from all its corresponding source stations, the server can synchronously adjust the correspondence information of the data according to each notification received to indicate that the data resource does not exist in the data source station.
[0124] For the acquisition scenario of live streaming media resources, the server responds to the client with real-time live streaming media data by pulling the stream from the source station and pushing the stream to the client. Therefore, when obtaining the client's resource request, the server can use real-time detection to obtain the resource status information of the target resource, so as to determine whether the live streaming media resource requested by the user exists in the data source station.
[0125] For the acquisition scenario of non-live data resources, real-time detection can be used to obtain the resource status information of the target resource, or the resource status information of the target resource can be obtained by querying the resource status information that matches the resource identifier of the target resource from the above-mentioned correspondence set.
[0126] Step 203: Determine whether there is a target resource corresponding to the resource request in the data source station according to the obtained resource status information.
[0127] Step 204: If the resource request does not exist, a response message indicating a resource request error is fed back to the client.
[0128] If it is determined based on the obtained resource status information that the target resource indicated by the resource request does not exist in the data source station, the server feeds back response information indicating the resource request error to the client to disconnect the connection with the client request.
[0129] The response information fed back by the server to the client to indicate a resource request error may be, but is not limited to, a common HTTP error status code, such as 404.
[0130] According to the above scheme, the resource request processing method provided by the present application determines whether the required resources exist in the source station by pre-detecting or real-time detecting the resource status information of the target resource. The resource status information used as the basis for determination is obtained by pre-detection, or by real-time detection of each data source station in parallel. Compared with the traditional method of initiating streaming media requests to each source station in a polling manner and making determinations based on a set timeout period, the time required for determination is reduced. When the required resources do not exist at the source station, information indicating a resource request error can be quickly responded to the user, reducing the user's waiting time.
[0131] Optionally, in one embodiment, in order to facilitate the initiation of a back-to-source request directly to the source station where the target resource exists when the target resource exists in the data source station (rather than initiating a back-to-source request to each source station in a polling manner), the resource status information of the determined target resource can not only represent whether the target resource exists in the data source station, but also, if the target resource exists, indicate at least one data source station where the target resource exists.
[0132] Specifically, the above function can be expanded for the resource status information by recording the source station information of at least one data source station of the target resource, such as the source station identifier or the source station address, in the resource status information of the target resource.
[0133] On this basis, see Figure 7 The flowchart of the method for processing resource requests shown in the figure, after step 203, the method of the present application may further include:
[0134] Step 701: If a target resource exists in a data source station, determine at least one data source station where the target resource exists according to resource status information of the target resource.
[0135] The existence of the target resource in the data source station means that the target resource exists in at least one data source station among all the data source stations corresponding to the target resource.
[0136] If it is determined that the target resource exists in the data source station based on the resource status information of the target resource, further based on the above-mentioned record information in the resource status information of the target resource, at least one data source station of the target resource is determined to facilitate the return of the target data on the server side.
[0137] Step 702: When there are multiple data source stations where the target resource exists, determine a target source station that meets the source station condition from the multiple data source stations where the target resource exists.
[0138] For the real-time detection method, in one embodiment, the above-mentioned source station conditions can be set as conditions related to the response time / response rate of the detection results of the source station, such as setting the source station conditions as: the feedback time of the detection results of the source station is the earliest, or the response rate of the detection results of the source station is the fastest. In another embodiment, the above-mentioned source station conditions can also be set as conditions related to the performance of the source station, such as specifically setting the condition to the source station having the best performance evaluation in any one or more aspects of response rate, congestion situation, load situation, stability, etc. Based on the above conditions, the source station with the earliest feedback time / fastest response rate of the detection results among multiple data source stations where the target resources exist, or the source station with the best performance, can be determined as the target source station for pulling the target data in the back-to-source processing.
[0139] For the pre-detection method, the source station conditions can be set to the above-mentioned conditions related to the performance of the source station, but are not limited to them. The purpose of setting the source station conditions is to determine a single source station from multiple data source stations where the target data exists for data backhaul. In specific application scenarios, the source station conditions are not limited to the several settings provided above, and can be flexibly configured according to actual needs.
[0140] When there is only one data source station with target resources, the data source station can be directly determined as the target source station for pulling target data in the back-to-source processing.
[0141] Step 703: Send a source request back to the target source station to obtain the target resource indicated by the resource request from the target source station, and feed back response information including at least part of the target resource to the client.
[0142] After selecting a target source station according to the source station conditions, the server further sends a back-to-source request carrying the resource identifier of the target resource to the target source station to request to pull the required target data from the target source station. The back-to-source request initiated to the target source station can be, but is not limited to, a streaming media request.
[0143] The server pulls the target data from the target source station, implements the back-to-source processing of the target data, and feeds back the response information including at least part of the target resources to the client. For example, it feeds back a picture pulled from the target source station to the client, or feeds back a video stream corresponding to the current playback progress of a certain on-demand video pulled from the target source station to the client. At the same time, for non-live broadcast scenarios, the server can also cache the target data pulled from the source station.
[0144] This embodiment directly locates the target source station for retrieval based on the resource status information obtained based on the detection and in combination with the source station conditions, rather than trying and locating the source station with the target resources one by one based on the polling method for retrieval. When the target resources exist in the source station, the data required by the user can be quickly pulled from the source station and the user can be quickly responded to, thereby reducing the waiting time for the client user to retrieve the required resources from the network.
[0145] In one embodiment, for a method of detecting resource status information of a target resource in real time, see Figure 8 The method of processing resource requests shown in the flowchart of the present application may also include:
[0146] Step 801: In the process of acquiring the target resource from the target source station to implement back-to-source processing, if a resource request for requesting the target resource is obtained again, determine whether the above back-to-source processing has timed out. If so, execute step 802; if not, go to execute step 803.
[0147] When the server responds to the client's requested target resource through back-to-source processing (pulling the target resource from the source station), the data pulling process may be interrupted due to network congestion or source station abnormalities. For the client, it usually manifests as pauses / stuttering in live / on-demand streaming media, download progress suspension, etc. In this case, the server will obtain a resource request for the target resource again. The resource request can be a retry request initiated by the client user by performing operations such as video refresh and re-downloading resources, or a request automatically triggered by the server based on the interruption event of data pulling, which is not restricted here.
[0148] When the server obtains a resource request for the target resource again, in order to control the number of detection requests initiated to the source station and prevent the concurrent request limit of the data source station from exceeding the limit, this embodiment first determines whether the most recent back-to-source processing of the target resource has timed out based on the set timeout duration, and performs different processing based on the different back-to-source situations of whether it has timed out or not.
[0149] Whether the most recent back-to-source processing of the target resource has timed out refers to the time from the moment the data fetching was interrupted to the current moment, and whether it is greater than the set timeout period.
[0150] Step 802: If the timeout occurs, the resource status information of the target resource is detected in real time.
[0151] Among them, if the most recent back-to-source processing of the target resource times out, the resource status information of the target resource is detected in real time for the resource request obtained this time. For example, through the Lua detection method, a detection request is initiated in parallel to each data source station corresponding to the target resource to detect whether the target resource exists at each data source station, etc. After that, the back-to-source processing of the target resource can be further performed based on the real-time detected resource status information.
[0152] Step 803: If it has not timed out, continue to obtain the target resource from the above-mentioned target source station.
[0153] On the contrary, if the most recent back-to-source processing of the target resource has not timed out, the resource status information of the target resource will no longer be detected in real time. For example, the server will no longer perform lua detection on each source station in real time for the resource request obtained this time, but will continue to pull the target data directly based on the back-to-source situation during the most recent back-to-source processing, that is, directly retry the target source station used in the most recent back-to-source processing to continue pulling the target resource from the target source station.
[0154] If the pulling process is interrupted when resources are returned to the source due to a failure of the target source station, continuing to pull data from the target resource will result in a pull failure (if the interruption is caused by network congestion or other reasons, the data can usually be successfully pulled by retrying the target source station). In this case, when the pull fails, the server can further initiate a detection request to each source station corresponding to the target resource except the above-mentioned target source station in real time, perform Lua detection, and then continue the return-to-source processing of the target resource based on the resource status information obtained by the detection.
[0155] Among them, for the interruption event of the data pulling process, when continuing to perform the back-to-source processing of the target resource, optionally, the complete resource data of the target resource can be re-pulled (for example, a complete video file can be re-pulled from the source station), or, optionally, based on the disconnection and resumption function, starting from the pulling progress corresponding to the interruption time, only continue to pull the part of the target resource that was not completed when the pulling was interrupted. For live broadcast scenarios, considering the real-time characteristics of live broadcast, preferably, the latter implementation method can be adopted, and for non-live broadcast scenarios, any implementation method can be adopted.
[0156] In response to a pull interruption event of a target resource, when a resource request for the target resource is obtained again, this embodiment determines whether the most recent back-to-source processing of the target resource has timed out based on the set timeout duration. If it has not timed out, the detection processing of the source station is not performed, and the back-to-source processing of the target data is continued directly according to the most recent back-to-source situation of the target resource. This can effectively control the number of requests in the detection process and avoid exceeding the limit on the concurrent request amount of the data source station.
[0157] Corresponding to the above method, the embodiment of the present application also discloses a resource request processing device, such as Fig. 9 As shown, the device comprises:
[0158] The first acquisition module 901 is used to acquire a resource request from a client;
[0159] The second acquisition module 902 is used to acquire the resource status information of the target resource detected in advance or in real time when the target resource indicated by the resource request is not cached locally; when there are multiple corresponding data source stations, the above-mentioned real-time detection includes: detecting in real time in parallel whether the target resource exists in each data source station to obtain the resource status information of the target resource;
[0160] The determination module 903 is used to determine whether the target resource exists in the data source station according to the above resource status information;
[0161] Feedback module 904, used for feeding back response information indicating a resource request error to the client when the determination result of determination module 903 is that the resource does not exist;
[0162] In the case that there are multiple corresponding data source stations, the target resource does not exist in the data source station means: the target resource does not exist in the multiple data source stations.
[0163] In one embodiment, the above-mentioned device further includes a detection module, which is used to detect resource status information of the target resource in advance or in real time.
[0164] The detection module is used to detect the resource status information of the target resource in real time:
[0165] Obtain the source station address of the data source station corresponding to the target resource in real time;
[0166] In the case where the target resource corresponds to multiple data source stations, a detection request carrying a resource identifier of the target resource is sent to each data source station in parallel according to the source station address of each data source station;
[0167] Obtaining detection results fed back by each data source station, where the detection results fed back by the data source station are used to indicate whether the target resource exists in the data source station;
[0168] Determine the resource status information of the target resource based on the detection results fed back by each data source station;
[0169] If all detection results corresponding to the target resource indicate that the target resource does not exist, the resource status information of the target resource indicates that the target resource does not exist in the data source station.
[0170] In one embodiment, when the detection module detects the resource status information of the target resource in advance, it is used to:
[0171] When the detection condition is met, a detection request carrying a resource identifier of a data resource is sent to each data source station corresponding to the data resource; the data resource is at least one data resource that has been cached locally;
[0172] Obtaining detection results fed back by each data source station corresponding to the data resource, where the detection results fed back by the data source station are used to indicate whether the data resource exists in the data source station;
[0173] Determine the resource status information of the data resource according to the obtained detection results; wherein, if all the detection results corresponding to the data resource indicate that the data resource does not exist, then the resource status information of the data resource indicates that the data resource does not exist in the data source station;
[0174] A correspondence between the metadata of the data resource and the resource status information is constructed and recorded; the metadata of the data resource at least includes the resource identifier of the data resource.
[0175] In one embodiment, when the detection module detects the resource status information of the target resource in advance, it is used to:
[0176] Obtaining a status notification of the data resource sent by each data source station corresponding to the data resource, the status notification being used to indicate that the corresponding data resource is online or offline at the data source station;
[0177] Determine resource status information of the data resource according to status notifications sent by each data source station corresponding to the data resource; wherein, if all data source stations corresponding to the data resource send status notifications indicating that the data resource is offline, then the resource status information of the data resource indicates that the data resource does not exist in the data source station;
[0178] The corresponding relationship between the metadata of the data resource and the resource status information is constructed and recorded; the metadata of the data resource at least includes the resource identifier of the data resource.
[0179] In one implementation, the second acquisition module 902, when acquiring the resource status information of the pre-detected target resource, is used to:
[0180] From the corresponding relationship set between the metadata of the data resource and the resource status information, the resource status information matching the resource identifier of the target resource is searched to obtain the resource status information of the target resource.
[0181] In one implementation, the detection request is an HTTP request or an HTTPS request generated based on a predetermined scripting language.
[0182] In one implementation, the target resource is a live streaming media resource or a non-live data resource;
[0183] When the target resource is a live streaming media resource, detecting resource status information of the target resource in real time;
[0184] In the case where the target resource is a non-live data resource, the resource status information of the target resource is detected in advance or in real time.
[0185] In one embodiment, the above-mentioned device also includes a back-to-source module, which is used to: if the target resource exists in the data source station, determine at least one data source station where the target resource exists according to the resource status information of the target resource; in the case where there are multiple data source stations where the target resource exists, determine the target source station that meets the source station conditions from the multiple data source stations where the target resource exists; send a back-to-source request to the target source station to obtain the target resource indicated by the resource request from the target source station, and feedback response information including at least part of the target resource to the client.
[0186] In one embodiment, the above-mentioned device also includes: a retry module, which is used to: in the process of obtaining the target resource from the target source station to implement back-to-source processing, if a resource request for requesting the above-mentioned target resource is obtained again, determine whether the above-mentioned back-to-source processing has timed out; if it has timed out, detect the resource status information of the target resource in real time; if it has not timed out, continue to obtain the target resource from the above-mentioned target source station.
[0187] As for the resource request processing device disclosed in the embodiment of the present application, since it corresponds to the resource request processing method disclosed in the above method embodiments, the description is relatively simple. For the relevant similarities, please refer to the description of the corresponding method embodiments above, which will not be described in detail here.
[0188] The present application also discloses an electronic device, which may be but is not limited to Figure 1 Servers in the architecture shown.
[0189] See also Fig.10 A schematic diagram of the structure of an electronic device is provided, the electronic device at least comprising:
[0190] Memory 1001, used for storing computer instruction sets;
[0191] The computer instruction set may be implemented in the form of a computer program.
[0192] The memory 1001 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.
[0193] The processor 1002 is used to implement the resource request processing method of any of the above method embodiments by executing an instruction set stored in the memory.
[0194] Among them, the processor 1002 can be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, etc.
[0195] In addition, the electronic device may also include components such as a communication interface and a communication bus. The memory, the processor and the communication interface communicate with each other through the communication bus.
[0196] The communication interface is used for communication between electronic devices and other devices (such as the central control device in the CDN). The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc.
[0197] In this embodiment, when the processor in the electronic device executes the computer instruction set stored in the memory, for the obtained resource request, if the target resource indicated by the request is not cached locally, it obtains the resource status information of the target resource detected in advance or in real time, determines whether the target resource exists in the data source station based on the detected resource status information, and feeds back the response information used to characterize the resource request error if it does not exist. When determining whether the required resources exist in the source station, the resource status information used as the basis for determination is obtained by pre-detection, or by detecting each source station in real time in parallel. Compared with the traditional method of initiating streaming media requests to each source station in a polling manner and determining in combination with a set timeout period, the time required for determination is reduced. When the required resources do not exist in the source station, information characterizing the resource request error can be quickly responded to the user, reducing the user's waiting time.
[0198] In addition, an embodiment of the present application further discloses a computer-readable storage medium, which stores a computer instruction set. When the computer instruction set is executed by a processor, the resource request processing method disclosed in any of the method embodiments above is implemented.
[0199] Specifically, when the computer instruction set in the computer-readable storage medium of this embodiment is executed by the processor, for the obtained resource request, when the target resource indicated by the request is not cached locally, the resource status information of the target resource detected in advance or in real time is obtained, and whether the target resource exists in the data source station is determined based on the detected resource status information, and when it does not exist, response information used to characterize the resource request error is fed back. When determining whether the required resources exist in the source station, the resource status information used as the basis for determination is obtained by pre-detection, or by detecting each source station in real time in parallel. Compared with the traditional method of initiating streaming media requests to each source station in a polling manner and determining in combination with a set timeout period, the time required for determination is reduced. When the required resources do not exist in the source station, information characterizing the resource request error can be quickly responded to the user, reducing the user's waiting time.
[0200] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0201] For the convenience of description, the above system or device is described by dividing it into various modules or units according to its functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.
[0202] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.
[0203] Finally, it should be noted that, in this article, relational terms such as first, second, third and fourth are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0204] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A method for processing a resource request, characterized in that: include: Get the client's resource request; If the target resource indicated by the resource request is not cached locally, obtaining resource status information of the target resource detected in advance or in real time; In the case where there are multiple corresponding data source stations, the real-time detection includes: detecting in real time in parallel whether the target resource exists in each data source station to obtain resource status information of the target resource; Determining whether the target resource exists in the data source station according to the resource status information; If not, feedback response information indicating a resource request error to the client; Wherein, in the case that there are multiple corresponding data source stations, the target resource does not exist in the data source station means: the target resource does not exist in any of the multiple data source stations; Real-time detection of resource status information of the target resource includes: Acquire the source station address of the data source station corresponding to the target resource in real time; In the case where the target resource corresponds to multiple data source stations, sending a detection request carrying a resource identifier of the target resource to each data source station in parallel according to the source station address of each data source station; Acquire the detection results fed back by each data source station, where the detection results fed back by the data source station are used to indicate whether the target resource exists in the data source station; Determine resource status information of the target resource according to the detection results fed back by each data source station; If all detection results corresponding to the target resource indicate that the target resource does not exist, the resource status information indicates that the target resource does not exist in the data source station.
2. The method according to claim 1, characterized in that Pre-detecting resource status information of the target resource includes: When the detection condition is met, a detection request carrying a resource identifier of the data resource is sent to each data source station corresponding to the data resource; the data resource is at least one data resource that has been cached locally; Acquire detection results fed back by each data source station corresponding to the data resource, where the detection results fed back by the data source station are used to indicate whether the data resource exists in the data source station; Determine the resource status information of the data resource according to the obtained detection results; wherein, if all detection results corresponding to the data resource indicate that the data resource does not exist, then the resource status information of the data resource indicates that the data resource does not exist in the data source station; A corresponding relationship between the metadata of the data resource and the resource status information is constructed and recorded; the metadata of the data resource at least includes a resource identifier of the data resource.
3. The method according to claim 1, characterized in that: Pre-detecting resource status information of the target resource includes: Obtaining a status notification of the data resource sent by each data source station corresponding to the data resource, wherein the status notification is used to indicate that the corresponding data resource is online or offline from the data source station; Determine resource status information of the data resource according to status notifications sent by each data source station corresponding to the data resource; wherein, if all data source stations corresponding to the data resource send status notifications indicating that the data resource has gone offline, then the resource status information of the data resource indicates that the data resource does not exist in the data source station; A corresponding relationship between the metadata of the data resource and the resource status information is constructed and recorded; the metadata of the data resource at least includes a resource identifier of the data resource.
4. The method according to claim 2 or 3, characterized in that: Acquiring the resource status information of the target resource detected in advance, including: From the corresponding relationship set between the metadata of the data resource and the resource status information, the resource status information matching the resource identifier of the target resource is searched to obtain the resource status information of the target resource.
5. The method according to claim 1 or 2, characterized in that: The detection request is a Hypertext Transfer Protocol (HTTP) request or a Hypertext Transfer Security (HTTPS) request generated based on a predetermined scripting language.
6. The method according to claim 1, characterized in that The target resource is a live streaming media resource or a non-live data resource; In the case where the target resource is a live streaming media resource, detecting resource status information of the target resource in real time; In the case that the target resource is a non-live data resource, the resource status information of the target resource is detected in advance or in real time.
7. The method according to claim 1, characterized in that Also includes: If the target resource exists in the data source station, determining at least one data source station where the target resource exists according to the resource status information; In the case that there are multiple data source stations where the target resource exists, determining a target source station that meets the source station condition from the multiple data source stations where the target resource exists; A source request is sent back to the target source station to obtain the target resource from the target source station, and response information including at least part of the target resource is fed back to the client.
8. The method according to claim 7, characterized in that In the case of detecting the resource status information of the target resource in real time, the method further includes: In the process of acquiring the target resource from the target source station to implement back-to-source processing, if a resource request for requesting the target resource is obtained again, determining whether the back-to-source processing has timed out; If the timeout occurs, the resource status information of the target resource is detected in real time; If it has not timed out, continue to obtain the target resource from the target source station.
9. A resource request processing device, characterized in that: include: A first acquisition module, used to acquire a resource request from a client; A second acquisition module is used to acquire resource status information of the target resource detected in advance or in real time when the target resource indicated by the resource request is not cached locally; In the case where there are multiple corresponding data source stations, the real-time detection includes: detecting in real time in parallel whether the target resource exists in each data source station to obtain resource status information of the target resource; A determination module, used to determine whether the target resource exists in the data source station according to the resource status information; A feedback module, configured to feed back response information representing a resource request error to the client when the determination result of the determination module is that the resource request does not exist; Wherein, in the case that there are multiple corresponding data source stations, the target resource does not exist in the data source station means: the target resource does not exist in any of the multiple data source stations; Real-time detection of resource status information of the target resource includes: Acquire the source station address of the data source station corresponding to the target resource in real time; In the case where the target resource corresponds to multiple data source stations, sending a detection request carrying a resource identifier of the target resource to each data source station in parallel according to the source station address of each data source station; Acquire the detection results fed back by each data source station, where the detection results fed back by the data source station are used to indicate whether the target resource exists in the data source station; Determine resource status information of the target resource according to the detection results fed back by each data source station; If all detection results corresponding to the target resource indicate that the target resource does not exist, the resource status information indicates that the target resource does not exist in the data source station.
10. An electronic device, characterized in that: include: A memory for storing at least one set of instruction sets; A processor, configured to implement the method for processing a resource request according to any one of claims 1 to 8 by executing an instruction set stored in the memory.
11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an instruction set, and when the instruction set is executed by a processor, the method for processing a resource request according to any one of claims 1 to 8 is implemented.
