System and method for data access

Abstract

To provide a system and a method for data access for reducing network latency in live streaming in multiple dimensions.SOLUTION: A method includes the steps for acquiring a request from a user of a user terminal, acquiring a type of the request and user's request tolerance data, and determining a delay time for transmitting the request to a backend server that corresponds to the request on the basis of the type of request and the user's request tolerance data.SELECTED DRAWING: Figure 13

Description

[Technical field]

[0001] The present invention relates to data access, and more particularly to data access by a server. [Background technology]

[0002] As exemplified by live streaming services, real-time interactions over the Internet have become a part of our daily lives. A variety of platforms and providers offer live streaming services, and competition is fierce. It is important for platforms to provide services that users want.

[0003] Chinese Patent Application Publication No. CN114268631A discloses a multi-dimensional reduction of network delay. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] China Patent Application Publication No. 114268631 Summary of the Invention

[0005] A method according to one embodiment of the present invention is a method for data access executed by one or more computers, comprising the steps of: obtaining a request from a user of a user terminal; obtaining status data of a backend server corresponding to the request; sending the request to the backend server corresponding to the request based on the status data; obtaining a response to the request from the backend server; determining that the response has an error; and determining a delay time for retrying sending the request to the backend server based on the status data.

[0006] A method according to one embodiment of the present invention is a method for data access executed by one or more computers, comprising the steps of obtaining a request from a user of a user terminal, obtaining a type of the request and request tolerance data of the user, and determining a delay time for sending the request to a backend server corresponding to the request based on the type of the request and the request tolerance data of the user.

[0007] A system according to one embodiment of the present invention is a system for data access including one or more processors, the one or more computer processors executing machine-readable instructions to perform the steps of obtaining a request from a user of a user terminal, obtaining status data of a backend server corresponding to the request, sending the request to the backend server corresponding to the request based on the status data, obtaining a response to the request from the backend server, determining that the response contains an error, and determining a delay time for retrying sending the request to the backend server based on the status data. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a schematic diagram showing a configuration of a live streaming system 1 according to some embodiments of the present invention. [Diagram 2] 2 is a block diagram illustrating the functionality and configuration of user terminal 30 of FIG. 1 in accordance with some embodiments of the present invention. [Diagram 3] 2 is a block diagram illustrating the functionality and configuration of the server of FIG. 1 in accordance with some embodiments of the present invention. [Figure 4] 4 is a table showing an example data structure of the stream DB 310 of FIG. 3. [Diagram 5] 4 is a table illustrating an example data structure of the user DB 312 of FIG. 3. [Figure 6] 4 is a table showing an example data structure of the gift DB 314 of FIG. 3. [Figure 7]FIG. 2 is a block diagram showing the functions and configurations of a server 10 and a user terminal 30 according to some embodiments of the present invention. [Figure 8] 1 is a table showing an exemplary data structure of a request DB 250. [Figure 9] 11 is a table showing an exemplary data structure of a user tolerance DB 252. [Figure 10] 11 is a table showing an exemplary data structure of a retry DB 254. [Figure 11] 1 is a table showing an exemplary data structure of a status DB 320. [Figure 12] 1 is an exemplary flow chart illustrating a method according to some embodiments of the present invention. [Figure 13] 1 is an exemplary flow chart illustrating a method according to some embodiments of the present invention. [Figure 14] 1 is an example of user tolerance data generation according to some embodiments of the present invention. [Figure 15] 13 is an example of determining transmission retry timing based on endpoint status data and the number of retries. [Figure 16] 2 is an exemplary timeline diagram according to some embodiments of the present invention. [Figure 17] 2 is an exemplary timeline diagram according to some embodiments of the present invention. [Figure 18] 4 is an exemplary data access flow according to some embodiments of the present invention. [Figure 19] 1 is a block diagram showing an example of a hardware configuration of an information processing device according to some embodiments of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the same or similar components, parts, steps, or signals shown in each drawing will be denoted by the same reference numerals in all drawings, and duplicated explanations will be omitted as appropriate. In addition, some parts that are not important in the explanation of each drawing will be omitted.

[0010] Data access via the Internet typically involves a user terminal (such as a smartphone, tablet, computer, etc.) used by a user, an application (or application software such as streaming application software) running on the user terminal, and a server (or application server) for the application that communicates with the user terminal.

[0011] A user can initiate a data request through a user interface of an application, which may involve clicking, tapping, or scrolling, and the request is sent from the user terminal to the server, which then sends a response to the request to the user terminal, completing the data access.

[0012] The state of the server (such as its load and other health status) may determine whether data access is successful. An access mechanism that can efficiently and dynamically utilize the server's resources without overloading the server is desirable.

[0013] FIG. 1 is a schematic diagram showing a configuration of a live streaming system 1 according to some embodiments of the present invention. The live streaming system 1 provides a live streaming service for streaming streamers (also called live streamers, anchors, distributors, and live streamers) LV and viewers (also called audiences) AU (AU1, AU2, ...) to interact or communicate in real time. As shown in FIG. 1, the live streaming system 1 includes a server 10, a user terminal 20, and a user terminal 30 (30a, 30b, ...). In some embodiments, the streamers and viewers may be collectively referred to as users. The server 10 may include one or more information processing devices connected to a network NW. The user terminals 20 and 30 may be, for example, mobile terminal devices such as smartphones, tablets, notebook PCs, recorders, portable game consoles, and wearable terminals, or stationary devices such as desktop PCs. The server 10, the user terminals 20, and the user terminals 30 are connected to each other so as to be able to communicate with each other via various wired or wireless networks NW.

[0014] The live streaming system 1 includes a distributor LV, a viewer AU, and an administrator (or an app provider, not shown) who manages the server 10. The distributor LV is a person who distributes the content in real time by recording the content on his / her user terminal 20 and uploading it directly or indirectly to the server 10. Examples of the content may be the distributor's own song, talk, performance, game play, or any other content. The administrator provides a platform for live streaming the content on the server 10, and mediates or manages real-time interactions between the distributor LV and the viewer AU. The viewer AU accesses the platform with his / her user terminal 30, selects and watches desired content. During the live streaming of the selected content, the viewer AU performs operations such as sending comments, cheers, and gifts via the user terminal 30. The distributor LV who distributes the content may respond to those comments, cheers, or gifts. The response is transmitted to the viewer AU by video and / or audio, and two-way communication is established.

[0015] The term "live streaming" may refer to a data transmission mode that allows the content recorded at the user terminal 20 of the distributor LV to be played back and viewed at the user terminal 30 of the viewer AU in substantially real time, or may refer to a live broadcast realized by such a transmission mode. The live streaming may be realized by utilizing existing live streaming technologies such as HTTP live streaming, CMAF (Common Media Application Format), WebRTC (Web Real-Time Communications), RTMP (Real-Time Messaging Protocol), MPEG DASH, etc. Live streaming includes a transmission mode that allows the viewer AU to view the content with a predetermined delay at the same time as the content is recorded by the distributor LV. The length of the delay may be a delay that allows interaction between the distributor LV and the viewer AU. The live streaming is distinguished from so-called on-demand distribution, in which all recorded data of the content is stored once on the server and then provided to the user from the server upon the user's request.

[0016] The term "video data" as used herein refers to data including image data (also called video data) generated using the imaging function of the user terminal 20 or 30, and audio data generated using the audio input function of the user terminal 20 or 30. The video data is played on the user terminal 20, 30 so that the user can view the content. In some embodiments, it is assumed that the video data is subjected to a process such as compression, decompression, encoding, decoding, transcoding, etc., to change its format, size, or data specifications between the generation of the video data on the distributor's user terminal and the playback of the video data on the viewer's user terminal. However, since the content (e.g., video and audio) represented by the video data does not substantially change before and after such a process, the video data after such a process is expressed in this specification as being the same as the video data before such a process. In other words, when video data is generated at the distributor's user terminal and then played at the viewer's user terminal via the server 10, the video data generated at the distributor's user terminal, the video data passing through the server 10, and the video data received and played at the viewer's user terminal are all the same video data.

[0017] In the example shown in Fig. 1, the distributor LV provides live streaming data. The user terminal 20 of the distributor LV records video and audio of the distributor LV to generate the streaming data, and the generated data is transmitted to the server 10 via the network NW. At the same time, the user terminal 20 displays the recorded video VD of the distributor LV on the display of the user terminal 20, allowing the user to check the live streaming content currently being provided by the distributor LV.

[0018] The user terminals 30a, 30b of the viewers AU1, AU2 who have requested the platform to view the live streaming of the distributor LV receive video data related to the live streaming (hereinafter, may be referred to as "live streaming video data") via the network NW, play the received video data, display the videos VD1, VD2 on the display, and output audio from the speaker. The videos VD1, VD2 displayed on the user terminals 30a, 30b are substantially the same as the video VD captured by the user terminal 20 of the distributor LV, and the audio output from the user terminals 30a, 30b is substantially the same as the audio recorded by the user terminal 20 of the distributor LV.

[0019] The recording of video and audio at the user terminal 20 of the distributor LV and the playback of video data at the user terminals 30a and 30b of the viewers AU1 and AU2 are performed substantially simultaneously. When the viewer AU1 inputs a comment on the content provided by the distributor LV to the user terminal 30a, the server 10 displays the comment on the user terminal 30a of the distributor LV in real time, and also displays the comment on the user terminals 30a and 30b of the viewers AU1 and AU2. When the distributor LV reads the comment and develops a talk corresponding to the comment, the video and audio of the talk are displayed on the user terminals 30a and 30b of the viewers AU1 and AU2, respectively. This interactive operation is recognized as a conversation being established between the distributor LV and the viewer AU1. As a result, the live streaming system 1 realizes live streaming that enables two-way communication, not one-way communication.

[0020] 2 is a block diagram showing the functions and configuration of the user terminal 30 of FIG. 1 according to some embodiments of the present invention. The user terminal 20 has the same or similar functions and configuration as the user terminal 30. Each block of FIG. 2 and the following block diagrams may be realized by hardware elements such as a computer CPU or a mechanical device, or by software elements such as a computer program. The functional blocks may be realized by the cooperation of these elements. Therefore, it will be understood by those skilled in the art that these functional blocks may be realized in various forms by combining hardware and software.

[0021] The distributor LV and the viewer AU may download a live streaming application program (hereinafter, referred to as a live streaming application) from a download site via the network NW and install it on the user terminals 20 and 30. Alternatively, the live streaming application may be pre-installed on the user terminals 20 and 30. When the live streaming application is executed on the user terminals 20 and 30, the user terminals 20 and 30 communicate with the server 10 via the network NW and implement or execute various functions. Hereinafter, functions implemented by the user terminals 20 and 30 (processors such as CPUs) on which the live streaming application is executed will be described as functions of the user terminals 20 and 30. These functions are actually realized by the live streaming application on the user terminals 20 and 30. In some embodiments, these functions may be realized by a computer program written in a programming language such as HTML (HyperText Markup Language), transmitted from the server 10 to a web browser of the user terminals 20 and 30 via the network NW, and executed by the web browser.

[0022] The user terminal 30 includes a distribution unit 100 and a viewing unit 200. The distribution unit 100 generates video data in which the video and audio of the user (or the user side) are recorded, and provides the video data to the server 10. The viewing unit 200 receives the video data from the server 10 and plays the video data. The user starts the distribution unit 100 when performing live streaming, and starts the viewing unit 200 when the user watches a video. The user terminal on which the distribution unit 100 is started is the terminal of the distributor, i.e., the user terminal that generates the video data. The user terminal on which the viewing unit 200 is started is the terminal of the viewer, i.e., the user terminal on which the video data is reproduced and played.

[0023] The delivery unit 100 includes an imaging control unit 102, an audio control unit 104, a video transmission unit 106, and a distributor-side UI control unit 108. The imaging control unit 102 is connected to a camera (not shown in FIG. 2) and controls imaging performed by the camera. The imaging control unit 102 acquires image data from the camera. The audio control unit 104 is connected to a microphone (not shown in FIG. 2) and controls audio input from the microphone. The audio control unit 104 acquires the audio data from the microphone. The video transmission unit 106 transmits the image data acquired by the imaging control unit 102 and the video data including the audio data acquired by the audio control unit 104 to the server 10 via the network NW. The video data is transmitted in real time by the video transmission unit 106. That is, the generation of the video data by the imaging control unit 102 and the audio control unit 104 and the transmission of the generated video data by the video transmission unit 106 are performed substantially simultaneously. The distributor's UI control unit 108 controls the UI (user interface) of the distributor. The distributor's UI control unit 108 may be connected to a display (not shown in FIG. 2) and displays a video on the display by playing the video data transmitted by the video transmission unit 106. The distributor's UI control unit 108 may display an operation object or an instruction permission object on the display and accept an input from the distributor who taps on the object.

[0024] The viewing unit 200 includes a viewer-side UI control unit 202, an overlay information generating unit 204, and an input information transmitting unit 206. The viewing unit 200 receives video data related to live streaming in which the distributor, the viewer who is the user of the user terminal 30, and other viewers participate from the server 10 via the network NW. The viewer-side UI control unit 202 controls the UI of the viewer. The viewer-side UI control unit 202 is connected to a display and a speaker (not shown in FIG. 2), plays the received video data, displays the video on the display, and outputs the audio from the speaker. The state in which the video is output to the display and the audio is output from the speaker can be called a "video data is being played" state. The viewer-side UI control unit 202 is also connected to an input means (not shown in FIG. 2) such as a touch panel, a keyboard, a display, etc., and acquires a user's input via the input means. The overlay information generating unit 204 overlays a predetermined frame image on an image generated from video data from the server 10. The frame image includes various user interface objects (hereinafter simply referred to as "objects") for receiving input from the user, comments entered by the viewer, and / or information acquired from the server 10. The input information transmitting unit 206 transmits the user input acquired by the viewer-side UI control unit 202 to the server 10 via the network NW.

[0025] 3 shows a block diagram illustrating the function and configuration of the server 10 in FIG. 1 according to some embodiments of the present invention. The server 10 includes a distribution information providing unit 302, a relay unit 304, a gift processing unit 306, a payment processing unit 308, a stream DB 310, a user DB 312, and a gift DB 314.

[0026] When a notification or request to start live streaming is received from the user terminal 20 on the distributor side via the network NW, the distribution information providing unit 302 registers in the stream DB 310 a stream ID for identifying the live streaming and the distributor ID of the distributor performing the live streaming.

[0027] When the distribution information providing unit 302 receives a request for providing information about live streams from the viewing unit 200 of the user terminal 30 on the viewer side via the network NW, the distribution information providing unit 302 obtains or checks currently available live streams from the stream DB 310, and creates a list of available live streams. The distribution information providing unit 302 transmits the created list to the requesting user terminal 30 via the network NW. The viewer side UI control unit 202 of the requesting user terminal 30 generates a live stream selection screen based on the received list, and displays it on the display of the user terminal 30.

[0028] When the input information sending unit 206 of the user terminal 30 receives the selection result of the viewer on the live stream selection screen, the input information sending unit 206 generates a delivery request including the stream ID of the selected live stream and sends the request to the server 10 via the network NW. The delivery information providing unit 302 starts providing the live stream specified by the stream ID included in the received delivery request to the requesting user terminal 30. The delivery information providing unit 302 updates the stream DB 310 so that the viewer ID of (or the corresponding) the stream ID includes the user ID of the viewer of the requesting user terminal 30.

[0029] The relay unit 304 relays the video data from the user terminal 20 on the distributor side to the user terminal 30 on the viewer side in the live streaming started by the distribution information providing unit 302. The relay unit 304 receives a signal representing a user input by a viewer from the input information transmitting unit 206 during the live streaming or during playback of the video data. The signal representing the user input may be an object designation signal that designates an object displayed on the display of the user terminal 30. The object designation signal may include a viewer ID of the viewer, a distributor ID of the distributor of the live stream that the viewer is viewing, and an object ID that identifies the object. When the object is a gift, the object ID is a gift ID. Similarly, the relay unit 304 receives a signal representing a user input made by a distributor during playback (or live streaming) of the video data from the distribution unit 100 of the user terminal 20. The signal may be an object designation signal.

[0030] Furthermore, the signal representing the user input may be a comment input signal including a comment input by a viewer to the user terminal 30 and the viewer ID of the viewer. Upon receiving the comment input signal, the relay unit 304 transmits the comment and the viewer ID included in the signal to the user terminal 20 of the distributor and the user terminals 30 of other viewers. In these user terminals 20, 30, the viewer-side UI control unit 202 and the overlay information generation unit 204 display the received comment on a display associated with the viewer ID that has also been received.

[0031] The gift processing unit 306 increases the points of the distributor based on the points of the gift specified by the gift ID included in the object designation signal, and updates the user DB 312. Specifically, the gift processing unit 306 refers to the gift DB 314 to specify the points to be given to the gift ID included in the received object designation signal. Then, the gift processing unit 306 updates the user DB 312 and adds the specified points to the points of the distributor ID (or corresponding to the distributor ID) included in the object designation signal.

[0032] In response to receiving the object designation signal, the payment processing unit 308 processes the payment of the gift from the viewer. Specifically, the payment processing unit 308 refers to the gift DB 314 to determine the price points of the gift identified by the gift ID included in the object designation signal. Then, the payment processing unit 308 updates the user DB 312 and subtracts the determined price points from the points of the viewer identified by the viewer ID included in the object designation signal.

[0033] FIG. 4 is a table showing an exemplary data structure of the stream DB 310 of FIG. 3. The stream DB 310 holds information about a live stream currently being performed. The stream DB 310 stores a stream ID, a broadcaster ID, and a viewer ID in a mutually associated manner. The stream ID is an ID for identifying a live stream in a live streaming platform provided by the live streaming system 1. The broadcaster ID is a user ID for identifying a broadcaster providing the live stream. The viewer ID is a user ID for identifying a viewer of the live stream. In a live streaming platform provided by the live streaming system 1 according to some embodiments, when a user starts a live stream, the user becomes a broadcaster, and when the same user watches a live stream broadcasted by another user, the user also becomes a viewer. Therefore, the distinction between a broadcaster and a viewer is not fixed, and a user ID registered as a broadcaster ID at one time may be registered as a viewer ID at another time.

[0034] FIG. 5 is a table showing an exemplary data structure of the user DB 312 of FIG. 3. The user DB 312 holds information about users. The user DB 312 stores the user ID and the points in a mutually associated manner. The user ID identifies a user. The points correspond to points held by the corresponding user. The points are electronic values ​​circulating within the live streaming platform. In some embodiments, when a broadcaster receives a gift from a viewer during a live broadcast, the broadcaster's points increase by a value corresponding to the gift. The points are used, for example, to determine the amount of compensation (such as money) that the broadcaster receives from the administrator of the live streaming platform. In some embodiments, when the broadcaster receives a gift from a viewer, an amount corresponding to the gift may be given instead of the points.

[0035] FIG. 6 is a table showing an exemplary data structure of the gift DB 314 of FIG. 3. The gift DB 314 holds information about gifts available to the viewer during the live streaming. Gifts are electronic data. Gifts may be purchased with points or money, or may be provided free of charge. Gifts may be given to the broadcaster by the viewer. Giving a gift to the broadcaster is also called using a gift, sending a gift, throwing a gift, etc. Some gifts can be used immediately after purchase, while others can be used at any time after purchase by the viewer who purchased the gift. When a viewer gives a gift to a broadcaster, the broadcaster is given an amount of points corresponding to the gift. When a gift is used, an effect related to the gift may occur as a result of the use. For example, an effect (such as a visual effect or an auditory effect) corresponding to the gift is displayed on the live streaming screen.

[0036] The gift DB 314 stores a gift ID, an assignment point, and a price point in association with each other. The gift ID is for identifying a gift. The assignment point is the amount of points that is assigned to a broadcaster when the gift is given to the broadcaster. The price point is the amount of points paid for the use (purchase) of the gift. A viewer can give a desired gift to a broadcaster by paying the price points of the desired gift while watching a live stream. The payment of the price points can be made by an appropriate electronic payment means. For example, the viewer may pay the price points to the administrator. Alternatively, payment by bank transfer or credit card may be used. The administrator can arbitrarily set the relationship between the assignment points and the price points. For example, it may be set as assignment points = price points. Alternatively, it may be set as the price points obtained by multiplying the assignment points by a predetermined coefficient such as 1.2, or as the price points obtained by adding a predetermined fee point to the assignment points.

[0037] FIG. 7 is a block diagram showing the functionality and configuration of the server 10 and the user terminal 30 according to some embodiments of the present invention.

[0038] The user terminal 30 includes an obtaining unit 220, a sending unit 222, a determining unit 224, a request DB 250, a user tolerance DB 252, and a retry DB 254. The server 10 includes a status DB 320 and a machine learning DB 324. In some embodiments, the machine learning DB 324 may be implemented in the user terminal 30.

[0039] The acquiring unit 220 is configured to acquire a request (or an API request) from a user of the user terminal 30. The request may be initiated by the user via a user interface of the user terminal 30. The acquiring unit 220 is configured to acquire status data (or status prediction) of a backend server (or an end point of a backend server) corresponding to the request. The backend server may be the server 10. The status data may be acquired from the status DB of the server 10.

[0040] The sending unit 222 is configured to send the request to the backend server (or the endpoint) corresponding to the request based on (or in response to) the status data. For example, the sending unit 222 may determine whether to send the request immediately or after a certain time period based on the health status of the backend server.

[0041] The obtaining unit 220 is configured to obtain a response of the request from the backend server.

[0042] The determining unit 224 is configured to determine whether the response has an error, and if the response has an error, the determining unit 224 determines a delay time for retrying to send the request to the backend server based on the status data.

[0043] In some embodiments, the determining unit 224 determines, based on the status data, the total number of retries to send the request to the backend server.

[0044] In some embodiments, the delay time is determined based on the status data and the retry number of the retry.

[0045] In some embodiments, the determining unit 224 determines whether the delay time is longer than the update period of the status data (in the status DB 320) of the backend server. If yes, the obtaining unit 220 obtains updated status data of the backend server, and the determining unit 224 updates the retry delay time based on the updated status data.

[0046] In some embodiments, the acquisition unit 220 acquires (or detects) the status of the user and acquires (or detects) the type of the request. The delay time may be determined according to the status data of the endpoint, the status of the user, and / or the type of the request. The status of the user may be stored in the user DB 312 and / or the request DB 250. The type of the request may be stored in the request DB 250.

[0047] In some embodiments, the acquisition unit 220 acquires the user's tolerance data and acquires the type of the request. The delay time may be further determined based on the user's tolerance data and the type of the request. The tolerance data may be stored in the user tolerance DB 252.

[0048] In some embodiments, the determining unit 224 determines to shorten the delay time if the user's tolerance is determined to be low for the type of request based on the tolerance data of the user, In some embodiments, the determining unit 224 determines to lengthen the delay time if the user's tolerance is determined to be high for the type of request based on the tolerance data of the user.

[0049] 8 is a table showing an exemplary data structure of the request DB 250. The request DB 250 stores a request ID, a request timing, a user ID, a user status, a request type, and a stream ID in mutual association.

[0050] The user status is the state of the user when the request is made by the user. For example, the request R1 (charge) is made by the user U1 when the user U1 is a viewer watching stream S1. The request R4 (charge) is made by the user U1 when the user U1 is a broadcaster broadcasting stream S3. The request type "charge" may be a request to charge money or points to the streaming platform. The request type "send gift" may be a request to send a gift in a chat room (or stream) of the streaming platform. The request type "comment" may be a request to post a comment in a chat room (or stream) of the streaming platform. Some request types, such as "charge", may be performed by the user when the user is watching a stream or broadcasting a stream. In some embodiments, a correct (or successful) response to a request may be an indication of task completion or an indication of a corresponding effect.

[0051] 9 is a table showing an exemplary data structure of the user tolerance DB 252. The user tolerance DB 252 stores a user ID, a request type, and a maximum waiting time (or a delay threshold time) in association with each other.

[0052] The maximum wait time is the maximum length of time the user can tolerate to complete the corresponding request type. For example, user U1 cannot tolerate a wait time longer than 6 seconds to send a gift. User U1 can wait 15 seconds for a "follow" request to complete. A "follow" request is a request to follow a broadcaster who has performed on the broadcaster's stream. The tolerance data may be generated by a machine learning algorithm in the machine learning DB 324.

[0053] In some embodiments, the tolerance data of each user may be sent to and stored by the server 10 for overall response allocation. For example, the server 10 (or an allocation unit within the server 10) may allocate response timings for requests from different users based on their respective tolerance data. For example, under high load on the server 10, a request R1 from a less tolerant user U1 may be responded to sooner than a request R2 from a more tolerant user U2. This mechanism can optimize the overall satisfaction of users and increase the overall revenue of the platform.

[0054] 10 is a table showing an exemplary data structure of the retry DB 254. The retry DB 254 stores retry data (or retry plan, retry schedule) for a request. The retry DB 254 stores a request ID, a user ID, a request waiting time (or delay time), a current retry count (current number of retry attempts), and a maximum retry count (total number of retry attempts) in correlation with each other.

[0055] The retry data is determined by the determination unit 224. In some embodiments, the retry data for a request is determined by status data of a corresponding endpoint, the status of the user, the type of the request, and / or tolerance data of the user.

[0056] 11 is a table showing an exemplary data structure of the status DB 320. The status DB 320 stores the endpoints (or API endpoints), the corresponding request types, and the health scores (or health statuses) in association with each other.

[0057] The health score may be calculated based on parameters of the endpoint. The calculation may be determined according to actual operations and the focus of the streaming platform operator. The calculation may be performed by a machine learning algorithm in the machine learning DB 324. Normalization may be performed in the calculation. Some examples of the parameters of the endpoint include past API response time, past response error rate, past throughput, past latency, CPU utilization of a server hosting the endpoint, memory utilization of a server hosting the endpoint, number of concurrent connections handled by a server hosting the endpoint, etc.

[0058] FIG. 12 is an exemplary flow chart illustrating a method according to some embodiments of the present invention.

[0059] In step S1100, the acquiring unit 220 acquires a request from a user U1, for example, via a UI of the user terminal.

[0060] In step S1102, the acquiring unit 220 acquires, from the status DB 320, the status data of the endpoint corresponding to the request.

[0061] In step S1104, the sending unit 222 sends the request to the endpoint. Before sending the request, the endpoint may be determined (by the determining unit 224 or the sending unit 222) to be in a sufficiently healthy state based on the status data.

[0062] In step S1106, the acquiring unit 220 acquires the response. The determining unit 224 determines that the response has an error.

[0063] In step S1108, the determining unit 224 determines the total number of retries for sending the request based on the status data.

[0064] In step S1110, the determination unit 224 determines (or updates) a delay time for each transmission retry (or an upcoming transmission retry) based on the status data (or the updated status data) and / or the retry count of the retry.

[0065] In step S1112, the determination unit 224 determines whether the delay time (determined in step S1110) is greater than the status update period of the status data. If yes, flow proceeds to step S1114. If no, flow proceeds to step S1116. In some embodiments, in step S1112, the determination unit 224 determines whether the status of the endpoint has been updated since the last determination of the delay time.

[0066] In step S1114, the acquiring unit 220 acquires the updated status data of the endpoint. Then, flow returns to step S1110, where the determining unit 224 updates the delay time of each transmission retry (or an upcoming transmission retry) based on the updated status data.

[0067] In step S1116, the sending unit 222 performs a retry of sending the request. The obtaining unit 220 obtains the response. The determining unit 224 determines whether the response is successful or unsuccessful (with an error). If successful, the flow proceeds to step S1120, and data access is completed. If no, the flow proceeds to step S1118.

[0068] In step S1118, the retry count number is updated, for example by a counter in the user terminal. Flow then returns to step S1110, where the determination unit 224 updates the delay time for each transmission retry (or upcoming transmission retry) based on the status data and the updated retry count for the retry.

[0069] FIG. 13 is an exemplary flow chart illustrating a method according to some embodiments of the present invention.

[0070] In step S130, the acquiring unit 220 acquires a request from a user U1.

[0071] In step S132, the acquiring unit 220 acquires, from the status DB 320, the status data of the endpoint corresponding to the request.

[0072] Step S134 includes three steps: S1340, S1342, and S1344.

[0073] In step S1340, the obtaining unit 220 obtains user status data of the user U1 from the request DB 250, for example.

[0074] In step S1342, the obtaining unit 220 obtains user tolerance data of the user U1 from the user tolerance DB 252, for example.

[0075] In step S1344, the obtaining unit 220 obtains or detects the request type of the request.

[0076] In step S136, the determination unit 224 determines the delay time for the request transmission (first time or retry) based on the status data of the endpoint, the user status, the user tolerance data, and / or the request type.

[0077] FIG. 14 is an example of user tolerance data generation according to some embodiments of the present invention.

[0078] The feature data input to the machine learning DB 324 includes each user's request type sequence, each user's waiting time sequence, each user's exit behavior data (such as time sequence of exiting from a streaming platform or exiting from a stream chat), and each user's feedback data (such as complaint data). The machine learning DB 324 then generates the user tolerance data indicating each user's tolerance (or threshold) for different request types. The user tolerance data may be as shown in the example of FIG. 9. The tolerance may be called request tolerance data for the user.

[0079] For example, the machine learning DB 324 may perform a correlation analysis of the feature data to determine for each user which request type's waiting time is most correlated with the exit behavior (and / or complaint behavior) and find the associated threshold. The threshold may be the maximum waiting time the user is willing to wait to exit the chat room (or platform) and thus can be set as the maximum value of the delay time. Different users may have different tolerances for different request types. The present invention provides a customized mechanism for determining the request sending delay time for each user.

[0080] In some embodiments, the determining unit 222 determines the total number of retries to be higher if the status data for the endpoint indicates a better state for the endpoint. In some embodiments, the determining unit 222 determines the total number of retries to be lower if the status data for the endpoint indicates a worse state for the endpoint. A better state may mean that the endpoint (or server) has more processing power available to process the request. A worse state may mean that the endpoint is overburdened and a higher number of retries may increase the risk of a crash.

[0081] FIG. 15 illustrates an example of determining when to retry a transmission based on endpoint status data and the number of retries.

[0082] In this embodiment, the delay time of a retry is determined based on delay time=k(n*n)+y, where k is the health score (status data) of the endpoint, n is the failure time of the API call, and y is the time it takes to send a request and receive a response. The numbers shown in FIG. 15 are calculated assuming [health score=0.8, y=1 second]. The health score k is normalized and is between 0 and 1. In this embodiment, a higher score indicates a worse endpoint condition. The delay time increases with the health score because a worse condition requires the server to process and balance the flow of requests. The delay time increases with the number of retries (number of retries). This is because the more failed retries there are, the worse the server condition is and the longer the server requires to recover and process. This increase is more rapid than a linear increase, and a more efficient rescue of the server can be achieved.

[0083] In some embodiments, the determining unit 224 determines the delay time of sending or retrying a request based on the potential revenue contribution of the request. The potential revenue contribution may be related to the status of the user or the request type. For example, a delay time of sending (or retrying) may be shorter (or a higher transmission priority) when the request type from the user watching the stream is "charging" compared to a request type of "charging" from the user broadcasting the stream. When the user is watching the stream, a "charging" operation may subsequently lead to a "gifting" operation that contributes more to the platform. In some embodiments, a delay time of sending (or retrying) may be shorter (or a higher transmission priority) when the request type from the user broadcasting the stream is "poke" compared to a request type of "poking" from the user watching the stream. Broadcasters can use poke operations to remind viewers to interact more or contribute more in a chat room.

[0084] In some embodiments, the machine learning DB 324 may utilize historical data regarding different request types, different user statuses, and resulting contribution data to determine correlations between the three and determine delays (or priorities) for request transmissions (or transmission retries).

[0085] In some embodiments, the calculation of the potential contribution may depend on the dependency or relationship between different request types. For example, a request to get a gift list from a viewer may lead to a subsequent gift request, and therefore a request to get a gift list may be considered to have a high potential contribution. For example, a request to get a viewer list from a broadcaster may lead to a subsequent poke request, which may increase the viewer's willingness to give gifts, and therefore the request to get a viewer list may be considered to have a high contribution. In some embodiments, the acquisition unit 220 acquires dependency data between different request types (e.g., from a request DB in the server 10), and the determination unit 224 determines the delay time (or priority) of transmission (or retransmission) based thereon. For example, a request with a higher contribution may have a shorter delay time (or a higher priority).

[0086] FIG. 16 is an exemplary timeline diagram according to some embodiments of the present invention.

[0087] The API score service (or status DB) on the backend (BE) side performs a polling process on the AI ​​system on the BE side to obtain the scores of the endpoints. The AI ​​system calculates and / or monitors the scores of the endpoints. The UI on the client side sends an API request to the API service on the client side. The API service checks the status of the corresponding endpoint with the API score service. If the status is OK, the API service sends the API request to the corresponding endpoint on the BE side. If the response received on the client side has an error, the counter on the client side increases the retry count by one, and the timer on the client side counts the delay time for retrying the transmission. The delay time may be calculated by the decision unit based on the method described above. After the retry, the correct data is received on the client side and presented on the UI.

[0088] FIG. 17 is an exemplary timeline diagram according to some embodiments of the present invention.

[0089] The API score service (or status DB) on the backend (BE) side performs a polling process on the AI ​​system on the BE side to obtain the scores of the endpoints. The AI ​​system calculates and / or monitors the scores of the endpoints. The UI on the client side sends an API request to the API service on the client side. The API service checks the status of the corresponding endpoint with the API score service. The response indicates that the endpoint is not ready to receive the request. The client holds off the request. The timer waits for a delay time and then sends the request to the endpoint. The delay time may be calculated by the decision unit based on the above-mentioned method. If the response received by the client side has an error, the counter on the client side increases the retry count by one, and the timer counts the delay time for retrying the transmission. The delay time may be calculated by the decision unit based on the above-mentioned method. After the retry, the correct data is received by the client side and presented on the UI.

[0090] 18 is an exemplary data access flow according to some embodiments of the present invention. Some steps are the same as those described in FIG. 16 and FIG.

[0091] The client invokes an API request to the API service, which checks the status data of the endpoint corresponding to the API request with the API score service.

[0092] In step S180, the decision unit 224 determines whether the endpoint is in good health based on the status data. If yes, the request is sent to the endpoint. If no, the flow proceeds to step S182.

[0093] In step S182, the determining unit 224 determines whether to wait a delay time before sending the request to the endpoint based on the severity status of the endpoint. If the status of the endpoint is not severe and there is no need to wait, the request is sent to the endpoint. If the status is severe, a timer is used to count the delay time before sending the request to the endpoint. The delay time may be determined based on the method described above.

[0094] In step S184, the decision unit 224 determines whether the response from the endpoint is correct. If the response is correct, the correct data is sent to the API service, which sends the data to the client. If the response contains an error, the flow proceeds to step S186.

[0095] In step S186, the decision unit 224 determines whether a retry is applicable to sending the API request. If yes, the flow proceeds to step S188. If no, an error message is sent to the API service, which passes the error message to the client.

[0096] In step S188, the decision unit 224 determines whether the number of retries exceeds a determined maximum number of retries (which may be determined according to the method described above). If no, the flow proceeds to step S190. If yes, an error message is sent to the API service, which passes the error message to the client.

[0097] In step S190, a counter increments the retry count by one and flow returns to step S182 where the updated status data of the endpoint is used to determine a retry waiting process for sending the request.

[0098] The hardware configuration of the information processing device will be described with reference to Fig. 19. Fig. 19 is a block diagram showing an example of the hardware configuration of an information processing device based on some embodiments of the present invention. The information processing device 900 shown in the figure can realize, for example, the server 10 and / or the user terminals 20 and 30 in some embodiments.

[0099] The information processing device 900 includes a CPU 901, a ROM (read only memory) 903, and a RAM (random access memory) 905. The information processing device 900 may also include a host bus 907, a bridge 909, an external bus 911, an interface 913, an input device 915, an output device 917, a storage device 919, a drive 921, a connection port 925, and a communication device 929. The information processing device 900 further includes an imaging device such as a camera (not shown). The information processing device 900 may also include a DSP (digital signal processor) or an ASIC (application specific integrated circuit) in addition to or instead of the CPU 901.

[0100] The CPU 901 functions as an arithmetic processing device and a control device, and controls all or part of the operation of the information processing device 900 according to various programs stored in the ROM 903, the RAM 905, the storage device 919, or the removable recording medium 923. For example, the CPU 901 controls the overall operation of each functional unit included in the server 10 and the user terminals 20 and 30 in some embodiments. The ROM 903 stores programs and arithmetic parameters used by the CPU 901. The RAM 905 functions as a main storage device that stores programs used when the CPU 901 is executed, parameters that change appropriately during execution, and the like. The CPU 901, the ROM 903, and the RAM 905 are interconnected by a host bus 907, which may be an internal bus such as a CPU bus. Furthermore, the host bus 907 is connected to an external bus 911 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 909.

[0101] The input device 915 may be a device operated by a user, such as a mouse, a keyboard, a touch panel, a button, a switch, or a lever, or a device that converts a physical quantity into an electric signal, such as a sound sensor represented by a microphone, an acceleration sensor, a tilt sensor, an infrared sensor, a depth sensor, a temperature sensor, or a humidity sensor. The input device 915 may be, for example, a remote control device that uses radio waves such as infrared rays, or an external connection device 927 such as a mobile phone that supports the operation of the information processing device 900. The input device 915 includes an input control circuit that generates an input signal based on information input by the user or a detected physical quantity and outputs the input signal to the CPU 901. The user operates the input device 915 to input various data to the information processing device 900 and instruct the information processing device 900 to operate.

[0102] The output device 917 is a device that can visually or audibly notify the user of acquired information. The output device 917 may be, for example, a display such as an LCD, a PDP, or an OLED, an audio output device such as a speaker or a headphone, or a printer. The output device 917 outputs the processing result by the information processing device 900 as a video such as text or an image, or an audio such as a voice.

[0103] The storage device 919 is a data storage device and is configured as an example of a storage unit of the information processing device 900. The storage device 919 is, for example, a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, or a magneto-optical storage device. The storage device 919 stores the programs executed by the CPU 901, various data, various data acquired from the outside, and the like.

[0104] The drive 921 is a reader / writer of a removable recording medium 923 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and is built into or externally attached to the information processing device 900. The drive 921 reads out information recorded on the attached removable recording medium 923, and outputs it to the RAM 905. Furthermore, the drive 921 writes information to the attached removable recording medium 923.

[0105] The connection port 925 is a port for directly connecting a device to the information processing device 900. The connection port 925 may be, for example, a USB (Universal Serial Bus) port, an IEEE 1394 port, or a SCSI (Small Computer System Interface) port. Furthermore, the connection port 925 may be an RS-232C port, an optical audio terminal, an HDMI (High Definition Multimedia Interface) port, or the like. By connecting the external connection device 927 to the connection port 925, various data can be exchanged between the information processing device 900 and the external connection device 927.

[0106] The communication device 929 is, for example, a communication interface formed by a communication device for connecting to the network NW. The communication device 929 may be, for example, a communication card for a wired or wireless local area network (LAN), Bluetooth (trademark), or wireless USB (WUSB). Furthermore, the communication device 929 may be a router for optical communication, a router for ADSL (asymmetric digital subscriber line), or a modem for various communications. The communication device 929 transmits and receives signals, etc., on the Internet or between other communication devices using a predetermined protocol such as TCP / IP. The communication network NW connected to the communication device 929 is a network connected by wire or wirelessly, for example, the Internet, a home LAN, infrared communication, radio wave communication, satellite communication, etc. The communication device 929 realizes a function as a communication device.

[0107] The imaging device (not shown) is a device that captures an image of real space using an imaging element such as a CCD (charge-coupled device) or a CMOS (complementary metal-oxide semiconductor) and various members such as a lens for controlling the formation of a subject image on the imaging element, and generates a captured image. The imaging device may capture still images or moving images.

[0108] The configuration and operation of the live streaming system 1 according to this embodiment have been described above. This embodiment is merely an example, and it will be understood by those skilled in the art that various modifications are possible with respect to the combination of each component and each process, and that such modifications are also within the scope of the present invention.

[0109] The processes and procedures described in the present invention can be realized by software, hardware, or any combination thereof, in addition to those explicitly described. For example, the processes and procedures described herein can be realized by implementing logic corresponding to the processes and procedures in a medium such as an integrated circuit, a volatile memory, a non-volatile memory, a non-transitory computer-readable medium, a magnetic disk, etc. Furthermore, the processes and procedures described herein can be realized as a computer program corresponding to the processes and procedures, and can be executed by various computers.

[0110] Furthermore, the system or method described in the above embodiments may be integrated into a program stored in a non-transitory computer-readable medium such as a solid-state storage device, an optical disk storage device, a magnetic disk storage device, etc. Alternatively, the program may be downloaded from a server via the Internet and executed by a processor.

[0111] Although the technical contents and features of the present invention have been described above, those skilled in the art can still make many variations and modifications without departing from the teachings and disclosure of the present invention. Therefore, the scope of the present invention is not limited to the embodiments already disclosed, but is within the scope of the claims, including other variations and modifications that do not depart from the present invention. [Explanation of symbols]

[0112] 1. Communication Systems 10 Server 20 User terminal 30, 30a, 30b User terminal LV Streamer AU1, AU2 viewers VD, VD1, VD2 Video NW Network 30 User terminals 100 Delivery Units 102 Imaging control unit 104 Audio control unit 106 Video transmission unit 108 Provider UI Control Unit 200 viewing units 202 Viewer side UI control unit 204 Superposition Information Generation Unit 206 Input information transmission unit 302 Distribution Information Unit 304 Relay unit 306 Gift Processing Unit 308 Payment Processing Unit 310 Stream DB 312 User DB 314 Gift DB 220 Acquired Units 222 Transmitting unit 224 Decision Unit 250 request DB 252 User Tolerance DB 254 Retry DB 320 Status DB 324 Machine Learning DB S1100, S1102, S1104, S1106, S1108, S1110, S1112, S1114, S1116, S1118, S1120 process S130, S132, S134, S136, S1340, S1342, S1344 process S180, S182, S184, S186, S188, S190 process 900 Information processing device 901 CPU 903 ROM 905 RAM 907 Host Bus 909 Bridge 911 External Bus 913 Interface 915 Input Device 917 Output Device 919 Storage Device 921 Drive 923 Removable Recording Media 925 Connection Port 927 External connection device 929 Communication Equipment

Claims

1. 1. A method of data access performed by a user terminal, comprising: receiving a request from a user of a user terminal; obtaining status data of a backend server corresponding to the request; sending the request to the backend server corresponding to the request based on the status data; obtaining a response to the request from the backend server; determining that the response is in error; determining a delay time for retrying sending the request to the backend server based on the status data; 1. A method of data access comprising:

2. 2. The method of claim 1, further comprising the step of determining a total number of retries to send the request to the backend server based on the status data.

3. 2. The method of claim 1, wherein the delay time is determined based on the status data and a retry count for the retry.

4. moreover, determining that the delay time is longer than an update period of the status data; obtaining updated status data; updating the delay time for the retry based on the updated status data; 2. The method of claim 1, comprising:

5. moreover, obtaining a status of the user; obtaining a type of the request; 2. The method of claim 1, further comprising: determining the delay time based on the status data, the status of the user, and the type of the request.

6. moreover, obtaining said user's tolerance data; obtaining a type of the request; 2. The method of claim 1, further comprising: determining said delay time based on said tolerance data of said user and said type of said request.

7. moreover, determining to shorten the delay time if it is determined based on the tolerance data of the user that the user has a low tolerance for the type of request; determining to increase the delay time if it is determined based on the tolerance data of the user that the user has a high tolerance for the type of request; 7. The method of claim 6, comprising:

8. 8. The method of data access of claim 7, wherein the requests are associated with the user's operations in a live streaming chat room, and the tolerance data indicates the user's dependency on different tolerance levels for different types of requests.

9. 1. A method of data access performed by a user terminal, comprising: receiving a request from a user of a user terminal; obtaining a type of the request; obtaining said user's request tolerance data; determining a delay time for transmitting the request to a backend server corresponding to the request based on the type of the request and the request tolerance data of the user; 1. A method of data access comprising:

10. moreover, obtaining a status of the user; 10. The method of claim 9, further comprising: determining the delay time based on the status of the user, the type of the request, and the request tolerance data of the user.

11. 1. A system for broadcaster analysis, comprising: one or more processors, the one or more processors executing machine-readable instructions to: receiving a request from a user of a user terminal; obtaining status data of a backend server corresponding to the request; sending the request to the backend server corresponding to the request based on the status data; obtaining a response to the request from the backend server; determining that the response is in error; determining a delay time for retrying sending the request to the backend server based on the status data; A system comprising:

Images