Video double-recording data transmission method, device, node, medium and program product

Abstract

The invention discloses a video double-recording data transmission method and device, a node, a medium and a program product. Comprising the following steps: receiving kth fragmented data sent by the client equipment; wherein the kth fragmented data is any data in K pieces of fragmented data obtained by fragmenting the video double-recording data by the client device; k is an integer greater than or equal to 1 and less than or equal to K; k is an integer greater than 1; determining a first node from a plurality of service nodes contained in the micro-service cluster at least based on a load balancing strategy; transmitting the kth fragmented data to the first node, so that the first node transmits the kth fragmented data to a first storage device; wherein the first storage device is used for integrating the first fragment data to the Kth fragment data to obtain and store the video double-recording data.

Description

Technical Field

[0001] The present application relates to the field of data processing in financial business scenarios, and in particular to a video dual-recording data transmission method, device, node, medium and program product. Background Art

[0002] The data volume of the video dual-recording file obtained by the online video dual-recording operation is large. If the video dual-recording file is transmitted by the traditional data transmission method, it is easy to cause transmission jams or even failures. Therefore, the video dual-recording file upload method in the related art cannot meet the actual business processing needs. Summary of the invention

[0003] Based on the above technical problems, the embodiments of the present application provide a video dual recording data transmission method, device, node, medium and program product.

[0004] The embodiment of the present application first provides a video dual-recording data transmission method, which is applied to a gateway device associated with a client device; the method includes:

[0005] Receive the kth fragment data sent by the client device; wherein the kth fragment data is any one of the K fragment data obtained by the client device fragmenting the video dual recording data; k is an integer greater than or equal to 1 and less than or equal to K; K is an integer greater than 1;

[0006] Determine a first node from a plurality of service nodes included in the microservice cluster based at least on a load balancing strategy;

[0007] The kth slice data is transmitted to the first node, so that the first node transmits the kth slice data to the first storage device; wherein the first storage device is used to integrate the first slice data to the Kth slice data to obtain and store the video dual recording data.

[0008] The embodiment of the present application also provides another video dual recording data transmission method, which is applied to a target node among multiple service nodes included in a microservice cluster; the method includes:

[0009] Receive the kth slice data transmitted by the gateway device; wherein the gateway device is used to receive the kth slice data sent by the client device, and determine the target node from the multiple service nodes based on at least a load balancing strategy; the kth slice data is any one of the K slice data obtained by the client device from slicing the video dual recording data; k is an integer greater than or equal to 1 and less than or equal to K; K is an integer greater than 1;

[0010] The kth slice data is transmitted to a first storage device, so that the first storage device integrates the first slice data into the Kth slice data to obtain and store the video dual recording data.

[0011] The embodiment of the present application further provides a gateway device, the gateway device comprising:

[0012] A first transceiver module is configured to receive the kth fragment data sent by the client device; wherein the kth fragment data is any one of the K fragment data obtained by the client device fragmenting the video dual recording data; k is an integer greater than or equal to 1 and less than or equal to K; K is an integer greater than 1;

[0013] A determination module, configured to determine a first node from a plurality of service nodes included in the microservice cluster based at least on a load balancing strategy;

[0014] The first transceiver module is also used to transmit the kth slice data to the first node, so that the first node can transmit the kth slice data to the first storage device; wherein the first storage device is used to integrate the first slice data to the Kth slice data to obtain and store the video dual recording data.

[0015] An embodiment of the present application also provides another gateway device, which includes a first processor and a first memory; a first computer program is stored in the first memory; when the first computer program is executed by the first processor, it can implement the video dual recording data transmission method applied to the gateway device as described in any of the above.

[0016] The embodiment of the present application further provides a target node, wherein the service node is set in a microservice cluster; the target node is determined by a gateway device from a plurality of service nodes included in the microservice cluster; the target node includes:

[0017] The second transceiver module is used to receive the kth slice data transmitted by the gateway device; wherein the gateway device is used to receive the kth slice data sent by the client device, and determine the target node from the multiple service nodes based on at least a load balancing strategy; the kth slice data is any one of the K slice data obtained by the client device from slicing the video dual recording data; k is an integer greater than or equal to 1 and less than or equal to K; K is an integer greater than 1;

[0018] The second transceiver module is further used to transmit the kth slice data to the first storage device, so that the first storage device integrates the first slice data into the Kth slice data to obtain and store the video dual recording data.

[0019] An embodiment of the present application also provides another target node, wherein the service node is set in a microservice cluster; the target node is determined by a gateway device from multiple service nodes included in the microservice cluster; the target node includes a second processor and a second memory; wherein a second computer program is stored in the second memory; when the second computer program is executed by the second processor, it can implement the video dual-recording data transmission method applied to the target node as described in any of the above.

[0020] An embodiment of the present application further provides a computer-readable storage medium, in which a third computer program is stored; when the third computer program is executed by a processor of an electronic device, it can implement any of the above-mentioned video dual-recording data transmission methods.

[0021] The embodiment of the present application further provides a computer program product, which includes a fourth computer program; when the fourth computer program is executed by a processor of an electronic device, it can implement any of the above-mentioned video dual-recording data transmission methods.

[0022] In the video dual-recording data transmission method applied to a gateway device provided in an embodiment of the present application, a gateway device associated with a client device receives the kth shard data sent by the client device, and the kth shard data is any data of the K shard data obtained by sharding the video dual-recording data by the client device. Thus, through the above operation, the gateway device can flexibly receive any data of the K shard data associated with the video dual-recording data sent by the client device; and, by determining the first node from the multiple service nodes included in the microservice cluster at least based on the load balancing strategy, the gateway device can flexibly determine the first node based on the load balancing strategy in the scenario of receiving the kth shard data. Thus, not only can the efficiency of the kth shard data being transmitted through the first node be improved, but also, Through the above operation, the probability that multiple shard data are directed to the same service node can be reduced, thereby overcoming the problem in the related art that the service node is overloaded due to all shard data being directed to the same service node; on this basis, the kth shard data is transmitted to the first node so that the first node can transmit the kth shard data to the first storage device, thereby improving the efficiency of transmitting the kth shard data from the client device to the first storage device via the gateway device and the first node; on the other hand, by integrating the first shard data to the Kth shard data by the first storage device to obtain and store video dual recording data, stable and efficient transmission of video dual recording data from the client device to the first storage device is achieved, thereby meeting the video dual recording data transmission requirements in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A video dual recording file transmission method applied to a gateway device provided in an embodiment of the present application;

[0024] Figure 2 A schematic diagram of a flow chart of a video dual recording data transmission method applied to a target node provided in an embodiment of the present application;

[0025] Figure 3 A schematic diagram of a network structure for video dual-recording data transmission provided in an embodiment of the present application;

[0026] Figure 4 Another schematic diagram of the process of video dual recording data transmission provided in an embodiment of the present application;

[0027] Figure 5 A statistical diagram of the consumption status of the central processing unit (CPU) resources of the service node during the video dual recording data transmission process provided in an embodiment of the present application;

[0028] Figure 6 A schematic diagram of the statistical results of the memory occupancy rate of the service node during the video dual recording data transmission process provided in an embodiment of the present application;

[0029] Figure 7 A schematic diagram of the statistical results of the data transmission volume of the service node during the video dual recording data transmission process provided in the embodiment of the present application;

[0030] Figure 8 A schematic diagram of another statistical result of the data transmission volume of the service node during the video dual recording data transmission process provided in an embodiment of the present application;

[0031] Fig. 9 A schematic diagram of the statistical results of the average response time of the service nodes provided in the embodiment of the present application;

[0032] Fig.10 A schematic diagram of the structure of a gateway device provided in an embodiment of the present application;

[0033] Fig.11 Another schematic diagram of the structure of the gateway device provided in the embodiment of the present application;

[0034] Fig.12 A schematic diagram of the structure of the target node provided in the embodiment of the present application;

[0035] Fig.13 Another structural diagram of a target node provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0037] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0038] With the explosive growth of digital media content and types, large file uploading has become an important guarantee for the stable operation of modern network applications in areas such as video sharing, cloud storage and big data processing.

[0039] In the technical field of financial business processing, the online video dual recording system has become a regular channel for private equity sales business. It can undertake the recording of video dual recording files for a large number of high-end customers every day. In addition, with the acceleration of business types, scopes and the update and iteration speed of mobile devices, the data volume of video dual recording files is becoming larger and larger. Therefore, efficient and stable transmission of video dual recording files can improve the stability and efficiency of financial business processing.

[0040] The technical solution for file uploading in the related art mainly relies on the direct interaction between the client including the browser and the server and the Hypertext Transfer Protocol (HTTP) to complete the file transmission. Among them, the above solution usually includes the following steps:

[0041] Step 1: Create an upload form:

[0042] Using HyperText Markup Language (HTML) <form>The tag creates a file upload form. The GET method is limited by the length of the Uniform Resource Locator (URL) and is not suitable for file upload. Therefore, the method attribute of the above form must be set to POST to provide conditions for uploading large files. The enctype attribute of the form must be set to multipart / form-data, which is used to instruct the browser how the form data should be encoded in order to carry the file content. In addition, add <input type＝"file"> Label that allows the user to select a file.

[0043] Step 2: User selects file:

[0044] The user accesses the upload page through a browser and selects a file selection box to select a large file to be uploaded from the local file system.

[0045] Step 3: Submit the form:

[0046] After the user fills out the form and clicks the Submit option, the browser will encode the form data, including large files, into the multipart / form-data format and send it to the server via an HTTP POST request.

[0047] Step 4: Server processing:

[0048] After receiving the HTTP POST request, the server parses the multipart / form-data data in the request, extracts the large file content and other form fields, saves the large file to the specified directory or database, and then processes other form fields as needed.

[0049] When the network environment is stable and the bandwidth is wide, the above scheme is simple, direct and easy to implement when transmitting small files. Therefore, the above scheme is suitable for most Web application scenarios of small files. However, the above scheme is suitable for synchronous upload scenarios, which means that the above scheme requires users to wait for the file upload to complete before performing other data transmission operations; and the above scheme relies on a direct connection between the client and the server, so it has high network requirements; at the same time, when dealing with large file uploads, the above scheme can easily cause the page to freeze and fail to respond for a long time; on the other hand, the data transmission speed of the above scheme is slow, the network bandwidth cannot be fully utilized, there is a single point problem, and there is no system fault tolerance.

[0050] Therefore, the above solution limits the upload efficiency and reduces the user experience when facing the transmission of large files of several GB or even TB level, such as video dual recording files; when the network conditions are unstable or the terminal environment changes frequently, the failure of video dual recording file upload executed by the above solution becomes the norm.

[0051] In summary, the file upload mechanism provided by the above solution is prone to blocking of synchronous processing, low transmission efficiency and potential single point failure risks when uploading large files.

[0052] In order to solve the above technical problems, the relevant technology also proposes an asynchronous upload strategy; however, although the asynchronous upload strategy alleviates the problem of synchronous waiting to a certain extent and improves the fluency of the user interface, it cannot fundamentally solve the problems of network fluctuations and single point failures.

[0053] The related art also provides a technical solution for slicing large files and transmitting the multiple data slicings obtained after slicing to network devices respectively. This solution can improve the transmission efficiency of large files to a certain extent and enhance fault tolerance. However, the slicing data obtained after slicing in the related art needs to be transmitted according to a strict request routing strategy, that is, all slicings are required to be strictly directed to the same service node. This centralized processing mode not only increases the load pressure of a single service node, but also the serialized processing method cannot fully utilize the network bandwidth, especially in the scenario of multi-user or multi-task concurrency, the file transmission efficiency of the above scheme is relatively low; and the above scheme is not robust enough. For example, if the service node used to receive multiple slicing data fails or restarts, the entire file upload process will be forced to interrupt, and the uploaded slicing data will face the risk of loss.

[0054] In summary, although the existing large file uploading solutions have their own advantages, they cannot fundamentally solve the problems faced in the video dual recording file uploading process, including transmission stability, single point failure and low bandwidth utilization.

[0055] In order to solve the above technical problems, the embodiments of the present application provide a video dual recording data transmission method, device, node, medium and program product.

[0056] Figure 1 The present invention provides a video dual recording file transmission method applied to a gateway device in an embodiment of the present invention, wherein the gateway device is associated with a client device.

[0057] like Figure 1 As shown, the method may include the following steps:

[0058] Step 101: Receive the kth fragment data sent by a client device.

[0059] The kth fragment data is any one of the K fragment data obtained by fragmenting the video dual recording data by the client device; k is an integer greater than or equal to 1 and less than or equal to K; and K is an integer greater than 1.

[0060] In one implementation, the client device may include a physical machine device or a virtual machine device.

[0061] In one embodiment, the client device may include a data collection device with a data collection function; exemplarily, the collection device may include any one of a computer device, a mobile terminal device, and a data collection device; exemplarily, the mobile terminal device may include a smart phone, and the data collection device may include a device with an image and / or sound collection function.

[0062] In one implementation, the association between the gateway device and the client device may indicate that a communication connection is established between the gateway device and the client device, so that the gateway device can achieve unidirectional or bidirectional data transmission with the client device through the communication connection.

[0063] In one implementation, the video dual recording data may be associated with at least one link corresponding to the securities business, that is, the video dual recording data may be used to accurately record the process details of the at least one link.

[0064] In one implementation, the video dual recording data may be related to the securities business type.

[0065] In one implementation, the video dual recording data may appear in the form of a single file, and the data volume of the video dual recording data may be greater than or equal to a data volume threshold.

[0066] In one implementation, K shard data may be obtained in the following manner:

[0067] The client device obtains a slicing parameter, and slices the video dual recording data based on the slicing parameter to obtain K sliced ​​data; illustratively, the slicing parameter can be determined according to at least one of the data volume of the video dual recording data, the network status, and the transmission requirement; wherein the network status may include the network bandwidth between the gateway device and the server device, the data transmission rate and its stability, etc., and the transmission requirement may include the target transmission efficiency for the video dual recording data.

[0068] In one embodiment, after obtaining K shard data, the client device may send the first shard data to the Kth shard data to the gateway device in sequence, may send K shard data to the gateway device at the same time, or may randomly send K shard data to the gateway device.

[0069] Step 102: Determine a first node from a plurality of service nodes included in the microservice cluster based at least on a load balancing strategy.

[0070] In one implementation, the multiple service nodes included in the microservice cluster may include physical machine devices or virtual machine devices.

[0071] In one implementation, a service node in a microservice cluster may provide at least one microservice, and the service node may be a server-side device in the aforementioned embodiment.

[0072] In one implementation, the microservice cluster may include multiple clusters, and the service nodes included in each cluster may have high availability characteristics. Exemplarily, the number of replicas of each service node may be 2.

[0073] In one embodiment, the gateway device obtains the load status of multiple service nodes included in the microservice cluster through the communication connection between it and the microservice cluster, and determines the first node from the multiple service nodes based on the above load status and the load balancing strategy. That is, the first node determined when the gateway device receives the kth shard data and the first node determined when it receives the k+1th shard data may not be the same service node.

[0074] In one implementation, the gateway device may also acquire the first node from multiple service nodes based on a random strategy.

[0075] In one embodiment, the gateway device may include Nginx.

[0076] Step 103: Transmit the kth shard data to the first node, so that the first node transmits the kth shard data to the first storage device.

[0077] The first storage device is used to integrate the first segment data to the Kth segment data to obtain and store the video dual recording data.

[0078] In one embodiment, after receiving the kth shard data, the first node can transmit the kth shard data to the target storage space in the first storage device; exemplarily, the target storage space may include at least one predetermined storage unit specifically used to store the shard data sent by the client device.

[0079] In one implementation, the first storage device may include a network-side storage device specifically used to store large data or large files; illustratively, the first storage device may include a network attached storage (NAS). In actual applications, NAS is a dedicated data storage server.

[0080] In one implementation, the first storage device may obtain the video dual recording data in the following manner:

[0081] After the gateway device or the first storage device determines that it has received K sliced ​​data sent by the client device and associated with the same business identifier, the K sliced ​​data are integrated based on the sequential numbers corresponding to the K sliced ​​data to obtain video dual recording data; wherein the business identifier can represent the number or name of the securities business associated with the video dual recording data.

[0082] Exemplarily, after obtaining the video dual-recording data, the first storage device may delete the K slice data stored therein.

[0083] As can be seen from the above, in the video dual-recording data transmission method applied to a gateway device provided by an embodiment of the present application, the gateway device associated with the client device receives the kth shard data sent by the client device, and the kth shard data is any data of the K shard data obtained by sharding the video dual-recording data by the client device. In this way, through the above operation, the gateway device can flexibly receive any data of the K shard data associated with the video dual-recording data sent by the client device; and, by determining the first node from the multiple service nodes included in the microservice cluster at least based on the load balancing strategy, the gateway device can flexibly determine the first node based on the load balancing strategy in the scenario of receiving the kth shard data. In this way, not only can the efficiency of the kth shard data being transmitted through the first node be improved, Moreover, through the above operation, the probability that multiple shard data are directed to the same service node can be reduced, thereby overcoming the problem in the related art that the service node is overloaded due to all shard data being directed to the same service node; on this basis, the kth shard data is transmitted to the first node so that the first node can transmit the kth shard data to the first storage device, thereby improving the efficiency of transmitting the kth shard data from the client device to the first storage device via the gateway device and the first node; on the other hand, by integrating the first shard data to the Kth shard data by the first storage device to obtain and store video dual recording data, stable and efficient transmission of video dual recording data from the client device to the first storage device is achieved, thereby meeting the video dual recording data transmission requirements in practical applications.

[0084] Based on the foregoing embodiment, in the video dual recording data transmission method applied to a gateway device provided in the embodiment of the present application, before receiving the kth fragment data sent by the client device, the following steps may also be performed:

[0085] Step A1: receiving a transmission start request sent by a client device.

[0086] In one implementation, the transmission initiation request may include a request sent by the client device to request transmission of the first fragment data, or may include a request by the client device to request transmission of other fragment data among the K fragment data except the first fragment data.

[0087] Step A2: determine a second node from a plurality of service nodes, and send a transmission start request to the second node to trigger the second node to determine a breakpoint resume indication.

[0088] The breakpoint resume transmission indication includes at least the fragment identifier of the kth fragment data; the breakpoint resume transmission indication is determined by the second node based on the transmission metadata and transmission status of the video dual recording data; the transmission metadata is at least used to characterize the fragment status corresponding to the video dual recording data.

[0089] In one implementation, the gateway device may determine the second node from a plurality of service nodes based on a load balancing strategy or a random strategy.

[0090] In one implementation, the first node and the second node may be the same or different service nodes; illustratively, if the first node and the second node are different, the clusters to which the first node and the second node are respectively located may be the same or different.

[0091] In one implementation, the fragment identifier of the kth fragment data may be used to indicate the number of the kth fragment data in the K fragment data.

[0092] In one implementation, the fragmentation status may include a first data volume of the video dual-recording data, a number of fragmented data obtained after fragmenting the video dual-recording data, and a second data volume of each fragmented data.

[0093] In one implementation, the transmission metadata may include, in addition to characterizing the segmentation status, the file name, concurrency, and storage address of the target storage space of the video dual-recording data.

[0094] In one implementation, the transmission metadata may be embodied in the form of shard metadata.

[0095] In one implementation, the transmission status may include the number of shard data that the client device has transmitted to the first storage device, and shard identifiers of the shard data that has been transmitted to the first storage device.

[0096] In one implementation, the breakpoint resume indication may be determined in the following manner:

[0097] If the transmission metadata is not empty and is associated with the client device, or the transmission metadata is associated with a designated securities business corresponding to the client device, the remaining identifier is determined based on the transmission status, and then a breakpoint resume indication is generated based on the remaining identifier.

[0098] Exemplarily, the remaining identifier may represent a set of shard identifiers of the shard data that have not been transmitted to the first storage device in the K shard data; exemplarily, the remaining identifier may be determined in the following manner:

[0099] If the transmission status indicates that the number of shard data transmitted from the client device to the first storage device is less than K, the remaining identifiers of the remaining shard data that have not been transmitted to the first storage device are determined based on the shard identifiers included in the transmission status.

[0100] In one implementation, the remaining identifier may include the fragment identifier of the kth fragment data.

[0101] Step A3: receiving and sending the breakpoint resume instruction sent by the second node to the client device, so as to instruct the client device to send the kth fragment data to the gateway device.

[0102] In one embodiment, after receiving the breakpoint resume indication, the gateway device can send the breakpoint resume indication to the client device, so that the client device can parse the breakpoint resume indication and obtain the fragment identifier of the kth fragment data, and then obtain and send the kth fragment data to the gateway device based on the fragment identifier of the kth fragment data.

[0103] As can be seen from the above, the video dual recording data transmission method applied to the gateway device provided in the embodiment of the present application is that after the gateway device receives and sends the start transmission request sent by the client device, the second node is determined from the service node, and the start transmission request is sent to the second node. In this way, through the above operation, the control of the operation of determining the second node is realized; and, by sending the start transmission request, the second node is triggered to determine the breakpoint resume indication, and the breakpoint resume indication at least includes the fragment identifier of the kth fragment data, and the breakpoint resume indication is determined by the second node based on the transmission metadata and transmission status of the video dual recording data. The transmission metadata is at least used to characterize the fragment status of the video dual recording data. In this way, through the above operation, not only the accuracy of the client device transmitting the kth fragment data can be improved, but also the stability and robustness of the client device sending the fragment data to the first storage device through the gateway device can be improved; on this basis, the breakpoint resume indication sent by the second node is received and sent to the client device to instruct the client device to send the kth fragment data to the gateway data, which can improve the stability and robustness of the transmission of the fragment data between the client device and the second node.

[0104] Based on the foregoing embodiment, in the video dual recording data transmission method applied to a gateway device provided in the embodiment of the present application, before receiving the kth fragment data sent by the client device, the following steps may also be performed:

[0105] Step B1: In response to a start transmission request sent by a client, a third node is determined from a plurality of service nodes.

[0106] The transmission start request at least includes a first hash value; the first hash value includes a hash value of the video dual recording data.

[0107] In one implementation, the gateway device may determine the third node from a plurality of service nodes based on a load balancing strategy or a random strategy.

[0108] In one embodiment, the third node may be the same as or different from the first node and the second node in the aforementioned example; for example, if the third node is different from the first node and the second node, then the clusters in which the third node and the first node and the second node are located may be the same as or different.

[0109] In one implementation, the first hash value may include a hash value of the video dual recording data to be transmitted by the client device; illustratively, the first hash value may include an MD5 value of the video dual recording data, and the first hash value may be determined by the client device processing the video dual recording data.

[0110] In one implementation, if the client device requests to transmit the first fragment data, a transmission start request may be generated based on the first hash value. That is, at this time, the transmission start request may carry the first hash value.

[0111] Step B2: Send a start transmission request to the third node to trigger the third node to determine the transmission metadata associated with the client device based on the first hash value.

[0112] The transmission metadata is at least used to characterize the fragmentation status corresponding to the video dual-recording data.

[0113] In one embodiment, after receiving the request to start transmission, the third node can determine whether there is transmission metadata corresponding to the client device or the securities business associated with the client device, and if the transmission metadata does not exist, determine the transmission metadata associated with the client device based on the first hash value.

[0114] Step B3: receiving and sending the transmission metadata sent by the third node to the client device, so that the client device can segment the video dual recording data based on the transmission metadata.

[0115] In one embodiment, after the gateway device sends the transmission metadata to the client device, the client device can parse the transmission metadata to determine the second data volume and concurrency, and then divide the video dual recording data into K slices of data through a cross-platform mobile application development framework including React Native (RN for short) set in the client device, and number each slice of data, and then send data transmission requests carrying a single slice of data to the gateway device in batches based on the concurrency limit, thereby sending the first slice of data to the Kth slice of data to the gateway device.

[0116] As can be seen from the above, in the video dual recording data transmission method applied to the gateway device provided in the embodiment of the present application, the gateway device responds to the start transmission request of the client device, determines the third node from multiple service nodes, and then sends the start transmission request to the third node. In this way, through the above operation, flexible determination of the third node is achieved; and the start transmission request at least includes a first hash value, and the first hash value includes a hash value of the video dual recording data. The third node determines the transmission metadata associated with the client device based on the first hash value, and the transmission metadata is at least used to characterize the sharding state corresponding to the video dual recording data. In this way, through the above operation, the third node can flexibly determine the transmission metadata; on this basis, the gateway device receives and sends the transmission metadata sent by the third node to the client device, so that the client device can shard the video dual recording data based on the transmission metadata. In this way, through the above operation, indirect control of the sharding operation of the client device by the third node is achieved, so that the sharding data obtained by the client device sharding the video dual recording data can match the network state between the client device and the microservice cluster, thereby improving the stability and robustness of the sharding data transmission.

[0117] Based on the foregoing embodiment, the video dual recording data transmission method applied to a gateway device provided in the embodiment of the present application may further perform the following steps:

[0118] Step C1: if a transmission end request sent by the client device is detected, the transmission end request is transmitted to the third node.

[0119] Among them, the third node is a node among multiple service nodes that receives a transmission start request sent by a client device; the third node is used to determine the transmission result of the video dual recording data based on at least the first hash value and the second hash value; the first hash value includes the hash value of the video dual recording data; the second hash value includes the hash value of the transmission data obtained after integrating the fragmented data stored in the first storage device.

[0120] Exemplarily, if the end transmission operation sent by the client device is not detected, the end transmission request may not be transmitted to the third node.

[0121] In one implementation, the end transfer request may be triggered by any of the following methods:

[0122] After the client device determines that it has transferred the K shard data to the first storage device, it generates a request to end the transfer.

[0123] An end transfer request is generated when the client device is in a fault state.

[0124] When the client device detects the data transmission termination operation, it generates a transmission termination request; illustratively, the data transmission termination operation may be triggered by a host device or a user of the client device.

[0125] In one implementation, after receiving the start transmission request carrying the first hash value, the gateway device may parse the start transmission request to obtain and store the first hash value.

[0126] In one embodiment, the transmission data may be determined by:

[0127] After receiving the data integration indication related to the end of data transmission sent by the gateway device, the first storage device obtains the received shard data associated with the client device from its target storage space, and sorts and integrates the shard data according to the shard identifiers of each shard data to obtain the transmission data.

[0128] In one implementation, the second hash value may be obtained in the following manner:

[0129] A hash value of the transmission data is determined as a second hash value.

[0130] The MD5 value of the transmission data is determined as the second hash value.

[0131] In one implementation, the transmission result may include the result that all K slice data corresponding to the video dual recording data are transmitted to the first storage device, and may also include the result that part of the K slice data corresponding to the video dual recording data is transmitted to the first storage device.

[0132] In one embodiment, the transmission result may be determined by:

[0133] The third node determines the transmission result based on the degree of match between the first hash value and the second hash value; for example, if the first hash value matches the second hash value, it can be determined that the transmission result is: all K fragmented data corresponding to the video dual recording data are transmitted; for another example, if the first hash value does not match the second hash value, it can be determined that the transmission result is: the fragmented data corresponding to the video dual recording data is not transmitted. At this time, the third node can generate and send a breakpoint resume indication to the gateway device through the method provided in the aforementioned embodiment, so that the gateway device can send the breakpoint resume indication to the client device.

[0134] Step C2: receiving and transmitting the transmission result sent by the third node to the client device to instruct the client device to perform a specified operation.

[0135] The specified operation includes a breakpoint resume operation or a transmission end operation.

[0136] In one implementation, the method provided in the aforementioned embodiment may be used to trigger the terminal device to perform a breakpoint resume operation.

[0137] In one embodiment, if the terminal device executes the end transmission operation, it can indicate that the terminal device has successfully transmitted all the sliced ​​data corresponding to the video dual recording data to the first storage device; in this case, the first storage device can sort and integrate the K sliced ​​data to obtain the transmission data, and delete the K sliced ​​data stored in the target storage space.

[0138] From the above, it can be seen that the video dual recording data transmission method applied to the gateway device provided in the embodiment of the present application, if the gateway device detects the end transmission request sent by the client device, the end transmission request is transmitted to the third node, and the third node is a node among multiple service nodes that receives the start transmission request sent by the client device, and the third node determines the transmission result of the video dual recording data based on at least the first hash value and the second hash value, and the second hash value includes the hash value of the transmission data obtained after the integration of the fragmented data stored in the first storage device. In this way, by sending the end transmission operation to the third node, the efficiency of determining the transmission result can be improved; and, the transmission result sent by the third node is received and transmitted to the client device to instruct the client device to perform a breakpoint resume operation or an end transmission operation, thereby realizing precise control of the fragmented data corresponding to the video dual recording data transmitted by the client device, and can also improve the stability and robustness of the video dual recording data transmission.

[0139] Based on the above embodiment, the embodiment of the present application also provides a video dual recording data transmission method applied to a target node in a microservice cluster. The target node may be a node determined by a gateway device among multiple node service nodes included in the microservice cluster; illustratively, the target node may include the first node, the second node, and the third node in the above embodiment.

[0140] Figure 2 A schematic diagram of a process flow of a video dual recording data transmission method applied to a target node provided in an embodiment of the present application, such as Figure 2 As shown, the process may include the following steps:

[0141] Step 201: Receive the kth fragment data sent by the gateway device.

[0142] Among them, the gateway device is used to receive the kth fragment data sent by the client device, and determine the target node from multiple service nodes based on at least the load balancing strategy; the kth fragment data is any one of the K fragment data obtained after the client device fragments the video dual recording data; k is or equal to 1 and less than or equal to K; K is an integer greater than 1.

[0143] Exemplarily, the target node at this time may be the first node.

[0144] Step 202: Transmit the kth slice data to the first storage device, so that the first storage device integrates the first slice data to the Kth slice data to obtain and store the video dual recording data.

[0145] In one embodiment, when the client device transmits all K slice data to the first storage device, the first storage device can integrate the first slice data to the Kth slice data to obtain and store video dual recording data, and delete the first slice data to the Kth slice data.

[0146] From the above, it can be seen that the video dual-recording data transmission method applied to the target node provided in the embodiment of the present application is that after the target node is a gateway device that receives the kth fragment data sent by the client device, it is determined from multiple service nodes at least based on the load balancing strategy that the kth fragment data is any one of the K fragment data obtained after the client device fragments the video dual-recording data. In this way, through the above operation, the probability of sending all the fragment data to the same service node and causing the service node to be overloaded can be reduced; and by transmitting the kth fragment data to the first storage device, so that the first storage device integrates the first fragment data to the Kth fragment data to obtain and store the video dual-recording data, it is possible to achieve stable, flexible and efficient transmission of the video dual-recording data.

[0147] Based on the foregoing embodiment, the video dual-recording data transmission method applied to the target node provided in the embodiment of the present application may further perform the following operations before receiving the kth fragment data transmitted by the gateway device:

[0148] Step D1: Receive a transmission start request sent by a gateway device, and obtain a transmission status associated with a client device.

[0149] Step D2: Obtain the storage status of the transmission metadata associated with the client device in the second storage device.

[0150] In one embodiment, the second storage device may be different from the first storage device. In the present application, the second storage device may be specifically used to store transmission metadata associated with each client device or associated with various securities services.

[0151] In one implementation, the second storage device may include a remote dictionary server (Remote Dictionary Server, Redis).

[0152] In one implementation, the storage status may include that the transmission metadata associated with the client device is stored in the second storage device, or that the transmission metadata associated with the client device is not stored in the second storage device.

[0153] In one embodiment, after the target node including the third node receives a start transmission request from a client device requesting to transmit the first slice data and generates transmission metadata, the transmission metadata can be stored in the second storage device.

[0154] Step D3: if the storage status indicates that the transmission metadata is stored in the second storage device, and the transmission status indicates that the client device has transmitted part of the fragmented data corresponding to the video dual recording data to the first storage device, generate and send a breakpoint resume indication to the gateway device, so that the gateway device can send the breakpoint resume indication to the client device.

[0155] The breakpoint resume transmission indication at least includes the segment identifier of the kth segment data; and the transmission metadata is at least used to characterize the segment status corresponding to the video dual recording data.

[0156] Exemplarily, if the storage status indicates that the transmission metadata is not stored in the second storage device, or the transmission status does not indicate that part of the fragmented data corresponding to the video dual recording data is transmitted to the first storage device, then no breakpoint resume indication may be generated and sent to the gateway device.

[0157] From the above, it can be seen that in the video dual-recording data transmission method applied to the target node provided in the embodiment of the present application, after the target node receives the start transmission request sent by the gateway device, it obtains the transmission status associated with the client device and the storage status of the transmission metadata associated with the client device in the second storage device. If the storage status indicates that the transmission metadata is stored in the second storage device, and the transmission status indicates that the client device has transmitted part of the fragmented data corresponding to the video dual-recording data to the first storage device, then a breakpoint resume indication is generated and sent to the gateway device. In this way, through the above method, the target node not only realizes close tracking of the status and progress of the fragmented data transmitted by the client device, but also can improve the stability and robustness of the fragmented data transmitted by the client device, thereby reducing the probability that the fragmented data cannot continue to be transmitted after the breakpoint.

[0158] Based on the foregoing embodiment, in the video dual recording data transmission method applied to the target node provided by the embodiment of the present application, before receiving the kth fragment data transmitted by the gateway device, the following operations may also be performed:

[0159] Step E1: receiving a transmission start request sent by a gateway device, and obtaining a storage status of transmission metadata associated with a client device in a second storage device.

[0160] Step E2: If the storage status indicates that the transmission metadata is not stored in the second storage device, obtain a first hash value from the transmission start request.

[0161] Exemplarily, if the storage state represents that the transmission metadata is stored in the second storage device, the first hash value may not be obtained from the start transmission request.

[0162] Exemplarily, if the storage status indicates that the transmission metadata is stored in the second storage device, a breakpoint resume indication may be generated by the method provided in the aforementioned embodiment.

[0163] Step E3: Generate transmission metadata based on the first hash value.

[0164] The transmission metadata is used to characterize the fragmentation status corresponding to the video dual recording data.

[0165] Step E4: determine a target storage address associated with the first storage device, so that the target node can transmit the shard data associated with the client device to the target storage space indicated by the target storage address.

[0166] In one implementation, the target storage address may include a slice root path of a storage unit in the first storage device for storing slice data corresponding to the video dual-recording data.

[0167] Step E5: Transmit the metadata to the gateway device.

[0168] The transmission metadata is used by the gateway device to send to the client device to instruct the client device to segment the video dual recording data.

[0169] As can be seen from the above, the video dual recording data transmission method applied to the target node provided by the embodiment of the present application is that after the target node receives the start transmission request sent by the gateway device, the storage status of the transmission metadata associated with the client device in the second storage device is obtained. If the storage status indicates that the transmission metadata is not stored in the second storage device, the first hash value is obtained from the start transmission request; thus, through the above operation, the control of the operation of obtaining the first hash value is realized; and by generating the transmission metadata based on the first hash value, the correlation between the transmission metadata and the first hash value is enhanced; on this basis, the target storage address associated with the first storage device is determined so that the target node stores the fragmented data associated with the client device to the target storage space indicated by the target storage address. Thus, through the above operation, the efficiency of the target node storing the fragmented data associated with the client device can be improved; on the other hand, by transmitting the metadata to the gateway device so that the gateway device sends the transmission metadata to the client device, the client device is triggered to fragment the video dual recording data. Thus, through the above operation, indirect control of the client for fragmenting the video dual recording data is realized, thereby improving the flexibility and controllability of the fragmentation operation performed by the client device.

[0170] Based on the foregoing embodiment, the video dual recording data transmission method applied to the target node provided in the embodiment of the present application may further include the following steps:

[0171] Step F1: If a transmission end request sent by the gateway device is received, determine a second hash value corresponding to the shard data associated with the client device in the first storage device.

[0172] Exemplarily, if the end transmission request is not received, the second hash value may not be determined.

[0173] Step F2: Obtain a first hash value from a second storage device.

[0174] The first hash value includes a hash value of the video dual recording data.

[0175] Step F3: Based on the first hash value and the second hash value, determine and send a transmission result to the gateway device, so that the gateway device triggers the client device to perform a specified operation based on the transmission result.

[0176] The specified operation includes a breakpoint resume operation or a transmission end operation.

[0177] From the above, it can be seen that in the video dual recording transmission method applied to the target node provided in the embodiment of the present application, when the target node receives the end transmission request sent by the gateway device, it determines the second hash value corresponding to the fragmented data associated with the client device in the first storage device, and based on the first hash value obtained from the second storage device, determines and sends the transmission result to the gateway device. In this way, through the above operation, the accuracy of the transmission result can be improved; and the transmission result is used to trigger the client device to perform a breakpoint resume operation or an end transmission operation. In this way, the stability of the client device transmitting the video dual recording device can be improved, and precise control of the data transmission operation performed by the client device can also be achieved.

[0178] Figure 3 A schematic diagram of a network structure for transmitting dual-recording video data provided in an embodiment of the present application, such as Figure 3 As shown, the network structure may include: a terminal device 301, a gateway device 302, a microservice cluster 303 and a storage device 304; wherein:

[0179] The terminal device 301 may include multiple clients; illustratively, the client device in the aforementioned embodiment may include a first client and / or a second client.

[0180] The gateway device 302 may be Nginx.

[0181] The microservice cluster 303 may include a first cluster and a second cluster; wherein the first cluster may include a first node set, and the second cluster may include a second node set; and the first node set and the second node set may constitute a plurality of service nodes in the aforementioned embodiment.

[0182] The storage device 304 may include NAS and Redis; wherein, NAS may be the first storage device in the aforementioned embodiment, and Redis may be the second storage device in the aforementioned embodiment.

[0183] The deployment platform of the above network structure can use Docker+Ops technology to achieve automated continuous integration (CI), continuous delivery (CD) and continuous deployment (CD); and the operating system and data upload application set in different client devices can be different.

[0184] Redis in the above network structure can be replaced by middleware Etcd or Zookeeper that also supports key-value storage, while NAS can be replaced by file storage systems such as S3 or Ceph.

[0185] Through the above network structure, the organic integration between client devices and gateway devices, microservice clusters and storage devices is achieved.

[0186] Figure 4 Another schematic diagram of the process of transmitting dual-recording video data provided in the embodiment of the present application is as follows: Figure 4 As shown, the process can be implemented by the cooperation of the client device 401, the gateway device 302, the target node 402, the NAS 403 and the Redis 404, as shown in FIG. Figure 4 As shown, the process may include the following steps:

[0187] Step G1, sending a transmission request.

[0188] Exemplarily, the client device 401 may send a transmission request to the gateway device 302 .

[0189] Exemplarily, the transmission request may be the start transmission request in the aforementioned embodiment. In actual application, the transmission request may be a start request.

[0190] Step G2: Random routing.

[0191] For example, the gateway device 302 may perform a random routing operation.

[0192] Exemplarily, the gateway device 302 may determine the first node from the microservice cluster by random or load balancing strategy, and may also send the transmission request to the first node.

[0193] Step G3: Determine the shard root path.

[0194] Exemplarily, the target node 402 including the first node may determine a shard root path; wherein the shard root path may be the target storage path in the aforementioned embodiment.

[0195] Exemplarily, after receiving the start transmission request, if the first node determines that the transmission request includes the first hash value, the shard root path in NAS 403 can be determined.

[0196] Step G4: Create transmission metadata.

[0197] Exemplarily, the target node 402 may create transmission metadata and store the transmission metadata in Redis 404 .

[0198] It should be noted that managing transmission metadata through Redis can improve the efficiency of cross-cluster sharing and updating of transmission metadata and reduce the probability of transmission metadata redundancy and conflict.

[0199] Step G5: Return the transmission metadata.

[0200] Exemplarily, the target node 402 may return the transmission metadata to the client device 401 via the gateway device 302 .

[0201] Step G6: Transmit the fragmented data.

[0202] Exemplarily, the client device 401 may segment the video dual recording data based on the transmission metadata to obtain K segment data, and transmit the kth segment data among the K segment data to the gateway device 302 .

[0203] Step G7: Random routing.

[0204] For example, the gateway device 302 may perform a random routing operation.

[0205] Exemplarily, the gateway device 302 may determine a target node including a second node and a third node from the microservice cluster based on a random or load balancing strategy, and transmit the kth shard data to the second node or the third node.

[0206] Step G8: Write the shard data.

[0207] Exemplarily, a target node including the second node or the third node may write the shard data.

[0208] Exemplarily, the second node or the third node may write the kth shard data into the target storage space of NAS 403 based on the shard root path.

[0209] In the above process, the client device can design a client upload engine based on RN technology and the producer-consumer model. By making full use of the concurrent processing capabilities of the service nodes in the microservice cluster, the large amount of video dual-recording data can be automatically segmented into multiple shard data according to the transmission metadata. Each shard data is transmitted and stored by the target node corresponding to the consumer role, realizing the concurrent uploading of shard data by dynamically adjusting the target node.

[0210] Step G9: Return the shard writing result.

[0211] Exemplarily, the second node or the third node may return the shard write result.

[0212] Exemplarily, the shard writing result may include the result that the kth shard data is successfully written into the target storage space or fails to be written.

[0213] It should be noted that in the above process, the method provided in the above embodiment can also be used to determine the breakpoint resume instruction to trigger the retransmission mechanism of the fragmented data, thereby realizing automatic retry of the transmission operation and reducing the probability of failure of the fragmented data transmission.

[0214] In the embodiment of the present application, by using NAS as transit storage, the transmission delay of shard data from different clusters and the consistency problem between shard data can be solved.

[0215] Step G10: Request to end transmission.

[0216] For example, the client device 401 may request to end the transmission.

[0217] Exemplarily, the request to end transmission may correspond to the end transmission request in the aforementioned embodiment.

[0218] Exemplarily, in the above process, the integrity of the shard data stored in the NAS may be determined based on the degree of matching between the first hash value and the second hash value.

[0219] Step G11: random routing.

[0220] By way of example, the gateway device 302 may perform random routing.

[0221] Exemplarily, the gateway device may send an end transmission request to the third node.

[0222] Step G12: Read metadata.

[0223] Exemplarily, the third node may read the metadata.

[0224] Exemplarily, the third node may read the transmission metadata from Redis 404 .

[0225] Step G13: Read the transmission status.

[0226] Exemplarily, the third node may read the transmission status.

[0227] Step G14: Determine the transmission result.

[0228] Exemplarily, the third node may determine the transmission result.

[0229] Step G15: Delete metadata.

[0230] Exemplarily, the third node may delete the metadata.

[0231] Exemplarily, if the third node determines that all K shard data have been transmitted to NAS 403, the transmission metadata may be deleted.

[0232] Step G16: Delete the shard data.

[0233] Exemplarily, the third node may instruct NAS 403 to delete K shard data.

[0234] Exemplarily, if the third node determines that all K fragmented data have been transmitted to NAS 403, then NAS 403 may be instructed to delete the K fragmented data.

[0235] Step G17, return the transmission result.

[0236] Exemplarily, the third node may return the transmission result through the gateway device 302, thereby completing the transmission of K slice data corresponding to the video dual recording data.

[0237] In the above process, in response to cross-cluster problems caused by changes in the client's Internet Protocol (IP) address, the gateway device can improve the stable and accurate transmission of shard data to the target node through the redirection mechanism of intelligent identification and random routing; and by adding verification logic for transmission metadata, it can increase the probability of re-uploading after the IP change.

[0238] Through the technical solution provided in the embodiment of the present application, in the process of the client device transmitting the fragmented data corresponding to the video dual recording data, by making full use of the breakpoint resume mechanism in the microservice cluster, after the fragmented data transmission terminal, the fragmented data with interrupted transmission can be continued to be uploaded through any node in the microservice cluster, thereby improving the fault tolerance and robustness of the fragmented data transmission process; and, by designing a distributed storage system, even if a storage node fails, the uploaded fragmented data will not be lost, and can be automatically migrated to other healthy storage nodes for continued processing.

[0239] The technical solution provided in the embodiment of the present application can support parallel uploading of multiple shard data, and can also intelligently adjust the transmission speed of the shard data according to the network status, thereby improving the utilization rate of the network bandwidth; and, with the help of the first storage device to automatically sort and integrate the shard data, the present solution can also realize random uploading of the shard data, thereby reducing the transmission delay of the shard data.

[0240] The technical solution provided in the embodiment of the present application intelligently determines the target node based on multiple factors such as the load status, geographical location, and network delay of the service node in the microservice cluster; and by introducing a load balancing mechanism, it can improve the efficiency of sharded data transmission and reduce the probability of overloading a single service node.

[0241] In summary, the technical solution provided by the embodiment of the present application realizes flexible, efficient and robust transmission of the segmented data corresponding to the video dual recording data, thereby meeting the actual transmission requirements of the video dual recording data.

[0242] Figure 5 A statistical diagram of the CPU resource consumption status of a service node during the video dual recording data transmission process provided in an embodiment of the present application. Figure 5 In the two-dimensional coordinate system shown, the horizontal axis represents the date in days, and the vertical axis represents the utilization of CPU resources in percentage.

[0243] like Figure 5 As shown, the first curve 501 and the second curve 502 are respectively used to represent the fluctuation state of the CPU resource utilization when different service nodes transmit video dual recording data during the period between July 1 and July 24; wherein, the maximum value, minimum value and instantaneous value of the CPU resource utilization represented by the first curve 501 are 3.58%, 0.587% and 0.994% respectively, and the maximum value, minimum value and instantaneous value of the CPU resource utilization represented by the second curve 502 may be 5.17%, 0.442% and 0.650% respectively.

[0244] It can be seen from the CPU resource utilization represented by the first curve 501 to the second curve 502 that the method provided in the embodiment of the present application occupies a smaller proportion of the CPU resources of the service node when transmitting video dual recording data, thereby reducing the probability of service node overload and improving the stability of video dual recording data transmission.

[0245] Figure 6 A schematic diagram of the statistical results of the memory occupancy rate of the service node during the video dual recording data transmission process provided in the embodiment of the present application. Figure 6 In the two-dimensional coordinate system shown, the horizontal axis represents the date in days, and the vertical axis represents the utilization rate of the memory resources in percentage.

[0246] like Figure 6 As shown, the third curve 601 and the fourth curve 602 are respectively used to represent the fluctuation state of memory resource utilization when different service nodes transmit video dual recording data during the period between July 1 and July 24; wherein, the maximum value, minimum value and instantaneous value of the memory resource utilization represented by the third curve 601 are 9.20%, 2.00% and 3.01% respectively, and the maximum value, minimum value and instantaneous value of the memory resource utilization represented by the fourth curve 602 may be 4.68%, 1.52% and 2.13% respectively.

[0247] It can be seen from the memory resource utilization represented by the third curve 601 to the fourth curve 602 that the method provided in the embodiment of the present application occupies a smaller proportion of the memory resources of the service node when transmitting video dual recording data, thereby reducing the probability of service node overload and improving the stability and robustness of video dual recording data transmission.

[0248] Figure 7 A schematic diagram of the statistical results of the data transmission volume of the service node during the video dual recording data transmission process provided by the embodiment of the present application. Figure 7 In the two-dimensional coordinate system shown, the horizontal axis represents the date in days, and the vertical axis represents the data transmission volume in MB.

[0249] like Figure 7 As shown, in the bandwidth range of 0 to 250MB, during the period from July 1 to July 24, the maximum data transmission volume of the service node obtained by counting in units of 5 minutes can be 211MB or 238MB, the average data transmission volume can be 4.89MB or 5.07MB, and the instantaneous data transmission volume can be 0.483MB or 0.373MB. In other words, the fluctuation state of data transmission volume of different service nodes when transmitting video dual recording data is basically consistent.

[0250] from Figure 7 It can be seen that through the method provided in the embodiment of the present application, in the bandwidth range of 0 to 250MB, when different service nodes transmit video dual recording data, their data transmission speeds show consistency and balance, thereby reducing the probability of overloading a single service node and improving the stability of video dual recording data transmission.

[0251] Figure 8 Another schematic diagram of the statistical results of the data transmission volume of the service node during the video dual recording data transmission process provided by the embodiment of the present application. Figure 8 In the two-dimensional coordinate system shown, the horizontal axis represents the date in days, and the vertical axis represents the data transmission volume in MB / GB.

[0252] like Figure 8 As shown, in the bandwidth range of 0 to 1 GB, during the period from July 1 to July 24, the maximum data transmission volume, average data transmission volume and instantaneous data transmission volume of the fifth curve 801 corresponding to the first service node, which are calculated in units of 5 minutes, may be 924 MB, 138 MB and 172 MB respectively; at the same time, the maximum data transmission volume, average data transmission volume and instantaneous data transmission volume of the sixth curve 802 corresponding to another service node may be 488 MB, 137 MB and 174 MB respectively.

[0253] from Figure 8 The results shown show that when different service nodes transmit video dual-recording data in the bandwidth range of 0 to 1 GB, the data transmission volume shows differences in magnitude, but the overall change pattern remains consistent, and the bandwidth resources can be fully utilized, thereby improving the efficiency of video dual-recording data transmission.

[0254] Fig. 9 A schematic diagram of the statistical results of the average response time of the service nodes provided in the embodiment of the present application, Fig. 9 The horizontal axis of the two-dimensional coordinate system shown represents the date in days, and the vertical axis represents the average response time in milliseconds (ms).

[0255] from Fig. 9 It can be seen that during the period between June 28 and July 25, the average response time of the service nodes was concentrated in the range of less than 1 second, and only in a few cases the average response time was greater than 1 second.

[0256] from Fig. 9 It can be seen from the average time statistics results shown that by adopting the technical solution provided in the embodiment of the present application, the average response time of the service node can be shortened, thereby improving the efficiency of the service node in transmitting video dual recording data.

[0257] Based on the foregoing embodiment, the embodiment of the present application further provides a gateway device, Fig.10 A schematic diagram of the structure of the gateway device provided in the embodiment of the present application, such as Fig.10 As shown, the gateway device 302 may include:

[0258] The first transceiver module 1001 is used to receive the kth fragment data sent by the client device; wherein the kth fragment data is any one of the K fragment data obtained by the client device fragmenting the video dual recording data; k is an integer greater than or equal to 1 and less than or equal to K; K is an integer greater than 1;

[0259] A determination module 1002 is used to determine a first node from a plurality of service nodes included in the microservice cluster based at least on a load balancing strategy;

[0260] The first transceiver module 1001 is also used to transmit the kth slice data to the first node, so that the first node can transmit the kth slice data to the first storage device; wherein the first storage device is used to integrate the first slice data to the Kth slice data to obtain and store video dual recording data.

[0261] In some embodiments, the first transceiver module 1001 is used to receive a start transmission request sent by a client device;

[0262] A determination module 1002, configured to determine a second node from a plurality of service nodes;

[0263] The first transceiver module 1001 is used to send a transmission start request to the second node to trigger the second node to determine a breakpoint resume transmission indication; wherein the breakpoint resume transmission indication at least includes a fragment identifier of the kth fragment data; the breakpoint resume transmission indication is determined by the second node based on the transmission metadata and transmission status of the video dual recording data; the transmission metadata is at least used to characterize the fragment status corresponding to the video dual recording data;

[0264] The first transceiver module 1001 is further used to receive and send a breakpoint resume instruction sent by the second node to the client device, so as to instruct the client device to send the kth fragment data to the gateway device.

[0265] In some embodiments, the determination module 1002 is used to determine a third node from a plurality of service nodes in response to a start transmission request sent by a client device; wherein the start transmission request at least includes a first hash value; and the first hash value includes a hash value of the video dual recording data;

[0266] The first transceiver module 1001 is used to send a start transmission request to the third node to trigger the third node to determine the transmission metadata associated with the client device based on the first hash value; wherein the transmission metadata is at least used to characterize the fragmentation state corresponding to the video dual recording data;

[0267] The first transceiver module 1001 is further used to receive and send the transmission metadata sent by the third node to the client device, so that the client device can segment the video dual recording data based on the transmission metadata.

[0268] In some embodiments, the first transceiver module 1001 is used to transmit the end transmission request to the third node if the end transmission request sent by the client device is detected; wherein the third node is a node among the multiple service nodes that receives the start transmission request sent by the client device; the third node is used to determine the transmission result of the video dual recording data based on at least the first hash value and the second hash value; the first hash value includes the hash value of the video dual recording data; the second hash value includes the hash value of the transmission data obtained after the fragmented data stored in the first storage device is integrated;

[0269] The first transceiver module 1001 is further used to receive and transmit the transmission result sent by the third node to the client device to instruct the client device to perform a specified operation; wherein the specified operation includes a breakpoint resume operation or an end transmission operation.

[0270] The embodiment of the present application also provides a gateway device, Fig.11 Another schematic diagram of the structure of the gateway device provided in the embodiment of the present application is as follows Fig.11 As shown, the gateway device 302 includes a first processor 1101 and a first memory 1102; a first computer program is stored in the first memory 1102; when the first computer program is executed by the first processor 1101, it can implement the video dual recording data transmission method applied to the gateway device as described in any of the above.

[0271] An embodiment of the present application further provides a target node, wherein the service node is arranged in a microservice cluster; the target node is determined by a gateway device from a plurality of service nodes included in the microservice cluster.

[0272] Fig.12 A schematic diagram of the structure of the target node provided in the embodiment of the present application, such as Fig.12 As shown, the target node 402 includes:

[0273] The second transceiver module 1201 is used to receive the kth slice data transmitted by the gateway device; wherein the gateway device is used to receive the kth slice data sent by the client device, and determine the target node from multiple service nodes based on at least the load balancing strategy; the kth slice data is any one of the K slice data obtained by the client device from slicing the video dual recording data; k is an integer greater than or equal to 1 and less than or equal to K; K is an integer greater than 1;

[0274] The second transceiver module 1201 is further used to transmit the kth slice data to the first storage device, so that the first storage device integrates the first slice data into the Kth slice data to obtain and store the video dual recording data.

[0275] In some embodiments, the target node further comprises a processing module; wherein:

[0276] The second transceiver module 1201 is used to receive a transmission start request sent by a gateway device;

[0277] A processing module is used to obtain a transmission status associated with a client device; obtain a storage status of transmission metadata associated with the client device in a second storage device; if the storage status indicates that the transmission metadata is stored in the second storage device, and the transmission status indicates that the client device has transmitted part of the fragmented data corresponding to the video dual-recording data to the first storage device, generate and send a breakpoint resume indication to the gateway device, so that the gateway device can send the breakpoint resume indication to the client device; wherein the breakpoint resume indication at least includes a fragment identifier of the kth fragmented data; the transmission metadata is at least used to indicate the fragment status corresponding to the video dual-recording data.

[0278] In some embodiments, the second transceiver module 1201 is used to receive a start transmission request sent by a gateway device;

[0279] A processing module, configured to obtain a storage status of the transmission metadata associated with the client device in the second storage device; if the storage status indicates that the transmission metadata is not stored in the second storage device, obtain a first hash value from the start transmission request; the first hash value includes a hash value of the video dual recording data;

[0280] The processing module is further used to generate transmission metadata based on the first hash value; wherein the transmission metadata is at least used to characterize the fragmentation state corresponding to the video dual recording data;

[0281] The processing module is further used to determine a target storage address associated with the first storage device, so that the target node can transmit the shard data associated with the client device to the target storage space indicated by the target storage address;

[0282] The second transceiver module 1201 is used to transmit transmission metadata to the gateway device; wherein the transmission metadata is used for the gateway device to send to the client device to instruct the client device to segment the video dual recording data.

[0283] In some embodiments, the processing module is used to determine the second hash value corresponding to the slice data associated with the client device in the first storage device if an end transmission request sent by the gateway device is received; obtain the first hash value from the second storage device; wherein the first hash value includes the hash value of the video dual recording data;

[0284] The processing module is also used to determine and send the transmission result to the gateway device based on the first hash value and the second hash value, so that the gateway device can trigger the client device to perform a specified operation based on the transmission result; wherein the specified operation includes a breakpoint resume operation or an end transmission operation.

[0285] The embodiment of the present application also provides another target node, which is a service node set in a microservice cluster; the target node is determined by a gateway device from multiple service nodes included in the microservice cluster; Fig.13 Another structural diagram of the target node provided in the embodiment of the present application is as follows: Fig.13 As shown, the target node 402 may include: a second processor 1301 and a second memory 1302; wherein the second memory 1302 stores a second computer program; when the second computer program is executed by the second processor 1301, it can implement the video dual recording data transmission method applied to the target node as described above.

[0286] An embodiment of the present application further provides a computer-readable storage medium, in which a third computer program is stored; when the third computer program is executed by a processor of an electronic device, it can implement any of the above-mentioned video dual-recording data transmission methods.

[0287] The embodiment of the present application further provides a computer program product, which includes a fourth computer program; when the fourth computer program is executed by a processor of an electronic device, it can implement any of the above-mentioned video dual-recording data transmission methods.

[0288] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0289] The methods disclosed in the various method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0290] The features disclosed in the various product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0291] The features disclosed in the various method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0292] It should be noted that the above-mentioned computer-readable storage medium can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory (Flash Memory), a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM) and other memories; it can also be various electronic devices including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0293] It should be noted that, in this article, the term "includes", "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0294] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0295] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus necessary general hardware nodes, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0296] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0297] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0298] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0299] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.< / form>

Claims

1. A video dual recording data transmission method, characterized in that: The method is applied to a gateway device associated with a client device; the method comprises: Receive the kth fragment data sent by the client device; wherein the kth fragment data is any one of the K fragment data obtained by the client device fragmenting the video dual recording data; k is an integer greater than or equal to 1 and less than or equal to K; K is an integer greater than 1; Determine a first node from a plurality of service nodes included in the microservice cluster based at least on a load balancing strategy; The kth slice data is transmitted to the first node, so that the first node transmits the kth slice data to the first storage device; wherein the first storage device is used to integrate the first slice data to the Kth slice data to obtain and store the video dual recording data.

2. The method according to claim 1, characterized in that Before receiving the kth fragment data sent by the client device, the method further includes: Receiving a transmission start request sent by the client device; Determine a second node from the multiple service nodes, send the start transmission request to the second node, so as to trigger the second node to determine a breakpoint resume transmission indication; wherein the breakpoint resume transmission indication at least includes a fragment identifier of the kth fragment data; the breakpoint resume transmission indication is determined by the second node based on the transmission metadata and transmission status of the video dual recording data; the transmission metadata is at least used to characterize the fragment status corresponding to the video dual recording data; Receive and send the breakpoint resume instruction sent by the second node to the client device to instruct the client device to send the kth fragment data to the gateway device.

3. The method according to claim 1, characterized in that Before receiving the kth fragment data sent by the client device, the method further includes: In response to a start transmission request sent by the client device, determining a third node from the multiple service nodes; wherein the start transmission request at least includes a first hash value; the first hash value includes a hash value of the video dual recording data; Sending the start transmission request to the third node to trigger the third node to determine the transmission metadata associated with the client device based on the first hash value; wherein the transmission metadata is at least used to characterize the fragmentation state corresponding to the video dual recording data; The transmission metadata sent by the third node is received and sent to the client device, so that the client device segments the video dual recording data based on the transmission metadata.

4. The method according to claim 1, characterized in that: The method further comprises: If a transmission end request sent by the client device is detected, the transmission end request is transmitted to a third node; wherein the third node is a node among the multiple service nodes that receives the transmission start request sent by the client device; the third node is used to determine the transmission result of the video dual recording data based on at least a first hash value and a second hash value; the first hash value includes a hash value of the video dual recording data; the second hash value includes a hash value of the transmission data obtained by integrating the fragmented data stored in the first storage device; Receive and transmit the transmission result sent by the third node to the client device to instruct the client device to perform a specified operation; wherein the specified operation includes a breakpoint resume operation or an end transmission operation.

5. A video dual recording data transmission method, characterized in that: The method is applied to a target node among a plurality of service nodes included in a microservice cluster; the method comprises: Receive the kth slice data transmitted by the gateway device; wherein the gateway device is used to receive the kth slice data sent by the client device, and determine the target node from the multiple service nodes based on at least a load balancing strategy; the kth slice data is any one of the K slice data obtained by the client device from slicing the video dual recording data; k is an integer greater than or equal to 1 and less than or equal to K; K is an integer greater than 1; The kth slice data is transmitted to a first storage device, so that the first storage device integrates the first slice data into the Kth slice data to obtain and store the video dual recording data.

6. The method according to claim 5, characterized in that Before receiving the kth fragment data transmitted by the gateway device, the method further includes: receiving a transmission start request sent by the gateway device, and acquiring a transmission status associated with the client device; Obtaining a storage status of the transmission metadata associated with the client device in a second storage device; If the storage status indicates that the transmission metadata is stored in the second storage device, and the transmission status indicates that the client device has transmitted part of the fragmented data corresponding to the video dual-recording data to the first storage device, a breakpoint resume transmission indication is generated and sent to the gateway device, so that the gateway device can send the breakpoint resume transmission indication to the client device; wherein the breakpoint resume transmission indication at least includes the fragment identifier of the kth fragmented data; and the transmission metadata is at least used to characterize the fragment status corresponding to the video dual-recording data.

7. The method according to claim 5, characterized in that Before receiving the kth fragment data transmitted by the gateway device, the method further includes: Receiving a transmission start request sent by the gateway device; Obtaining a storage status of the transmission metadata associated with the client device in a second storage device; If the storage state indicates that the transmission metadata is not stored in the second storage device, obtaining a first hash value from the transmission start request; the first hash value includes a hash value of the video dual recording data; Generate the transmission metadata based on the first hash value; wherein the transmission metadata is at least used to characterize the fragmentation state corresponding to the video dual recording data; Determine a target storage address associated with the first storage device, so that the target node can transmit the sliced ​​data associated with the client device to the target storage space indicated by the target storage address; The transmission metadata is transmitted to the gateway device; wherein the transmission metadata is used by the gateway device to send to the client device to instruct the client device to segment the video dual recording data.

8. The method according to claim 5, characterized in that The method further comprises: If a transmission end request sent by the gateway device is received, determining a second hash value corresponding to the shard data associated with the client device in the first storage device; Obtaining a first hash value from a second storage device; wherein the first hash value includes a hash value of the video dual recording data; Based on the first hash value and the second hash value, a transmission result is determined and sent to the gateway device, so that the gateway device triggers the client device to perform a specified operation based on the transmission result; wherein the specified operation includes a breakpoint resume operation or an end transmission operation.

9. A gateway device, characterized in that: The gateway device comprises: A first transceiver module is configured to receive the kth fragment data sent by the client device; wherein the kth fragment data is any one of the K fragment data obtained by the client device fragmenting the video dual recording data; k is an integer greater than or equal to 1 and less than or equal to K; K is an integer greater than 1; A determination module, configured to determine a first node from a plurality of service nodes included in the microservice cluster based at least on a load balancing strategy; The first transceiver module is also used to transmit the kth slice data to the first node, so that the first node can transmit the kth slice data to the first storage device; wherein the first storage device is used to integrate the first slice data to the Kth slice data to obtain and store the video dual recording data.

10. A gateway device, characterized in that: The gateway device includes a first processor and a first memory; a first computer program is stored in the first memory; when the first computer program is executed by the first processor, the video dual-recording data transmission method as described in any one of claims 1 to 4 can be implemented.

11. A target node, characterized in that: The service node is set in the microservice cluster; The target node is determined by a gateway device from a plurality of service nodes included in the microservice cluster; The target node includes: The second transceiver module is used to receive the kth slice data transmitted by the gateway device; wherein the gateway device is used to receive the kth slice data sent by the client device, and determine the target node from the multiple service nodes based on at least a load balancing strategy; the kth slice data is any one of the K slice data obtained by the client device from slicing the video dual recording data; k is an integer greater than or equal to 1 and less than or equal to K; K is an integer greater than 1; The second transceiver module is further used to transmit the kth slice data to the first storage device, so that the first storage device integrates the first slice data into the Kth slice data to obtain and store the video dual recording data.

12. A target node, characterized in that: The service node is set in a microservice cluster; the target node is determined by a gateway device from multiple service nodes included in the microservice cluster; the target node includes a second processor and a second memory; wherein a second computer program is stored in the second memory; when the second computer program is executed by the second processor, it can implement the video dual-recording data transmission method as described in any one of claims 5 to 8.

13. A computer-readable storage medium, characterized in that: A third computer program is stored in the storage medium; when the third computer program is executed by the processor of the electronic device, the video dual-recording data transmission method as described in any one of claims 1 to 4 or 5 to 8 can be implemented.

14. A computer program product, characterized in that The program product includes a fourth computer program; when the fourth computer program is executed by a processor of an electronic device, it can implement the video dual-recording data transmission method as described in any one of claims 1 to 4 or 5 to 8.

Images