Data recovery method and device, equipment, medium and product

Abstract

The invention discloses a data recovery method and device, equipment, a medium and a product, and relates to the technical field of data processing, and the method comprises the steps: obtaining at least one recovery request message sent by a consumer and / or a multicast network; analyzing the recovery request message to obtain a consumer port identifier, a request type and a difference data packet sequence corresponding to the consumer port identifier; performing de-duplication processing on the difference data packet sequence based on the consumer port identifier and the request type to obtain at least one recovery data packet sequence and a sending identifier corresponding to the recovery data packet sequence; and sending the recovery data packet to the consumer and / or the multicast network based on the sending identifier and the consumer port identifier. By adopting the technical scheme, the recovery request message is subjected to deduplication and merging processing, so that repeated uploading of the recovery request is effectively filtered, the resource occupancy is reduced, and the problems of repeated transmission of the recovery request and repeated issuing of the recovery data are solved.

Description

A data recovery method, apparatus, device, medium, and product Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a data recovery method, apparatus, device, medium, and product. Background Technology

[0002] In scenarios with high concurrency, low latency, and extremely high fairness requirements, such as trading systems and market data systems, the industry widely adopts multicast based on UDP (User Datagram Protocol) as a market data distribution mechanism to simultaneously broadcast the same data to all subscribers and maintain a consistent view.

[0003] To handle network packet loss and sequence gaps, the mainstream approach is to use a mechanism of "multicast recovery request + multicast recovery data". Multicast recovery requests tend to multiply as the number of consumers increases, requiring producers to handle a large number of duplicate requests and retransmit the entire group, which can cause the buffer to fill up quickly or even overflow. In addition, consumers often receive data that is irrelevant to themselves, increasing the overhead of unpacking and discarding.

[0004] Therefore, there is an urgent need for a data recovery method that can flexibly deduplicate recovery requests from multicast broadcasts or consumer unicast uploads, thereby improving recovery efficiency and robustness. Summary of the Invention

[0005] This invention provides a data recovery method, apparatus, device, medium, and product that solves the problems of repeated transmission of recovery requests and repeated delivery of recovery data. It performs deduplication and merging processing on recovery request messages from multicast network broadcasts and / or consumer unicasts, avoiding consumers from frequently receiving data irrelevant to themselves, which leads to increased unpacking and discarding overhead. It effectively achieves intelligent filtering of repeated uploads of recovery requests and reduces resource consumption.

[0006] According to one aspect of the present invention, a data recovery method is provided, applied to a producer, comprising: acquiring at least one recovery request message sent by a consumer and / or a multicast network; parsing the recovery request message to obtain a consumer port identifier, a request type, and a differential data packet sequence corresponding to the consumer port identifier; performing deduplication processing on the differential data packet sequence based on the consumer port identifier and the request type to obtain at least one recovery data packet sequence and a sending identifier corresponding to the recovery data packet sequence; determining a recovery data packet based on the recovery data packet sequence and a database; and sending the recovery data packet to the consumer and / or the multicast network based on the sending identifier and the consumer port identifier.

[0007] According to another aspect of the present invention, a data recovery method is provided, applied to a consumer, comprising: generating a recovery request message based on a source data packet sequence and a consumer-end data packet sequence; the recovery request message including a consumer port identifier and a differential data packet sequence number; uploading the recovery request message to a producer, or uploading the recovery request message to a multicast network, so that the multicast network sends the recovery request message to the producer, so that the producer determines a recovery data packet based on the recovery request message received from the consumer and / or the multicast network, and sends the recovery data packet to the consumer or the multicast network; and receiving the recovery data packet sent by the producer or the multicast network.

[0008] According to one aspect of the present invention, a data recovery apparatus is provided, applied to a producer, comprising: an acquisition module, configured to acquire at least one recovery request message sent by a consumer and / or a multicast network; a parsing module, configured to parse the recovery request message to obtain a consumer port identifier, a request type, and a sequence of differential data packets corresponding to the consumer port identifier; a deduplication module, configured to perform deduplication processing on the sequence of differential data packets based on the consumer port identifier and the request type to obtain at least one recovery data packet sequence and a sending identifier corresponding to the recovery data packet sequence; a determination module, configured to determine a recovery data packet based on the recovery data packet sequence and a database; and a sending module, configured to send the recovery data packet to the consumer and / or the multicast network based on the sending identifier and the consumer port identifier.

[0009] According to one aspect of the present invention, a data recovery apparatus is provided, applied to a consumer, comprising: a generation module, configured to generate a recovery request message based on a source data packet sequence and a consumer-end data packet sequence; the recovery request message including a consumer port identifier and a differential data packet sequence number; an uploading module, configured to upload the recovery request message to a producer, or upload the recovery request message to a multicast network, so that the multicast network sends the recovery request message to the producer, so that the producer determines a recovery data packet based on the recovery request message received from the consumer and / or the multicast network, and sends the recovery data packet to the consumer or the multicast network; and a receiving module, configured to receive the recovery data packet sent by the producer or the multicast network.

[0010] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the data recovery method according to any embodiment of the present invention.

[0011] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the data recovery method described in any embodiment of the present invention.

[0012] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the data recovery method according to any embodiment of the present invention.

[0013] The technical solution of this invention performs deduplication and merging processing on recovery request messages of multicast network broadcast and / or consumer unicast, avoiding consumers from frequently receiving data that is irrelevant to themselves, which leads to increased unpacking and discarding overhead. It solves the problems of repeated transmission of recovery requests and repeated delivery of recovery data, effectively achieving intelligent filtering of repeated uploads of recovery requests and reducing resource consumption.

[0014] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 is a flowchart of a data recovery method according to an embodiment of the present invention; Figure 2 is a flowchart of a data recovery method according to an embodiment of the present invention; Figure 3 is a flowchart of a data packet sequence determination method according to an embodiment of the present invention; Figure 4 is a flowchart of a data recovery method according to an embodiment of the present invention; Figure 5 is a schematic diagram of a data recovery device according to an embodiment of the present invention; Figure 6 is a schematic diagram of a data recovery device according to an embodiment of the present invention; Figure 7 is a schematic diagram of an electronic device implementing the data recovery method of the present invention. Detailed Implementation

[0017] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0018] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification, claims and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product or device.

[0019] Furthermore, it should be noted that the information collected in the technical solution of this invention is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data all comply with the relevant laws, regulations and standards of relevant countries and regions, necessary confidentiality measures have been taken, and public order and good morals are not violated. Corresponding operation entry points are provided for users to choose to authorize or refuse.

[0020] Figure 1 is a flowchart of a data recovery method provided by an embodiment of the present invention. This embodiment is applicable to data recovery of recovery request messages sent by multicast networks and / or consumers, and is particularly suitable for deduplicating repeatedly uploaded recovery request messages to complete data recovery. This method can be executed by a data recovery device, which can be implemented in hardware and / or software and can be configured in the producer, such as a server or cloud device. As shown in Figure 1, the method includes: S110, acquiring at least one recovery request message sent by a consumer and / or multicast network.

[0021] In this multicast distribution environment, the producer, consumer, and multicast network consist of one producer and at least one consumer. The multicast method involves the producer or consumer sending a data delivery request or data recovery request to the multicast network, which then replicates it to all consumer terminals or producers. The producer can be a cloud device or server, delivering data packets to the terminal devices or multicast network to each consumer. The consumer can be a terminal device used to receive recovery data packets from the producer or multicast network. The recovery request message is a retransmission request initiated by the consumer after detecting a gap in the packet sequence number in the terminal. It includes the missing data packet sequence number and the consumer to which the missing data packet belongs. If the consumer sends it directly to the producer, it is a unicast transmission; if the consumer sends it to the multicast network, which then performs the multicast request, it is a multicast transmission.

[0022] Specifically, it acquires at least one recovery request message sent by a consumer in unicast type; and / or at least one recovery request message sent by a multicast network in multicast type.

[0023] In an optional embodiment of the present invention, before obtaining the recovery request message, when constructing the multicast distribution environment, the producer can bind a listening port for receiving unicast recovery requests. This port is used to receive recovery request messages sent by each consumer in unicast type and upload them to the multicast network heartbeat packet, carrying the unicast listening port for receiving unicast recovery request messages. Simultaneously, when the multicast network receives the heartbeat packet uploaded by the producer, it directly multicasts and replicates it to all consumer terminals in the multicast distribution environment. This allows consumers to directly send recovery request messages based on the unicast listening port, achieving precise location of the producer.

[0024] S120. Parse the recovery request message to obtain the consumer port identifier, request type, and difference data packet sequence.

[0025] Among them, the consumer port identifier is used to distinguish each consumer and represents the recovery data receiving port of each consumer; the request type is used to distinguish whether the recovery message request is sent by the consumer or the recovery message request sent by the multicast network. The recovery message request sent by the consumer indicates that the request type is unicast; the recovery message request sent by the multicast network indicates that the request type is multicast; the difference packet sequence is the packet sequence number missing by each consumer, and the difference packet sequence may include at least one packet sequence number.

[0026] Specifically, the recovery request message sent by the consumer is parsed to obtain the consumer port identifier and the difference data packet sequence, which further identifies the request type of the recovery request message as unicast; the recovery request message sent by the multicast network is parsed to obtain the difference data packet sequence, which identifies the request type of the recovery request message as multicast.

[0027] In an optional embodiment of the present invention, the recovery request message sent by the consumer may also carry a "unicast recovery indication" flag to indicate that it is unicast. If the recovery request message is further parsed to carry a "unicast recovery indication" flag, it indicates that the request type of the recovery request message is unicast. If it does not carry a "unicast recovery indication" flag, it indicates that the request type of the recovery request message is multicast. By setting the unicast recovery indication flag, accurate deduplication of the subsequent differential data packet sequence is ensured, avoiding repeated transmission of multicast recovery data, which would cause consumers to frequently receive data irrelevant to themselves, and increase the overhead of unpacking and discarding.

[0028] S130. Based on the consumer port identifier and request type, perform deduplication on the differential data packet sequence to obtain at least one recovery data packet sequence and the sending identifier corresponding to the recovery data packet sequence.

[0029] The recovered data packet sequence is obtained by deduplicating the differential data packet sequence. The deduplication process involves discarding duplicate sequence numbers between differential data packet sequences and merging different sequence numbers. The sending identifier includes multicast sending or unicast sending. If the sending identifier is multicast sending, it indicates that the data is sent to a multicast network. If the sending identifier is unicast sending, it indicates that the data is sent to the corresponding consumer.

[0030] Specifically, the sequence numbers in multiple differential data packet sequences are deduplicated based on the consumer port identifier and request type to obtain at least one recovery data packet sequence; and a corresponding sending identifier is set for the recovery data packet sequence according to the request type. For example, if the request type is multicast, its corresponding sending identifier can be multicast sending; if the request type is unicast, its corresponding sending identifier can be unicast sending.

[0031] In an optional embodiment of the present invention, a recovery queue can be set up to deduplicate the differential data packet sequence. For example, received recovery request messages are enqueued, and the consumer port identifier, request type, and differential data packet sequence obtained from parsing the recovery request message are associated and enqueued. When a new recovery request message is received, the new consumer port identifier, request type, and differential data packet sequence are deduplicated in the queue. When the request type of the new recovery request message is unicast, the new differential data packet sequence is fully compared with the deduplication queue; when the request type of the new recovery request message is multicast, it is only compared with "multicast" in the deduplication queue; if the sequence number in the new differential data packet sequence completely overlaps with or is contained within the deduplication queue, it is discarded, and only the new differential data packet sequence number is saved. Compared to coarse-grained or semantically indiscriminate deduplication mechanisms, setting up a deduplication queue significantly reduces duplicate requests and retransmissions, reducing CPU and buffer pressure.

[0032] S140. Determine the recovery data packets based on the recovery data packet sequence and the database.

[0033] The database stores the entire source data packet, and each source data packet contains at least one source data packet sequence number; the recovery data packet is the source data packet corresponding to each recovery data packet sequence number in the recovery data packet sequence.

[0034] Specifically, the corresponding source data packet is searched in the database based on the sequence number in the recovery data packet sequence, and the source data packets are combined in sequence according to the sequence number to obtain the recovery data packet.

[0035] S150, based on the sending identifier and the consumer port identifier, will send the recovery packet to the consumer and / or multicast network.

[0036] Specifically, the sending type is determined based on the sending identifier. If the sending identifier is unicast, the recovery data packet is sent directly to the consumer based on the consumer port identifier; if the sending identifier is multicast, the recovery data packet is sent directly to the multicast network.

[0037] Optionally, the recovery data packet is sent to the consumer and / or multicast network based on the sending identifier and the consumer port identifier, including: if the sending identifier is unicast sending, then determining the consumer port address based on the consumer port identifier; sending the recovery data packet to the consumer based on the consumer port address; if the sending identifier is multicast sending, then sending the recovery data packet to the multicast network so that the multicast network sends the recovery data packet to all consumers.

[0038] If the sending identifier is unicast, the consumer port address is determined based on the consumer port identifier, and the recovery data packet is sent to the consumer terminal to which the consumer port address belongs. If the sending identifier is multicast, the recovery data packet is sent to the multicast network so that the multicast network can directly copy the recovery data packet to all consumers in the multicast distribution environment.

[0039] Understandably, by setting a sending identifier for each recovery data packet, we can avoid the situation where all multicast broadcasts cause other consumers to receive useless data packets, resulting in packet loss and consuming a lot of resources on the consumer side. This enables the introduction of "unicast recovery" self-discovery in the multicast distribution environment. By effectively deduplicating the sequence of different data packets, we can significantly avoid multicast recovery storms, reduce data duplication, and improve recovery efficiency and robustness.

[0040] In an optional embodiment of the present invention, the consumer port identifier is obtained by hashing the consumer port address of the unicast recovery port for receiving unicast transmission types when the producer uploads heartbeat packet data to the multicast network and the consumer receives the unicast listening port copied by the multicast network broadcast. This allows the producer to find the consumer port address based on the consumer port identifier, enabling targeted transmission of recovery data and preventing other consumers from receiving useless recovery data, thus avoiding unnecessary occupation of consumer resources.

[0041] This invention addresses the issue by acquiring at least one recovery request message sent by a consumer and / or multicast network; parsing the recovery request message to obtain a consumer port identifier, request type, and a sequence of differential data packets corresponding to the consumer port identifier; deduplicating the differential data packet sequence based on the consumer port identifier and request type to obtain at least one recovery data packet sequence and a sending identifier corresponding to the recovery data packet sequence; determining the recovery data packet based on the recovery data packet sequence and a database; and sending the recovery data packet to the consumer and / or multicast network based on the sending identifier and consumer port identifier. This technical solution performs deduplication and merging processing on recovery request messages broadcast by multicast networks and / or unicast by consumers, preventing consumers from frequently receiving data irrelevant to themselves, thus avoiding increased unpacking and discarding overhead. It also solves the problems of repeated transmission and delivery of recovery requests and recovery data, effectively achieving intelligent filtering of repeated uploads of recovery requests and reducing resource consumption.

[0042] Figure 2 is a flowchart of a data recovery method according to an embodiment of the present invention. This embodiment supplements the deduplication method for differential data packet sequence numbers based on the above embodiments. It should be noted that for parts not detailed in this embodiment, please refer to the relevant descriptions in other embodiments. As shown in Figure 2, the method includes: S210, obtaining at least one recovery request message sent by the consumer and / or multicast network.

[0043] S220. Parse the recovery request message to obtain the consumer port identifier, request type, and difference data packet sequence.

[0044] S230. Perform overlap judgment on at least one differential data packet sequence, and divide at least one differential data packet sequence into an overlapping sequence set and a non-overlapping sequence set.

[0045] The overlapping sequence set contains at least one differential data packet sequence, and the sequence numbers in the differential data packet sequence are repeated; the non-overlapping sequence set contains at least one differential data packet sequence, and the sequence numbers in the differential data packet sequence are not repeated.

[0046] Specifically, overlap determination is performed on at least one differential data packet sequence, and at least one differential data packet sequence in which there is an overlap relationship between the sequence numbers is constructed as an overlapping sequence set; at least one differential data packet sequence in which there is no overlap relationship between the sequence numbers is constructed as a non-overlapping sequence set.

[0047] S240. Based on the consumer port identifier and request type, perform deduplication on the overlapping sequence set to obtain the recovered data packet sequence and the sending identifier corresponding to the recovered data packet sequence.

[0048] Specifically, the overlapping sequence numbers in the overlapping sequence set can be deduplicated according to the consumer port identifier to obtain the sequence numbers that do not overlap. The sequence numbers that do not overlap are then integrated to obtain the recovery data packet sequence. The sending identifier of the recovery data packet sequence is marked based on whether the request type is multicast or unicast.

[0049] Optionally, as shown in Figure 3, a method for determining the recovery data packet sequence involves deduplicating the overlapping sequence set based on the consumer port identifier and request type to obtain the recovery data packet sequence and the sending identifier corresponding to the recovery data packet sequence, including: S241, determining whether the request type corresponding to the overlapping sequence set is a multicast type.

[0050] Specifically, determine whether the request type corresponding to at least one differential data packet sequence in the overlapping sequence set is a multicast type.

[0051] S242. If there is at least one request type that is multicast, then at least one differential data packet sequence is merged to obtain a recovery data packet sequence, and the sending identifier of the recovery data packet sequence is marked as multicast sending.

[0052] Specifically, if at least one differential data packet sequence in the overlapping sequence set corresponds to a multicast request type, then all differential data packet sequences in the overlapping sequence set are merged, and duplicate differential data packet sequence numbers are deduplicated. The merged and deduplicated differential data packet sequence numbers are integrated to obtain a recovered data packet sequence, and the sending identifier of the recovered data packet sequence is marked as multicast sending.

[0053] S243. If there is no multicast type, the overlapping sequence set is deduplicated based on the consumer port identifier to obtain the recovery data packet sequence and the sending identifier corresponding to the recovery data packet sequence.

[0054] Specifically, if the request type corresponding to all the different data packet sequences in the overlapping sequence set is not multicast but unicast, then the consumer port identifier is used to determine whether it is a duplicate transmission from the same consumer. The overlapping sequence set is then deduplicated based on the consumer port identifier to obtain the recovered data packet sequence, and the transmission identifier corresponding to the recovered data packet sequence is further marked based on the consumer port identifier.

[0055] Understandably, by setting up a refined deduplication process, accurate deduplication of the recovered data packet sequence can be achieved, effectively avoiding the repeated transmission of data packets with the same data sequence number, which would otherwise consume a large amount of resources on the consumer side.

[0056] Optionally, the overlapping sequence set is deduplicated based on the consumer port identifier to obtain the recovered data packet sequence and the corresponding transmission identifier, including: merging different differential data packet sequences in the overlapping sequence set to obtain the recovered data packet sequence; determining whether different differential data packet sequences in the overlapping sequence set belong to the same consumer based on the consumer port identifier; if they belong to the same consumer, the transmission identifier of the recovered data packet sequence is marked as unicast transmission; if they do not belong to the same consumer, the transmission identifier of the recovered data packet sequence is marked as multicast transmission.

[0057] Specifically, the different differential data packet sequences in the overlapping sequence set are merged and deduplicated to obtain recovery data packet sequences with non-overlapping sequence numbers. Then, based on the consumer port identifier, it is determined whether the different differential data packet sequences in the overlapping sequence set belong to the same consumer. If they belong to the same consumer, it indicates that the different differential data packet sequences in the overlapping sequence set are duplicate transmissions of the consumer, and other consumers are not involved in the missing recovery data packet. In this case, the transmission identifier of the recovery data packet sequence is directly marked as unicast transmission. If they do not belong to the same consumer, it indicates that at least two consumers are missing the recovery data packet. In this case, the transmission identifier of the recovery data packet sequence is marked as multicast transmission.

[0058] Understandably, further consumer identification through consumer port identifiers ensures that repeated transmission of missing data packet sequences by the same consumer does not affect the repeated acquisition by other consumers, effectively preventing consumers from frequently receiving data unrelated to themselves, thus increasing the overhead of unpacking and discarding.

[0059] S250. Take the different difference data packet sequences in the non-overlapping sequence set as the recovery data packet sequences, and mark the recovery data packet sequences based on the request type to obtain the sending identifier corresponding to the recovery data packet sequences.

[0060] Specifically, different differential data packet sequences in the non-overlapping sequence set are used as recovery data packet sequences, and the recovery data packet sequences are marked based on the request types corresponding to the different differential data packet sequences to obtain the sending identifiers corresponding to the recovery data packet sequences; for example, if the request type corresponding to the differential data packet sequence is unicast, then the recovery data packet sequence is marked with the sending identifier as unicast; if the request type corresponding to the differential data packet sequence is multicast, then the recovery data packet sequence is marked with the sending identifier as multicast.

[0061] S260. Determine the recovery data packets based on the recovery data packet sequence and the database.

[0062] S270, based on the sending identifier and the consumer port identifier, will send the recovery packet to the consumer and / or multicast network.

[0063] This invention deduplicates data packet sequences based on consumer port identifiers and request types, effectively preventing the buffer from being quickly filled or even overflowing when the number of consumers increases exponentially. This avoids the need for producers to handle a large number of duplicate requests and retransmit the entire group. Furthermore, the consumer port identifier-based determination effectively avoids the situation where consumers often receive data unrelated to themselves due to the undirected nature of multicast recovery data, thus reducing the overhead of unpacking and discarding. This significantly avoids multicast recovery storms, reduces data duplication, and improves recovery efficiency and robustness.

[0064] Figure 4 is a flowchart of a data recovery method provided by an embodiment of the present invention. This embodiment is applicable to data recovery of recovery request messages sent by multicast networks and / or consumers, and is particularly suitable for deduplicating repeatedly uploaded recovery request messages to complete data recovery. This method can be executed by a data recovery device, which can be implemented in hardware and / or software and can be configured in the consumer, such as a terminal device. As shown in Figure 4, the method includes: S310, generating a recovery request message based on the source data packet sequence and the consumer data packet sequence; the recovery request message includes a consumer port identifier and a differential data packet sequence.

[0065] Among them, the source data packet sequence is the source sequence corresponding to the data packet, including at least one source sequence number; the consumer data packet sequence is the sequence number of the data packet corresponding to the consumer's terminal integrated; and the difference data packet sequence number is the sequence obtained by integrating the sequence number gap with the source data packet sequence.

[0066] Specifically, the consumer end compares the sequence with the source data packet sequence in real time. If there are missing sequence numbers, the missing sequence numbers are integrated to obtain the difference data packet sequence.

[0067] Optionally, a recovery request message is generated based on the source data packet sequence and the consumer data packet sequence, including: obtaining the initial data packet sequence sent by the multicast network; supplementing the consumer data packet sequence with the initial data packet sequence to obtain an updated data packet sequence; determining whether a differential data packet sequence exists based on the updated data packet sequence and the source data packet sequence; if it exists, generating a recovery request message based on the consumer port identifier and the differential data packet sequence.

[0068] The initial data packet sequence is the sequence of data packets first sent by the multicast network; the updated data packet sequence is the data packet sequence obtained by the consumer supplementing the existing data packet sequence based on the sent data packet sequence; and the difference data packet sequence is the sequence with missing sequence numbers.

[0069] Specifically, the process involves obtaining the initial data packet sequence sent by the multicast network; supplementing the consumer data packet sequence with the initial data packet sequence to obtain the updated data packet sequence; determining whether there are missing sequence numbers based on the updated data packet sequence and the source data packet sequence; if there are missing sequence numbers, integrating the missing sequence numbers to obtain the differential data packet sequence; and generating a recovery request message based on the consumer port identifier and the differential data packet sequence.

[0070] Understandably, after detecting a gap in the packet sequence number, consumers compare it with the source data packet sequence to determine the difference data packet sequence, thus accurately and effectively identifying the difference data packet and improving the accuracy of subsequent missing data delivery.

[0071] S320. Upload the recovery request message to the producer, or upload the recovery request message to the multicast network, so that the multicast network sends the recovery request message to the producer, so that the producer determines the recovery data packet based on the recovery request message received from the consumer and / or the multicast network, and sends the recovery data packet to the consumer or the multicast network.

[0072] Specifically, the recovery request message is uploaded to the producer, or the recovery request message is uploaded to the multicast network, so that the multicast network sends the recovery request message to the producer. The producer then parses the received recovery request message to obtain the consumer port identifier and the corresponding difference data packet sequence. Whether or not the consumer port identifier is carried determines the request type of the recovery request message. Based on the consumer port identifier and the request type, the difference data packet sequence is deduplicated and merged. Then, the recovery data packet is determined according to the database, and the sending method is determined based on the sending identifier. Based on the sending method, the recovery data packet is sent to the consumer or the multicast network.

[0073] Optionally, the recovery request message is uploaded to the producer, or the recovery request message is uploaded to the multicast network, including: repeatedly uploading at least one recovery request message according to a preset interval; if the number of times the recovery request message is uploaded reaches a threshold, an interval update mechanism is triggered; the request interval for the next round of recovery request message upload is determined based on the exponential coefficient, the jitter coefficient, and the preset interval; if the request interval reaches the interval threshold, the request interval is updated to the target interval; and the recovery request message is uploaded to the producer or the multicast network based on the target interval.

[0074] Among them, the preset interval is the upload interval of the recovery request message; the number threshold is the upload threshold of the consumer unicast recovery request message; the interval update mechanism is an exponential backoff strategy and a random jitter mechanism; the interval threshold is a time interval threshold used to maintain the upload interval of the recovery request message; and the target interval is the retransmission interval to stop exponential growth.

[0075] Specifically, at least one recovery request message is repeatedly uploaded at a preset interval; if the number of times the recovery request message is uploaded reaches the threshold, the interval update mechanism is triggered; the request interval for the next round of recovery request message upload is determined based on the exponential coefficient, jitter coefficient and preset interval; if the request interval reaches the interval threshold, the request interval is updated to the target interval; the recovery request message is uploaded to the producer or multicast network according to the target interval.

[0076] The request interval is calculated as follows: in, This is the starting interval; The common ratio; For the first Try again; For the request interval; for ; This is the interval threshold.

[0077] Understandably, by adding a retry mechanism that combines exponential backoff and random perturbation, this mechanism, compared to fixed-interval or unlimited exponential backoff strategies, integrates random jitter and exponential backoff strategies. The exponential backoff strategy suppresses high-frequency retries by gradually extending the retry interval, which helps to automatically reduce the request rate and optimize the system's adaptive recovery capability during network congestion or temporary failures. Random jitter introduces random perturbations into the backoff interval to break the synchronicity of retry timing. This effectively avoids the problem that if all consumers adopt the same deterministic backoff mode, retries will tend to be synchronized during large-scale system restarts or global failures, forming periodic traffic peaks.

[0078] In one optional embodiment of the invention, different data packets correspond to different interval thresholds, while the initial interval threshold setting is the same for all. The threshold is determined based on the allowed number of fast retries, backoff parameters, service level, data type, and latency mathematical model. Since different data types often simultaneously possess different characteristics: latency, loss tolerance, message rate and burstiness, recovery cost, and fairness / consistency requirements, the interval threshold can be optimized based on the total time taken to reach the threshold. This effectively reduces peak requests and improves the smoothness and robustness of the system.

[0079] S330: Receive recovery data packets sent by the producer or multicast network.

[0080] Specifically, missing data sequences are supplemented by receiving recovery data packets sent by the producer through the unicast recovery port, or by receiving recovery data packets sent by the multicast network broadcast.

[0081] This invention generates a recovery request message based on the source data packet sequence and the consumer data packet sequence. The recovery request message includes a consumer port identifier and a differential data packet sequence number. The recovery request message is then uploaded to the producer, or uploaded to the multicast network, so that the multicast network sends the recovery request message to the producer. The producer then determines the recovery data packet based on the recovery request message received from the consumer and / or the multicast network, and sends the recovery data packet to the consumer or the multicast network. The producer also receives the recovery data packet sent by the producer or the multicast network. This technical solution determines the differential data packet sequence and combines unicast transmission from the consumer with multicast transmission from the multicast network. The consumer can switch between unicast request and unicast recovery in real time, and dynamically switches back to multicast recovery after unicast recovery fails, thus ensuring fault tolerance.

[0082] Figure 5 is a schematic diagram of the structure of a data recovery device provided in an embodiment of the present invention. The present invention is applicable to the situation of data recovery of recovery request messages sent by multicast networks and / or consumers, especially to the situation of deduplication of repeatedly uploaded recovery request messages, thereby completing data recovery. The data recovery device can be implemented in hardware and / or software and can be configured in the producer, such as a server or cloud device. As shown in Figure 5, the data recovery device 400 includes an acquisition module 410, a parsing module 420, a deduplication module 430, a determination module 440, and a sending module 450: the acquisition module 410 is used to acquire at least one recovery request message sent by the consumer and / or multicast network; the parsing module 420 is used to parse the recovery request message to obtain the consumer port identifier, request type, and differential data packet sequence; the deduplication module 430 is used to perform deduplication processing on the differential data packet sequence based on the consumer port identifier and request type to obtain at least one recovery data packet sequence and a sending identifier corresponding to the recovery data packet sequence; the determination module 440 is used to determine the recovery data packet based on the recovery data packet sequence and a database; and the sending module 450 is used to send the recovery data packet to the consumer and / or multicast network based on the sending identifier and the consumer port identifier.

[0083] This invention acquires at least one recovery request message sent by a consumer and / or multicast network; parses the recovery request message to obtain a consumer port identifier, request type, and a sequence of differential data packets corresponding to the consumer port identifier; performs deduplication processing on the differential data packet sequence based on the consumer port identifier and request type to obtain at least one recovery data packet sequence and a sending identifier corresponding to the recovery data packet sequence; determines the recovery data packet based on the recovery data packet sequence and a database; and sends the recovery data packet to the consumer and / or multicast network based on the sending identifier and consumer port identifier. This technical solution performs deduplication and merging processing on recovery request messages broadcast by multicast networks and / or unicast by consumers, avoiding consumers frequently receiving data irrelevant to themselves, which increases the overhead of unpacking and discarding. It solves the problems of repeated transmission of recovery requests and repeated delivery of recovery data, effectively achieving intelligent filtering of repeated uploads of recovery requests and reducing resource consumption.

[0084] Optionally, the deduplication module 430 includes an overlap judgment unit, an overlap sequence set deduplication unit, and a non-overlap sequence set deduplication unit. The overlap judgment unit is used to perform overlap judgment on at least one differential data packet sequence, dividing the at least one differential data packet sequence into an overlap sequence set and a non-overlap sequence set. The overlap sequence set deduplication unit is used to perform deduplication processing on the overlap sequence set based on the consumer port identifier and the request type, to obtain a recovery data packet sequence and a corresponding sending identifier. The non-overlap sequence set deduplication unit is used to take different differential data packet sequences in the non-overlap sequence set as recovery data packet sequences, and to mark the recovery data packet sequences based on the request type, to obtain a corresponding sending identifier.

[0085] Optionally, the overlapping sequence set deduplication unit is also used to determine whether the request type corresponding to the overlapping sequence set is a multicast type; if at least one request type is a multicast type, then at least one differential data packet sequence is merged to obtain a recovery data packet sequence, and the sending identifier of the recovery data packet sequence is marked as multicast sending; if no multicast type exists, then the overlapping sequence set is deduplicated based on the consumer port identifier to obtain the recovery data packet sequence and the sending identifier corresponding to the recovery data packet sequence.

[0086] Optionally, the overlapping sequence set deduplication unit is also used to merge different differential data packet sequences in the overlapping sequence set to obtain a recovered data packet sequence; and to determine whether different differential data packet sequences in the overlapping sequence set belong to the same consumer based on the consumer port identifier; if they belong to the same consumer, the sending identifier of the recovered data packet sequence is marked as unicast transmission; if they do not belong to the same consumer, the sending identifier of the recovered data packet sequence is marked as multicast transmission.

[0087] Optionally, the sending module 450 is further configured to, if the sending identifier is unicast sending, determine the consumer port address based on the consumer port identifier; send the recovery data packet to the consumer based on the consumer port address; and if the sending identifier is multicast sending, send the recovery data packet to the multicast network so that the multicast network sends the recovery data packet to all consumers.

[0088] Figure 6 is a schematic diagram of a data recovery device provided in an embodiment of the present invention. This embodiment is applicable to data recovery of recovery request messages sent by multicast networks and / or consumers, particularly for deduplicating repeatedly uploaded recovery request messages to complete data recovery. The data recovery device can be implemented in hardware and / or software and can be configured in a consumer, such as a terminal device. As shown in Figure 6, the data recovery device 500 includes a generation module 510, an uploading module 520, and a receiving module 530: The generation module 510 is used to generate a recovery request message based on the source data packet sequence and the consumer data packet sequence; the recovery request message includes a consumer port identifier and a differential data packet sequence number; the uploading module 520 is used to upload the recovery request message to the producer, or upload the recovery request message to the multicast network, so that the multicast network sends the recovery request message to the producer, so that the producer determines the recovery data packet based on the recovery request message received from the consumer and / or the multicast network, and sends the recovery data packet to the consumer or the multicast network; the receiving module 530 is used to receive the recovery data packet sent by the producer or the multicast network.

[0089] This invention generates a recovery request message based on the source data packet sequence and the consumer data packet sequence. The recovery request message includes a consumer port identifier and a differential data packet sequence number. The recovery request message is then uploaded to the producer, or uploaded to the multicast network, so that the multicast network sends the recovery request message to the producer. The producer then determines the recovery data packet based on the recovery request message received from the consumer and / or the multicast network, and sends the recovery data packet to the consumer or the multicast network. The producer also receives the recovery data packet sent by the producer or the multicast network. This technical solution determines the differential data packet sequence and combines unicast transmission from the consumer with multicast transmission from the multicast network. The consumer can switch between unicast request and unicast recovery in real time, and dynamically switches back to multicast recovery after unicast recovery fails, thus ensuring fault tolerance.

[0090] Optionally, the generation module 510 is also used to obtain the initial data packet sequence sent by the multicast network; supplement the consumer data packet sequence with the initial data packet sequence to obtain the updated data packet sequence; determine whether there is a differential data packet sequence based on the updated data packet sequence and the source data packet sequence; if there is, generate a recovery request message based on the consumer port identifier and the differential data packet sequence.

[0091] Optionally, the receiving module 530 is further configured to repeatedly upload at least one recovery request message according to a preset interval; if the number of times the recovery request message is uploaded reaches a threshold, an interval update mechanism is triggered; the request interval for the next round of recovery request message upload is determined based on the exponential coefficient, the jitter coefficient, and the preset interval; if the request interval reaches the interval threshold, the request interval is updated to the target interval; and the recovery request message is uploaded to the producer or multicast network based on the target interval.

[0092] The data recovery device provided in this embodiment of the invention can execute the data recovery method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0093] According to embodiments of the present invention, the present invention also provides an electronic device, a readable storage medium, and a computer program product.

[0094] Figure 7 illustrates a schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0095] As shown in Figure 7, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0096] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0097] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as data recovery methods.

[0098] In some embodiments, the data recovery method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the data recovery method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the data recovery method by any other suitable means (e.g., by means of firmware).

[0099] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0100] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0101] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0102] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0103] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0104] A computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product within the cloud computing service system. This addresses the shortcomings of traditional physical hosts and dedicated virtual services, such as high management difficulty and weak business scalability.

[0105] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0106] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A data recovery method, characterized in that, Applied to producers, the method includes: acquiring at least one recovery request message sent by a consumer and / or a multicast network; parsing the recovery request message to obtain a consumer port identifier, a request type, and a differential data packet sequence; performing deduplication processing on the differential data packet sequence based on the consumer port identifier and the request type to obtain at least one recovery data packet sequence and a sending identifier corresponding to the recovery data packet sequence; determining a recovery data packet based on the recovery data packet sequence and a database; and sending the recovery data packet to the consumer and / or the multicast network based on the sending identifier and the consumer port identifier.

2. The method according to claim 1, characterized in that, The step of deduplicating the differential data packet sequences based on the consumer port identifier and the request type to obtain at least one recovery data packet sequence and a corresponding sending identifier for the recovery data packet sequence includes: determining the overlap of at least one differential data packet sequence and dividing the at least one differential data packet sequence into an overlapping sequence set and a non-overlapping sequence set; deduplicating the overlapping sequence set based on the consumer port identifier and the request type to obtain the recovery data packet sequence and a corresponding sending identifier for the recovery data packet sequence; using different differential data packet sequences from the non-overlapping sequence set as the recovery data packet sequence, and marking the recovery data packet sequence based on the request type to obtain a corresponding sending identifier for the recovery data packet sequence.

3. The method according to claim 2, characterized in that, The step of deduplicating the overlapping sequence set based on the consumer port identifier and the request type to obtain the recovery data packet sequence and the corresponding transmission identifier of the recovery data packet sequence includes: determining whether the request type corresponding to the overlapping sequence set is a multicast type; if at least one request type is a multicast type, then merging at least one differential data packet sequence to obtain the recovery data packet sequence, and marking the transmission identifier of the recovery data packet sequence as multicast transmission; if no multicast type exists, then deduplicating the overlapping sequence set based on the consumer port identifier to obtain the recovery data packet sequence and the corresponding transmission identifier of the recovery data packet sequence.

4. The method according to claim 3, characterized in that, The overlapping sequence set is then deduplicated based on the consumer port identifier to obtain the recovered data packet sequence and the corresponding transmission identifier. This includes: merging different differential data packet sequences in the overlapping sequence set to obtain the recovered data packet sequence; determining whether the different differential data packet sequences in the overlapping sequence set belong to the same consumer based on the consumer port identifier; if they belong to the same consumer, the transmission identifier of the recovered data packet sequence is marked as unicast transmission; if they do not belong to the same consumer, the transmission identifier of the recovered data packet sequence is marked as multicast transmission.

5. The method according to claim 1, characterized in that, Sending the recovery data packet to the consumer and / or the multicast network based on the sending identifier and the consumer port identifier includes: if the sending identifier is unicast sending, determining the consumer port address based on the consumer port identifier; sending the recovery data packet to the consumer based on the consumer port address; if the sending identifier is multicast sending, sending the recovery data packet to the multicast network so that the multicast network sends the recovery data packet to all consumers.

6. A data recovery method, characterized in that, For consumers, the process includes: generating a recovery request message based on a source data packet sequence and a consumer data packet sequence; the recovery request message includes a consumer port identifier and a differential data packet sequence number; uploading the recovery request message to a producer, or uploading the recovery request message to a multicast network, so that the multicast network sends the recovery request message to the producer, so that the producer determines a recovery data packet based on the recovery request message received from the consumer and / or the multicast network, and sends the recovery data packet to the consumer or the multicast network; and receiving the recovery data packet sent by the producer or the multicast network.

7. The method according to claim 6, characterized in that, The step of generating a recovery request message based on the source data packet sequence and the consumer data packet sequence includes: obtaining an initial data packet sequence sent by the multicast network; supplementing the consumer data packet sequence with the initial data packet sequence to obtain an updated data packet sequence; determining whether a differential data packet sequence exists based on the updated data packet sequence and the source data packet sequence; and if it exists, generating a recovery request message based on the consumer port identifier and the differential data packet sequence.

8. The method according to claim 6, characterized in that, Uploading the recovery request message to the producer, or uploading the recovery request message to the multicast network, includes: repeatedly uploading at least one recovery request message at a preset interval; if the number of times the recovery request message is uploaded reaches a threshold, triggering an interval update mechanism; determining the request interval for the next round of recovery request message uploads based on an exponential coefficient, a jitter coefficient, and the preset interval; if the request interval reaches an interval threshold, updating the request interval to a target interval; and uploading the recovery request message to the producer or multicast network based on the target interval.

9. A data recovery device, characterized in that, Applied to producers, the system includes: an acquisition module for acquiring at least one recovery request message sent by a consumer and / or a multicast network; a parsing module for parsing the recovery request message to obtain a consumer port identifier, a request type, and a sequence of differential data packets corresponding to the consumer port identifier; a deduplication module for deduplicating the sequence of differential data packets based on the consumer port identifier and the request type to obtain at least one recovery data packet sequence and a sending identifier corresponding to the recovery data packet sequence; a determination module for determining recovery data packets based on the recovery data packet sequence and a database; and a sending module for sending the recovery data packets to the consumer and / or the multicast network based on the sending identifier and the consumer port identifier.

10. A data recovery device, characterized in that, Applied to consumers, the system includes: a generation module for generating a recovery request message based on a source data packet sequence and a consumer data packet sequence; the recovery request message includes a consumer port identifier and a differential data packet sequence number; an uploading module for uploading the recovery request message to a producer, or uploading the recovery request message to a multicast network, so that the multicast network sends the recovery request message to the producer, so that the producer determines a recovery data packet based on the recovery request message received from the consumer and / or the multicast network, and sends the recovery data packet to the consumer or the multicast network; and a receiving module for receiving the recovery data packet sent by the producer or the multicast network.

11. An electronic device, characterized in that, The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the data recovery method of any one of claims 1-5 or 6-8.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the data recovery method of any one of claims 1-5 or 6-8.

13. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the data recovery method according to any one of claims 1-5 or 6-8.

Images