Method and system for achieving zero data loss in network failure scenarios

Abstract

The present disclosure relates to method and system for achieving zero data loss in network failure scenarios The present disclosure encompasses receiving, by a receiving unit [102], a data stream from a source device [101] over a data transfer session between the source device [101] and a destination device [103]; transmitting, by a transmitting unit [104], the received data stream to the destination device [103]; storing, by a first storing unit [106], a set of metadata corresponding to each instant of the receiving and the transmitting of the data stream successfully; determining, by a determining unit [108], failure in the data transfer session; storing, by a second storing unit [110], the received data stream; and resuming, by the transmitting unit [104], upon restoration of the data transfer session, transmission of the received data stream, wherein resuming the transmission of the data stream is based on the set of metadata.

Description

METHOD AND SYSTEM FOR ACHIEVING ZERO DATA LOSS IN NETWORK FAILURE SCENARIOSTECHNICAL FIELD

[0001] The present disclosure generally relates to network performance management systems. More particularly, the present disclosure relates to a method and system for achieving zero data loss in network failure scenarios.BACKGROUND

[0002] The following description of the related art is intended to provide background information pertaining to the field of the disclosure. This section may include certain aspects of the art that may be related to various features of the present disclosure. However, it should be appreciated that this section is used only to enhance the understanding of the reader with respect to the present disclosure, and not as admissions of the prior art.

[0003] Wireless communication technology has rapidly evolved over the past few decades, with each generation bringing significant improvements and advancements. The first generation of wireless communication technology was based on analog technology and offered only voice services. However, with the advent of the second-generation (2G) technology, digital communication and data services became possible, and text messaging was introduced. 3G technology marked the introduction of high-speed internet access, mobile video calling, and location-based services. The fourth-generation (4G) technology revolutionized wireless communication with faster data speeds, better network coverage, and improved security. Currently, the fifth-generation (5G) technology is being deployed, promising even faster data speeds, low latency, and the ability to connect multiple devices simultaneously. With each generation, wireless communication technology has become more advanced, sophisticated, and capable of delivering more services to its users.

[0004] Network performance management systems typically track network elements and data from network monitoring tools and combine and process such data to determine key performance indicators (KPI) of the network. Integrated performance management systems provide the means to visualize the network performance data so that network operators and other relevant stakeholders are able to identify the service quality of the overall network, and individual / groupednetwork elements. By having an overall as well as detailed view of the network performance, the network operators can detect, diagnose, and remedy actual service issues, as well as predict potential service issues or failures in the network and take precautionary measures accordingly.

[0005] Existing solutions in data transfer between network nodes or servers primarily rely on protocols like SSH (Secure Shell) or SFTP (Secure File Transfer Protocol) to manage data flows from source to destination systems. However, these traditional systems face significant challenges in maintaining data integrity during network disruptions. In existing solutions, data transmission resumes from the point of reconnection, ignoring the data not transferred during the outage. Moreover, existing methodologies often require manual interventions to resolve data inconsistencies caused by network failures. The manual intervention not only increases the workload for IT staff but also introduces delays in data availability and can lead to errors in data handling. Furthermore, the lack of robust audit mechanisms in traditional systems means that discrepancies between source and destination data during outages are not always detected or resolved promptly, compounding the risk of data loss. The reliance on manual processes and the absence of an automated, resilient data synchronization and recovery system highlights a significant gap in current network data management practices. This gap can affect businesses' efficiency, data reliability, and ultimately, their ability to make informed decisions based on complete and accurate data sets.

[0006] Thus, there exists an imperative need in the art to provide a solution that provides zero data loss in network failure scenarios in a network system, allows automatic transfer of the data and overcomes above stated and other limitations of the existing solutions.OBJECTS OF THE INVENTION

[0007] Some of the objects of the present disclosure, which at least one embodiment disclosed herein satisfies are listed herein below.

[0008] It is an object of the present disclosure to provide zero data loss in network failure scenarios in a network system.

[0009] It is another object of the present disclosure to provide a solution that can allow automatic transfer of the data by eliminating the need of manual intervention involved in making the data available for the time of network failure occurrence.

[0010] It is another object of the present disclosure to provide a solution that ensures that the data is not lost even in case of network failure (connectivity loss) of the source as well as the destination system, without any need of manual intervention needed for making data available.

[0011] It is yet another object of the present disclosure to provide a solution that can provide functionality of Audit Scheduler (Periodic as well as On Demand) which synchronizes data between source and destination, thereby avoiding any possibility of losing data.SUMMARY OF THE DISCLOSURE

[0012] This section is provided to introduce certain aspects of the present disclosure in a simplified form that are further described below in the detailed description. This summary is not intended to identify the key features or the scope of the claimed subject matter.

[0013] An aspect of the present disclosure provides a method for achieving zero data loss in an event of network failure. The method includes receiving, by a receiving unit via an ingestion layer, a data stream from a source device over a data transfer session between the source device and a destination device. The method further includes transmitting, by a transmitting unit via the ingestion layer, the received data stream to the destination device. The method further includes storing, by a first storing unit via the ingestion layer, a set of metadata corresponding to each instant of the receiving and the transmitting of the data stream successfully. The method further includes determining, by a determining unit, failure in the data transfer session. The method further includes storing, by a second storing unit via the ingestion layer, the received data stream. Thereafter, the method includes resuming, by the transmitting unit via the ingestion layer, upon restoration of the data transfer session, transmission of the received data stream, wherein resuming the transmission of the data stream is based on the set of metadata.

[0014] In an aspect, the data transfer session utilizes at least one protocol selected from a group consisting of Secure Shell (SSH), Secure File Transfer Protocol (SFTP), Transmission Control Protocol (TCP), and Internet Protocol (IP).

[0015] In an aspect, the method comprises performing, by an audit unit, an audit on the stored set of metadata and the received data stream at the destination device periodically after a pre-definedperiod of time to determine a delta between the stored set of metadata and the received data stream at the destination device.

[0016] In an aspect, the method further comprises synchronizing, by a synchronizing unit, the received data stream at the destination device and the received data stream based on the determined delta.

[0017] In an aspect, the method further comprises auditing, by the audit unit, the stored set of metadata and the received data stream at the source device to ensure consistency between the source and destination devices.

[0018] In an aspect, the method further comprises synchronizing, by the synchronising unit, the data stream between the source device and the destination device using the determined delta to maintain data consistency across devices.

[0019] In an aspect, the method comprises rendering, by a display unit, a user interface on a display device to receive a user input to trigger the audit.

[0020] In an aspect, the determination of the data transfer session failure includes monitoring for at least one of a network failure or a disk failure between at least one of the source devices and the ingestion layer, or the ingestion layer and the destination device.

[0021] Another aspect of the present disclosure provides a system for achieving zero data loss in an event of network failure. The system includes an ingestion layer. The ingestion layer includes a receiving unit configured to receive a data stream from a source device over a data transfer session between the source device and a destination device. The ingestion further includes a transmitting unit configured to transmit the received data stream to the destination device. The ingestion further includes a first storing unit configured to store a set of metadata corresponding to each instant of the receiving and the transmitting of the received data stream successfully. The ingestion further includes a determining unit configured to determine failure in the data transfer session. The ingestion further includes a second storing unit configured to store the received data stream. The ingestion further includes the transmitting unit configured to resume transmission of the received data stream, wherein resuming the transmission of the data stream is based on the set of metadata upon restoration of the data transfer session.

[0022] Yet another aspect of the present invention relates to a non-transitory computer-readable storage medium storing instructions for achieving zero data loss in an event of network failure. The instructions include an executable code, which when executed by the system, may cause receiving by a receiving unit via an ingestion layer, a data stream from a source device over a data transfer session between the source device and a destination device. The instructions further include transmitting, by a transmitting unit via the ingestion layer, the received data stream to the destination device. Further, the instructions include storing, by a first storing unit via the ingestion layer, a set of metadata corresponding to each instant of the receiving and the transmitting of the data stream successfully. The instructions further include determining, by a determining unit, failure in the data transfer session, storing, by a second storing unit via the ingestion layer, the received data stream and resuming, by the transmitting unit via the ingestion layer, upon restoration of the data transfer session, transmission of the received data stream, wherein resuming the transmission of the data stream is based on the set of metadata.

[0023] Yet another aspect of the present invention relates to a user equipment (UE) for achieving zero data loss in an event of network failure. The UE comprising a processor to receive, by a receiving unit via an ingestion layer, a data stream from a source device over a data transfer session between the source device and a destination device. The processor may further transmit, by a transmitting unit via the ingestion layer, the received data stream to the destination device. The process may further store, by a first storing unit via the ingestion layer, a set of metadata corresponding to each instant of the receiving and the transmitting of the data stream successfully. Furthermore, the processor may determine, by a determining unit, failure in the data transfer session. The processor may further store, by a second storing unit via the ingestion layer, the received data stream. Further, the processor may resume, by the transmitting unit via the ingestion layer, upon restoration of the data transfer session, transmission of the received data stream, wherein resuming the transmission of the data stream is based on the set of metadata.BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings, which are incorporated herein, and constitute a part of this disclosure, illustrate exemplary embodiments of the disclosed methods and systems in which like reference numerals refer to the same parts throughout the different drawings. Components in the drawings are not necessarily to scale, emphasis instead being placed upon clearly illustrating the principles of the present disclosure. Also, the embodiments shown in the figures are not to be construed as limiting the disclosure, but the possible variants of the method and system accordingto the disclosure are illustrated herein to highlight the advantages of the disclosure. It will be appreciated by those skilled in the art that disclosure of such drawings includes disclosure of electrical components or circuitry commonly used to implement such components.

[0025] FIG. 1 illustrates an exemplary block diagram of a system for achieving zero data loss in an event of network failure, in accordance with exemplary embodiments of the present disclosure.

[0026] FIG. 2 illustrates an exemplary block diagram of a network performance management system, in accordance with the exemplary embodiments of the present invention.

[0027] FIG. 3 illustrates an exemplary architecture for implementation of a system for providing zero data loss in network failure scenarios in a network system, in accordance with the exemplary embodiments of the present invention.

[0028] FIG. 4 illustrates an exemplary method flow diagram indicating the process for providing zero data loss in network failure scenarios in a network system, in accordance with the exemplary embodiments of the present invention.

[0029] FIG. 5 illustrates an exemplary block diagram of a computing device upon which an embodiment of the present disclosure may be implemented.

[0030] The foregoing shall be more apparent from the following more detailed description of the disclosure.DESCRIPTION

[0031] In the following description, for the purposes of explanation, various specific details are set forth in order to provide a thorough understanding of embodiments of the present disclosure. It will be apparent, however, that embodiments of the present disclosure may be practiced without these specific details. Several features described hereafter may each be used independently of one another or with any combination of other features. An individual feature may not address any of the problems discussed above or might address only some of the problems discussed above.

[0032] The ensuing description provides exemplary embodiments only, and is not intended to limit the scope, applicability, or configuration of the disclosure. Rather, the ensuing description of theexemplary embodiments will provide those skilled in the art with an enabling description for implementing an exemplary embodiment. It should be understood that various changes may be made in the function and arrangement of elements without departing from the spirit and scope of the disclosure as set forth.

[0033] Specific details are given in the following description to provide a thorough understanding of the embodiments. However, it will be understood by one of ordinary skill in the art that the embodiments may be practiced without these specific details. For example, circuits, systems, processes, and other components may be shown as components in block diagram form in order not to obscure the embodiments in unnecessary detail.

[0034] Also, it is noted that individual embodiments may be described as a process which is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations may be performed in parallel or concurrently. In addition, the order of the operations may be re-arranged. A process is terminated when its operations are completed but could have additional steps not included in a figure.

[0035] The word “exemplary” and / or “demonstrative” is used herein to mean serving as an example, instance, or illustration. For the avoidance of doubt, the subject matter disclosed herein is not limited by such examples. In addition, any aspect or design described herein as “exemplary” and / or “demonstrative” is not necessarily to be construed as preferred or advantageous over other aspects or designs, nor is it meant to preclude equivalent exemplary structures and techniques known to those of ordinary skill in the art. Furthermore, to the extent that the terms “includes,” “has,” “contains,” and other similar words are used in either the detailed description or the claims, such terms are intended to be inclusive — in a manner similar to the term “comprising” as an open transition word — without precluding any additional or other elements.

[0036] As used herein, “a user equipment”, “a user device”, “a smart-user-device”, “a smartdevice”, “an electronic device”, “a mobile device”, “a handheld device”, “a wireless communication device”, “a mobile communication device”, “a communication device” may be any electrical, electronic and / or computing device or equipment, capable of implementing the features of the present disclosure. The user equipment / device may include, but is not limited to, a mobile phone, smart phone, laptop, a general-purpose computer, desktop, personal digital assistant, tablet computer, wearable device or any other computing device which is capable ofimplementing the features of the present disclosure. Also, the user device may contain at least one input means configured to receive an input from at least one of a transceiver unit, a processing unit, a storage unit, a detection unit and any other such unit(s) which are required to implement the features of the present disclosure.

[0037] As used herein, “storage unit” or “memory unit” refers to a machine or computer-readable medium including any mechanism for storing information in a form readable by a computer or similar machine. For example, a computer-readable medium includes read-only memory (“ROM”), random access memory (“RAM”), magnetic disk storage media, optical storage media, flash memory devices or other types of machine-accessible storage media. The storage unit stores at least the data that may be required by one or more units of the system to perform their respective functions.

[0038] As discussed in the background section, existing solutions in data transfer between network nodes or servers primarily rely on protocols like SSH (Secure Shell) or SFTP (Secure File Transfer Protocol) to manage data flows from source to destination systems. However, these traditional systems face significant challenges in maintaining data integrity during network disruptions. In existing solutions, data transmission resumes from the point of reconnection, ignoring the data not transferred during the outage. Moreover, existing methodologies often require manual interventions to resolve data inconsistencies caused by network failures. The manual intervention not only increases the workload for IT staff but also introduces delays in data availability and can lead to errors in data handling. Furthermore, the lack of robust audit mechanisms in traditional systems means that discrepancies between source and destination data during outages are not always detected or resolved promptly, compounding the risk of data loss. The reliance on manual processes and the absence of an automated, resilient data synchronization and recovery system highlights a significant gap in current network data management practices. This gap can affect businesses' efficiency, data reliability, and ultimately, their ability to make informed decisions based on complete and accurate data sets.

[0039] To overcome these and other inherent problems in the art, the present disclosure proposes a solution of an advanced data management system that ensures zero data loss during network failures by employing a robust and automated data backup and synchronization methodology. The present disclosure leverages an ingestion layer that not only transmits and receives data streams between source and destination devices but also records and stores metadata for each data transfer event. The metadata helps the system identify the exact point at which the data transfer wasinterrupted, thereby enabling a precise resumption of data flow once network connectivity is restored. The proposed solution enhances traditional data transfer protocols such as SSH, SFTP, TCP, and IP with additional layers of security and reliability. By storing the data stream during a network failure and utilizing the stored metadata for resuming transmission, the system effectively mitigates the risk of data loss. Thus, the proposed solution provides a significant improvement over existing systems, which typically start transferring data only from the point at which the network becomes available again, without addressing the data lost during the outage. Moreover, the present disclosure includes a dynamic auditing mechanism that periodically checks the consistency and integrity of the data stored at the destination against the metadata. The audit can be triggered periodically or on demand, ensuring ongoing vigilance and proactive maintenance of data integrity. The audit process helps in identifying any discrepancies between the source and destination, which are then corrected by synchronizing the data based on the identified delta.

[0040] It would be appreciated by the person skilled in the art that the present disclosure addresses the existing challenges in data transfer during network disruptions by providing a system and method that ensure continuous data availability and integrity. The proposed solution functions autonomously, reduce downtime, and maintain a consistent and accurate data flow between network nodes, thereby greatly enhancing the efficiency and reliability of data management practices in networked environments.

[0041] Hereinafter, exemplary embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0042] FIG. 1 illustrates an exemplary block diagram of a system

[0100] for achieving zero data loss in an event of network failure, in accordance with exemplary embodiments of the present disclosure. Referring to FIG. 1, the system

[0100] comprises an ingestion layer [100A], In an embodiment, the ingestion layer [100A] includes a receiving unit

[0102] , a transmitting unit

[0104] , a first storing unit

[0106] , a determining unit

[0108] , and a second storing unit

[0110] , wherein all the components are assumed to be connected to each other in a manner as obvious to the person skilled in the art for implementing features of the present disclosure.

[0043] In an exemplary embodiment, the ingestion layer [100 A] maybe at the IPM [200a], The system

[0100] for achieving zero data loss in an event of network failure comprises the ingestion layer [100A], The ingestion layer [100A] includes the receiving unit

[0102] , which is configured to receive a data stream from a source device

[0101] over a data transfer session between the sourcedevice

[0101] and a destination device

[0103] , In an exemplary implementation, the data stream refers to a performance counter data from the source device

[0101] , The source device

[0101] may be at least one of a probe, a network node, and the like. The destination device

[0103] maybe a normalization layer, anomaly detection layer, computation layer, mapping layer, and the like. In an example, the data stream may be an alarm, a configuration data, a node data such as a Session Detail Record (SDR), a call detail record (CDR), a log, a trace, and the like. This data transfer session facilitates the continuous flow of data from the source device

[0101] to the destination device

[0103] , even in scenarios where network connectivity might be disrupted. During normal operation, the receiving unit

[0102] receives and processes the incoming data stream from the source device

[0101] , The data transfer session utilizes at least one protocol selected from a group consisting of Secure Shell (SSH), Secure File Transfer Protocol (SFTP), Transmission Control Protocol (TCP), and Internet Protocol (IP).

[0044] The transmitting unit

[0104] is communicatively coupled to the receiving unit

[0102] within the ingestion layer [100A] of the system

[0100] , The transmitting unit

[0104] is configured to transmit the received data stream to the destination device

[0103] , Once the receiving unit

[0102] has successfully received and pre-processed the data stream from the source device

[0101] , the transmitting unit

[0104] forwards the received data stream accurately and promptly to the destination device

[0103] ,

[0045] The source device

[0101] can be any device or server that originates data to be transmitted. Examples of source devices include network element servers, such as routers, switches, or base stations that generate real-time operational or performance data. Another example could be a database server hosting financial transactions or customer data, which needs to be regularly backed up or synchronized with another location. Further, the destination device

[0103] is the recipient of the data stream transmitted from the source device

[0101] that could be a data centre server where data from multiple network elements are aggregated for processing and analysis. The destination device

[0103] may also be a cloud storage service where data is sent for redundancy and disaster recovery purposes. Additionally, the destination device could be an enterprise server that receives updates from various branch office servers across different geographic locations to maintain a centralized data repository.

[0046] The first storing unit

[0106] is communicatively coupled to the transmitting unit

[0104] within the ingestion layer [100 A] of the system

[0100] , In an implementation of the present disclosure, the first storing unit

[0106] may be the caching layer [200c], the Distributed data lake[200u], the Distributed File System [200j], and the like. The first storing unit

[0106] is configured to store a set of metadata corresponding to each instant of the receiving and the transmitting of the received data stream successfully. The metadata includes but not limited only to timestamps, data packet sizes, source and destination addresses, and status indicators, which are essential for tracking the progress and integrity of the data transmission process. The metadata provides a reliable audit trail that can be used for troubleshooting, performance monitoring, and ensuring compliance with data governance standards. Further, in the event of a network failure, the metadata facilitates in resuming the data transmission from the exact point of interruption without loss. It would be appreciated by the person skilled in the art that the metadata allows the system to identify precisely what data has already been successfully transmitted and what has not, enabling an efficient and accurate recovery process.

[0047] The determining unit

[0108] is communicatively coupled to the first storing unit

[0106] within the ingestion layer [100A] of the system

[0100] , The determining unit

[0108] is configured to determine failure in the data transfer session. The determining unit

[0108] continuously monitoring the data transfer process and using various metrics and indicators such as interruption in connectivity, unexpected latency increases, or error rates that exceed predefined thresholds. The determination of the data transfer session failure comprises monitoring for at least one of a network failure or a disk failure between at least one of the source device

[0101] or the ingestion layer [100A], and between the ingestion layer [100A] and the destination device

[0103] ,

[0048] The second storing unit

[0110] is communicatively coupled to the determining unit

[0108] within the ingestion layer [100 A] of the system

[0100] , In an implementation of the present disclosure, the second storing unit

[0110] may be the caching layer [200c], the Distributed data lake [200u], the Distributed File System [200j], and the like. The second storing unit

[0110] is configured to store the received data stream. When such a failure occurs, the determining unit signals the second storing unit to immediately begin storing the incoming data stream, effectively preserving the data that has not yet been transmitted successfully to the destination device. The second storing unit

[0110] acts as a buffer or temporary holding area for data during disruptions in the network. The data stored in the second storing unit

[0110] includes not just the raw data received but may also encompass any processing or partial transmissions that were underway at the time of the interruption.

[0049] The transmitting unit

[0104] is further configured to resume transmission of the received data stream, wherein resuming the transmission of the data stream is based on the set of metadataupon restoration of the data transfer session. When network connectivity or system functionality is restored after a failure, the transmitting unit

[0104] , leveraging the metadata, determines the last successful data transmission instant. This information includes timestamps, sequence numbers, and other pertinent data specifics that are essential for precisely aligning the continuation of the data stream with the point at which it was disrupted.

[0050] The ingestion layer [100A] further comprises the audit unit

[0112] , The audit unit

[0112] is configured to perform an audit on the stored set of metadata and the received data stream at the destination device periodically after a pre-defined period of time to determine a delta between the stored set of metadata and the received data stream at the destination device

[0103] , The purpose of these audits is to verify that all data intended to be transmitted from the source device

[0101] has been successfully and accurately received by the destination device

[0103] without any corruption or loss. By comparing the metadata, which includes detailed records of what was transmitted, with the actual data received at the destination, the audit unit

[0112] identifies any discrepancies or anomalies such as delta, represent differences or gaps in what was sent versus what was received.

[0051] The audit unit

[0112] is further configured to audit the stored set of metadata and the received data stream at the source device

[0101] to ensure consistency between the source and destination devices

[0103] , By performing audits at both the source and the destination, the audit unit

[0112] provides a comprehensive review of the data integrity throughout the entire transfer process. The audits at the source device

[0101] involve checking the metadata against the data that was initially sent out for transmission. This helps in verifying that the data dispatched from the source matches the records stored in the metadata. Similarly, by comparing this data with what is received at the destination device

[0103] , the audit unit

[0112] ensures that any discrepancies are identified not only post-transmission but also pre-transmission. The dual-check system significantly reduces the risk of data inconsistencies and errors propagating through the network.

[0052] The ingestion layer [100A] comprises a synchronizing unit

[0114] , The synchronizing unit

[0114] is configured to synchronize the received data stream at the destination device

[0103] and the received data stream based on the determined delta. The determined delta, identified by the audit unit

[0112] , indicates the specific differences or gaps between what was sent and what was actually received. Furthermore, the synchronizing unit

[0114] is further configured to synchronize the data stream between the source device

[0101] and the destination device

[0103] using the determined delta to maintain data consistency across devices. This means that the synchronizing unit not onlycorrects discrepancies at the destination but also ensures that any future transmissions take into account the corrected data state, thereby maintaining a continuous and consistent data flow.

[0053] The ingestion layer [100A] comprises the display unit

[0116] , The display unit

[0116] is configured to render a user interface to receive a user input to trigger the audit. Through the user interface, users can specify parameters for the audit, such as the time period to be reviewed or particular data streams to focus on, thereby facilitating targeted and effective monitoring of data consistency and integrity.

[0054] FIG. 2 illustrates an exemplary block diagram of a network performance management system

[0200] , in accordance with the exemplary embodiments of the present invention. Referring to FIG. 2, the network performance management system

[0200] comprises various sub-systems such as: integrated performance management system [200a], normalization layer [200b], computation layer [200d], anomaly detection layer [200o], streaming engine

[2001] , load balancer [200k], operations and management system [200p], API gateway system [200r], analysis engine [200h], parallel computing framework [200i], forecasting engine [200t], distributed file system [200j], mapping layer [200s], distributed data lake [200u], scheduling layer [200g], reporting engine [200m], message broker [200e], graph layer [200f], caching layer [200c], service quality manager [200q] and correlation engine[200n]. Exemplary connections between these subsystems are also as shown in FIG.2. However, it will be appreciated by those skilled in the art that the present disclosure is not limited to the connections shown in the diagram, and any other connections between various subsystems that are needed to realise the effects are within the scope of this disclosure.

[0055] Following are the various components of the network performance management system

[0200] , the various components may include:

[0056] Integrated performance management (TPM) system [200a] comprises of one or more 5G performance engine [200v] and one or more 5GKey Performance Indicator (KPI) Engine [200w],

[0057] 5G Performance Management Engine [200v]: The 5G Performance Management engine [200v] is a crucial component of the integrated system, responsible for collecting, processing, and managing performance counter data from various data sources within the network. The gathered data includes metrics such as connection speed, latency, data transfer rates, and many others. This raw data is then processed and aggregated as required, forming a comprehensive overview ofnetwork performance. The processed information is then stored in a Distributed Data Lake [200u], a centralized, scalable, and flexible storage solution, allowing for easy access and further analysis. The 5G Performance Management engine [200v] also enables the reporting and visualization of this performance counter data, thus providing network administrators with a real-time, insightful view of the network's operation. Through these visualizations, operators can monitor the network's performance, identify potential issues, and make informed decisions to enhance network efficiency and reliability.

[0058] 5G Key Performance Indicator (KPI) Engine [200w]: The 5G Key Performance Indicator (KPI) Engine is a dedicated component tasked with managing the KPIs of all the network elements. It uses the performance counters, which are collected and processed by the 5G Performance Management engine from various data sources. These counters, encapsulating crucial performance data, are harnessed by the KPI engine [200w] to calculate essential KPIs. These KPIs might include data throughput, latency, packet loss rate, and more. Once the KPIs are computed, they are segregated based on the aggregation requirements, offering a multi-layered and detailed understanding of network performance. The processed KPI data is then stored in the Distributed Data Lake [200u], ensuring a highly accessible, centralized, and scalable data repository for further analysis and utilization. Similar to the Performance Management engine, the KPI engine [200w] is also responsible for reporting and visualization of KPI data. This functionality allows network administrators to gain a comprehensive, visual understanding of the network's performance, thus supporting informed decision-making and efficient network management.

[0059] Ingestion layer [not shown]: The Ingestion layer forms a key part of the Integrated Performance Management system. Its primary function is to establish an environment capable of handling diverse types of incoming data. This data may include Alarms, Counters, Configuration parameters, Call Detail Records (CDRs), Infrastructure metrics, Logs, and Inventory data, all of which are crucial for maintaining and optimizing the network's performance. Upon receiving this data, the Ingestion layer processes it by validating its integrity and correctness to ensure it is fit for further use. Following validation, the data is routed to various components of the system, including the Normalization layer, Streaming Engine, Streaming Analytics, and Message Brokers. The destination is chosen based on where the data is required for further analytics and processing. By serving as the first point of contact for incoming data, the Ingestion layer plays a vital role in managing the data flow within the system, thus supporting comprehensive and accurate network performance analysis.

[0060] Normalization layer [200b]: The Normalization Layer [200b] serves to standardize, enrich, and store data into the appropriate databases. It takes in data that has been ingested and adjusts it to a common standard, making it easier to compare and analyse. This process of "normalization" reduces redundancy and improves data integrity. Upon completion of normalization, the data is stored in various databases like the Distributed Data Lake [200u], Caching Layer [200c], and Graph Layer [200f], depending on its intended use. The choice of storage determines how the data can be accessed and used in the future. Additionally, the Normalization Layer [200b] produces data for the Message Broker, a system that enables communication between different parts of the performance management system through the exchange of data messages. Moreover, the Normalization Layer [200b] supplies the standardized data to several other subsystems. These include the Analysis Engine [200h] for detailed data examination, the Correlation Engine [200n] for detecting relationships among various data elements, the Service Quality Manager [200q] for maintaining and improving the quality of services, and the Streaming Engine

[2001] for processing real-time data streams. These subsystems depend on the normalized data to perform their operations effectively and accurately, demonstrating the Normalization Layer's [200b] critical role in the entire system.

[0061] Caching layer [200c]: The Caching Layer [200c] in the Integrated Performance Management system plays a significant role in data management and optimization. During the initial phase, the Normalization Layer [200b] processes incoming raw data to create a standardized format, enhancing consistency and comparability. The Normalizer Layer then inserts this normalized data into various databases. One such database is the Caching Layer [200c], The Caching Layer [200c] is a high-speed data storage layer which temporarily holds data that is likely to be reused, to improve speed and performance of data retrieval. By storing frequently accessed data in the Caching Layer [200c], the system significantly reduces the time taken to access this data, improving overall system efficiency and performance. Further, the Caching Layer [200c] serves as an intermediate layer between the data sources and the sub-systems, such as the Analysis Engine [200h], Correlation Engine [200n], Service Quality Manager, and Streaming Engine. The Normalization Layer [200b] is responsible for providing these sub-systems with the necessary data from the Caching Layer [200c],

[0062] Computation layer [200d]: The Computation Layer [200d] in the Integrated Performance Management system serves as the main hub for complex data processing tasks. In the initial stages, raw data is gathered, normalized, and enriched by the Normalization Layer [200b], The Normalizer Layer then inserts this standardized data into multiple databases including the Distributed DataLake [200u], Caching Layer [200c], and Graph Layer [200f], and also feeds it to the Message Broker [200e], Within the Computation Layer [200d], several powerful sub-systems such as the Analysis Engine [200h], Correlation Engine [200n], Service Quality Manager, and Streaming Engine, utilize the normalized data. These systems are designed to execute various data processing tasks. The Analysis Engine performs in-depth data analytics to generate insights from the data. The Correlation Engine [200n] identifies and understands the relations and patterns within the data. The Service Quality Manager assesses and ensures the quality of the services. And the Streaming Engine processes and analyses the real-time data feeds. In essence, the Computation Layer [200d] is where all major computation and data processing tasks occur. It uses the normalized data provided by the Normalization Layer [200b], processing it to generate useful insights, ensure service quality, understand data patterns, and facilitate real-time data analytics.

[0063] Message broker [200e]: The Message Broker [200e], an integral part of the Integrated Performance Management system, operates as a publish-subscribe messaging system. It orchestrates and maintains the real-time flow of data from various sources and applications. At its core, the Message Broker [200e] facilitates communication between data producers and consumers through message-based topics. This creates an advanced platform for contemporary distributed applications. With the ability to accommodate a large number of permanent or ad-hoc consumers, the Message Broker [200e] demonstrates immense flexibility in managing data streams. Moreover, it leverages the filesystem for storage and caching, boosting its speed and efficiency. The design of the Message Broker [200e] is centred around reliability. It is engineered to be fault-tolerant and mitigate data loss, ensuring the integrity and consistency of the data. With its robust design and capabilities, the Message Broker [200e] forms a critical component in managing and delivering real-time data in the system.

[0064] Graph layer [200f] : The Graph Layer [200f], serving as the Relationship Modeler, plays a pivotal role in the Integrated Performance Management system. It can model a variety of data types, including alarm, counter, configuration, CDR data, Infra-metric data, 5G Probe Data, and Inventory data. Equipped with the capability to establish relationships among diverse types of data, the Relationship Modeler offers extensive modelling capabilities. For instance, it can model Alarm and Counter data, Vprobe and Alarm data, elucidating their interrelationships. Moreover, the Modeler should be adept at processing steps provided in the model and delivering the results to the system requested, whether it be a Parallel Computing system, Workflow Engine, Query Engine, Correlation System [200n], 5G Performance Management Engine, or 5G KPI Engine [200w], With its powerful modelling and processing capabilities, the Graph Layer [200f] forms anessential part of the system, enabling the processing and analysis of complex relationships between various types of network data.

[0065] Scheduling layer [200g]: The Scheduling Layer [200g] serves as a key element of the Integrated Performance Management System, endowed with the ability to execute tasks at predetermined intervals set according to user preferences. A task might be an activity performing a service call, an API call to another microservice, the execution of an Elastic Search query, and storing its output in the Distributed Data Lake [200u] or Distributed File System or sending it to another micro- service. The versatility of the Scheduling Layer [200g] extends to facilitating graph traversals via the Mapping Layer to execute tasks. This crucial capability enables seamless and automated operations within the system, ensuring that various tasks and services are performed on schedule, without manual intervention, enhancing the system's efficiency and performance. In sum, the Scheduling Layer [200g] orchestrates the systematic and periodic execution of tasks, making it an integral part of the efficient functioning of the entire system.

[0066] Analysis Engine [200h]: The Analysis Engine [200h] forms a crucial part of the Integrated Performance Management System, designed to provide an environment where users can configure and execute workflows for a wide array of use-cases. This facility aids in the debugging process and facilitates a better understanding of call flows. With the Analysis Engine [200h], users can perform queries on data sourced from various subsystems or external gateways. This capability allows for an in-depth overview of data and aids in pinpointing issues. The system's flexibility allows users to configure specific policies aimed at identifying anomalies within the data. When these policies detect abnormal behaviour or policy breaches, the system sends notifications, ensuring swift and responsive action. In essence, the Analysis Engine [200h] provides a robust analytical environment for systematic data interrogation, facilitating efficient problem identification and resolution, thereby contributing significantly to the system's overall performance management.

[0067] Parallel Computing Framework [200i] : The Parallel Computing Framework [200i] is a key aspect of the Integrated Performance Management System, providing a user-friendly yet advanced platform for executing computing tasks in parallel. This framework showcases both scalability and fault tolerance, crucial for managing vast amounts of data. Users can input data via Distributed File System (DFS) [200j] locations or Distributed Data Lake (DDL) indices. The framework supports the creation of task chains by interfacing with the Service Configuration Management (SCM) Sub-System. Each task in a workflow is executed sequentially, but multiplechains can be executed simultaneously, optimizing processing time. To accommodate varying task requirements, the service supports the allocation of specific host lists for different computing tasks. The Parallel Computing Framework [200i] is an essential tool for enhancing processing speeds and efficiently managing computing resources, significantly improving the system's performance management capabilities.

[0068] Distributed File System [200j] : The Distributed File System (DFS) [200j] is a critical component of the Integrated Performance Management System, enabling multiple clients to access and interact with data seamlessly. This file system is designed to manage data files that are partitioned into numerous segments known as chunks. In the context of a network with vast data, the DFS [200j] effectively allows for the distribution of data across multiple nodes. This architecture enhances both the scalability and redundancy of the system, ensuring optimal performance even with large data sets. DFS [200j] also supports diverse operations, facilitating the flexible interaction with and manipulation of data. This accessibility is paramount for a system that requires constant data input and output, as is the case in a robust performance management system.

[0069] Load Balancer [200k]: The Load Balancer (LB) [200k] is a vital component of the Integrated Performance Management System, designed to efficiently distribute incoming network traffic across a multitude of backend servers or microservices. Its purpose is to ensure the even distribution of data requests, leading to optimized server resource utilization, reduced latency, and improved overall system performance. The LB [200k] implements various routing strategies to manage traffic. These include round-robin scheduling, header-based request dispatch, and contextbased request dispatch. Round-robin scheduling is a simple method of rotating requests evenly across available servers. In contrast, header and context-based dispatching allow for more intelligent, request-specific routing. Header-based dispatching routes requests based on data contained within the headers of the Hypertext Transfer Protocol (HTTP) requests. Context-based dispatching routes traffic based on the contextual information about the incoming requests. For example, in an event-driven architecture, the LB [200k] manages event and event acknowledgments, forwarding requests or responses to the specific microservice that has requested the event. This system ensures efficient, reliable, and prompt handling of requests, contributing to the robustness and resilience of the overall performance management system.

[0070] Streaming Engine

[2001] : The Streaming Engine

[2001] , also referred to as Stream Analytics, is a critical subsystem in the Integrated Performance Management System. This engineis specifically designed for high-speed data pipelining to the User Interface (UI). Its core objective is to ensure real-time data processing and delivery, enhancing the system's ability to respond promptly to dynamic changes. Data is received from various connected subsystems and processed in real-time by the Streaming Engine

[2001] , After processing, the data is streamed to the UI, fostering rapid decision-making and responses. The Streaming Engine

[2001] cooperates with the Distributed Data Lake [200u], Message Broker [200e], and Caching Layer [200c] to provide seamless, real-time data flow. Stream Analytics is designed to perform required computations on incoming data instantly, ensuring that the most relevant and up-to-date information is always available at the UI. Furthermore, this system can also retrieve data from the Distributed Data Lake [200u], Message Broker [200e], and Caching Layer [200c] as per the requirement and deliver it to the UI in real-time. The streaming engine's

[2001] ultimate goal is to provide fast, reliable, and efficient data streaming, contributing to the overall performance of the management system.

[0071] Reporting Engine [200m]: The Reporting Engine [200m] is a key subsystem of the Integrated Performance Management System. The fundamental purpose of designing the Reporting Engine [200m] is to dynamically create report layouts of API data, catered to individual client requirements, and deliver these reports via the Notification Engine (not shown). The REM serves as the primary interface for creating custom reports based on the data visualized through the client's dashboard. These custom dashboards, created by the client through the User Interface (UI), provide the basis for the Reporting Engine [200m] to process and compile data from various interfaces. The main output of the Reporting Engine [200m] is a detailed report generated in Excel format. The Reporting Engine’s [200m] unique capability to parse data from different subsystem interfaces, process it according to the client's specifications and requirements, and generate a comprehensive report makes it an essential component of this performance management system. Furthermore, the Reporting Engine [200m] integrates seamlessly with the Notification Engine (not shown) to ensure timely and efficient delivery of reports to clients via email, ensuring the information is readily accessible and usable, thereby improving overall client satisfaction and system usability.

[0072] Further, referring to FIG. 3 that illustrates an exemplary architecture

[0300] for implementation of a system for providing zero data loss in network failure scenarios in a network system, in accordance with the exemplary embodiments of the present invention. As shown in the FIG. 3, the architecture

[0300] include source device

[0101] , an ingestion layer [101 A], a file system

[0304] , a graphical user interface

[0302] , destination device

[0103] , and a database

[0306] , The architecture

[0300] may further include a successful data transfer flow

[0310] , a network failure flow

[0312] , and a restoration flow

[0314] , Also, all of the components / units of the architecture

[0300] are assumed to be connected to each other unless otherwise indicated below. Also, in FIG. 3 only a few units are shown, however, the architecture

[0300] may comprise multiple such units or the architecture

[0300] may comprise any such numbers of said units, as required to implement the features of the present disclosure.

[0073] The ingestion layer [100 A] facilitates in managing the flow of data from the source device

[0101] to the destination device

[0103] , ensuring the data's integrity and correctness during the transfer process. The ingestion layer [100 A] may scan the source device

[0101] with meta data for audit via a periodic audit flow

[0318] . The periodic audit flow

[0318] may be an audit by the ingestion layer [100A] at predefined time intervals. The predefined time intervals may be defined by the user. The ingestion layer [100A] is configured to receive a data stream from the source device

[0101] over a data transfer session between the source device

[0101] and the destination device

[0103] via the successful data transfer flow

[0310] , The successful data transfer flow refers to the process of receiving the data stream by the ingestion layer [100A], The successful data transfer may ensure that the data stream moves seamlessly from the source device

[0101] to the ingestion layer [100A], The ingestion layer [100A] could be utilizing data transfer protocols such as SSH, SFTP, TCP, or IP. Once the data stream is received, the ingestion layer [100A] validates and processes the data stream for transfer to the destination device

[0103] , The ingestion layer [100 A] further stores metadata that corresponds to each instant of successful data transmission, thereby enabling the system to track the data flow accurately and facilitate a quick recovery and resumption of data transfer in the event of a network failure via the network failure flow

[0312] , The network failure may occur between at least one of the source device

[0101] or the ingestion layer [100A], and between the ingestion layer [100 A] and the destination device

[0103] ,

[0074] The network connectivity or system functionality maybe restored after the network failure via the restoration flow

[0314] , wherein the transmission of the data stream may be resumed based on the set of metadata upon restoration of the data transfer session.

[0075] Additionally, the architecture

[0300] includes a database

[0306] that stores the metadata as well as potentially other information required for the system's auditing processes. Further, the architecture

[0300] includes the graphical user interface

[0302] , The graphical user interface

[0302] enables the users to initiate audits, on demand or scheduled via an on-demand audit flow

[0316] , The on-demand audit flow

[0316] refers to a process to allow the users to initiate audits when required by the user. The audits can verify the integrity and consistency of the data between thesource device

[0101] and the destination device

[0103] , particularly following any detected failures in the data transfer session.

[0076] Referring to FIG. 4 depicts an exemplary method

[0400] flow diagram indicating the process for providing zero data loss in network failure scenarios in a network system, in accordance with the exemplary embodiments of the present invention. In an implementation, the method

[0400] is performed by the system

[0100] , As shown in FIG. 4, the method

[0400] starts at step

[0402] ,

[0077] At step

[0404] , the method

[0400] as disclosed by the present disclosure comprises receiving, by a receiving unit

[0102] via an ingestion layer [100A], a data stream from a source device

[0101] over a data transfer session between the source device

[0101] and a destination device

[0103] , The data transfer session facilitates the continuous flow of data from the source device

[0101] to the destination device

[0103] , even in scenarios where network connectivity might be disrupted. During normal operation, the receiving unit

[0102] receives and processes the incoming data stream from the source device

[0101] , The data transfer session utilizes at least one protocol selected from a group consisting of Secure Shell (SSH), Secure File Transfer Protocol (SFTP), Transmission Control Protocol (TCP), and Internet Protocol (IP).

[0078] At step

[0406] , the method

[0400] as disclosed by the present disclosure comprises transmitting, by a transmitting unit

[0104] via the ingestion layer [100A], the received data stream to the destination device

[0103] , Once the receiving unit

[0102] has successfully received and pre- processed the data stream from the source device

[0101] , the transmitting unit

[0104] forwards the received data stream accurately and promptly to the destination device

[0103] ,

[0079] Next at step

[0408] , the method

[0400] as disclosed by the present disclosure comprises storing, by a first storing unit

[0106] via the ingestion layer [100A], a set of metadata corresponding to each instant of the receiving and the transmitting of the data stream successfully. The metadata includes but not limited only to timestamps, data packet sizes, source and destination addresses, and status indicators, which are essential for tracking the progress and integrity of the data transmission process. The metadata provides a reliable audit trail that can be used for troubleshooting, performance monitoring, and ensuring compliance with data governance standards. Further, in the event of a network failure, the metadata facilitates in resuming the data transmission from the exact point of interruption without loss. It would be appreciated by the person skilled in the art that the metadata allows the system to identify precisely what data hasalready been successfully transmitted and what has not, enabling an efficient and accurate recovery process.

[0080] Next at step

[0410] , the method

[0400] as disclosed by the present disclosure comprises determining, by a determining unit

[0108] , failure in the data transfer session. The determining unit

[0108] continuously monitoring the data transfer process and using various metrics and indicators such as interruption in connectivity, unexpected latency increases, or error rates that exceed predefined thresholds. The determination of the data transfer session failure comprises monitoring for at least one of a network failure or a disk failure between at least one of the source device

[0101] or the ingestion layer [100A], and between the ingestion layer [100A] and the destination device

[0103] ,

[0081] Next at step

[0412] , the method

[0400] as disclosed by the present disclosure comprises storing, by a second storing unit

[0110] via the ingestion layer [100A], the received data stream. When the failure occurs, the determining unit signals the second storing unit to immediately begin storing the incoming data stream, effectively preserving the data that has not yet been transmitted successfully to the destination device. The second storing unit

[0110] acts as a buffer or temporary holding area for data during disruptions in the network. The data stored in the second storing unit

[0110] includes not just the raw data received but may also encompass any processing or partial transmissions that were underway at the time of the interruption.

[0082] At step

[0414] , the method

[0400] as disclosed by the present disclosure comprises resuming, by the transmitting unit

[0104] via the ingestion layer [100A], upon restoration of the data transfer session, transmission of the received data stream, wherein resuming the transmission of the data stream is based on the set of metadata. When network connectivity or system functionality is restored after a failure, the transmitting unit

[0104] , leveraging the metadata, determines the last successful data transmission instant. The information includes timestamps, sequence numbers, and other pertinent data specifics that are essential for precisely aligning the continuation of the data stream with the point at which it was disrupted.

[0083] Thereafter, the method

[0400] terminates at step

[0416] ,

[0084] FIG. 5 illustrates an exemplary block diagram of a computing device

[0500] (also referred to herein as a computer system

[0500] ) upon which an embodiment of the present disclosure may be implemented. In an implementation, the computing device implements the method forachieving zero data loss in an event of network failure using the system

[0100] , In another implementation, the computing device itself implements the method for achieving zero data loss in an event of network failure by using one or more units configured within the computing device, wherein said one or more units are capable of implementing the features as disclosed in the present disclosure.

[0085] The computing device

[0500] may include a bus

[0502] or other communication mechanism for communicating information, and a processor

[0504] coupled with bus

[0502] for processing information. The processor

[0504] may be, for example, a general-purpose microprocessor. The computing device

[0500] may also include a main memory

[0506] , such as a random-access memory (RAM), or other dynamic storage device, coupled to the bus

[0502] for storing information and instructions to be executed by the processor

[0504] , The main memory

[0506] also may be used for storing temporary variables or other intermediate information during execution of the instructions to be executed by the processor

[0504] , Such instructions, when stored in non-transitory storage media accessible to the processor

[0504] , render the computing device

[0500] into a special-purpose machine that is customized to perform the operations specified in the instructions. The computing device

[0500] further includes a read only memory (ROM)

[0508] or other static storage device coupled to the bus

[0502] for storing static information and instructions for the processor

[0504] ,

[0086] A storage device

[0510] , such as a magnetic disk, optical disk, or solid-state drive is provided and coupled to the bus

[0502] for storing information and instructions. The computing device

[0500] may be coupled via the bus

[0502] to a display

[0512] , such as a cathode ray tube (CRT), for displaying information to a computer user. An input device

[0514] , including alphanumeric and other keys, may be coupled to the bus

[0502] for communicating information and command selections to the processor

[0504] , Another type of user input device may be a cursor controller

[0516] , such as a mouse, a trackball, or cursor direction keys, for communicating direction information and command selections to the processor

[0504] , and for controlling cursor movement on the display

[0512] , This inputs device typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allow the device to specify positions in a plane.

[0087] The computing device

[0500] may implement the techniques described herein using customized hard-wired logic, one or more Application-Specific Integrated Circuits (ASICs) or Field Programmable Gate Arrays (FPGAs), firmware and / or program logic which in combination with the computing device

[0500] causes or programs the computing device

[0500] to be a specialpurpose machine. According to one embodiment, the techniques herein are performed by thecomputing device

[0500] in response to the processor

[0504] executing one or more sequences of one or more instructions contained in the main memory

[0506] , Such instructions may be read into the main memory

[0506] from another storage medium, such as the storage device

[0510] , Execution of the sequences of instructions contained in the main memory

[0506] causes the processor

[0504] to perform the process steps described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions.

[0088] The computing device

[0500] also may include a communication interface

[0518] coupled to the bus

[0502] , The communication interface

[0518] provides a two-way data communication coupling to a network link

[0520] that is connected to a local network

[0522] , For example, the communication interface

[0518] may be an integrated services digital network (ISDN) card, cable modem, satellite modem, or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, the communication interface

[0518] may be a local area network (LAN) card to provide a data communication connection to a compatible LAN. Wireless links may also be implemented. In any such implementation, the communication interface

[0518] sends and receives electrical, electromagnetic, or optical signals that carry digital data streams representing various types of information.

[0089] The computing device

[0500] can send messages and receive data, including program code, through the network(s), the network link

[0520] and the communication interface 518. In the Internet example, a server

[0530] might transmit a requested code for an application program through the Internet

[0528] , the Internet Service Provider (ISP)

[0526] , host

[0524] , the local network

[0522] and the communication interface

[0518] , The received code may be executed by the processor

[0504] as it is received, and / or stored in the storage device

[0510] , or other non-volatile storage for later execution.

[0090] Yet another aspect of the present invention relates to a non-transitory computer-readable storage medium storing instructions for achieving zero data loss in an event of network failure. The instructions include an executable code, which when executed by the system, may cause receiving by a receiving unit via an ingestion layer, a data stream from a source device over a data transfer session between the source device and a destination device. The instructions further include transmitting, by a transmitting unit via the ingestion layer, the received data stream to the destination device. Further, the instructions include storing, by a first storing unit via the ingestion layer, a set of metadata corresponding to each instant of the receiving and the transmitting of the data stream successfully. The instructions further include determining, by a determining unit,failure in the data transfer session, storing, by a second storing unit via the ingestion layer, the received data stream and resuming, by the transmitting unit via the ingestion layer, upon restoration of the data transfer session, transmission of the received data stream, wherein resuming the transmission of the data stream is based on the set of metadata.

[0091] Yet another aspect of the present invention relates to a user equipment (UE) for achieving zero data loss in an event of network failure. The UE comprising a processor to receive a data stream from a source device

[0101] over a data transfer session between the source device

[0101] and a destination device

[0103] ; transmit the received data stream to the destination device

[0103] , wherein for achieving zero data loss in an event of network failure the process comprises: storing a set of metadata corresponding to each instant of the receiving and the transmitting of the data stream successfully; determining failure in the data transfer session; storing the received data stream; and resuming upon restoration of the data transfer session, transmission of the received data stream, wherein resuming the transmission of the data stream is based on the set of metadata.

[0092] Various advantages of the present disclosure include:

[0093] Zero loss of data in case of any network failures (connectivity loss) between the source and / or destination system.

[0094] The ingestion layer eliminates the need of manual intervention involved in making the data available for the time of network failure occurrence.

[0095] The Audit Scheduler Module allows the data transfer to done at pre-defined regular intervals or on-demand which synchronizes data between source and destination, thereby avoiding any possibility of losing data.

[0096] Reduction in overall time required for transfer of data in case of network failure during transfer.

[0097] Further, in accordance with the present disclosure, it is to be acknowledged that the functionality described for the various components / units can be implemented interchangeably. While specific embodiments may disclose a particular functionality of these units for clarity, it is recognized that various configurations and combinations thereof are within the scope of the disclosure. The functionality of specific units, as disclosed in the disclosure, should not be construed as limiting the scope of the present disclosure. Consequently, alternative arrangementsand substitutions of units, provided they achieve the intended functionality described herein, are considered to be encompassed within the scope of the present disclosure.

[0098] While considerable emphasis has been placed herein on the disclosed embodiments, it will be appreciated that many embodiments can be made and that many changes can be made to the embodiments without departing from the principles of the present disclosure. These and other changes in the embodiments of the present disclosure will be apparent to those skilled in the art, whereby it is to be understood that the foregoing descriptive matter to be implemented is illustrative and non-limiting.

Claims

We Claim:

1. A method for achieving zero data loss in an event of network failure, the method comprising: receiving, by a receiving unit [102] via an ingestion layer [100 A], a data stream from a source device [101] over a data transfer session between the source device [101] and a destination device [103]; transmitting, by a transmitting unit [104] via the ingestion layer [100A], the received data stream to the destination device [103]; storing, by a first storing unit [106] via the ingestion layer [100 A], a set of metadata corresponding to each instant of the receiving and the transmitting of the data stream successfully; determining, by a determining unit [108], failure in the data transfer session; storing, by a second storing unit [110] via the ingestion layer [100 A], the received data stream; and resuming, by the transmitting unit [104] via the ingestion layer [100A], upon restoration of the data transfer session, transmission of the received data stream, wherein resuming the transmission of the data stream is based on the set of metadata.

2. The method as claimed in claim 1, wherein the data transfer session utilizes at least one protocol selected from a group consisting of Secure Shell (SSH), Secure File Transfer Protocol (SFTP), Transmission Control Protocol (TCP), and Internet Protocol (IP).

3. The method as claimed in claim 1, wherein the method comprises performing, by an audit unit [112], an audit on the stored set of metadata and the received data stream at the destination device periodically after a pre-defined period of time to determine a delta between the stored set of metadata and the received data stream at the destination device.

4. The method as claimed in claim 3, comprises synchronizing, by a synchronizing unit, the received data stream at the destination device [103] and the received data stream based on the determined delta.

5. The method as claimed in claim 3, further comprising auditing, by the audit unit [112], the stored set of metadata and the received data stream at the source device [101] to ensure consistency between the source [101] and destination devices [103],6. The method as claimed in claim 5, comprises rendering, by a display unit, a user interface on a display device to receive a user input to trigger the audit.

7. The method as claimed in claim 1, wherein the determination of the data transfer session failure includes monitoring for at least one of a network failure or a disk failure between at least one of the source device [101] and the ingestion layer [100A], or the ingestion layer [100 A] and the destination device [103],8. A system [100] for achieving zero data loss in an event of network failure, the system [100] comprising: an ingestion layer [100A] comprising: a receiving unit [102] configured to receive a data stream from a source device [101] over a data transfer session between the source device [101] and a destination device [103]; a transmitting unit [104] configured to transmit the received data stream to the destination device; a first storing unit [106] configured to store a set of metadata corresponding to each instant of the receiving and the transmitting of the received data stream successfully; a determining unit [108] configured to determine failure in the data transfer session; a second storing unit [110] configured to store the received data stream; and the transmitting unit [104] configured to resume transmission of the received data stream, wherein resuming the transmission of the data stream is based on the set of metadata upon restoration of the data transfer session.

9. The system as claimed in claim 8, wherein the data transfer session utilizes at least one protocol selected from a group consisting of Secure Shell (SSH), Secure File Transfer Protocol (SFTP), Transmission Control Protocol (TCP), and Internet Protocol (IP).

10. The system as claimed in claim 8, wherein an audit unit [112] is configured to perform audit on the stored set of metadata and the received data stream at the destination device periodically after a pre-defined period of time to determine a delta between the stored set of metadata and the received data stream at the destination device [103],11. The system as claimed in claim 10, wherein a synchronizing unit [114] is configured to synchronize the received data stream at the destination device [103] and the received data stream based on the determined delta.

12. The system as claimed in claim 10, wherein the audit unit [112] is further configured to audit the stored set of metadata and the received data stream at the source device [101] to ensure consistency between the source and destination devices [103],13. The system as claimed in claim 12, wherein a display unit [116] is configured to render a user interface to receive a user input to trigger the audit.

14. The system as claimed in claim 8, wherein the determination of the data transfer session failure comprises monitoring for at least one of a network failure or a disk failure between at least one of the source device [101] or the ingestion layer [100A], and between the ingestion layer [100 A] and the destination device [103],15. A user equipment (UE) for achieving zero data loss in an event of network failure comprising: a processor configured to: receive a data stream from a source device [101] over a data transfer session between the source device [101] and a destination device [103];- transmit the received data stream to the destination device [103], wherein for achieving zero data loss in an event of network failure the process comprises: o storing a set of metadata corresponding to each instant of the receiving and the transmitting of the data stream successfully; o determining failure in the data transfer session; o storing the received data stream; and o resuming upon restoration of the data transfer session, transmission of the received data stream, wherein resuming the transmission of the data stream is based on the set of metadata.

16. A non-transitory computer-readable storage medium storing instruction for achieving zero data loss in an event of network failure, the storage medium comprising executable code which, when executed by one or more units of a system, causes:a receiving unit [102] to receive a data stream from a source device [101] over a data transfer session between the source device [101] and a destination device [103]; a transmitting unit [104] to transmit the received data stream to the destination device; a first storing unit [106] to store a set of metadata corresponding to each instant of the receiving and the transmitting of the received data stream successfully; a determining unit [108] to determine failure in the data transfer session; a second storing unit [110] to store the received data stream; and the transmitting unit [104] to resume transmission of the received data stream, wherein resuming the transmission of the data stream is based on the set of metadata upon restoration of the data transfer session.

Images