Information system for monitoring data transport

Abstract

The invention relates to a system for analyzing and supervising data transport traces between applications of an information system. The innovation concerns the analysis of traces along 3 axes: their description at runtime, the mapping of the information system and their temporality. Figure for the abstract: Fig. 1

Description

Title of the invention: Information system for monitoring data transport Technical field of the invention

[0001] The present invention relates to an information system for monitoring data transport. It applies, in particular, to the overall architecture of the information system, including hardware and software components, servers, communication devices, databases, and user interfaces.

[0002] The present invention covers methods and systems for monitoring and supervising data transports, including event detection, error handling, transfer traceability, and report generation.

[0003] The field may include aspects related to data security, such as encryption, authentication, protection against computer threats and attacks, and access management.

[0004] The field may encompass techniques and algorithms for optimizing the efficiency of data transfers, including compression, latency reduction, bandwidth management, and data prioritization.

[0005] The field may also address methods for tracking and recording information from each data transfer, thus enabling full traceability and auditability for compliance and performance management. Thus the field may include aspects relating to seamless integration with different applications and systems, thus ensuring compatibility and interoperability.

[0006] Finally, the domain may encompass aspects related to the user interface of the system, including real-time monitoring functionality, configuration, reporting, and user interaction. Prior art

[0007] Some monitoring solutions are directly integrated into intermediation tools, such as Enterprise Service Bus (ESB), Extract-Transform-Load (ETL), and API Management offered by renowned publishers such as Oracle, IBM, Google and Microsoft, registered trademark. These monitoring consoles provide information on running programs, deployed programs and the status of unit processing.

[0008] Other external solutions focus on collecting logs and metrics from operating systems. These solutions read text files or performance data and are used to monitor the overall health of systems. It is important to note that the content of log files is generally free and is not not necessarily structured in a specific way.

[0009] To track technical transactions, tracking tools are available, whether offered by commercial vendors or in "open source" versions. The use of agents that automatically add "markers" to application behaviors is a common method for linking technical calls with protocol markers. These solutions are grouped under the term "Application Performance Management" (APM).

[0010] It is important to note that these solutions focus primarily on performance and technical dependencies and do not provide functional information or in-depth analysis.

[0011] The many existing solutions are aimed at technology experts and administrators of these tools. Strong IT skills are a prerequisite for using these solutions. Presentation of the invention

[0012] The present invention aims to remedy these drawbacks with a completely innovative approach having as its main objective the reconstruction of the functional and conceptual vision of a data transport between a source application and a set of target applications from technical data.

[0013] The essence of this technology lies in its ability to reconstruct and deduce the initial purpose of data transport as well as its current state, based on the individual transport traces collected.

[0014] In other words, this system, hereinafter referred to as the invention, aims to establish a complete understanding of the data transport process by reconstructing the initial vision of the operation, including the underlying intentions and objectives. It carefully analyzes the information collected during the transport, the processing carried out along the way and the interactions with the different target applications to develop a representation

[0015] These objectives, as well as others which will appear subsequently, are achieved, using an information system for supervising data transport, comprising at least two servers linked together by a communication network, computer client applications connecting to said servers, remarkable in that it comprises: - transport traces having at least the following information from a server - a unique identifier corresponds to a sequence of characters; - a message transported with at least one time information corresponding to a date, a time with milliseconds and a time slot; - a transport route corresponding to a character string defining the processing in charge of the transport; - a current status corresponding to one of the following values: information, success, timeout, alert, and error; - a list of references of the transported data corresponding to a type and an identifier; - a database of transport traces, a database of the exchange mapping of the supervised information system and a database for the exchanges calculated from the traces; - means for processing transport traces and exchanges in the transport traces database and the calculated exchange database, called calculated exchange; said means for processing transport traces and exchanges modifying the status of the exchange calculated according to a server if the transport trace has not arrived at the corresponding server in the exchange database then the current status is modified and one of the following values ​​is modified: information, success, out of time, alert, and error; - means of visualizing transport traces and calculated exchanges.

[0016] The invention is based on a fundamental technical architecture which encompasses several key elements, each of which is of capital importance for achieving efficient supervision of data transport.

[0017] Connecting servers to each other through a communication network is fundamental to establishing a robust communication infrastructure. This network allows servers to exchange data, coordinate operations, and ensure the consistency of transmitted information. It also offers the possibility of two-way communication between servers and client applications.

[0018] Computer client applications play a vital role in the use of the system by end users. These applications allow users to connect to servers and interact with the system to monitor and manage data transport. The presence of these computer client applications responds to functionally rendered services and guarantees a user-friendly and intuitive interface for system operators.

[0019] The invention thus offers a new perspective on data transport by allowing users to better understand information flows, to identify possible deviations from the initial objective and to make more informed decisions regarding the management and optimization of this transport.

[0020] The invention represents a significant advance in the field of monitoring and analysis of data transfers, opening up new possibilities for improve the efficiency, quality and security of communication operations between applications.

[0021] The main objective of the invention is the supervision of data transport. The combination of servers, the communication network and client applications makes it possible to monitor data flows in real time, identify potential problems, generate alerts and facilitate decision-making. This supervision function is crucial to guarantee the performance, security and reliability of data transfers.

[0022] The database of transport traces received by the invention makes it possible to store in an organized and structured manner all the information relating to the transport traces, while the exchange database defines the type of each transport trace. This organization guarantees complete traceability and allows precise management of the transported data.

[0023] The means for processing transport traces and exchanges provide an automated processing function, making it possible to track the status of transport traces and take appropriate action based on their progress. Automating this process improves efficiency and reduces the possibility of human error.

[0024] The processing means allow real-time monitoring of data transports. If a transport trace does not reach the corresponding server in the exchange database, this may indicate a problem or a deviation from the planned route. Changing the current status, for example by replacing "success" with "error", allows potential problems to be quickly identified and corrective actions to be initiated.

[0025] The current status deduction feature is innovative. It deduces the status of functional exchanges, “success”, “error”, “alert”, “information” or “out of time”, in relation to the states of the transport traces received by the servers. This feature helps improve the quality of service by categorizing anomalies and data transfer incidents. This allows for a rapid response to minimize negative impacts on the system.

[0026] The functionality of automatically replacing the current status with “out of time”, when a transport trace does not reach the corresponding server within the given time, helps to improve the quality of service by quickly reporting shortages and missing processing within the monitored system. This allows a rapid reaction to minimize negative impacts on the system.

[0027] The invention finds application in various complex computing environments, including those involving sensitive data transfers, distributed systems, or large-scale networks. The ability to adapt the current status of the exchange based on the statuses of the numerous transport traces is a technical feature that enhances the robustness and usefulness of the system in various scenarios.

[0028] The invention is advantageously implemented according to the embodiments and variants set out below, which are to be considered individually or according to any technically effective combination.

[0029] In one embodiment, the system references the exchanges with the characteristics of the source applications, the target applications and its allocated time in a mapping database.

[0030] In one embodiment, the system assigns the transport traces to elements called Mediation, defined by at least one Mediation name, and a Mediation time limit, and represented in the mapping database as well as in the calculated exchange database as calculated Mediation, using the route first and then the transported data first.

[0031] In one embodiment, the system assigns Mediations to elements called Flows, defined by at least one Flow name, and a Flow deadline, represented in the mapping database, as well as in the database of calculated exchanges as a calculated Flow, according to the dependencies defined in the mapping database, and the Mediation deadlines.

[0032] In one embodiment, the system assigns the calculated Flows to calculated exchanges according to the dependencies defined in the mapping database, and the given Flow deadlines.

[0033] In one embodiment, the status of the calculated Mediation is determined according to the status of the oldest affected transport trace, said traces ordered on a time axis relative to a clock of a microcontroller used to generate the transport trace.

[0034] According to embodiments, the status of the calculated Flow is determined according to the statuses of each assigned calculated Mediation. If all the calculated Mediations have a successful status, the calculated Flow has a successful status. If one of the calculated Mediations has an alert status, the calculated Flow has an alert status. If one of the calculated Mediations has an error status, the calculated Flow has an error status. The status of the calculated Exchange is determined according to the statuses of each assigned calculated Flow. If all the calculated Flows have a successful status, the calculated Exchange has a successful status. If one of the calculated Flows has an alert status, the calculated Exchange has an alert status. If one of the calculated Flows has an error status, the calculated Exchange has an error status. Brief description of the figures

[0035] Other advantages, aims and characteristics of the present invention emerge from the following description given, for explanatory and in no way limiting purposes, with regard to the attached drawings, in which:

[0036] [Fig.l] represents a block diagram of an information system;

[0037] [Fig.2] represents an example of transporting data XY to B and C;

[0038] [Fig.3] represents the operation for emitting traces;

[0039] [Fig.4] represents a schematic diagram with the operation with bases of data. Description of the embodiments

[0040] [Fig.l] shows a block diagram of an information system composed of 3 applications: A, B and C.

[0041] This system allows the efficient and secure movement of data generated, xy, by a source application, A, to 2 target applications: B and C.

[0042] The data transport process involves several essential steps, including the generation of content by the source application in the form of a file or message. This data is then routed to dedicated intermediary applications, which play a key role in managing this information.

[0043] Intermediation applications are responsible for reading, adapting, transforming, and distributing content between other intermediation applications, as well as to the final destination applications. This mediation process ensures that data is presented in a manner appropriate for use by the target applications, whether similar or different from the source application. These applications are represented in [Fig.2].

[0044] There are technical products that are exchanged on a site. Application A then transfers data xy to application B and C with ids such as 0001, 0002, 0003, etc.

[0045] The expected result is a list of customer and product data exchanges between applications A, B and C.

[0046] List of executed exchanges:

[0047] 20230731 10:34: A -> B: data: xy, id:0001 successful

[0048] 20230731 10:35: A -> B: data: xy, id:0002 in error

[0049] 20230731 10:35: A -> C: data: xy, id:0002 successful

[0050] Naturally, the invention is described in the foregoing by way of example. It is it is understood that a person skilled in the art is able to carry out different variant embodiments of the invention without departing from the scope of the invention.

[0051] [Fig.2] shows an example of XY data transport design from application A to applications B and C.

[0052] The transport is organized in a so-called “Y” flow. That is to say in 3 distinct stages, 1 for the extraction of data from its source format, from A. Then 2 for the pro- paging data to their target format, to B and C.

[0053] Each of these steps is supported by technological bases, which can be different. In this example: an ETL (acronym for: Extract-Transform-Load in English for Extract the transformation and load), then an ESB (acronym for: Enterprise Services Bus in English for Enterprise Services Bus), then an API Management (acronym for: Application Programming Interface Management in English for Application Programming Interface Management).

[0054] [Fig.3] shows an example of implementation of XY data transport from application A to applications B and C.

[0055] This figure makes it possible to transcend the technical and often arduous nature of data transport and consequently of its tracking by transport traces. The invention translates them into a more accessible concept: an exchange of data between different applications, such as in [Fig.l]. This simplification process is essential to allow users, whether experts or not, to quickly and easily grasp the meaning of transport traces and to integrate them into their overall understanding of the operation of computer systems.

[0056] Four individual processes or functions are implemented to transport “xy”:

[0057] fl: generation of the file, by an ETL platform.

[0058] f2: reading the file then separating it into 1 message per line, by an ETL platform.

[0059] f3: output no. if to B: reading of the message, by an ESB, then call of a service HTTP, acronym for Hypertext Transfer Protocol for standard communication protocol, exposed by an API Management

[0060] f4: output n° s2 to C: reading of the message, by an ESB, then generation of a file.

[0061] [Fig.3] shows the operation for generating transport traces.

[0062] The format of the data ingested by this trace management system, commonly referred to as "transport traces", represents the first essential element of the system. These traces are generated by various intermediary applications, illustrated here by applications fl to f4, covering the families commonly encountered in the transport field. At the output of the applications there may be fi files, or mes or m messages. The objective is to have one or more t traces at the output of the application.

[0063] The transport trace format is structured into five individual blocks, each playing a specific role in managing and representing this data.

[0064] These five blocks are as follows:

[0065] Generality Block: This block contains crucial information about the trace, including the transport status, which can be classified as success, error, alert or information. Additionally, it may contain information about the environment where the trace was produced, such as "Production" or "Recipe".

[0066] "Message" block: In this block, we record information such as the date of support, including the date, time, milliseconds and time slot. In addition, technical headers can be included, as can the body of the message. Each message is also associated with a so-called correlation identifier, optional but recommended, and a unique identifier, a free but certainly unique character string for each message.

[0067] "Route" block: This block specifies the details of the route taken by the message. It can include the name of the process, its version and its unique identifier, a free but unique character string for each transport process, whether it is a mediation or a function.

[0068] "Business" block: For each piece of data transported, this block records its type and identifier. This allows for a conceptual but precise definition of the data handled during the transport process.

[0069] "Exception" block: When an error occurs during processing, this block is used to describe in detail the error encountered. It includes a unique code, the type of the class involved, the details of the error and possibly the call stack of the code.

[0070] "Infrastructure" block: This block contains information relating to the infrastructure, such as the identifier of the server where the processing took place and the identifier of the instance where the processing took place.

[0071] "Error" block: Finally, the last block is the error block, which records the code associated with the error that occurred and an explanatory message.

[0072] This five-block structure ensures a complete and consistent representation of transport traces, which facilitates their management and analysis within the system.

[0073] Within the system of the invention, four matching processes are implemented to result in the discovery of an exchange:

[0074] Processing by transported business data,

[0075] Processing by mapping the information system concerned,

[0076] A treatment by the temporal concordance of the traces between them,

[0077] A treatment by the management of a correlation identifier.

[0078] [Fig.4] shows a block diagram with the operation with databases.

[0079] We find the treatments fl to f4, of [Fig.3], which generate traces t to tn.

[0080] These traces, tl to tn, are then ingested by a trace database bdd tn to be analyzed by one or more R reconciliations to constitute a database bdd E of E exchanges.

[0081] The rapprochement by mapping:

[0082] Information system mapping provides a set of crucial elements that play a determining role in the organization and effective management of the information system. These elements include:

[0083] Application Repository: This repository lists the different applications involved in the system, thus identifying the software, platforms, and application modules necessary for its operation.

[0084] Flow Repository: This repository reveals the complete mapping of data and information flows within the system, thus allowing precise visualization of the paths taken by data between different parts of the system.

[0085] Mediation Reference: This reference lists the intermediation processes, namely the IT functions executed on servers or dedicated platforms, which play a central role in the transport and transformation of data within the system.

[0086] Data Repository: This repository centralizes all information on the data handled by the system, in particular their structure, format, and semantics, thus facilitating their management and use.

[0087] Service Contract Reference: This reference defines the service contracts that govern the interactions between the system components, thus guaranteeing smooth and reliable coordination.

[0088] What most strongly characterizes the organization of this information system are the close and complex dependencies between the different key elements.

[0089] These dependencies include:

[0090] Intermediation Processing and Flows: Intermediation processing is intrinsically linked to data flows, as it is responsible for the management and transformation of these flows.

[0091] Flows and Applications: Data flows are closely linked to applications, as they transport information between the different applications in the system.

[0092] Flows and Data: Data flows are also dependent on the data they carry, as this data determines their content and structure.

[0093] Intermediation Processing and Service Contracts: Intermediation processing interacts directly with service contracts to ensure execution in accordance with the agreed agreements.

[0094] In order to better organize the elements of representation of a map, we have introduced three fundamental notions:

[0095] Exchange: This is the logical transport of one or more pieces of information between several systems. This concept offers a simplified view of the process.

[0096] Flow: The flow represents the complete technical transport of an application module to a other. It encompasses all technological aspects of this transmission such as formatting, transformation, routing and protocol adaptation. In the literature, it is commonly assimilated to the concept of half-stream.

[0097] Mediation: Mediation is a computer function or process executed on a server or platform provided. Each mediation exposes and uses interfaces and protocols to consume and produce data. It is the smallest identifiable element in the transport chain.

[0098] These notions are interconnected by logical dependencies: exchange -> flow -> mediation.

[0099] According to an example, there is also the reconciliation by Data.

[0100] An essential aspect of the present invention lies in the reconciliation by the transported data, which answers the crucial question of what is transported. However, it is important to note that the types of transported data vary depending on the source and target applications. This is why we have introduced a more abstract notion called Information, necessary for the exchanges. The repository ensures the correspondence between the Data Types and the Information Types.

[0101] Correspondence between Mediation and Exchange

[0102] The transport traces communicated by the processes in charge of part of the transport, the mediations, which manipulate the corresponding data are selected to be part of the same exchange, thus guaranteeing efficient management of the transported data.

[0103] According to an example, there is also the approximation by Correlation.

[0104] A key mechanism for establishing a direct link between mediations is the correlation identifier. An identifier is added to the headers of transported messages, allowing distinct calls to be linked together, which facilitates the management of intermediation processing.

[0105] According to an example, there is also the Temporal rapprochement

[0106] Temporal connections are essential but complex to determine. It is imperative to define a maximum execution time for each concept: exchange, flow and mediation. Based on their dependencies and these maximum deadlines, the traces are brought together in sets which execute within an acceptable gap, thus ensuring optimized time management.

[0107] There then exist the dependencies of the Rapprochements

[0108] Following successive reconciliations in accordance with the steps described, the exchanges are discovered and made available. Each exchange contains crucial information, including its state, its source and target applications, as well as the data transported.

[0109] It is emphasized that all the characteristics, as they emerge for a A person skilled in the art from this description, the drawings and the attached claims, even if concretely they have only been described in relation to other determined features, both individually and in any combinations, may be combined with other features or groups of features disclosed herein, provided that this has not been expressly excluded or that technical circumstances make such combinations impossible or meaningless.

Claims

Claims

1. Information system for monitoring data transport, comprising at least two servers connected to each other by a communication network, computer client applications connecting to said servers, characterized in that it comprises: - transport traces having at least the following information from a server: - a unique identifier corresponding to a character sequence; - a message transported with at least one time information corresponding to a date, a time with milliseconds and a time slot; - a transport route corresponding to a character string defining the processing in charge of the transport; - a current status corresponding to one of the following values: information, success, out of time, alert, and error; - a list of references of the transported data corresponding to a type and an identifier;- a database of transport traces, a database of the exchange mapping of the supervised information system and a database for the exchanges calculated from the traces; - means for processing transport traces and exchanges in the database of transport traces and the database of calculated exchanges, called calculated exchange; said means for processing transport traces and exchanges modifying the status of the calculated exchange according to a server if the transport trace has not arrived at the corresponding server in the database of exchanges then the current status is modified and one of the following values ​​is modified: information, success, out of time, alert, and error; - means for viewing transport traces and calculated exchanges.;

2. The system of claim 1, wherein the system references the exchanges with the characteristics of the source applications, the target applications and its time limit in a mapping database.

3. The system of claim 2, wherein the system assigns the transport traces to elements called Mediation, defined by at

4.

5.

6. least one Mediation name, and a Mediation time limit, and represented in the mapping database as well as in the calculated exchange database as calculated Mediation, using the route first and then the transported data first. The system of claim 3, wherein the system assigns Mediations to elements called Flows, defined by at least one Flow name, and a Flow deadline, represented in the mapping database, as well as in the database of calculated exchanges as a calculated Flow, according to the dependencies defined in the mapping database, and the Mediation deadlines. The system of claim 4, wherein the system assigns the calculated Flows to calculated exchanges according to the dependencies defined in the mapping database, and the given Flow deadlines. The system of claim 5, wherein the status of the calculated Mediation is determined based on the status of the oldest affected transport trace, said traces ordered on a time axis relative to a clock of a microcontroller used to generate the transport trace.

Images