Computer-implemented method and system for determining the origin of a data set
The computer-implemented process and system enhance data traceability by adding origin information entries at each data interface in digital processes, addressing the challenge of tracing data origins and transformations in complex digital architectures.
Patent Information
- Application Number
- DE102023210941
- Authority / Receiving Office
- DE · DE
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-11-03
- Publication Date
- 2025-05-08
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In digital processes, tracing the origin and intermediate transformations of data records is challenging due to complex architectures and the lack of intermediate data versions, making it difficult to identify issues with data quality or processing speed.
A computer-implemented process and system that determine the origin of a data record by adding original information entries at each data interface along a data path, creating a traceable data record that includes the origin information of each module involved in the data processing.
This approach significantly improves data traceability, simplifies troubleshooting and analysis, reduces costs and resources, and allows for better understanding and reproduction of data processing workflows.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
[0001] The invention relates to a computer-implemented method and a system for determining the origin of a data set.
[0002] In general, digital processes involve the creation, processing, and transmission of data. These processes encompass various methods, particularly data creation, editing, and similar operations, which transform input data into desired output data. For example, a camera can capture an image, and the associated image data set can be processed using various sequential editing techniques. This often occurs automatically. The output data provided to the user can be an edited image, which might, for instance, exhibit improved contrast.
[0003] The methods, networks, computers, and processors used in digital processes can have a complex architecture. If, for example, the output data is not of the desired quality or is generated too slowly, finding the responsible part of the architecture can prove difficult. Furthermore, usually only the output data is provided, meaning that intermediate versions between input and output data are not available.
[0004] It is an object of the present invention to provide a computer-implemented method and a system that at least improve upon or even eliminate one or more of the aforementioned disadvantages. In particular, it is an object of the present invention to make a process traceable from the input data to the output data.
[0005] The task is solved, according to a first aspect, by a computer-implemented method for determining the origin of a data record. The method comprises determining a first data interface between two modules of a first predetermined data path, wherein the first module transmits a first data record to the second module via the first data interface. The method further comprises adding origin information with a first entry to the first data record, where the first entry points to the first module as the origin of the first data record. The method further comprises creating, using the second module, a second data record based on the first data record, which differs at least partially from the first data record and contains the origin information with the first entry.
[0006] Using the proposed method, an entry for each module can be added to one, several, or all interfaces along a data path, indicating that module is the origin. Consequently, after traversing multiple modules, the origin information provides clues as to which modules have been involved so far. As described below, the entries can characterize various pieces of information, which are user-defined and / or predefined. This significantly improves data traceability. Furthermore, troubleshooting and analysis are simplified, and costs and resources are reduced.
[0007] The first data record, the second data record, and / or any subsequently introduced data records can comprise the source information and characterize content information. For example, in the case of an image capture, the content information can characterize image content information, particularly color data for the camera pixels. The second data record differs at least partially from the first data record, with the content information differing at least partially. This can also apply to at least one of the subsequently introduced data records. The source information can be modified by adding a second entry to the source information when the second data record is transferred to a subsequent module. Consequently, a distinction can be made between the source information and the content information for a given data record.
[0008] Multiple and / or all data interfaces along the first data path and / or each subsequent data path can be defined. Furthermore, entries from the transmitting modules can be added to the source information according to these data interfaces.
[0009] The first data path and / or each subsequent data path can characterize a sequence of modules designed to create and / or edit respective data records. A data path is not limited to one-dimensional transmission; that is, a transmitting module can send a data record not only to one receiving module, but to two or more receiving modules. A data path can have a different origin than another data path and / or terminate within another data path and / or intersect it. For example, multiple data paths can share a common module and otherwise contain different modules.
[0010] Creating a dataset can involve editing and / or generating that dataset. For example, the first module could be a camera that creates the first dataset in the form of a captured image and transfers it to the second module. The second module could be designed to segment the image and consequently create a segmented dataset based on the image.
[0011] Adding an entry to the original information can occur before creating the data record to be generated by a subsequent module. For example, the entry for the first unit can be added to the original information before the second data record is created.
[0012] The method can further include determining a second data interface between the second module and a third module of the first data path or a second predetermined data path, wherein the second module transmits the second data record to the third module via the second data interface. The method can further include adding a second entry to the source information of the second data record, wherein the second entry points to the second module as an source of the second data record. The method can further include creating, by means of the third module, a third data record based on the second data record, wherein the third data record is at least partially different from the first and second data records, and wherein the third data record contains the source information with the first and second entries.The procedure can further include the creation, using the third module, of a third data set that is at least partially different from the first and second data sets, based on the second data set, wherein the third data set contains the original information with the first and second entries.
[0013] Accordingly, the original information can include the entries of the involved modules of the respective data paths, thus significantly improving traceability.
[0014] The procedure can be designed such that, for predetermined data interfaces, a respective entry of the transmitting module is added to the original information.
[0015] The method can further include determining a fourth data interface between the second or third module and a fourth module of the first, second, or a third predetermined data path, wherein the fourth module transmits a fourth data record to the second or third module via the fourth data interface. The first through third data paths can be at least partially different from each other. The method can further include adding a fourth entry to the origin information of the fourth data record, wherein the fourth entry points to the fourth module as an origin of the fourth data record.The creation of the second or third data set may further be based on the fourth data set, and the second or third data set based on the fourth data set may further be at least partially different from the fourth data set, wherein the second or third data set further based on the fourth data set further includes the source information with the fourth entry.
[0016] If the fourth data set was transmitted to the second module, the second data set may include the source information with the first, second, and fourth entries when the second data set is transmitted to the third module. The third module may be configured to create one or the third data set based on the second data set. The method may further include adding a third entry to the source information of the third data set, such that the source information of the third data set includes the first to fourth entries.
[0017] The preceding and subsequent designations—first to fourth data interfaces, first to fifth modules, first to fifth entries, and first to fifth data records—can be understood as unique identifiers for the respective data interface, module, entry, and data record. For example, a third entry, depending on the implementation, can imply that it is already the third entry in the source information. Alternatively, depending on the implementation, the third entry in the source information can be the only entry or, considered in total, for example, the second or fourth entry when referencing the third module.
[0018] Consequently, entries from different data paths, as well as the data records from different data sets, can be combined. The resulting combined data set can therefore significantly improve traceability across multiple data paths. Any combination of modules and / or data paths can be referenced as the origin using the entries in the source information.
[0019] The method can further include determining a third data interface between the third module and a fifth module of the data path of the third module, wherein the third module transmits the third data record to the fifth module via the third interface. The method can further include adding a third entry to the origin information of the third data record, wherein the third entry points to the third module as an origin of the third data record. The method can further include creating, by means of the fifth module, a fifth data record based on the fourth data record, wherein the fifth data record is at least partially different from the third data record, and wherein the fifth data record contains the origin information with the first three entries or with the first four entries.Alternatively or additionally, the procedure can further include the creation, using the third module, of a third data set that is at least partially different from the first and second data sets, based on the second data set, wherein the third data set contains the original information with the first and second entries.
[0020] At least one of the first five entries can further characterize at least one of the following: the origin of the respective data record, the time of transmission, reception, and / or creation of the respective data record, the duration of transmission and / or creation of the respective data record, at least one variable used to create the respective data record, a method for creating the respective data record, at least one module system parameter of the respective module, diagnostic information of the respective module relating to the respective data record, and / or at least user information of a user of the respective module. The origin can be a MAC address and / or another identifier of the respective module, in particular a unique identifier. One of the methods for creating the respective data record can be an image processing method, for example, a segmentation method.The module system parameter can characterize at least one system parameter of the module that was measured and / or determined during the creation of the respective data set with this module, for example, processor load, memory utilization, etc. The diagnostic information can characterize a diagnostic result from the creation of the respective data set of the module, for example, the color distribution of an image, brightness, blurriness, erroneous data, etc. If the data sets are image data sets, at least one of the first five entries can further characterize a frame processed by the respective module. The type of information in the first five entries can be determined based on user input and / or be predefined.
[0021] The first, second, third, fourth, and / or fifth entries can identify at least partially, and in particular completely, identical and / or different types of information. Consequently, for example, all entries can specify an origin, a method used to create the respective data record, and a processor load. This allows for a subsequent comparison of the methods and processor loads of the different modules. In particular, the type of information can be predetermined and / or predetermined via user input.
[0022] Adding at least one of the first five entries can further include storing the data record associated with and / or to be overwritten by that entry in the original information. This makes it possible later to simulate the exact same state of a system with the respective data paths, without having to reuse the modules and data paths. Accordingly, processes can be easily reproduced. Furthermore, various measures can be taken, such as modifying a module, and subsequently tested to see if this leads to an improvement in the system.
[0023] The original information, in particular at least one entry (here, one of the first five entries) and / or the data record stored in the original information and belonging to that entry (here, one of the first five data records), can be retained when creating the respective subsequent data record. In other words, the content information of the respective data records can be at least partially modified when creating a data record based on that data record, while the original information, especially the entries, is retained.
[0024] At least one of the first to fifth modules can be one of a computer-implemented algorithm, a computer-implemented method for creating the respective data set, a device, a processor, and / or a network node of a network. In particular, the modules and data paths can be such modules and data paths of a complex system, so that the proposed invention enables traceability of the data origin.
[0025] At least some of the first five data sets can be image data sets. Additionally or alternatively, at least some of the first five modules can be devices and / or methods designed to create and transmit the respective image data sets.
[0026] The method may further include determining the origin of at least one of the first to fifth data records based on the origin information of the at least one data record, and in particular, the method may further include outputting the origin and / or the origin information to a user.
[0027] According to a second aspect, the task is solved by a system for determining the origin of a data record. The system can be a closed system, such as an internal system of a company. Alternatively or additionally, the system can be a network system, for example, with multiple computers. Alternatively or additionally, the system can include cloud applications. The system includes a means for determining a first data interface between two modules of a first predetermined data path, wherein the first module transmits a first data record to the second module via the first data interface. The system further includes a means for adding origin information with a first entry to the first data record, wherein the first entry indicates the first module as the origin of the first data record.Furthermore, the system includes a means of creating a second data set that is at least partially different from the first data set, based on the first data set, with the second data set containing the original information with the first entry.
[0028] The aforementioned means may be one or more processors, devices, computers, and / or network nodes. Consequently, the system may be implemented, for example, in the form of a single computer or two or more computers. Two or more computers may communicate with each other via wireless and / or wired communication, for example, via a network. The means may comprise the respective modules and / or have them stored in a memory of the means.
[0029] The system may further include the first and second modules, and / or the first data path. The system may include additional modules and / or data paths, in particular those described for the procedure according to the first aspect.
[0030] Process features described in relation to the first process can be implemented as device features of the system according to the second aspect and vice versa.
[0031] According to a third aspect, the task is solved by a computer program product comprising instructions which, when the program is executed by a system according to the second aspect, cause it to execute the procedure according to the first aspect.
[0032] Preferred embodiments are explained by way of example with reference to the accompanying figures. These show: Fig. 1 a schematic embodiment of a computer-implemented method for determining the origin of a data set; Fig. 2 a schematic embodiment of the method with first to third modules; Fig. 3 a schematic embodiment of the method with first to fourth modules; Fig. 4 a schematic embodiment of the method with modules one to four and storage of predetermined data sets; and Fig. 5 a schematic embodiment of a system for determining an origin.
[0033] In the figures, identical or essentially functionally identical or similar elements are designated by the same reference numerals.
[0034] Fig. 1 shows a schematic embodiment of a computer-implemented method 100 for determining an origin of a data set. The method 100 may be stored in the form of a computer program product, which, when executed by one or more processors and / or a system 200 according to the Fig. 5 cause these to carry out the procedural steps of procedure 100.
[0035] The method 100 comprises determining 110 a first data interface DS1 between two modules M1, M2 of a first predetermined data path DP1, wherein the first module M1 transmits a first data set via the first data interface DS1 to the second module M2, see for example Fig. 2.
[0036] The first data path DP1, as well as each additional data path, can determine a sequence of modules for each creation of a data set. Each of the modules can be one of a method, a device, a processor, and / or a network node, each configured to create respective data sets based on input data. The input data can be one or more data sets from previous modules. Respective data interfaces are provided between the modules to transfer the data sets from one module to the next.
[0037] In complex systems with various data paths and various modules along the respective data paths, a resulting final data set can be output at the end, which is based on the previous editing, creation, and / or processing of the modules of the data paths. In order to be able to trace how this final data set came about, the method 100 further comprises adding 120 a piece of source information U with a first entry E1 to the first data set, wherein the first entry E1 indicates the first module M1 as an origin of the first data set. In this case, the first entry E1 can, in particular, characterize further relevant information of the first module M1 in order to improve traceability.For example, the first entry E1 can contain a unique identifier for module M1, such as a MAC address, a method used to create the first data record, one or more variables used to create the first data record, and other information. This type of information can be predefined and / or specified by a user via user input.
[0038] Procedure 100 further comprises the creation 130, using the second module M2, of a second data record based on the first data record, which differs at least partially from the first data record and contains the original information U with the first entry E1. Although the second data record is now available, the information from the first module M1, deemed relevant and predetermined by the user, can be read using the first entry E1.
[0039] According to the Fig. 2. A data path can have two or more modules, in this case three or more modules M1-3. The second module M2 is configured to transfer the second data record to a third module M3 via a specific second data interface DS2. A second entry E2 is added to the origin information U at the second data interface DS2, indicating module M2 as the origin. The second entry E2 can contain the same type of information and / or at least partially different types of information compared to the first entry E1. Consequently, one type of information, such as origin, applied method, processor load, and / or memory utilization, can be identical in the entries, but the specified MAC addresses of the respective origins, the applied methods, the processor load, and / or the memory utilization can differ.
[0040] Module M3 is further configured to create a third data set and can transmit it via a specific third data interface DS3 to another downstream module along the first data path DP1. Here, too, another entry, in this case the third entry E3, is added to the source information U, which indicates module M3 as the source.
[0041] Consequently, using the source information U of the third data set, further inferences can be made about the modules M1 and M2 that influence the third data set. This can simplify or even enable an analysis of the first data path DP1.
[0042] According to the Fig. 3 shows a second data path DP2 with a fourth module M4. The invention is not limited thereto, and the second data path DP2 may have more modules. Fig. Figure 3 shows that there is a fourth interface between the fourth module M4 and the third module M3. Consequently, the third module M3 is configured to receive the second data record from the second module M2, containing the origin information U and entries E1 and E2, and also to receive a fourth data record from the fourth module M4 via the specified fourth data interface DS4. The entries E1, E2, and E4 of the second and fourth data records can be combined by the third module M3, so that the origin information U contains the first, second, and fourth entries E1, E2, and E4. The entry E4 indicates the module M4 as the origin. If the third data record is transmitted via the third data interface DS3, the third entry E3 can be added to the origin information.
[0043] Consequently, arbitrarily complex system architectures for processing and / or creating data sets can be designed in a comprehensible manner using the proposed invention.
[0044] Fig. Figure 4 shows an advantageous embodiment in which, in addition to entries E1 to E4, the first and fourth data records are also stored in the original information U. Consequently, the state of these data records can be preserved, and simulations can be performed subsequently without a complete re-run of this system architecture. This can be advantageous if, for example, one or more modules are modified. Thus, a simple analysis can be performed to determine whether an improvement in the system architecture has been achieved.
[0045] The illustrated embodiments show that a respective entry is added at each data interface between two modules, although the invention is not limited thereto. The data interfaces at which a respective entry is added to the source information U can be predefined and / or set based on user input. For example, only the first and third entries E1, E3 may be of interest, so that no entry is added for the second module of the source information U.
[0046] Fig.Figure 5 shows a system 200 for determining the origin of a data record. The system 200 comprises a means 210 for determining a first data interface DS1 between two modules M1 and M2 of a first predetermined data path DP1, wherein the first module M1 transmits a first data record to the second module M2 via the first data interface DS1. The system 200 further comprises a means 220 for adding origin information U with a first entry E1 to the first data record, wherein the first entry E1 indicates the first module M1 as an origin of the first data record. The system 200 further comprises a means 230 for creating a second data record based on the first data record, which is at least partially different from the first data record, and wherein the second data record contains the origin information U with the first entry E1.
[0047] The system 200 further includes an optional memory 240 configured to store information, data sets, and / or the computer program product.
[0048] The means 210-230 can be individual processors, devices, virtual computers, and / or virtual cores, each configured to execute one or more of the modules M1-M4 and / or comprising the modules M1-M4. For example, a single computer can be used to process a data set characterizing an image. The respective processing steps can be executed on this single computer using various modules in the form of different procedures, program sections, and / or programs. Consequently, the computer could comprise only one processor to execute procedure 100. Alternatively, different modules can be executed, at least partially, by different processors, devices, virtual computers, and / or virtual cores of other computers and / or networks. Reference symbol 100 Computer-implemented method for determining an origin of a data set 110 Determining a first data interface 120 Adding source information with a first entry 130 Creating a second record and adding it to the first record M1-M4 first to fifth modules U Origin information E1-E4 first to fifth entries DS1-DS4 first to fifth interfaces DP1 first data path DP2 second data path 200 systems 210 Means for determining a first data interface 220 Means of adding origin information 230 means of creating a data set 240 storage
Claims
[1] A computer-implemented method (100) for determining an origin of a data set, comprising: Determining (110) a first data interface (DS1) between two modules (M1, M2) of a first predetermined data path (DP1), wherein the first module (M1) transmits a first data set to the second module (M2) via the first data interface (DS1); Adding (120) an item of origin information (U) with a first entry (E1) to the first data set, wherein the first entry (E1) indicates the first module (M1) as an origin of the first data set; Creating (130), by means of the second module (M2), a second data set which is at least partially different from the first data set based on the first data set, wherein the second data set has the original information (U) with the first entry (E1). [2] The method (100) of claim 1, further comprising: Determining a second data interface (DS2) between the second module (M3) and a third module (M3) of the first data path (DP1) or a second predetermined data path (DP2), wherein the second module (M2) transmits the second data set to the third module (M3) via the second data interface (DS2); Adding a second entry (E2) to the origin information (U) of the second data set, wherein the second entry (E2) points to the second module (M2) as an origin of the second data set; Creating, by means of the third module (M3), a third data set based on the second data set, wherein the third data set is at least partially different from the first and second data sets, wherein the third data set comprises the source information (U) with the first and second entries (E1, E2). [3] Method (100) according to one of the preceding claims, further comprising: Determining a fourth data interface (DS4) between the second or third module (M2, M3) and a fourth module (M4) of the first, the second or a third predetermined data path (DP1, DP2), wherein the fourth module (M4) transmits a fourth data set to the second or third module (M2, M3) via the fourth data interface (DS4); Adding a fourth entry (E4) to the origin information (U) of the fourth data set, wherein the fourth entry (E4) points to the fourth module (M4) as an origin of the fourth data set; wherein the creation of the second or third data set is further based on the fourth data set and the second or third data set based on the fourth data set is further at least partially different from the fourth data set, wherein the second or third data set further based on the fourth data set further comprises the source information (U) with the fourth entry (E4). [4] Method (100) according to one of the preceding claims, further comprising: Determining a third data interface (DS3) between the third module (M3) and a fifth module of the data path (DP1) of the third module (M3), wherein the third module (M3) transmits the third data set to the fifth module via the third interface (DS3); Adding a third entry (E3) to the origin information (U) of the third data set, wherein the third entry (E3) points to the third module (M3) as an origin of the third data set; Creating, by means of the fifth module, a fifth data set based on the fourth data set, wherein the fifth data set is at least partially different from the third data set, wherein the fifth data set comprises the source information (U) with the first to third entries (E1-3) or with the first to fourth entries. [5] Method (100) according to one of the preceding claims, wherein at least one of the first to fifth entries (E1-4) further comprises at least one of: an origin of the respective data set, a time of transmission, reception and / or creation of the respective data set, a duration of transmission and / or creation of the respective data set, at least one variable for creating the respective data set, a method for creating the respective data set, at least one module system parameter of the respective module (M1-4), diagnostic information of the respective module (M1-4) relating to the respective data set and / or at least one user information of a user of the respective module (M1-4). [6] Method (100) according to one of the preceding claims, wherein the first entry (E1), the second entry (E2), the third entry (E3), the fourth entry (E4) and / or the fifth entry identify at least partially, in particular completely identical and / or different information, wherein in particular the information is predetermined and / or can be predetermined by means of a user input. [7] Method (100) according to one of the preceding claims, wherein adding at least one of the first to fifth entries (E1-4) further comprises storing the data record associated with the at least one entry (E1-4) in the source information (U). [8] Method (100) according to one of the preceding claims, wherein the original information (U), in particular the at least one entry (E1-4) and / or the data record stored in the original information (U) and associated with the at least one entry (E1-4) is retained when the respective subsequent data record is created. [9] Method (100) according to one of the preceding claims, wherein at least one of the first to fifth modules (M1-4) is one of a computer-implemented algorithm, a computer-implemented method for creating the respective data set, a device, a processor and / or a network node of a network. [10] Method (100) according to one of the preceding claims, wherein at least partially the first to fifth data sets comprise and / or characterize image data sets, and / or wherein at least some of the first to fifth modules (M1-4) are designed in the form of devices and / or methods for creating and transmitting the respective image data sets. [11] Method (100) according to one of the preceding claims, further comprising: Determining an origin of at least one of the first to fifth data sets based on the origin information (U) of the at least one data set, wherein the method (100) further comprises outputting the origin and / or the origin information to a user. [12] A system (200) for determining an origin of a data set, comprising: means (210) for determining a first data interface (DS1) between two modules (M1, M2) of a first predetermined data path (DP1), wherein the first module (M1) transmits a first data set to the second module (M2) via the first data interface (DS1); means (220) for adding source information (U) with a first entry (E1) to the first data set, the first entry (E1) indicating the first module (M1) as an origin of the first data set; means (230) for creating a second data set which is at least partially different from the first data set based on the first data set, wherein the second data set comprises the original information (U) with the first entry (E1). [13] The system (200) of claim 12, further comprising the first and second modules (M1, M2), and / or the first data path (DP1). [14] A computer program product comprising instructions which, when executed by a system (200) according to claim 12 or 13, cause the system (200) to carry out the method (100) according to any one of claims 1 to 11.
Citation Information
Patent Citations
Unique identifier for log message sources in vehicles
DE102022113104A1
Method and apparatus for sending data associated with a data source
DE102022205002A1
Authenticating multimedia data, especially image and / or sound data, involves manipulation-proof combination of data with authentication information with source information item(s) for first source that has previously influenced data state
DE10358144A1
Secured network bridge
EP3152881B1