Data preservation with augmentation

Abstract

A computer-implemented method for preserving data in a data preservation system 100 comprises receiving, by the data preservation system, first data 102 for preservation, e.g. a text document, image, video, or audio file. The first data is transmitted, by the data preservation system, to a data augmentation system 110 which augments the first data, e.g. by redacting it to remove sensitive information. The data preservation system receives second data 104 from the data augmentation system, the second data comprising one or more data augmentations to the first data. The data preservation system maintains a first set of data permissions 122 for the first data and a second set of data permissions 124 for the second data. The first set of data permissions permits a different level of access than the second set of data permissions. The first and second data are stored in the data preservation system for later retrieval.

Description

Technical Field

[0001] The following relates to computer-implemented methods for preservation of digital data and in particular to augmenting data for preservation. Background

[0002] As of 2023, it had been estimated that 90% of the world’s data was generated in the preceding two years and that 330 million Terabytes of data is created each day. The overwhelming majority of that data is stored digitally. In addition to the rate of data generation, the environment in which that data is stored is highly dynamic. New file formats are regularly created to encode data (such as the HEIC image format) and existing formats may be updated to improve performance or address faults. Some information requires several digital files to completely and appropriately represent the information contained within. Within this dynamic environment, digital data can become lost or unreadable. Old formats may become unreadable as tools used to create, view or edit those formats become obsolete and fall out of use. Some information may be required for access long after creation, but it is likely that file formats in widespread use at the time of content creation will not be in use at a desired time of access, maybe tens or hundreds of years later.

[0003] While solutions such as storage and back-up can prevent some hardware related issues such as bit-decay (or bit-rot), the dynamic environment in which digital data is produced, stored and edited means that storage and back-up cannot mitigate all causes of loss of access to digital data. Digital preservation refers to activities performed to ensure continued availability of access to digital data and refers to all of the actions required to maintain access to digital materials beyond the limits of media failure or technological and organisational change.

[0004] Digital preservation systems often store data from multiple sources of varying sensitivity. That is, digital preservation systems may store data which is readily available to the public, in addition to strictly confidential documents. It is thus desirable to provide a data preservation system that respects the sensitivity of information preserved within. Summary

[0005] There is described herein a computer-implemented method for automatically preserving data in a data preservation system. The method comprises receiving, by the data preservation system, first data for preservation. The method further comprises transmitting, by the data preservation system, the first data to a data augmentation system configured to augment the first data. The method further comprises receiving, by the data preservation system from the data augmentation system, second data for preservation, the second data comprising one or more data augmentations to the first data. The data preservation system maintains a first set of data permissions for the first data and maintains a second set of data permissions for the second data. The first set of data permissions permits a different level of access than the second set of data permissions. The method further comprises storing, by the data preservation system, the first and second data in the data preservation system for later retrieval.

[0006] The first and second data may comprise text data, image data, video data, and / or audio data.

[0007] The data augmentations may comprise the removal of sensitive information from the first data or the addition of sensitive information to the first data.

[0008] The sensitive information may comprise personally identifiable information, financial information, health information, biometric information, confidential information, legal information, location information, personally generated information, government identification information, and / or employment information.

[0009] The first data and / or the second data may be a document file, an image file, a video file, an audio file, a Microsoft® SharePoint List Record, a Microsoft® Teams Message, a Microsoft® SharePoint Library Record, a Microsoft® Exchange Email, or a Lotus Notes Database Record.

[0010] The data augmentation system may be a system external to the data preservation system.

[0011] The transmitting and the receiving may comprise interfacing with an application programming interface of the data augmentation system.

[0012] The data augmentation system may comprise a Microsoft Azure® service and / or an Amazon Web Services® service.

[0013] The data augmentation system may comprise one or more machinelearning models configured to process the first data for augmentation.

[0014] The second data may be generated by the data augmentation system augmenting the first data. The augmentation may comprise redacting, obscuring, translating, transcribing, enhancing, classifying, and / or captioning the first data to generate the second data.

[0015] The first data may be stored in a first location in the data preservation system different to a second location in the data preservation system.

[0016] The respective locations may be file directories in the data preservation system.

[0017] The sets of data permissions may comprise one or more of read access permissions, write access permissions, delete access permissions, and / or time-limited access permissions.

[0018] The method may further comprise generating link data indicating a link between the first data and the second data. The method may further comprise retrieving, based upon the link data, the first or second data.

[0019] The method may further comprise receiving, by the data preservation system, a request to retrieve the first or second data from the data preservation system. The method may further comprise determining a level of access for the respective first or second data based on the respective first or second set of data permissions. The method may further comprise identifying the respective second or first data preserved in the data preservation system using the link data. The method may further comprise determining a level of access for the respective second or first data based on the respective second or first set of data permissions. Retrieving the first or second data may be based upon the level of access for the respective first or second data.

[0020] The link data may indicate the first and second location of the respective first and second data in the data preservation system. Retrieving the first or second data may be further based upon the respective first or second location indicated by the link data.

[0021] The method may further comprise receiving, by the data preservation system, a request to update the second data. The method may further comprise transmitting, by the data preservation system, the first data to the data augmentation system. The method may further comprise receiving, by the data preservation system from the data augmentation system, an updated version of the second data for preservation. The method may further comprise storing, by the data preservation system, the updated version of the second data.

[0022] The request to update the second data may be based upon identifying one or more errors in the second data.

[0023] The request to update the second data may be based upon identifying an update to the data augmentation system.

[0024] The request to update the second data may be based upon determining that the quality of the second data is below a threshold quality level.

[0025] Storing the updated version of the second data may comprise replacing the second data with the updated version of the second data in the data preservation system.

[0026] The second data may be an augmented version of the first data.

[0027] The second data may comprise metadata corresponding to the first data.

[0028] There is described a computing system comprising one or more processors and one or more non-transitory computer-readable media storing computer-readable instructions. The computer-readable instructions are configured to cause the one or more processors to perform the method as described above.

[0029] There is described a computer-readable storage medium comprising computer-readable instructions. The computer-readable instructions, when executed by a processor, cause the processor to carry out the method as described above. Brief Description of the Drawings

[0030] Detailed discussion directed to one of ordinary skill in the art is set forth in the specification, which makes reference to the appended figures, in which:

[0031] Figure 1 schematically depicts an example computing system that performs data preservation with augmentation.

[0032] Figure 2 schematically depicts a data augmentation system that performs data augmentation.

[0033] Figure 3 schematically depicts a data preservation system that performs data preservation.

[0034] Figure 4 schematically depicts a flow diagram of a data preservation process performed by a data preservation system.

[0035] Figure 5 schematically depicts a flow diagram of a data preservation process performed by a data preservation system.

[0036] Figure 6 is a schematic illustration of an exemplary computer system on which aspects described herein may be implemented.

[0037] Reference numerals that are repeated across plural figures are intended to identify the same features in various implementations. Detailed Description

[0038] Figure 1 schematically depicts an example system in which techniques described herein may be implemented.

[0039] A data preservation system 100 is a computing system configured to preserve (e.g. store) data. The data preservation system 100 may comprise components that enable one or more users to preserve data. That is, the data preservation system 100 enables data to be stored in such a way that ensures continued access to the data overtime. For example, the data preservation system 100 may monitor for ongoing changes in operating system environments and perform preservation actions on the data in order to ensure that the data remains compatible. More generally, the data preservation system 100 may preserve data in accordance with US patent publication number US 20230409521 A1 titled “Automatic Preservation”. Further detail regarding how the data preservation system 100 may preserve data is described in the description of UK patent application number 2305689.8 titled “Data Preservation”. The data preservation system 100 is configured to receive data for preservation, for example, first data 102. The first data 102 may comprise image data, video data, audio data, textual data, or any other suitable form of data for preservation. The first data 102 may be any data suitable for preservation. The first data 102 may be a document file, an image file, a video file, an audio file, or any other suitable file. For example, the first data 102 may be a .docx file, a.png file, a .mp4 file, a .wav file, etc. That is, the first data 102 may be a single-part asset. A single-part asset is a single data item (e.g. a .docx file). Equally, the first data 102 may be a multipart asset. A multi-part asset is a data item comprising a plurality of other files (e.g. a .png file, a .wav file, and a .txt file). For example, the first data 102 may be a Microsoft® SharePoint List Record, a Microsoft® Teams Message, a Microsoft® SharePoint Library Record, a Microsoft® Exchange Email, or a Lotus Notes Database Record. Other suitable data for preservation are envisaged.

[0040] The data preservation system 100 may be configured to perform various operations on data. These operations may serve the purpose of ensuring that users have ongoing access to the data as mentioned. In some cases, the data preservation system 100 may perform operations for some other or related purpose, for example, to manage and / or facilitate access to information in data comprising sensitive information. In another example, the data preservation system 100 may perform actions aimed at enhancing or improving content represented by preserved data. In the example of Figure 1, the data preservation system 100 is configured to transmit data, for example the first data 102, to a data augmentation system 110 for augmentation. The data preservation system 100 may comprise a network interface to facilitate transmitting and receiving data over a network, such as the internet. The data augmentation system 110 is configured to receive, augment, and transmit new data based on the received data). For example, the data augmentation system 110 may receive and transmit data over a network using an application programming interface (API) of the data augmentation system 110. The data augmentation system 110 may comprise one or more Software as a Service (SaaS), Infrastructure as a Service (laaS), or Platform as a Service (PaaS) services, such as a Microsoft® Azure service or an Amazon Web Services® service. The augmentation operations performed by the data augmentation system 110 may be highly specific, complex and / or generally only able to be performed by certain entities. As such, the data augmentation system 110 may be implemented by one or more servers adapted to perform the data augmentation. The transmission of data to and / or from the data augmentation system 110 may subject to one or more security operations. For example, the data may be encrypted. Alternatively, the data preservation system 100 itself may comprise the data augmentation system 110. That is, the data preservation system 100 may not be required to transmit and receive data to the data augmentation system 110 over the internet.

[0041] In response to receiving the first data 102, the data augmentation system 110 augments the first data 102 to generate second data 104 using one or more machine learning models 112. While the machine learning models 112 may be used to augment the first data 102, the data augmentation system 110 may, also or alternatively, use any other suitable means for augmenting the first data 102. The second data 104 comprises one or more data augmentations. That is, the second data 104 comprises at least one augmentation made to the first data 102. A data augmentation may also include new data generated based upon other data. The second data 104 may be an augmented version of the first data 102. That is, the second data 104 may be generated based upon the first data 102 and generally represents the first data 102 but for the one or more augmentations. For example, the data augmentation system 110 may augment the first data 102 in order to remove sensitive information in the first data 102. In this example, the second data 104 is an augmented version of the first data 102 in which the mentioned sensitive information is redacted. While this example describes removing sensitive information from the first data 102, other types of augmentation are envisaged. More particularly, the second data 104 may comprise augmentations in the form of redactions, obscurations, translations, transcriptions, enhancements, classifications, captions, or any other suitable augmentation of the first data 102. The second data 104 may comprise image data, video data, audio data, textual data, or any other suitable form of data for preservation.

[0042] The second data 104 may be a document file, an image file, a video file, an audio file, or any other suitable file. For example, the second data 104 may be a .docx file, a.png file, a .mp4 file, a .wav file, etc. In another example, the second data 104 may be a Microsoft® SharePoint List Record, a Microsoft® Teams Message, a Microsoft® SharePoint Library Record, a Microsoft® Exchange Email, or a Lotus Notes Database Record. That is, the second data 104 may be a single-part or multi-part asset as described above. Other suitable data for preservation are envisaged.

[0043] It will be appreciated that the nature of the augmentation may be depend upon the nature of the data being augmented. To this end, the machine learning models 112 used to generate the second data 104 may be determined based upon the first data 102. In particular, the data preservation system 100 or the data augmentation system 110 may identify one or more types (e.g. image, text, etc.) of data for the first data 102, the types of data being used to determine the machine learning models 112 used to generated the second data 104. Each one of the machine learning models 112 may be configured to perform a different type of data augmentation to the first data 102. For example, a first machine learning model may be configured to perform redaction, while a second machine learning model may be configured to perform obscuration. In some examples, the data preservation system 100 or the data augmentation system 110 may use a separate machine learning model (not depicted) to identify the one or more types of data for the first data 102. Multiple types of data may be identified for the first data 102 which may indicate that multiple ones of the machine learning models 112 may be used to generate the second data 104. The data preservation system 100 or the data augmentation system 110 may determine a machine learning model of the machine learning models 112 for each type of data identified for the first data 102. For example, the first data 102 may comprise image and text data. In this example, a first and second machine learning model that are configured to receive image and text data respectively may be determined. A machine learning model may be a support vector machine (SVM), an artificial neural network (ANN), a recurrent neural network (RNN), a convolutional neural network (CNN), an attention-based neural network (e.g. a generative pre-trained transformer (GPT)), or any other suitable type of machine learning model. Generally, any type of machine learning model is suitable if the machine learning model permits some data (e.g. image, text, etc.) in the first data 102 to be received as input. However, certain types of machine learning model may be identified for their improved performance on certain types of data in the first data 102 (e.g. a CNN may be identified for first data 102 comprising image data). Alternatively, the data preservation system 100 or the data augmentation system 110 may determine a machine learning model of the machine learning models 112 for multiple different types of data identified in the first data 102. For example, a multi-modal machine learning model may be identified which is configured to receive both image and text data as input. Furthermore, some (e.g. the multi-modal machine learning model) of the machine learning models 112 may be configured to perform multiple different types of data augmentation (e.g. both redaction and obscuration). In this way, different machine learning models may be used to process different types of data, thereby providing different types of data augmentation for the first data 102.

[0044] The second data 104 may comprise metadata based upon the first data 102. That is, the second data 104 may include information relating to, or indicative of, the first data 102. For example, the second data 104 may indicate that the first data 102 represents an image depicting “A person”. In this example, the caption “A person” is a data augmentation (i.e. new data generated based upon the first data 102) and may be considered metadata corresponding to the first data 102. In another example, the second data 104 may indicate a location of sensitive information in text data included in the first data 102. The metadata may be any data suitable for indicating information relating to the first data 102.

[0045] Once the data augmentation system 110 has generated the second data 104, the data augmentation system 110 transmits the second data 104 to the data preservation system 100. The data preservation system 100, after receiving the second data 104 from the data augmentation system 110, preserves the second data 104 in the data preservation system 100. The data preservation system 100 also preserves the first data 102 in the data preservation system 100. The first data 102 may be preserved in the data preservation system 100 at any point once it has been received by the data preservation system 100. For example, the first data 102 may be preserved before or after a copy of the first data 102 is transmitted to the data augmentation system 110 for augmentation. The first 102 and second 104 data may be preserved in the data preservation system 100 with data indicating a link between the first 102 and second 104 data. For example, third data (i.e. {0: {original: first_data, augmented: second_data}}) may be preserved in the data preservation system 100 indicating a link between the first 102 and second 104 data. The data indicating a link may be used to locate and retrieve one version of the data (e.g. augmented) based upon the other (e.g. original). Specific details on the data indicating a link are given with reference to Figure 3 below. Once the respective first 102 and second 104 data are preserved, the user 120 may access the first data 102 and second data 104 preserved in the data preservation system 100 according to respective first 122 and second 124 set of data permissions.

[0046] The data preservation system 100 may restrict or permit access by the user 120 to the first 102 and second 104 data using the first 122 and second 124 set of data permissions respectively. For example, it may be desirable to prevent the user 120 from having read access to the first data 102 if the first data 102 comprises sensitive information. In this example, read access to the first data 102 may be restricted by the first set of data permissions 122. In another example, it may be desirable to prevent the user 120 from writing the first data 102. In this example, write access to the first data 102 may be restricted by the first set of data permissions 122. Other suitable restrictions / permissions may be imposed upon the user 120 through the use of the first 122 and second 124 set of data permissions. While a single user 120 is depicted for clarity, there may be a plurality of users 120, each permitted / restricted by the sets of data permissions.

[0047] The first 122 and / or second 124 sets of data permissions may be determined by the data preservation system 100. For example, the data preservation system 100 may determine to restrict access by the user 120 to the first data 102, whilst permitting access by the user 120 to the second data 104. In some examples, a user (not depicted) providing the first data 102 to the data preservation system 100 for preservation may determine the first set of data permissions 122 for the first data 102 and / or the second set of data permissions 124 for the second data 104. Alternatively, the first 122 and / or second 124 sets of data permissions may be determined by the data augmentation system 110. For example, the data augmentation system 110 may determine that the first data 102 comprises sensitive information, and therefore determine to restrict access by the user 120 to the first data 102. In this example, the data augmentation system 110 may determine that the generated second data 104 may not comprise the sensitive information, and therefore permit access by the user 120 to the second data 104. In other examples, the data augmentation system 110 provides the data preservation system 100 an indication whether the first 102 and / or second data 104 comprises sensitive information (e.g. {0: {name: “first_data”}, {sensitivejnfo: true}}, {1: {name: “second_data”}, {sensitive_info: false}}). The data preservation system 100 may therefore determine the first 122 and / or second 124 sets of data permissions accordingly.

[0048] The second data 104 comprising one or more data augmentations may comprise information in the first data 102. For example, both the first 102 and second 104 data may comprise “Paris is the capital city of France”. In this example, the information is innocuous and may not be considered sensitive information. The second data 104 may be used to provide access to certain information in the first data 102 without providing full access to all information in the first data 102. For example, the first data 102 may comprise both sensitive and non-sensitive information. In this example, the user 120 may be provided with access to the non-sensitive information in the first data 102, without being provided access to the sensitive information, by being provided with access to the second data 104. In detail, the data preservation system 100 may limit access by the user 120 to the second data 104 according to a second set of data permissions 124. The second set of data permissions 124 may provide the user 120 with more permissive access to the second data 104 than the user 120 has to the first data 102. As explained in more detail with reference to Figure 3, in some cases the reverse may be true. For example, the data augmentation system 110 may augment the first data 102 to generate second data 104 such that the second data 104 comprises sensitive information which is not present in the first data 102. By providing respective sets of data permissions for original and augmented data, the data preservation system 100 may limit or permit access to the first data 102 and second data 104 as required. The particular permissions applied in each case may depend upon the nature of the information within the first data 102 and second data 104 respectively.

[0049] The data preservation system 100 may perform its ongoing data preservation functions on the first 102 and / or second 104 data in order to ensure that the data is readily accessible to the user 120 in light of the ongoing changes to the operating system environment as discussed above. Specific details as to how the first data 102 and second data 104 are preserved and accessed in the data preservation system 100 will be described with reference to Figure 3 below.

[0050] The second data 104 may be required to be updated. For instance, the user 120 or the data preservation system 100 itself may require the second data 104 to be updated. The data preservation system 100 may receive a request to update the second data 104. For example, the user 120 may provide the request, or the data preservation system 100 may provide the request to itself as part of its regular and ongoing data preservation actions as described above. The request to update the second data 104 may be based upon an identification of one or more errors in the second data 104. For example, the user 120 or the data preservation system 100 may determine that the second data 104 comprises sensitive information which should be augmented. Additionally, or alternatively, the request to update the second data 104 may be based upon identifying an update to the data augmentation system 110. For example, the user 120 or the data preservation system 100 may identify that the data augmentation system 110 comprises a new machine learning model 112 that was not available when the second data 104 was initially generated. It may be determined that the new machine learning model 112 is capable of one or more augmentations to the first data 102 and / or second data 104 that were not possible using previously available machine learning models 112. Additionally, or alternatively, the request to update the second data 104 may be based upon a determination that the quality of the second data 104 is below a threshold quality level. For example, the second data 104 may comprise image data enhanced by the data augmentation system 110. In this example, the data preservation system 100 or the user 120 may determine that the quality of the enhancement is below a threshold quality level or below a level that is available by a newly available machine learning model 112. The determination of quality and comparison to a threshold quality level may be made by a user 120, or using standard measurements, such as a Peak Signal-to-Noise Ratio (PSNR).

[0051] In response to the request to update the second data 104, the data preservation system 100 may transmit (or retransmit) the preserved copy of the first data 102 to the data augmentation system 110. The data augmentation system 110 may generate an updated version of the second data 104 based on the first data 102 and / or the second data 104. By transmitting the preserved copy of the first data 102, it can be ensured that updated versions of the second data 104 do not retain errors present within the second data 104. However, the data preservation system 100 may also, or alternatively, transmit the second data 104. This may be advantageous, for example, where multiple machine learning models 112 have been used to perform respective augmentations on the first data 102. Where only a single machine learning model 112 has been updated, it may be beneficial for the data augmentation system 110 to have access to the originally generated second data 104. For example, it may be determined that it is unnecessary to re-perform all of the augmentations on the first data 102 and that the updated version of the second data 104 may be obtained by processing at least part of the originally generated second data 104.

[0052] The updated version of the second data 104 may be generated by the data augmentation system 110 in the same manner as for the second data 104 described above. Generating the updated version of the second data 104 may be consistent with the description of Figure 2 provided below. The data preservation system 100 may receive from the data augmentation system 110 the updated version of the second data 104 for preservation. The updated version of the second data 104 may then be preserved in the data preservation system 100. In some examples, the updated version of the second data 104 replaces, or merges with, the second data 104. In other examples, the updated version of the second data 104 is stored alongside (i.e. in the same file directory) or separately (i.e. in separate file directories) from the originally generated second data 104 preserved in the data preservation system 100, thereby maintaining each updated version of the second data 104. By updating the second data 104 as described, an optimal version of the second data 104 is preserved in the data preservation system 100. For example, versions of the second data 104 which do not include errors and are of the highest possible quality in light of the augmentation capabilities of the data augmentation system 110 are maintained.

[0053] Figure 2 schematically depicts the data augmentation system 110 in which techniques described herein may be implemented.

[0054] The data augmentation system 110 receives the first data 102 and processes it with one or more machine learning models 112 to generate the second data 104. As described above, the data augmentation system 110 may process the first data 102 to generate the second data 104 using any suitable means, and not necessarily using the one or more machine learning models 112. For illustration purposes, the first data 102 represents a communication.

[0055] The first data 102 comprises image data 210, audio data 220, and text data 200. The image data 210 comprises sensitive information 230. In this case, the sensitive information is personally identifiable information (Pll). More particularly, the sensitive information 230 is a face of a person. The text data 200 also comprises sensitive information 240, 250, 260. In this case, the sensitive information 240, 250, 260 are a name, address, and account number respectively. The audio data 220 may also comprise sensitive information (not depicted). For instance, the audio data may be data representing an audio recording of a person speaking “My name is John Smith. I called your agents last Tuesday. I had very poor service”. In this case, the sensitive information may be “John Smith”.

[0056] The machine learning models 112 may process the first data 102 to generate the second data 104. The machine learning models 112 may comprise a different machine learning model for each function required by the data augmentation system 110. For example, the machine learning models 112 may comprise a machine learning model specifically adapted for obscuring sensitive information in image data, and a machine learning model specifically adapted for redacting sensitive information in text data. Generally, the machine learning models 112 may comprise a machine learning model specifically adapted for the function of redaction, obscuration, translation, transcription, enhancement, classification, captioning, or any other data augmentation function required by the data augmentation system 110. It is envisaged that an individual machine learning model may be capable of performing a plurality of the foregoing functions.

[0057] The second data 104 being generated by the data augmentation system 110 comprises one or more data augmentations. As depicted, the second data 104 comprises augmented image data 212, augmented audio data 222, and augmented text data 202. The augmented image data 212 is an augmented version of the image data 210 and represents an image depicting the same person as represented by the image data 210. However, the augmented image data 212 no longer comprises the sensitive information 230 (i.e. the face of the person). In this case, the face of the person has been obscured (e.g. blurred, pixelated, replaced, removed, etc.) using the machine learning models 112 to replace the face of the person with a blurred face 232 (i.e. a data augmentation). The augmented image data 212 further comprises a bounding box 234 indicating a location of the person’s obscured face in the image represented by the augmented image data 212. The second data 104 further comprises caption data 236 indicating a caption describing at least a portion of the image represented by the image data 212, the caption data 236 also generated by the machine learning models 112. Likewise, the augmented text data 202 is an augmented version of the text data 200. The augmented text data 202 no longer comprises the sensitive information 240, 250, 260. In particular, the augmented text data 202 comprises three redactions (i.e. redactions 242, 252, 262) corresponding to the sensitive information 240, 250, 260 in the text data 200. Likewise, the augmented audio data 222 is an augmented version of the audio data 220. The augmented audio data 222 no longer represents audio including the name “John Smith”. As depicted, the second data 104 further comprises transcription data 224 (e.g. text data) indicating a transcription of the audio represented by the augmented audio data 222.

[0058] As mentioned above, the first data 102 may comprise sensitive information 230, 240, 250, 260. Sensitive information is any information that could potentially be used to cause, for example, harm, damage, embarrassment, or discrimination to an individual or entity if disclosed, accessed, or used without authorization. The sensitive information may include personally identifiable information, financial information, health information, biometric information, confidential information, legal information, location information, personally generated information, government identification information, and / or employment information. Personally identifiable information (PII) is any data that can be used to identify, contact, or locate an individual or entity, either on its own or when combined with other information. This includes but is not limited to names, addresses, phone numbers, or email addresses. Financial information is any information regarding the finances of a person or entity including bank account numbers, credit card numbers, income, revenue, profits, and financial statements. Health information may be any information regarding health, such as medical history, conditions, treatments, medical records, prescriptions, and test results. Biometric information is any information regarding unique physical or behavioural characteristics of an individual such as fingerprints, facial recognition data, iris scans, or voiceprints. Confidential information is any information that is not intended to be disclosed to unauthorized parties such as trade secrets or proprietary information. Legal information is any information regarding legal matters such as court records, case details, attorney-client communications, or any data protected by attorney-client privilege. Location information is any information regarding a geographic location of an individual or entity such as GPS coordinates, IP addresses, or location history. Personally generated information is any information that an individual or entity creates or generates themselves such as social media posts, online activity, personal notes, user-generated content (e.g. image / video content items). Government identification information is any information regarding official government-issued identification documents such as driver's license numbers, passport numbers, or social security numbers. Employment information is any information regarding an individual’s employment, such as job titles, salary details, performance evaluations, and employee records. Non-sensitive information may include any information not falling within the definition of sensitive information provided above.

[0059] The data augmentation system 110 may in some examples, as outlined above, generate additional information for inclusion in the second data 104, such as information expressed by the bounding box 234, caption data 236, and transcription data 224. In another example, the first data 102 may comprise image data representing an image depicting a plurality of individuals. The image may not be clear enough to discern the faces of the individuals depicted. It is envisaged that the data augmentation system 110 may generate enhanced image data for inclusion in the second data 104. That is, the quality of the image represented by the enhanced image data may be increased in comparison with that included in the first data 102. The information within the image data may then become sensitive information. This is described in further detail below with reference to Figure 3 with regard to the use of two different sets of data permissions.

[0060] Advantageously, the data augmentation system 110 may augment the first data 102 to generate the second data 104 such that the second data 104 does not include certain information, such as sensitive information. Equally, the data augmentation system 110 may augment the first data 102 to generate the second data 104 such that the second data 104 includes additional information not present in the first data 102. Thus, the data augmentation system 110 provides control over the information included in the second data 104.

[0061] Figure 3 schematically depicts an example implementation of the data preservation system 100.

[0062] As described above, the data preservation system 100 may preserve the first data 102 and the second data 104. The first 102 and second 104 data may be preserved in different locations of the data preservation system 100. For example, the data preservation system 100 may comprise a file system 300. The file system 300 may comprise one or more file directories, such as a first file directory 300a and a second file directory 300b. As described above, the first data 102 has a first set of data permissions 122 and the second data 104 has a second set of data permissions 124. The first and second set of data permissions 122, 124 may determine (i.e. permit or restrict) access permissions of the user 120 to the first and second file directories 300a, 300b, respectively. For example, the second set of data permissions 124 may permit the user 120 read access to the second file directory 300b, while the first set of data permissions 124 may restrict read access by the user 120 to the first file directory 300a. Thus, the user 120 may be able to read the second data 104 preserved in the second file directory 300b, but not be able to read the first data 102 preserved in the first file directory 300a. Alternatively, the sets of data permissions may restrict or permit access to data on a file-by-file basis, rather than a directory-by-directory basis.

[0063] By way of further example, the data augmentation system 110 may generate fourth data 108 by augmenting third data 106. The fourth data 108 may be received and preserved by the data preservation system 100. In this example, the third data 106 does not comprise sensitive information. However, by virtue of the fact that the fourth data 108 has been augmented by the data augmentation system 110, the fourth data 108 now does comprise sensitive information. For instance, the data augmentation system 110 may augment image data in the third data 106 to enhance its quality and reveal facial information. Such facial information may be sensitive information, such as personally identifiable information. As depicted, the third data 106 is preserved in the first file directory 300a and the fourth data 108 is preserved in the second file directory 300b. Thus, the data preservation system 100 may permit the user 120 read access to the third data 106 based upon the first set of data permissions 122 and restrict read access by the user 120 to the fourth data 108 based upon the second set of data permissions 124. In this way, a different level of access to the original (e.g. third data 106) and augmented data (fourth data 108) is provided to the user 120. A level of access may be a subset of the total possible permissions that may be provided to the user 120. It will be appreciated that while read and write access are described above, other access permissions, such as delete access or time-limited permissions may also be provided. For example, a level of access may be {read access, write access} - i.e. not including other types of access permissions such as delete access. Different levels of access provided to the user 120 may be used to account for the nature of the information in the data, for example, if sensitive information is present in either original or augmented copies.

[0064] Depending upon the nature of the original or augmented data, different levels of access may be required in order to restrict or permit access to such data. Therefore, the first 122 and second 124 sets of data permissions may determine read, write, delete, and / or time-limited access permissions by the user 120 to the data in question for any particular user. Read access permissions may be permissions granted to the user 120 (or entity) permitting the user 120 to view or read the content of the data in question. Write access permissions may be permissions granted to the user 120 (or entity) permitting the user 120 to add to or modify the content of the data in question. Delete access permissions may be permissions granted to the user 120 (or entity) permitting the user 120 to delete the data in question. Time-limited access permissions are permissions granted to the user 120 (or entity) permitting the user 120 any one of the foregoing permissions restricted to one or more predetermined period(s) of time.

[0065] The data preservation system 100 may generate link data indicating a link between original and augmented data. A link may indicate that one piece of data (e.g. second data 104) is associated with (e.g. is based upon) the other (e.g. first data 102). For example, first link data 304 may be generated to indicate a link between the first data 102 and the second data 104. Likewise, second link data 306 may be generated to indicate a link between the third data 106 and the fourth data 108. The first link data 304 may be used to retrieve the second data 104 located in the file system 300. For example, the user 120 may wish to gain read access the first data 102. In this example, the first set of data permissions 122 may restrict read access by the user 120 to the first data 102. In this case, the first link data 304 may indicate that the first data 102 is linked to (i.e. corresponds to) the second data 104. The link data may indicate a location of a respective data item preserved in the data preservation system 100. For example, the first link data 304 may indicate that the first data 102 is preserved in the first file directory 300a of the file system 300, and that the second data 104 is preserved in the second file directory 300b of the file system 300. Alternatively, the link data may provide a means for conducting a search of the data preservation system 100 for a respective data item. For example, the link data may indicate that first data 102 is linked to the second data 104, and further provide their respective filenames for use in a search function of the data preservation system 100. Thus, the second data 104 may be located in the data preservation system 100and retrieved. This may be useful if, for example, the second set of data permissions 124 provides the level of access required by the user 120. The first link data 304 may be provided alongside (e.g. in the same directory) or as part of the first 102 or second 104 data. Link data may additionally be used upon determining that augmented data needs updating, as described above. In this way, the original data on which the augmented data may be retrieved to allow for new augmentations to be generated based upon the original data.

[0066] The link data may have none or minimal user access restrictions in order to allow the user 120 to readily locate data in the file system 300 of the data preservation system 100. Thus, if the user 120 is not permitted read access to the first data 102 according to the first set of data permissions 122, the first link data 304 provides a means for readily locating and retrieving (e.g. accessing) an augmented version of the first data 102 (i.e. the second data 104) which the user 120 may be permitted to access according to the second set of data permissions 124. This improves data accessibility within the data preservation system 100, whilst respecting data permissions which may, for example, have been put in place to protect sensitive information in the first 102 or second 104 data. The link data may comprise both the first 304 and second 306 link data. For example, the link data may be data indicating a plurality of one-to-one relationships, links, correspondence, etc. between an original and an augmented version of the original data. In other examples, the link data may be data indicating a plurality of one-to-many relationships, links, correspondence, etc. between original and multiple different augmented versions of the original data. The link data may be a database record (e.g. a record in an SQL database). For example, the link data may be {“123456, first_data.png, / path / to / first_data.png, second_data.png, / path / to / second_data.png”} for corresponding fields {“ID, first_filename, firstjocation, second_filename, second_location“} in the SQL database. Equally, the link data may be a JSON file, or any other suitable data. For example, link data may represent a JSON file: {original: “first_data.png”, augmented: [“second_data_v1.png”, “second_data_v2.png”, ...]}. The link data may be stored centrally within the data preservation system 100, thereby acting as a manifest for the data preserved in the data preservation system 100. Thus, a means for identifying augmented or original version(s) of data based upon a respective original or augmented version(s) of said data is provided.

[0067] Figure 4 depicts a flow diagram of a data preservation process that may be carried out by a data preservation system such as the data preservation system 100 described herein.

[0068] At step 400, first data is received by the data preservation system for preservation.

[0069] The first data may comprise text data, image data, video data, audio data, and / or any other suitable form of data.

[0070] The first data may comprise sensitive information or non-sensitive information. The sensitive information may comprise personally identifiable information, financial information, health information, biometric information, confidential information, legal information, location information, personally generated information, government identification information, employment information, and / or any other suitable form of sensitive information.

[0071] The first data may be a document file, an image file, a video file, an audio file, or any other suitable file. The first data may be a Microsoft® SharePoint List Record, a Microsoft® Teams Message, a Microsoft® SharePoint Library Record, a Microsoft® Exchange Email, a Lotus Notes Database Record, or any other suitable form of data for preservation. More generally, the first data may be as described anywhere in this specification.

[0072] At step 402, the first data is transmitted by the data preservation system to a data augmentation system, for example, the data augmentation system 110 described herein.

[0073] The data augmentation system may be a system external to the data preservation system. For example, the data augmentation system may not be an internal system / process of the data preservation system. As such, the transmission of the first data to the data augmentation system may comprise transmission over the internet.

[0074] Transmitting and receiving data to and from the data augmentation system may comprise using an application programming interface (API) of the data augmentation system.

[0075] The data augmentation system may comprise a Microsoft Azure® service, an Amazon Web Services® service, and / or any other suitable Software as a Service (SaaS) that is configured to augment the first data.

[0076] The data augmentation system may comprise one or more machine learning models configured to process the first data for augmentation.

[0077] At step 404, second data is received by the data preservation system from the data augmentation system for preservation. The second data comprises one or more data augmentations. The first data corresponds to a first set of data permissions that is different than a second set of data permissions corresponding to the second data (i.e. provides a different level of access). The second data may comprise information that enables the data preservation system to internally identify the first data stored within the data preservation system, such as an identification of the first data. The second data is an augmented version of the first data. The second data may generally take the form describe above. The second data is generated by the data augmentation system augmenting the first data. As described above, the augmenting may comprise redacting, obscuring, translating, transcribing, enhancing, classifying, and / or captioning the first data to generate the second data. The second data may be a document file, an image file, a video file, an audio file, or any other suitable file. The second data may be a Microsoft® SharePoint List Record, a Microsoft® Teams Message, a Microsoft® SharePoint Library Record, a Microsoft® Exchange Email, a Lotus Notes Database Record, or any other suitable form of data for preservation. More generally, the second data may be as described anywhere in this specification.

[0078]

[0079] At step 406, the first and second data are stored by the data preservation system for later retrieval. The storage may be as described above.

[0080] For example, the first data is stored in a first location of the data augmentation system and the second data is stored in a second location of the data augmentation system different to the first location.

[0081] The data preservation process may (but need not) further include step 408. At step 408, link data indicating a link between the first data and the second data is generated. The first or second data may then be retrieved from the data preservation system based upon the link data indicating the link.

[0082] The data preservation process may also further include (but need not) step 410. At step 410, a request is received by the data preservation system to update the second data. The first data may then be transmitted to the data augmentation system by the data preservation system for updating. In response, the data preservation system may receive an updated version of the second data for preservation system from the data augmentation system. In response, the data preservation system may then store the updated version of the second data.

[0083] Figure 5 depicts a flow diagram of a data preservation process that may be carried out by a data preservation system such as the data preservation system 100 described herein.

[0084] At step 500, a request is received by the data preservation system to retrieve first data preserved in the data preservation system. The request may be received from a user or entity (e.g. computer system) of the data preservation system.

[0085] At step 502, a level of access for the first data is determined. The level of access may be for the user or entity. For example, a first set of data permissions maintained by the data preservation system for the user may indicate that the user has no access permissions to the first data.

[0086] At step 504, link data indicating a link between the first and second data is used to identify second data preserved in the data preservation system. For example, the link data may be included in the first data, or stored alongside the first data, and may indicate a location of the second data preserved in the data preservation system. More generally, the link data may be as described anywhere in this specification.

[0087] At step 506, a level of access for the second data is determined. The level of access may be for the user or entity. For example, a second set of data permissions maintained by the data preservation system for the user may indicate that the user has read access permissions to the second data.

[0088] At step 508, the preserved second data is retrieved from the data preservation system. The preserved second data may be retrieved in response to determining that the user has certain access permissions to the second data. The preserved second data may be retrieved in response to determining that the user has different (e.g. more permissive) access permissions to the second data than to the first data. The link data may indicate a location of the second data (e.g. a file directory in which the second data is preserved). The retrieval of the second data may be based upon the location indicated by the link data. For example, the link data may indicate that the second data is preserved at “path / to / second_data”.

[0089] Figure 6 schematically illustrates an exemplary arrangement of components which may provide a computing system 4 used to implement all or part of the preservation system 100 or the data augmentation system 110.

[0090] A processor, in this case in the form of a CPU 4a, configured to read and execute instructions stored in a volatile memory 4b which takes the form of a random access memory. It will be appreciated that the processor may take other forms, such as, for example, a GPU. The volatile memory 4b stores instructions for execution by the CPU 4a and data used by those instructions. For example, the instructions may include instructions for causing the preservation system 100 to carry out the processing described above with reference to the respective Figures.

[0091] The computing system 4 comprises a storage device 5. It will be appreciated that the storage device 5 may be implemented in any way, such as for example, a hard disk drive, a solid state drive, etc. The storage device 5 may provide the means for preserving data in the data preservation system 100. The computing system 4 further comprises an I / O interface 4d to which are connected peripheral devices used in connection with the computing system. More particularly, a display 4e is configured so as to display output. Input devices are also connected to the I / O interface 4d. Such input devices include a keyboard 4f and a mouse 4g which allow user interaction with the computing system 4. A network interface 4h allows the computing system 4 to be connected to appropriate computer networks, such as the Internet 6, and so as to be able to send and receive from and to other computing devices. For example, to send and receive between the data preservation system 100 and the data augmentation system 110. The network interface 4h may provide the network interface 112. The CPU 4a, volatile memory 4b, the storage device 5, I / O interface 4d, and network interface 4h, are connected together by a bus 4i.

[0092] The techniques described above may be implemented in hardware, firmware, software, or any combination thereof. The techniques may also be implemented as instructions stored on a machine-readable medium, which may be read and executed by one or more processors. A machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computing device). For example, a machine-readable medium may include read only memory (ROM); random access memory (RAM); magnetic storage media; optical storage media; flash memory devices; electrical, optical, acoustical or other forms of propagated signals (e.g. carrier waves, infrared signals, digital signals, etc.), and others. Further, firmware, software, routines, instructions may be described herein as performing certain actions. However, it should be appreciated that such descriptions are merely for convenience and that such actions in fact result from computing devices, processors, controllers, or other devices executing the firmware, software, routines, instructions, etc. and in doing that may cause actuators or other devices to interact with the physical world.

[0093] While specific embodiments of the invention have been described above, it will be appreciated that the invention may be practiced otherwise than as described. The descriptions above are intended to be illustrative, not limiting. Thus it will be apparent to one skilled in the art that modifications may be made to the invention as described without departing from the spirit of the invention.

Claims

1. A computer-implemented method for automatically preserving data in a data preservation system, comprising at one or more computing devices: receiving, by the data preservation system, first data for preservation;transmitting, by the data preservation system, the first data to a data augmentation system configured to augment the first data;receiving, by the data preservation system from the data augmentation system, second data for preservation, the second data comprising one or more data augmentations to the first data;wherein the data preservation system maintains a first set of data permissions for the first data and maintains a second set of data permissions for the second data, and wherein the first set of data permissions permits a different level of access than the second set of data permissions; andstoring, by the data preservation system, the first and second data in the data preservation system for later retrieval.

2. The method of claim 1, wherein the first and second data comprises text data, image data, video data, and / or audio data.

3. The method of any preceding claim, wherein the data augmentations comprise the removal of sensitive information from the first data or the addition of sensitive information to the first data.

4. The method of claim 3, wherein the sensitive information comprises personally identifiable information, financial information, health information, biometric information, confidential information, legal information, location information, personally generated information, government identification information, and / or employment information.

5. The method of any preceding claim, wherein the first data and / or the second data is a document file, an image file, a video file, an audio file, aMicrosoft® SharePoint List Record, a Microsoft® Teams Message, a Microsoft® SharePoint Library Record, a Microsoft® Exchange Email, or a Lotus Notes Database Record.

6. The method of any preceding claim, wherein the data augmentation system is a system external to the data preservation system.

7. The method of any preceding claim, wherein the transmitting and the receiving comprises interfacing with an application programming interface of the data augmentation system.

8. The method of any preceding claim, wherein the data augmentation system comprises a Microsoft Azure® service and / or an Amazon Web Services® service.

9. The method of any preceding claim, wherein the data augmentation system comprises one or more machine-learning models configured to process the first data for augmentation.

10. The method of any preceding claim, wherein the second data is generated by the data augmentation system augmenting the first data, the augmenting comprising:redacting, obscuring, translating, transcribing, enhancing, classifying, and / or captioning the first data to generate the second data.

11. The method of any preceding claim, wherein the first data is stored in a first location in the data preservation system different to a second location in the data preservation system.

12. The method of claim 11, wherein the respective locations are file directories in the data preservation system.

13. The method of any preceding claim, wherein the sets of data permissions comprise one or more of read access permissions, write access permissions, delete access permissions, and / or time-limited access permissions.

14. The method of any preceding claim, further comprising:generating link data indicating a link between the first data and the second data; andretrieving, based upon the link data, the first or second data.

15. The method of claim 14, further comprising:receiving, by the data preservation system, a request to retrieve the first or second data from the data preservation system;determining a level of access for the respective first or second data based on the respective first or second set of data permissions; identifying the respective second or first data preserved in the data preservation system using the link data;determining a level of access for the respective second or first data based on the respective second or first set of data permissions; andwherein retrieving the first or second data is based upon the level of access for the respective first or second data.

16. The method of claims 14 or 15 when dependent upon claims 11 to 13, wherein the link data indicates the first and second location of the respective first and second data in the data preservation system; andwherein retrieving the first or second data is further based upon the respective first or second location indicated by the link data.

17. The method of any preceding claim, further comprising:receiving, by the data preservation system, a request to update the second data;transmitting, by the data preservation system, the first data to the data augmentation system;receiving, by the data preservation system from the data augmentation system, an updated version of the second data for preservation; andstoring, by the data preservation system, the updated version of the second data.

18. The method of claim 17, wherein the request to update the second data is based upon identifying one or more errors in the second data.

19. The method of any of claims 17 to 18, wherein the request to update the second data is based upon identifying an update to the data augmentation system.

20. The method of any of claims 17 to 19, wherein the request to update the second data is based upon determining that the quality of the second data is below a threshold quality level.

21. The method of any of claims 17 to 20, wherein storing the updated version of the second data comprises replacing the second data with the updated version of the second data in the data preservation system.

22. The method of any preceding claim, wherein the second data is an augmented version of the first data.

23. The method of any preceding claim, wherein the second data comprises metadata corresponding to the first data.

24. A computing system comprising:one or more processors; andone or more non-transitory computer-readable media storing computer-readable instructions configured to cause the one or more processors to perform a method according to any one of the preceding claims.

25. A computer-readable storage medium comprising computer-readable instructions, the computer-readable instructions, when executed by a processor, cause the processor to carry out the method of any of claims 1 to 23.

Images