Computer-implemented method for training a generative artificial intelligence, computer-implemented method for generating a masked image data matrix, and computer-implemented method for converting an input image data set into a compressible masked image data matrix.

Abstract

The invention relates to a computer-implemented method for training a generative artificial intelligence, a computer-implemented method for generating a masked image data matrix, and a computer-implemented method (201) for converting an input image data set (C) into a compressible masked image data matrix (209; 209').

Description

[0001] The invention relates to a computer-implemented method for training a generative artificial intelligence, a computer-implemented method for generating a masked image data matrix, and a computer-implemented method for converting an input image data set into a compressible masked image data matrix according to the preamble of claim 1 or 3 or 6.

[0002] It is known from the prior art to generate an image data matrix from an input image data set using an encoder unit of an artificial intelligence and to convert this image data matrix into an output image data set, which corresponds to the input image data set, using a decoder unit of the artificial intelligence.

[0003] The invention is based on the objective of developing a computer-implemented method for training a generative artificial intelligence, or by means of which a generative artificial intelligence can be trained to generate a masked image data matrix for an image described by an input image data set, the storage requirement of which is reduced compared to an unmasked image data matrix, but which allows it to be converted back into an output image data set by means of a decoder unit of an artificial intelligence, which is equivalent to the input image data set with regard to a desired or specified information content.

[0004] The invention also aims to develop a computer-implemented method for generating a masked image data matrix, by means of which a masked image data matrix can be generated for an image described by an input image data set, the storage requirement of which is reduced compared to an unmasked image data matrix, but which allows it to be converted back into an output image data set by means of a decoder unit of an artificial intelligence, which is equivalent to the input image data set with regard to a desired or specified information content.

[0005] The invention further aims to develop a computer-implemented method for converting an input image data set into a compressible masked image data matrix, by means of which transmission with a small file size without significant loss of information is possible.

[0006] This problem is solved by means of the characterizing features of claim 1, 3, or 6, respectively, starting from the features of the preamble of claim 1, 3, or 6. Advantageous and expedient embodiments are specified in the respective dependent claims.

[0007] A data matrix can be configured as a single-column matrix, a single-row matrix or vector, or as a matrix consisting of multiple rows and multiple columns, or even as a matrix of at least three dimensions. For the purposes of this invention, the term "matrix" can alternatively also refer to the term "tensor."

[0008] In the computer-implemented method according to the invention for training a generative artificial intelligence, it is provided that the generative artificial intelligence (AI-1) is implemented in a data processing device, wherein the data processing device comprises an encoder unit, a buffer, a masking unit, a decoder unit and a comparison unit, wherein the following steps are provided: - Input of an initial image data set into the encoder unit to generate an initial image data matrix; - Output of the first image data matrix generated for the first input image data set; - Generation of a first masked image data matrix from the first image data matrix using the masking unit such that a part of the first image data matrix is ​​masked according to a systematic pattern; - Input of the masked image data matrix into the decoder unit to generate a first output image data set; - Comparison of the first input image data set and the first output image data set using the comparison unit in such a way as to determine whether the two image data sets meet the matching criteria; - Repeating the aforementioned steps with a second to nth input image data set such that the generative artificial intelligence learns, by means of pattern recognition, to generate output image data sets which are optimized by being structured in such a way that their elements relevant for fulfilling the matching criteria are arranged in areas not intended for systematic masking, in particular columns.

[0009] Through such a training method, generative artificial intelligence can be trained to generate masked image data matrices from source image datasets. Despite varying complexity, these masked matrices largely retain the information content of the original image datasets. In particular, depending on the image complexity, a desired or predetermined level of information can be preserved, which, adapted to a technical application such as object recognition or use for moving image sequences, is considered sufficient.

[0010] In the computer-implemented method according to the invention for generating a masked image data matrix using a generative artificial intelligence, in which the generative artificial intelligence is implemented in a data processing unit and in which the data processing unit comprises an encoder unit and a masking unit, the following steps are provided: - Encoding an input image data set into an image data matrix using the endoder unit such that the image data matrix is ​​structured in such a way that its elements relevant for fulfilling matching criteria are arranged in areas (columns) which are not intended for systematic masking, - Generating a masked image data matrix using the masking unit by systematically masking elements of the image data matrix, - Output of a masked image data matrix for compression and subsequent transmission.

[0011] This makes it possible to transfer image data sets with reduced file sizes, and therefore faster and more cost-effectively.

[0012] In the computer-implemented method according to the invention for converting an input image data set into a compressible masked image data matrix using artificial intelligence, wherein the artificial intelligence comprises a generative artificial intelligence and a further, evaluative artificial intelligence, wherein both artificial intelligences are implemented in a data processing unit, wherein the data processing unit for this purpose comprises an encoder unit and a masking unit for the generative artificial intelligence and wherein the data processing unit for this purpose comprises a comparison unit and a suggestion unit for the further artificial intelligence, the following steps are provided: - Encoding an input image data set into an image data matrix using the endoder unit such that the image data matrix is ​​structured in such a way that its elements relevant for fulfilling matching criteria are arranged in areas (columns) which are not intended for systematic masking, - Compressing the image data matrix using the masking unit by systematic masking into a masked image data matrix, - Decoding a masked image data matrix into the original image data set, - Transfer of the input image data set and the output image data set to the comparison unit of the further artificial intelligence, - wherein the comparison unit checks the two image data sets for sufficient similarity and wherein ◯ either, if no sufficient match is found, a modified masking is requested from the generative artificial intelligence using the suggestion unit, or an alternatively masked image data matrix is ​​forwarded by the further artificial intelligence for compression and transmission, , ◯ or, if a sufficient match is found, the generated masked image data matrix is ​​forwarded for compression and transmission.

[0013] This makes it possible to transfer image data sets with reduced file sizes, and therefore faster and more cost-effectively.

[0014] In principle, according to the invention, it is provided that an image data matrix or image data vector, for example in the form of a single-column matrix or a single-row matrix, or a multi-column and multi-row matrix, is generated from an input image data set using artificial intelligence, and that this is compressed using artificial intelligence without losing essential information. In particular, depending on the image complexity, a desired or predetermined information content can be retained that is considered sufficient for a technical application, e.g., object recognition or use for moving image sequences.

[0015] Here, the number of elements contained in a corresponding matrix is ​​adjusted to the complexity of the image described by the input image data set.

[0016] When training a generative artificial intelligence, matrices with 32 columns and 5 rows or with 64 columns and 5 rows were used.

[0017] In order to train the artificial intelligence to recognize the essential information and later arrange it in a "front" area of ​​a corresponding matrix, the last 5 columns or the last 10 columns were statistically exchanged or masked, starting from a rear end of the matrix.

[0018] During the training process, the AI ​​must then convert the masked image data set back into an output image data set using a decoder unit and compare it with the input image data set, attempting to improve the result with regard to minimizing the loss of essential information.

[0019] The core of the training is to introduce disturbances, e.g., through masking, into a defined area of ​​the initial image data matrix in order to motivate the artificial intelligence to "rank" the elements of the matrix in such a way that the elements relevant for the essential information are arranged in areas that are not affected by the occurrence of disturbances.

[0020] After sufficient training, an image data matrix generated by the AI's encoder unit can be considered "relevantly sorted." After masking the image data matrix, it can be checked whether the reconstruction of an original image data set from the masked image data matrix is ​​still considered sufficient compared to the original image data set.

[0021] The size of the matrix therefore depends on the complexity of the input image data set. For input image data sets describing simple images, a comprehensively masked image data matrix is ​​sufficient. For input image data sets describing complex images, a more memory-intensive matrix is ​​required.

[0022] This allows, in the process of converting an input image dataset into an output image dataset, the number of columns to be compressed by a second, evaluating artificial intelligence to be selected based on the image's complexity, starting with the first column. In this process, the masked columns are either deleted before compression or not included in the compression process.

[0023] Generative artificial intelligence, which constitutes the first artificial intelligence, essentially comprises an encoder unit and a decoder unit. The encoder unit generates image data matrices, which are sorted and masked, and passes these to the decoder unit for conversion back into an output image data set.

[0024] A comparison of the input image data set with the output image data set generated by the first artificial intelligence is not performed via image subtraction and averages, but rather the aforementioned second artificial intelligence is used.

[0025] The second artificial intelligence compares the respective input image dataset with the respective output image dataset, particularly with regard to the criteria "Visual Loss" and / or "Preceptual Loss." This second artificial intelligence is specifically pre-trained. Its reactions to the respective input image dataset and the output image dataset, which the first artificial intelligence generated from the masked image data matrix, are observed. Based on the behavior of the second artificial intelligence, it is then determined, for example, whether the two image datasets are sufficiently similar with respect to "L-PIPS Loss."

[0026] The second artificial intelligence can decide how closely the input image dataset and the output image dataset should be related. The second artificial intelligence can request modified masking from the first artificial intelligence. The second artificial intelligence can also independently decide to what extent elements should be removed from the image data matrix created by the first artificial intelligence before compression.

[0027] Further details of the invention are described in the drawing with reference to schematically illustrated embodiments.

[0028] This shows: Fig. 1: a schematic representation of the state of the art as described in the introductory section; Fig. 2: a schematic representation of a computer-implemented method according to the invention for training a generative artificial intelligence in schematic form; Fig. 3: a feedback loop for the [project / project] in the Fig. 2 schematically represented training exercises; Fig. 4: a schematic representation of a computer-implemented method according to the invention for generating a masked image data matrix and Fig. 5: a schematic representation of a computer-implemented method according to the invention for converting an input image data set into a compressible masked image data matrix.

[0029] As in the Fig. As shown in Figure 1, it is known from the prior art to generate an image data matrix 904 from an input image data set F using a data processing unit 901, on which an encoder unit 902 of an artificial intelligence 903 operates. The generated image data matrix 904 is then converted into an output image data set G, corresponding to the input image data set F, by means of a decoder unit 905 of the artificial intelligence 903. To clearly separate the encoding and decoding steps, the schematic flow diagram of the Fig. 1 The image data matrix 904 and the data processing unit 901 are shown twice.

[0030] The Fig. Figure 2 shows a schematic representation of a computer-implemented method 1 according to the invention for training a generative artificial intelligence AI-1 in schematic form.

[0031] The generative artificial intelligence AI-1 is implemented in a data processing unit 2. The data processing unit 2 comprises an encoder unit 3, a buffer 4, a masking unit 5, a decoder unit 6, and a comparison unit 7.

[0032] The process includes the following steps: Input of an initial image data set A into the encoder unit 3 to generate an initial image data matrix 8, also known as a "feature matrix". Naturally, the initial image data matrix 8 can also be in the form of a tensor. Furthermore, it is possible for the encoder to convert multiple input image data sets simultaneously.

[0033] Output of the first “image data matrix” 8 generated for the first input image data set A, which can be stored in the intermediate memory 4 of the data processing device 2.

[0034] Generation of a first masked image data matrix 9 from the first image data matrix 8 using the masking unit 5 such that a part of columns SP8-3, SP8-4 of the first image data matrix 8 is masked according to a system, so that the masked image data matrix 9 with masks M9-3 and M9-4 is obtained.

[0035] Input of the masked image data matrix 9 into the decoder unit 6 to generate a first output image data set a.

[0036] Comparison of the first input image data set A and the first output image data set a using the comparison unit 7, such that it is determined whether the two image data sets A, a meet the matching criteria. For this purpose, reference is also made to the representation of the Fig. 3 referred to, in which a cuboid 10 depicted next to the artificial intelligence AI-1 symbolizes part of the learning process insofar as the masking carried out by the masking unit 5 - either the comparison unit determines that the information is "ok" and therefore any information classified as irrelevant is lost through masking - or is deemed "not ok" by the comparison unit, and thus information classified as relevant is lost due to the masking.

[0037] If the comparison unit 7 returns a "not ok" result, the learning process can be considered incomplete. Therefore, the encoder must be further trained regarding the sorting of the columns, and the masking unit 5 regarding the number of masked columns. Accordingly, in the Fig. Figure 3 also shows that the finding is forwarded to encoder 3 and masking unit 5, as further optimization of the coding is necessary depending on the finding.

[0038] To compare the two image datasets A and a, a variety of loss functions can be used in the comparison unit to evaluate the encoding quality. This evaluation is then fed back into the training process.

[0039] In the illustrated embodiment, the comparison unit 7 is implemented in the first artificial intelligence AI-1. According to an embodiment not shown, a comparison unit from another artificial intelligence can also be used. In particular, the comparison unit 7 can be pre-trained.

[0040] The process described above with the aforementioned steps is repeated for learning purposes with a second to nth input image data set in such a way that the generative artificial intelligence AI-1 learns by means of pattern recognition to generate output image data sets which are optimized by being structured in such a way that their elements relevant for fulfilling the matching criteria are arranged in areas (columns) not intended for systematic masking.

[0041] In the Fig. Figure 4 shows a computer-implemented method 101 according to the invention for generating a masked image data matrix 109. This is performed using the method already described in the Fig. 2 and Fig. The 3 described generative artificial intelligence AI-1 was executed, whose learning process is now at least sufficiently complete.

[0042] The generative artificial intelligence AI-1 is implemented in the data processing unit 2, as mentioned. The data processing unit 2 comprises the encoder unit 3 and the masking unit 5. Procedure 101 comprises the following steps: Encoding an input image data set B into an image data matrix 108 using the endoder unit 3 such that the image data matrix 108 is structured in such a way that its elements relevant for fulfilling matching criteria are arranged in front, left columns SP108-1, SP108-2, which are not intended for systematic masking.

[0043] Generating a masked image data matrix 109 using the masking unit 5 by systematically masking elements of the image data matrix 108.

[0044] Output of a masked image data matrix 109 for compression and subsequent transmission.

[0045] Above the image data matrix 8, its structure is symbolically represented by a curve 151. Here, the curve 151 lies in a coordinate system 152, whose x-axis 153 increasingly symbolizes columns 1 to n of the image data matrix 8 to the right, and whose y-axis 154 symbolizes the relevance of the columns ascending upwards with values ​​from, for example, 0 to 100. Accordingly, the relevance of columns SP8-1 to SP8-4 of the image data matrix 108 decreases to the right, so that a right-to-left masking of the columns has a lesser impact on the overall relevance of the image data matrix 108 with regard to a match with input image data set B than a right-to-left masking would have.

[0046] In the Fig. Figure 5 shows a schematic representation of a computer-implemented method 201 according to the invention for converting an input image data set C into a compressible masked image data matrix 209, 209' using artificial intelligence.

[0047] The artificial intelligence comprises a generative artificial intelligence AI-1 and another, evaluative artificial intelligence AI-2. Both artificial intelligences AI-1 and AI-2 are integrated into the system derived from the Fig. 2, Fig. 3 to Fig. 4 known data processing facilities 2 are implemented. Data processing facility 2 includes the already existing data processing facilities. Fig. 2, Fig. 3 to Fig. 4 mentioned generative artificial intelligence AI-1, the known encoder unit 3 and the known masking unit 5.

[0048] The data processing unit 2 additionally includes the further, evaluative artificial intelligence AI-2, a comparison unit 210 and a suggestion unit 211.

[0049] Procedure 201 has the following sequence, the first two steps of which are essentially the same as those described below. Fig. The procedure described in section 4 corresponds to: 101 Encoding an input image data set C into an image data matrix 208 using the endoder unit 3 such that the image data matrix 208 is structured in such a way that its elements relevant for fulfilling matching criteria are arranged in columns SP208-1 to SP208-2, which are not intended for systematic masking.

[0050] Compressing the image data matrix 208 using the masking unit 5 by systematic masking into the masked image data matrix 209.

[0051] Decoding a masked image data matrix 209 using the decoder unit 6 into an output image data set c.

[0052] Transfer of the input image data set C and the output image data set c to the comparison unit 210 of the further artificial intelligence AI-2. In particular, depending on the image complexity, the comparison unit can provide a desired or predetermined information content for preservation, which is considered sufficient for a technical application purpose, e.g. object recognition or use for moving image sequences.

[0053] Here, the comparison unit 210 checks the two image data sets C, c for sufficient similarity and wherein - either, if no sufficient match is found, a modified masking is requested from the generative artificial intelligence AI-1 using the proposal unit 211, or the image data matrix 209' alternatively masked by the further artificial intelligence AI-2 is forwarded for compression and transmission, e.g. to a cloud 212, - or, if a sufficient match is found, the generated masked image data matrix 209 is forwarded for compression and transmission, e.g. to a cloud 212.

[0054] During compression, the masked areas are deleted. During subsequent decompression for decoding, the originally masked areas are refilled to obtain a matrix of the original size in terms of column and row count.

[0055] According to the Fig.5. The compressed masked image data matrix 209' can be loaded from the cloud 212 into a decoder unit 806 of another artificial intelligence AI-X, in order to be converted by it into an output image data set c'. In this process, the decoder unit 806 uses so-called filler words or filler tokens to refill the masked areas of the generated image data matrix, in order to ensure a uniform size of the matrices to be processed and thus also to be able to process several matrices in parallel. Reference symbol list 1 Procedure 2 Data processing equipment 3 encoder units 4 intermediate storage 5 masking unit 6 decoder units 7 comparison units. 8 Image Data Matrix 9 masked image data matrix 10 cuboids A Input image data set a source image data set SP8-3, SP8-4 Column of 8 M9-3, M9-4 Mask of 9 101 procedures 108 Image Data Matrix 109 masked image data matrix B Input image data set SP108-1 Column SP108-2 Column SP108-3 Column SP108-4 Column 151 Curve 152 Coordinate system 153 x-axis 154 y-axis 201 procedures C Input image data set c Source image data set AI-1 generative artificial intelligence AI-2 is another, evaluative artificial intelligence. 208 Image Data Matrix 209 masked image data matrix 209' alternatively masked image data matrix 210 comparison unit 211 Proposal Unit 212 Cloud AI-X other artificial intelligence 806 decoder unit from KI-X F Input image data set G Source image data set 901 Data processing unit 902 Encoder Unit 903 artificial intelligence 904 Image Data Matrix 905 Decoder Unit

Claims

[1] Computer-implemented method (1) for training a generative artificial intelligence (AI-1), - wherein the generative artificial intelligence (AI-1) is implemented in a data processing device (2), - wherein the data processing device (2) comprises an encoder unit (3), a buffer (4), a masking unit (5), a decoder unit (6) and a comparison unit (7), - the procedure includes the following steps: - Input of a first input image data set (A) into the encoder unit (3) to generate a first image data matrix (8); - Output of the first image data matrix (8) generated for the first input image data set (A); - Generation of a first masked image data matrix (9) from the first image data matrix (8) using the masking unit (5) such that a part of the first image data matrix (8) is masked according to a system; - Input of the masked image data matrix (9) into the decoder unit (6) to generate a first output image data set (a); - Comparison of the first input image data set (A) and the first output image data set (a) using the comparison unit (7) such that it is determined whether the two image data sets (A, a) meet the matching criteria; - Repeating the aforementioned steps with a second to nth input image data set such that the generative artificial intelligence (AI-1) learns by means of pattern recognition to generate output image data sets (a) which are optimized by being structured in such a way that their elements relevant for fulfilling the matching criteria are arranged in areas not intended for systematic masking. [2] Computer-implemented method according to claim 1, characterized by, that the output of the image data matrix (8) is to a buffer (4) of the data processing device (2). [3] Computer-implemented method (101) for generating a masked image data matrix (108) using a generative artificial intelligence (AI-1), - wherein the generative artificial intelligence (AI-1) is implemented in a data processing device (2), - wherein the data processing unit (2) comprises an encoder unit (3) and a masking unit (5), comprising the steps: - Encoding an input image data set (B) into an image data matrix (108) using the endoder unit (3) such that the image data matrix (108) is structured in such a way that its elements relevant for fulfilling matching criteria are arranged in areas (columns) which are not intended for systematic masking, - Generating a masked image data matrix (109) using the masking unit (5) by systematically masking elements of the image data matrix (108), - Output of a masked image data matrix (109) for compression and subsequent transmission. [4] Computer-implemented method according to claim 3, characterized by , that the masked elements (M9-3, M9-4) of the masked image data matrix (109) are deleted, such that the image data matrix (109) is output free of masked elements. [5] Computer-implemented method according to at least one of claims 3, 4, characterized by that the generative artificial intelligence (AI-1) was trained by a method according to claim 1. [6] Computer-implemented method (201) for converting an input image data set (C) into a compressible masked image data matrix (209; 209') using artificial intelligence, - where the artificial intelligence comprises a generative artificial intelligence (AI-1) and another, evaluative artificial intelligence (AI-2), - wherein both artificial intelligences (AI-1, AI-2) are implemented in a data processing unit (2), - wherein the data processing unit (2) for generative artificial intelligence (AI-1) comprises an encoder unit (3) and a masking unit (5), - wherein the data processing unit (2) for further artificial intelligence (AI-2) comprises a comparison unit (210) and a suggestion unit (211), comprising the steps: - Encoding an input image data set (C) into an image data matrix (208) using the endoder unit (3) such that the image data matrix (208) is structured in such a way that its elements relevant for fulfilling matching criteria are arranged in areas (columns) which are not intended for systematic masking, - Compressing the image data matrix (208) using the masking unit (5) by systematic masking into a masked image data matrix (209), - Decoding a masked image data matrix (209) into the output image data set (c), - Transfer of the input image data set (C) and the output image data set (c) to the comparison unit (210) of the further artificial intelligence (AI-2), - wherein the comparison unit (210) checks the two image data sets (C; c) for sufficient similarity and wherein o either, if no sufficient match is found, a modified masking is requested from the generative artificial intelligence (AI-1) using the suggestion unit (211) or an alternatively masked image data matrix (209') is forwarded by the further artificial intelligence (AI-2) for compression and transmission, , ◯ or, if a sufficient match is found, the generated masked image data matrix (209) is forwarded for compression and transmission. [7] Computer-implemented method according to claim 6, characterized by, that the comparison unit (210) performs a comparison based on an LPIPS - Learned Perceptual Image Patch Similarity - algorithm, which compares the perceptual similarity of individual images based on a higher order image structure, and / or compares on the basis of a Visual Loss algorithm and / or compares on the basis of a Perceptual Loss algorithm. [8] Computer-implemented method according to claim 6 or claims 6 and 7, characterized by that the generative artificial intelligence (AI-1) was trained by a method according to claim 1 or according to claims 1 and 2. [9] Computer-implemented method according to at least one of the preceding claims 6 to 8, characterized by, that the transmitted masked image data matrix (209; 209') is converted into an output image data set (c') by means of a decoder unit (806) of another artificial intelligence (AI-X), wherein it is provided in particular that the decoder unit (806) refills the masked areas to ensure a uniform size of the matrices to be processed.

Images