Program, method, information processing device, and trained model

Abstract

Identify the data layout. [Solution] A program that causes a computer to perform the following actions: acquire a first data record; input input data based on the first data record into a trained model that has been trained to take training input data based on the second data record as input and output a second data layout showing the arrangement of data types in the second data record; and acquire a first data layout showing the arrangement of data types in the first data record from the trained model.

Description

Technical Field

[0001] The present invention relates to a program, a method, an information processing apparatus, and a trained model.

Background Art

[0002] Legacy migration is being performed to transfer an existing system that operates based on source code written in a certain programming language to a new system. Legacy migration enables operational improvements and addition of features that did not exist in the existing system. When transferring the system, conversion of the source code is required. Regarding source code conversion, Patent Document 1 describes a system that converts first source code written in a first programming language into second source code written in a second programming language by structuring the constituent elements of the first source code as an aggregate of nodes.

Prior Art Literature

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problem to be Solved by the Invention

[0004] The existing system to be migrated performs operations such as data processing using a database used by the existing system. When the system is migrated after rewriting the source code by converting the source code of the system from a first programming language to a second programming language, it is necessary to perform data migration for converting data into a format compatible with the new system.

[0005] When migrating data, it is necessary to maintain the data types and data structures used in the current system. During data migration, it is also necessary to convert the character encoding used to describe the data to match the data types and data structures of the data being migrated, to a character encoding compatible with the new system. To convert the character encoding, it may be necessary to identify the data layout, which indicates the data delimiters, data types, and the order of data types, and then convert the character encoding accordingly.

[0006] Thus, in data migration, it is sometimes necessary to identify the data layout. However, Patent Document 1 does not disclose how to identify the data layout.

[0007] Therefore, the present invention aims to provide an information processing device, program, and method capable of identifying a data layout. [Means for solving the problem]

[0008] A program according to one aspect of this disclosure causes a computer to perform the following actions: to acquire a first data record; to input input data based on the first data record into a trained model that has been trained to take training input data based on the second data record as input and output a second data layout showing the arrangement of data types in the second data record; and to acquire a first data layout showing the arrangement of data types in the first data record from the trained model.

[0009] Another aspect of the present disclosure involves a computer acquiring a first data record and inputting the input data based on the first data record into a trained model that has been trained to take training input data based on the second data record as input and to output a second data layout showing the arrangement of data types in the second data record, thereby acquiring a first data layout showing the arrangement of data types in the first data record from the trained model.

[0010] An information processing device according to another aspect of the present disclosure includes a data acquisition unit that acquires a first data record, and a data layout acquisition unit that takes input data based on the first data record as input to a trained model that has been trained to take training input data based on the second data record as input and to output a second data layout that shows the arrangement of data types in the second data record as output, and acquires a first data layout that shows the arrangement of data types in the first data record from the trained model.

[0011] A trained model according to another aspect of the present disclosure is trained to take training input data based on a second data record different from a first data record as input, and to output a second data layout showing the arrangement of data types in the second data record, and to take input data based on a first data record as input, and to output a first data layout showing the arrangement of data types in the first data record. [Effects of the Invention]

[0012] According to the present invention, it is possible to provide an information processing device, a program, and a method that can identify the data layout in a data record. [Brief explanation of the drawing]

[0013] [Figure 1] This is a schematic diagram of a data layout identification system. [Figure 2] This is a block diagram of the data layout identification device and each database in the data layout identification system. [Figure 3] This is a diagram illustrating the data records before conversion. [Figure 4] This is a diagram illustrating the training data records. [Figure 5] This diagram illustrates the input data used for training and the trained model. [Figure 6] This diagram illustrates an example of splitting data records before conversion. [Figure 7]This is a diagram showing an example of rules used for data record division. [Figure 8] This is a diagram for explaining the operation of a trained model. [Figure 9] This is a flowchart of processing in a data layout identification apparatus. [Figure 10] This is a diagram for explaining an example of character code conversion by the data layout identification apparatus. [Figure 11] This is a diagram for explaining another example of character code conversion. [Figure 12] This is a diagram for explaining an example of data record division. [Mode for Carrying Out the Invention]

[0014] Preferred embodiments of the present invention will be described with reference to the attached drawings. Note that in each drawing, components denoted by the same reference numeral have the same or corresponding configurations.

[0015] Figure 1 shows a block diagram illustrating an overview of a data layout identification system 10 according to the present embodiment. The data layout identification system 10 includes a data layout identification apparatus 101, a migration source database 201, and a migration destination database 301.

[0016] The data layout identification system 10 is a system that, for example, identifies the data layout of data records used by a system on a mainframe (hereinafter simply referred to as "layout identification") during legacy migration. Here, in the present embodiment, a "data record" refers to, for example, a unit of data acquired or processed by the data layout identification apparatus 101, and includes, for example, data for one record recorded in a data file that is actually used by a system on a mainframe. Further, in the present embodiment, "data layout" refers to information indicating the arrangement of data types in each data record.

[0017] Layout identification can also be performed using design information and source code. However, in recent mainframes, systems have become increasingly black-boxed, design information is often not well-organized, and the readability of the source code itself may be degraded. Furthermore, with respect to files used for external cooperation and form output, the layout information of such files may not exist in the source code of the own system. For this reason, layout identification based on design information and source code requires a great deal of man-hours, which becomes a factor that significantly reduces the feasibility of legacy migration. The design information and source code are, for example, specifications and programs describing data structures and processing logic, and may not be maintained. Therefore, it is not easy to identify the data layout of data used by a system.

[0018] The data layout identification device 101 according to the present embodiment of the present invention identifies a layout from a data file actually used by a system on a mainframe (hereinafter referred to as "actual file") without depending on design information or source code. With such a configuration, even when the design information is incomplete or for an external cooperation file not described in the source code, it is possible to efficiently grasp the structure of the data file, and the reliability of migration work can be improved while significantly reducing the work load.

[0019] The actual file is, for example, a data file that is actually read and written in a system operated on a mainframe. Examples of the actual file include a file including data records recorded in the EBCDIC format. The actual file retains a data structure that reflects the actual operation result of the system, and is less likely to have update omissions or description deficiencies unlike design information and source code. Therefore, the actual file is a highly reliable information source that allows direct observation of the actual state of the data structure, and unlike a black box, can be used to grasp the data layout of data records by performing appropriate analysis processing.

[0020] In the data layout identification system 10, the data layout identification device 101 and the source database 201 are connected via a network. Furthermore, the data layout identification device 101 and the destination database 301 are also connected via a network.

[0021] The data layout identification system 10 identifies the data layout of the data stored in the source database 201, which is a mainframe database that operates based on source code written in a first programming language (e.g., COBOL). The data stored in the source database 201 is converted into data that can be used in the destination database 301, which is a database of the new system whose source code has been rewritten in a second programming language (e.g., Java®).

[0022] The data layout identification device 101 is an information processing device such as a server that has a computer that performs predetermined processing by executing a predetermined program stored in the data layout identification device 101.

[0023] Source database 201 is the database used in the source system. Data records in source database 201 are recorded using character encodings such as EBCDIC (Extended Binary Coded Decimal Interchange Code).

[0024] The target database 301 is the database used by the target system. Data records in the target database 301 are recorded using character encoding, for example, ASCII (American Standard Code for Information Interchange).

[0025] Referring to Figure 2, the data layout identification device 101 and the data stored in the source database 201 will be described.

[0026] The data layout identification device 101 includes a communication unit 1011, a storage unit 1012, a data acquisition unit 1013, a data layout acquisition unit 1014, a delimiter position identification unit 1015, an input data generation unit 1016, and a data record conversion unit 1017.

[0027] The communication unit 1011 controls the communication between the data layout identification device 101 and the source database 201 and the destination database 301.

[0028] The memory unit 1012 stores various types of information used in processing by the data layout identification device 101. The memory unit 1012 stores the delimiter rules 10121 and the trained model 10122. The delimiter rules 10121 and the trained model 10122 will be described later.

[0029] The data acquisition unit 1013 retrieves the pre-conversion data record 2011 stored in the source database 201 from the source database 201. Alternatively, the data acquisition unit 1013 may obtain the pre-conversion data record 2011 through another information processing device that can access the source database 201.

[0030] Here, the pre-conversion data record (first data record) refers to a data record whose data layout (i.e., information such as the type, arrangement, and number of digits of data items constituting the record) is unknown. In this embodiment, the data type and structure contained in the first data record are estimated by a trained model 10122 that has been learned through machine learning.

[0031] The data layout acquisition unit 1014 inputs the input data generated based on the pre-conversion data record 2011 into the trained model 10122. The data layout acquisition unit 1014 then acquires the data layout of the pre-conversion data record 2011 based on the input data from the trained model 10122.

[0032] The delimiter position identification unit 1015 identifies the delimiter position where the data records contained in the pre-conversion data record 2011 are divided into predetermined partial data, based on the delimiter rule 10121. The delimiter position identification unit 1015 identifies the delimiter position based on the string contained in the pre-conversion data record 2011 and the delimiter rule 10121 applied to the string.

[0033] The input data generation unit 1016 generates tokens, which are data partitioned from the pre-conversion data record 2011, based on the pre-conversion data record 2011 and the delimiter positions identified by the delimiter position identification unit 1015. Each token is a partial data having at least one data type. The input data generation unit 1016 generates input data based on the tokens. The input data refers to data provided to the trained model 10122, generated based on the pre-conversion data record 2011. For example, it includes the result of representing the byte sequence constituting the data record in hexadecimal according to a predetermined character code (e.g., EBCDIC), and then performing tokenization based on a predetermined rule.

[0034] The data record conversion unit 1017 converts the character code of the pre-conversion data record 2011 from a first character code (e.g., EBCDIC) to a different character code (e.g., ASCII) based on the data layout of the pre-conversion data record 2011 acquired by the data layout acquisition unit 1014, and generates the post-conversion data record 3011. The data record conversion unit 1017 sends the generated post-conversion data record 3011 to the destination database 301.

[0035] Referring to Figure 3, the pre-conversion data record 2011 will be explained. The data on the mainframe is stored in the source database 201 as continuous data on a single line without delimiters, as shown in pre-conversion data record 2011A. Depending on the data length of the data stored in pre-conversion data record 2011, it can be represented in a sorted format, as shown in pre-conversion data record 2011B.

[0036] Data written as "Pre-Conversion Data Record 2011B" can be written as "Pre-Conversion Data Record 2011C" using the character encoding of "Pre-Conversion Data Record 2011".

[0037] In the example in Figure 3, it is assumed that the data record 2011A before conversion contains consecutive data represented as characters: "1114AB817, 11115 706, and 111632696". When these data are sorted and represented according to the data length "7", they are represented as the data record 2011B before conversion. "1114AB817", "11115 706", and "111632696" are represented as "F1F1F1F4C1C2817C", "F1F1F1F54040706C", and "F1F1F1F6F3F2696C" in EBCDIC notation (first hexadecimal notation). The data layout identification device 101 performs character code conversion of the data record 2011C before conversion in EBCDIC notation.

[0038] Referring to Figures 4 and 5, the training data used for training with the trained model 10122 and the trained model 10122 will be described.

[0039] The trained model 10122 is trained using a combination of training data generated based on training data record 4012 and the data layout 4011 of training data record 4012.

[0040] Training data records (second data records) are data records with a known data layout. The second data records have pre-assigned data types and placements for each item, and are used as training data.

[0041] In the example in Figure 4, the learning data record 4012A is a data record containing the data "2225CD928", "222643817", and "2227 707". The data layout identification device 101 has data types in COBOL format, such as a zone type in which each digit of a decimal number is represented by 1 byte (8 bits), a packed type in which each digit of a decimal number is represented by 4 bits, and a character type (X) that represents a fixed-length string in EBCDIC.

[0042] The data stored in the learning data record 4012A is associated with a data layout 4011 that indicates the data delimiter, the data type of each data, and the order in which the data types are arranged. In the example in Figure 4, the data types "9(4)", "X(2)", and "S9(2) COMP3" are arranged in this order in the data layout 4011. In EBCDIC, in the case of zone type, each digit is recorded as 1 byte, so the data layout 4011 has "9(4)" indicating that a 4-digit number is recorded as 4 bytes, "X(2)" indicating that a string of arbitrary characters is recorded as 2 bytes, and "S9(2) COMP3" indicating that a signed decimal number is recorded as 2 bytes. Data layout 4011 indicates that in the data "2225CD928", "2225" is a 4-digit number (zoned type), "CD" is a 2-character string (character type), and "928" is a signed decimal number (COMP-3), with the sign bit stored in the last 4 bits.

[0043] Training data record 4012A can be represented as training data record 4012B in EBCDIC. "2225CD928", "222643817", and "2227 707" are represented as "F2F2F2F5C3C4928C", "F2F2F2F6F4F3817C", and "F2F2F2F74040707C" in EBCDIC.

[0044] The training data record 4012B, written in EBCDIC format, is divided based on the data layout 4011 to generate tokens 4013A, 4013B, and 4013C. Token 4013A corresponds to the data type "9(4)", token 4013B corresponds to the data type "X(2)", and token 4013C corresponds to the data type "S9(2) COMP3".

[0045] The trained model 10122 is trained using a set of training input data 5011 and data layout 4011, which have tokens 4013A, 4013B, and 4013C, as shown in Figure 5. In addition to the set of training input data 5011 and data layout 4011, several other sets of training input data and data layout are available.

[0046] The trained model 10122 is trained to output data layout 4011, for example, in response to training input data 5011 having tokens 4013A, 4013B, and 4013C.

[0047] In the trained model 10122, when input data 5021 is input to the trained model 10122, the data layout of the input data 5021 is output, for example, as data layout 5022. In the example in Figure 5, it is shown that "9(4) X(2)S9(2)COMP3" is obtained as data layout 5022.

[0048] Thus, the data layout identification system 10 uses the trained model 10122 to identify the data layout of the data contained in the pre-transformation data record 2011.

[0049] In the data layout identification device 101, the input data generation unit 1016 generates input data 5021 based on the pre-conversion data record 2011. In the generation of input data, the delimiter position identification unit 1015 identifies the data delimiter positions in the pre-conversion data record 2011.

[0050] As shown in Figure 6, the delimiter position identification unit 1015 identifies the delimiter position D in the pre-conversion data record 2011C, which is expressed in EBCDIC format, based on the rule shown in Figure 7. The rule shown in Figure 7 is a rule for identifying the delimiter position based on the type of a predetermined character in the string and the type of characters surrounding the predetermined character. Note that the rule for identifying the delimiter position is not limited to this, and for example, if part of the data layout of the data to be converted is known, the delimiter position may be identified based on other rules, such as dividing the data within a predetermined data length range.

[0051] The delimiter position identification unit 1015 scans the characters represented in hexadecimal in 2-byte increments, and if it matches the delimiter rule 10121 shown in Figure 7, it delimits the data up to that character as a single sub-data. For example, the delimiter position identification unit 1015 scans the characters starting from the top left of the pre-conversion data record 2011C. In the example in Figure 6, the scan is performed up to "F1F1F1F4C1C281" in the first row, and when "7C" is scanned, there is no data to the right of "7C". At this time, the delimiter position identification unit 1015 aggregates all the data in the up and down directions of the pre-conversion data record 2011C corresponding to "7C". If the occurrence rate of "0C" is 10% or more in all directions, the delimiter position identification unit 1015 determines that rule number "4" in Figure 7 is satisfied. Therefore, the delimiter position identification unit 1015 identifies delimiter position D1 as the delimiter of packed data to the right of the 8th column of the data.

[0052] Next, when scanning the second row up to "F1", "F1", "F1", and "F5", the value to the right of "F5" is "40". At this point, the delimiter position identification unit 1015 aggregates all the data in the vertical direction in the pre-conversion data record 2011C corresponding to "F5". If the occurrence rate of "F0" is 10% or more in all vertical directions, the delimiter position identification unit 1015 determines that rule number "1" in Figure 7 is satisfied. Therefore, the delimiter position identification unit 1015 identifies delimiter position D2, assuming that there is a zone-type data delimiter between the 4th and 5th columns of the data.

[0053] Delimiter D1 indicates that the data located between the left side of delimiter D1 and other delimiter locations is considered packed data and should be delimited. Delimiter D2 indicates that the data located between the left side of delimiter D1 and other delimiter locations (including the data start position) is considered zoned data and should be delimited.

[0054] Based on the delimiter positions identified by the delimiter position identification unit 1015, the input data generation unit 1016 divides the pre-conversion data record 2011C into tokens and generates input data 5021. The input data 5021 has tokens 6011A divided based on the data start position and delimiter position D2, and tokens 6011B divided based on delimiter position D1 and delimiter position D2.

[0055] Here, if we were to attempt to directly identify the data layout from the tokens separated by the delimiter position identification unit 1015 and the input data generation unit 1016, it would appear that the data type of token 6011A separated by the zone-type delimiter position D2 is zone-type 9(4), and the data type of token 6011B separated by the pack-type delimiter position D1 is pack-type S9(4) COMP3. In other words, the identified data layout would be "9(4) S9(4) COMP3". However, since the first four bytes of token 6011B in the second row are "40" and "40", it contains character data, and therefore the data layout "9(4) S9(4) COMP3" is not correctly identified. This is because rule-based delimiters cannot completely identify character data types that can contain strings, numbers, half-width spaces, etc.

[0056] Thus, the tokens divided by the delimiter position identification unit 1015 and the input data generation unit 1016 may contain a mixture of multiple data types. Therefore, in order to correctly identify the data layout, it is necessary to utilize the trained model 10122.

[0057] As shown in Figure 8, the data layout acquisition unit 1014 inputs the input data 5021 into the trained model 10122. Since the trained model 10122 can identify data types on a token-by-token basis, it can identify multiple data types contained in token 6011B and output a data layout.

[0058] In the example in Figure 8, data layout 8011 is output in response to input data 5021. Data layout 8011 is "9(4) X(2)S9(2)COMP3". This correctly identifies the data layout of the pre-conversion data record 2011C.

[0059] Figure 9 shows a flowchart of the processing in the data layout identification device 101, including the series of processes described above.

[0060] In Figure 9, in step S901, the data acquisition unit 1013 acquires the pre-conversion data record 2011 from the source database 201.

[0061] In step S902, the delimiter position identification unit 1015 refers to the delimiter rule 10121.

[0062] In step S903, the delimiter position identification unit 1015 identifies the delimiter position of the data record 2011 before conversion.

[0063] In step S904, the input data generation unit 1016 generates input data in which the pre-conversion data record 2011 is separated into tokens based on the delimiter position.

[0064] In step S905, the data layout acquisition unit 1014 inputs the input data into the trained model 10122.

[0065] In step S906, the data layout of the pre-transformation data record 2011 is obtained from the trained model 10122.

[0066] In step S907, the data record conversion unit 1017 converts the character code of the data record 2011 before conversion based on the acquired data layout and generates the data record 3011 after conversion.

[0067] In step S908, the data record conversion unit 1017 sends the converted data record 3011 to the destination database 301 and records it. Alternatively, the data record conversion unit 1017 may only generate the converted data record 3011 and have the converted data record 3011 recorded in the destination database 301 through another information processing device.

[0068] Using the flow shown in Figure 9, layout identification can be performed from the actual file (pre-conversion data record 2011). The data layout identification device 101 reads the EBCDIC data in the actual file and visualizes the binary structure of the data by converting the character code to hexadecimal. Next, the data layout identification device 101 performs a predetermined rule-based tokenization process on the converted hexadecimal data to divide the data sequence into meaningful units. Then, the data layout identification device 101 infers the data type and layout structure of each token from the tokenized data sequence, for example, using a trained model 10122. In this process, training data based on token sequences obtained from data files with known layouts is used to generate the trained model 10122.

[0069] This makes it possible to identify the data layout of data records using data records actually used in the system on the mainframe. Furthermore, it eliminates the need to identify the layout based on design information or source code as in the past, and the layout identification work necessary for data migration can be performed even if the design information or source code is incomplete or nonexistent. In addition, in this embodiment, the character code of the actual file is obtained in hexadecimal format, and after rule-based tokenization processing is performed, machine learning is used to infer the data layout of the input data 5021 based on the token sequence. This enables highly accurate layout identification even for files whose structure is not explicitly stated. This makes it possible to simultaneously reduce the workload and improve feasibility in legacy migration. In this embodiment, the data layout identification device 101 inferred the data type of each token and the data layout of the data record from the tokenized data sequence using a trained model 10122, but the inference of data type and data layout is not limited to this. For example, tokens may be divided using a rule-based method, and the data type of each token and the data layout of the data record may be inferred.

[0070] Referring to Figure 10, we will now explain the data conversion based on the data layout identified by the data layout identification device 101. Here, we will explain only the data from the first row of the data record. Assume that the data record 2011 before conversion has been identified as having the data layout of data layout 8011 as a result of processing by the data layout identification device 101.

[0071] At this time, the data record conversion unit 1017 converts the character code of the data record 2011C before conversion from a first hexadecimal character code (e.g., EBCDIC) to a second hexadecimal character code (e.g., ASCII), and generates the converted data record 10011A. When the converted data record 10011A is represented using the characters indicated by the second hexadecimal character code, it is represented as the converted data record 10011B. When the converted data record 10011B and the data record 2011A before conversion are compared, the two data records match. Therefore, the data record is correctly identified and the conversion is performed.

[0072] Refer to Figure 11 to illustrate an example of character code conversion when the data layout is not specified. If the data layout is not specified, the data in each byte of the original data record 2011C will be converted as a string. The data converted in this way becomes the converted data record 11011A. When the converted data record 11011A is represented using the characters indicated by the second hexadecimal character code, it will appear as the converted data record 11011B. When comparing the converted data record 11011B with the original data record 2011A, the two data sets do not match.

[0073] Specifically, the packed data types "81" and "7C" were converted as separate characters, becoming "61" and "40" respectively, which then converted to "a" and "@" corresponding to the ASCII characters "61" and "40". "81" and "7C" should correctly be converted to "817".

[0074] This is because, while character and zone data types can be converted to character codes as strings, signed data formats, packed formats, or COMP formats require character code conversion that takes into account that these data types are signed data formats, packed formats, or COMP formats. In the example in Figure 10, "81" and "7C" are converted to ASCII codes, and the converted data record 10011 is generated without them being converted to ASCII codes, thus demonstrating that the conversion is correct.

[0075] In the data layout identification system 10, the data layout identification device 101 correctly identifies the data layout of the pre-conversion data record 2011 using the trained model 10122. Correctly identifying the data layout ensures that character code conversion is performed appropriately, reducing the workload of data migration in legacy migration. Furthermore, the data layout identified by the data layout identification system 10 can be used not only for data conversion but also, for example, to evaluate the correctness of source code conversion in legacy migration.

[0076] The advantages of dividing the data into tokens using the delimiter position identification unit 1015 and the input data generation unit 1016, and then inputting them into the trained model 10122 to identify the data layout will be explained with reference to Figure 12.

[0077] As shown in Figure 12, assume that there are data records such as pre-conversion data record 12011, pre-conversion data record 12012, and pre-conversion data record 12013.

[0078] Data record 12011 before conversion contains data arranged in the following order: half-width characters, packed numbers, half-width characters. Data record 12012 before conversion contains data arranged in the following order: full-width characters, packed numbers, full-width characters. Data record 12013 before conversion contains data arranged in the following order: zoned numbers, packed numbers, zoned numbers.

[0079] At this time, when the delimiter position identification unit 1015 and the input data generation unit 1016 divide each data record into tokens, packed digits have the characteristic that the last 4 bits indicate the sign, so the delimiter position of a packed digit can be identified with a high probability. Therefore, when "data of other data types" other than packed digits are included to the right of a "packed digit", the data type of the "data of other data types" can be identified based on the rules shown in Figure 7. In this case, when training the trained model 10122, the data layout can be trained for tokens with a data type where a "packed digit" is located to the right of the "data of other data types", and tokens where "data of other data types" other than packed digits is located to the right of the "packed digit" can be excluded from training the trained model 10122. Therefore, the variation of the training input data 5011 can be reduced, and the training cost of machine learning can be reduced. In addition, by not training unnecessary data, the prediction accuracy of the data layout can be increased. Therefore, by incorporating rule-based predictions where possible, the prediction accuracy can be improved.

[0080] In the data layout identification device 101, the data of the pre-conversion data record 2011 was divided into tokens by the delimiter position identification unit 1015 and the input data generation unit 1016, and the generated input data 5021 was input to the trained model 10122 to identify the data layout. However, in the data layout identification system 10, the pre-conversion data record 2011 may be input to the trained model without being divided to identify the data layout. In this case, the trained model 10122 is trained to use the undivided training data record as training input data and to output the data layout of the training data record. The data layout acquisition unit 1014 may input the pre-conversion data record 2011 as input data to the trained model 10122 and acquire the data layout of the pre-conversion data record 2011 from the trained model 10122.

[0081] Furthermore, the data layout identification device 101 may represent each of the data, including the pre-conversion data record 2011, the post-conversion data record 3011, the training input data 5011, and the input data 5021, based on predetermined representation rules corresponding to the data values. For example, half-width digits may be replaced with "X1" regardless of their value, unsigned zoned digits may be replaced with "Z1" regardless of their value, and packed digits may be replaced with "P1PP (PP is the sign part)" regardless of their value. This makes it possible to accurately train the trained model 10122 and infer data layouts using the trained model 10122 in a manner that does not depend on the specific values ​​of the data.

[0082] The embodiment of this model has been described above. The data layout identification device 101 according to this embodiment includes a data acquisition unit 1013 that acquires a data record 2011 before conversion, and a data layout acquisition unit 1014 that takes input data 5021 based on the data record 2011 as input to a trained model 10122 which has been trained to take training input data 5011 based on a training data record 4012 as input and output a data layout 4011 showing the arrangement of data types in the training data record 4012 as output, and acquires a data layout 8011 showing the arrangement of data types in the data record 2011 from the trained model 10122.

[0083] This allows the trained model 10122 to identify the data layout of the pre-conversion data record 2011. The data layout identification device 101 further comprises: a delimiter identification unit 1015 that identifies at least one delimiter position where the pre-conversion data record 2011 is delimited based on a string contained in the pre-conversion data record 2011 and a predetermined rule for the string; and an input data generation unit 1016 that generates input data 5021 containing at least one first token having at least one data type, which is delimited based on the pre-conversion data record 2011 and at least one delimiter position. The training input data 5011 is the training data The data record 4012 is data containing at least one second token having at least one data type, separated based on a second data layout, and the trained model 10122 is trained to output a data layout 4011 showing the arrangement of data types contained in each of the at least one second tokens, given the input of the training input data 5011, and the trained model 10122 may output a data layout 8011 showing the arrangement of at least one data type contained in each of the at least one first token, given the input of the input data 5012.

[0084] By dividing data records into tokens, the variations in the input data used for inferring data layouts can be reduced, thereby lowering the training cost of machine learning. Furthermore, by avoiding training with unnecessary data, the accuracy of data layout predictions can be improved. Therefore, incorporating rule-based predictions where possible can potentially improve prediction accuracy.

[0085] In the above embodiment, the predetermined rule may be a rule that identifies at least one delimiter position based on the type of a predetermined character in the string and the type of characters surrounding the predetermined character.

[0086] This makes it possible to identify delimiter positions based on the character characteristics of the data type, enabling prediction of the data layout with a high degree of accuracy.

[0087] In the above embodiment, the data layout identification device 101 may further include a data record conversion unit 1017 that converts the first character code of the data record 2011 before conversion based on the data layout 8011 and generates a converted data record 3011 written in a character code different from the first character code.

[0088] This ensures that character encoding conversions are performed based on the properly identified data layout, allowing for the correct conversion of data records. [Explanation of Symbols]

[0089] 10...Data layout identification system, 101...Data layout identification device, 201...Source database, 301...Destination database, 1013...Data acquisition unit, 1014...Data layout acquisition unit, 1015...Delimiter position identification unit, 1016...Input data generation unit, 1017...Data record conversion unit, 10122...Trained model

Claims

1. On the computer, To obtain the first data record, A program that takes input data based on the first data record as input to a trained model that has been trained to take training input data based on the second data record as input and output a second data layout showing the arrangement of data types in the second data record as output, and then takes input data based on the first data record as input to obtain a first data layout showing the arrangement of data types in the first data record from the trained model.

2. The program according to claim 1, To the aforementioned computer, Identifying at least one delimiter position where the first data record is separated based on a string contained in the first data record and a predetermined rule for the string, Further, the system generates input data that includes at least one first token having at least one data type, separated from the first data record, based on the first data record and the at least one delimiter position. The learning input data is data in which the second data record includes at least one second token having at least one data type, and the second data record is separated based on the second data layout. The trained model is trained to output a second data layout that shows the arrangement of data types contained in each of the at least one second token, in response to the input of the training input data. The trained model is a program that, in response to the input of the input data, outputs a first data layout that shows the arrangement of at least one data type contained in each of the at least one first tokens.

3. The program according to claim 2, A program in which the predetermined rule is a rule that identifies the at least one delimiter position based on the type of predetermined character in the string and the type of characters surrounding the predetermined character.

4. A program according to any one of claims 1 to 3, To the aforementioned computer, A program that further performs the following actions based on the first data layout: convert the first character code of the first data record and generate a third data record written in a character code different from the first character code.

5. Computers To obtain the first data record, A method comprising: inputting input data based on the first data record into a trained model that has been trained to take training input data based on the second data record as input and output a second data layout showing the arrangement of data types in the second data record, and obtaining a first data layout showing the arrangement of data types in the first data record from the trained model.

6. A data acquisition unit that acquires the first data record, A data layout acquisition unit takes input data based on the first data record as input to a trained model that has been trained to take training input data based on the second data record as input and output a second data layout showing the arrangement of data types in the second data record as output, and acquires a first data layout showing the arrangement of data types in the first data record from the trained model. An information processing device equipped with the following features.

7. Computers, It is trained to take training input data based on a second data record different from the first data record as input, and to output a second data layout that shows the arrangement of data types in the second data record. A trained model that, in response to input data based on the first data record, outputs a first data layout showing the arrangement of data types in the first data record.

Images