Data migration method, device, processor and electronic device

Abstract

The application discloses a data migration method and device, a processor and an electronic device. The method comprises the following steps: obtaining source data in an initial platform and target data in a target platform, wherein the source data is used to represent data to be migrated from the initial platform, and the target data is used to represent data conforming to a standard form in the target platform; determining a mapping relationship and a conversion rule between the source data and the target data, wherein the mapping relationship is used to represent an association relationship between data elements in the source data and data elements in the target data, and the conversion rule is used to represent a rule of migrating the source data from the initial platform to the target platform and performing conversion on the source data in the target platform; converting the source data into structured data corresponding to the target data in the target platform based on the mapping relationship and the conversion rule; and migrating the structured data from the initial platform to the target platform. The application solves the technical problem that data cannot be effectively migrated.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a data migration method, apparatus, processor, and electronic device. Background Technology

[0002] Currently, in traditional data migration scenarios, there are often problems such as inconsistent table data formats and incompatible interfaces between different platforms.

[0003] In related technologies, data migration often involves operators manually transferring data between the source and target platform interfaces by copying and pasting. This requires manual identification of field mappings, format adjustment, and outlier removal, making the process cumbersome and prone to errors. Therefore, technical problems remain regarding the effective migration of data.

[0004] There is currently no effective solution to the aforementioned technical problems. Summary of the Invention

[0005] This application provides a data migration method, apparatus, processor, and electronic device to at least solve the technical problem of ineffective data migration.

[0006] According to one aspect of the embodiments of this application, a data migration method is provided. The method includes: acquiring source data in an initial platform and target data in a target platform, wherein the source data represents data to be migrated from the initial platform, and the target data represents data conforming to a specified format in the target platform; determining a mapping relationship and transformation rules between the source data and the target data, wherein the mapping relationship represents the association between data elements in the source data and data elements in the target data, and the transformation rules represent the rules for migrating the source data from the initial platform to the target platform and performing transformations on the source data on the target platform; based on the mapping relationship and transformation rules, converting the source data into structured data conforming to the target data in the target platform; and migrating the structured data from the initial platform to the target platform.

[0007] Optionally, determining the mapping relationship between source data and target data includes: matching fields and / or values ​​in the source data and target data to obtain the mapping relationship; determining the transformation rules between source data and target data includes: determining a first transformation rule between source data and target data, and a second transformation rule between source data and target data, wherein the first transformation rule is used to represent explicit rules describing the association relationship and transformation logic between source data and target data in natural language form, and the second transformation rule is used to represent implicit transformation logic inferred using an inference model, the inference model being trained using source data samples and target data samples, the source data samples representing data samples to be migrated from the initial platform, and the target data samples representing data samples conforming to the standard form; and fusing the first transformation rule and the second transformation rule to obtain the transformation rule.

[0008] Optionally, based on the mapping relationship and transformation rules, the source data is transformed into structured data that corresponds to the target data in the target platform, including: binding the data elements in the source data with the data elements in the target data based on the mapping relationship, and performing serialization transformation on the bound source data according to the transformation rules to obtain structured data.

[0009] Optionally, the method further includes: verifying the transformation rules to obtain a verification result; in response to the verification result indicating that the data obtained after transforming the source data conforms to structured data, extracting the structured data to obtain an extraction result, wherein the extraction result includes at least one of the following: the field structure, data type, value mode, and transformation logic of the structured data; and outputting the extraction result as a structured file, wherein the structured file is used to represent a structured description file containing a field mapping relationship table and a data processing flow.

[0010] Optionally, the method further includes: in response to the verification result that the data obtained after transforming the source data does not conform to structured data, modifying the transformation rules, wherein the accuracy of the modified transformation rules is higher than the accuracy of the original transformation rules.

[0011] Optionally, the method further includes: generating an independently executable front-end application file based on the structured file, wherein the front-end application file represents a file that runs directly through a browser in a serverless environment and outputs structured data to the target platform.

[0012] Optionally, the front-end application file includes a data acquisition module, a transformation engine module, an output module, a display module, and a configuration module. The data acquisition module is used to acquire source data and target data; the transformation engine module is used to generate mapping relationships and transformation rules; the output module is used to convert source data into structured data that conforms to the target platform's specifications and send it to the target platform's interface, as well as generate a string of structured data; the display module is used to display the structured data; and the configuration module is used to configure the security of the structured data.

[0013] Optionally, the method further includes: acquiring and / or recording structured data according to a target time period.

[0014] According to another aspect of the embodiments of this application, a data migration apparatus is also provided. The data migration apparatus may include: an acquisition unit, configured to acquire source data in an initial platform and target data in a target platform, wherein the source data represents data to be migrated from the initial platform, and the target data represents data conforming to a specified format in the target platform; a determination unit, configured to determine a mapping relationship and conversion rules between the source data and the target data, wherein the mapping relationship represents the association between data elements in the source data and data elements in the target data, and the conversion rules represent the rules for migrating the source data from the initial platform to the target platform and performing conversion on the source data on the target platform; a conversion unit, configured to convert the source data into structured data conforming to the target data in the target platform based on the mapping relationship and the conversion rules; and a migration unit, configured to migrate the structured data from the initial platform to the target platform.

[0015] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided. The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform the method of the embodiments of this application.

[0016] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the method of the embodiments of this application when it runs.

[0017] According to another aspect of the embodiments of this application, a processor is also provided. The processor is used to run a program, wherein the program executes the methods of the embodiments of this application during runtime.

[0018] According to another aspect of the embodiments of this application, a computer program product is also provided. This computer program product includes a computer program that, when executed by a processor, implements the methods described in the embodiments of this application.

[0019] According to another aspect of the embodiments of this application, a vehicle is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of this application when it runs.

[0020] In this embodiment, if data migration is required, source data from the initial platform and target data from the target platform can be obtained. The source data represents the data to be migrated from the initial platform, and the target data represents data conforming to the specified format in the target platform. The mapping relationship and transformation rules between the source and target data can be determined. The mapping relationship represents the association between data elements in the source data and data elements in the target data, and the transformation rules represent the rules for migrating the source data from the initial platform to the target platform and performing transformations on the source data on the target platform. Based on the mapping relationship and transformation rules, the source data can be converted into structured data corresponding to the target data in the target platform. The structured data can then be migrated from the initial platform to the target platform. In other words, this embodiment determines the mapping relationship and transformation rules between the source and target data by parsing the structural features and semantic intent of the source and target data, and generates structured data corresponding to the target data in the target platform based on this. This achieves automated conversion and migration of source data, overcoming the limitations of related technologies that require manual coding or complex deployment, thus solving the technical problem of ineffective data migration and achieving the technical effect of effective data migration. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of this application and constitute a part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0022] Figure 1 This is a flowchart of a data migration method according to an embodiment of this application;

[0023] Figure 2 This is a schematic diagram of an AI-driven data docking system according to an embodiment of this application;

[0024] Figure 3 This is a flowchart of a rule configuration and verification method according to an embodiment of this application;

[0025] Figure 4 This is a flowchart of a program generation and deployment method according to an embodiment of this application;

[0026] Figure 5 This is a flowchart of a procedure and operation monitoring method according to an embodiment of this application;

[0027] Figure 6 This is a schematic diagram of a data migration apparatus according to an embodiment of this application. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] According to an embodiment of this application, an embodiment of a data migration method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0031] Figure 1 This is a flowchart of a data migration method according to an embodiment of this application, such as... Figure 1 As shown, the method may include the following steps.

[0032] Step S102: Obtain the source data from the initial platform and the target data from the target platform.

[0033] In the technical solution provided in step S102 of this application, the source data can be used to represent the data to be migrated from the initial platform, and the target data can be used to represent the data that conforms to the specification form in the target platform.

[0034] In this embodiment, the aforementioned source data refers to the data to be migrated from the initial platform, which may exist in a non-standardized, semi-structured, or even unstructured form. The initial platform may be an old system, such as an old version of an enterprise warehouse management system (WMS) or an outdated enterprise resource planning (ERP) system.

[0035] Optionally, the source data mentioned above can be Excel reports exported from an older version of the enterprise's Warehouse Management System (WMS). These Excel reports may contain merged cells, multi-level headers, Chinese field names (e.g., location codes), and inconsistent date formats (e.g., mixing "May 1, 2023", "2023 / 5 / 1", and "2023-05-01"). Another example is manually copied table text from screenshots of older Enterprise Resource Planning (ERP) systems, or log files in text format. While these source data contain complete business information, their inconsistent formatting and ambiguous semantics prevent them from being directly recognized by the target platform (e.g., a new system).

[0036] Alternatively, the aforementioned target data can refer to a sample of structured data specifications required by the target platform (e.g., a new WMS system). Target data can be an example of an Application Programming Interface (API) response, such as JavaScript Object Notation (JSON) format; a standard import template, such as Comma-Separated Values ​​(CSV) format conforming to International Organization for Standardization (ISO) standards; or a data structure specification document provided by the target platform's backend.

[0037] Optionally, for source data, users can obtain it by directly uploading Excel, CSV, or text (TXT) files, or by uploading PDFs containing tables, screenshots, or by pasting and copying table text. For target data, users can upload examples of the target platform's interface documentation, the standard CSV import template provided by the target platform, or manually fill in a sample that conforms to the target format as a reference.

[0038] In the embodiments of the present application, by obtaining the source data in the initial platform and the target data in the target platform, an accurate data basis can be provided for determining the mapping relationship and conversion rules between the subsequent source data and target data.

[0039] Step S104: Determine the mapping relationship and conversion rules between the source data and the target data.

[0040] In the technical solution provided in step S104 of the present application, the mapping relationship can be used to represent the association relationship between the data elements in the source data and the data elements in the target data. The conversion rules can be used to represent the rules for migrating the source data from the initial platform to the target platform and performing conversions on the source data on the target platform.

[0041] In this embodiment, after obtaining the source data in the initial platform and the target data in the target platform, the mapping relationship and conversion rules between the source data and the target data can be determined. The mapping relationship can refer to the semantic correspondence relationship between each data element in the source data (for example, "order number", "storage location", "product status" in Excel) and the corresponding fields in the target data (for example, order_id, warehouse_zone, status in JSON).

[0042] Optionally, the above semantic correspondence relationship is not a simple name match, but requires an understanding of semantic consistency and structure conversion requirements. For example, the value of the "storage location" field in the source data is "A-01-02-03", while the target platform requires it to be split into four independent fields: "zone", "rack", "layer", "position". At this time, the mapping relationship is no longer "storage location → storage location", but a one-to-many split mapping of "storage location → [zone, rack, layer, position]". Another example is that the "order date" in the source data is in Chinese format "May 1, 2023", and the target platform requires it to be in ISO standard timestamp "2023-05-01T00:00:00Z", and the mapping relationship needs to include format conversion logic.

[0043] Optionally, the above conversion rules can be the specific operation logics for processing, calculating, cleaning, judging, and routing the source data during the source data migration process. For example, "if the product status is 'to be inspected', then automatically fill the quality inspection warehouse as 'ZJ-01'", "multiply the order amount by 1.13 (tax-inclusive) and write it into the total amount field", "if the supplier code is empty, then default to use 'DEFAULT-SUPP'", etc. The above conversion rules can be input by the user in natural language form, such as "if the status is to be inspected, set the storage location to the quality inspection warehouse", "multiply the amount by 1.13 and round it off".

[0044] Optionally, the above conversion rules can be manual rules or conversion rules obtained by combining artificial intelligence (AI) engines with manual rules.

[0045] Optionally, the aforementioned conversion rules can be implemented by having the user input several natural language commands, with AI automatically completing over 80% of the mapping and rule generation, thus reducing configuration time to just a few minutes. AI can not only identify obvious field correspondences but also discover cross-column logic. For example, complex rules such as "automatically enable special packaging labels when 'quantity' > 100 and 'type' is 'large item'" are extremely difficult to configure with traditional tools, but AI can accurately capture them through semantic association and pattern learning.

[0046] In the embodiments of this application, by determining the mapping relationship and transformation rules between source data and target data, an accurate data foundation can be provided for the subsequent migration of source data from the initial platform to the target platform. This overcomes the limitations of related technologies, which require manual configuration of dozens of parameters, are time-consuming, and have complex rules that are extremely difficult to configure with traditional tools.

[0047] Step S106: Based on the mapping relationship and transformation rules, the source data is transformed into structured data that corresponds to the target data in the target platform.

[0048] In the technical solution provided by step S106 of this application, based on the mapping relationship and conversion rules, the source data is converted into structured data that corresponds to the target data in the target platform. This does not rely on external servers or dedicated software platforms for execution. Instead, the mapping relationship and conversion rules can be directly compiled into front-end JavaScript code logic by AI when generating the final HyperText Markup Language (HTML) program, and embedded in the automatically generated single-file webpage.

[0049] In this embodiment, the above steps mean that when the user finally runs the HTML program, the source data conversion is completed locally in the browser without connecting to the backend service, which meets the enterprise's core requirements for data security and lightweight deployment.

[0050] Optionally, the transformation of source data can be a multi-dimensional, multi-level automated processing flow. For example, field-level mapping and structural reorganization can be performed. When the mapping relationship determines that "Location: A-01-02-03" needs to be split into four fields: "Area," "Shelf," "Layer," and "Location," the embedded JavaScript in the program can call string processing functions to decompose the original field values ​​using regular expressions or delimiters (such as "-") and assign them to the four independent fields in the structured data. If there are misaligned data across rows due to merged cells in the source data (for example, "Product Name" spans two rows in a table), the AI-generated parsing engine can first identify the cell attributes, intelligently merge or complete missing rows, ensure data alignment between rows, and avoid field misalignment caused by formatting issues.

[0051] Optionally, data cleaning and format standardization can also be performed. If the conversion rules include instructions such as "dates should be formatted to ISO", "amounts should be rounded to two decimal places", and "spaces and hyphens should be removed from phone numbers", the corresponding conversion functions can be automatically called. For example, "May 1, 2023" can be recognized as a Date object by the built-in date parser and then formatted as "2023-05-01T00:00:00Z"; "¥1,234.56" can be converted to the floating-point number 1234.56 after removing the currency symbol and thousands separator.

[0052] Optionally, conditional statements and business logic can also be embedded. When the conversion rule involves "if the product status is 'pending inspection,' then the storage location is automatically set to 'ZJ-01'," conditional statements (if-else) can be generated to iterate through each source data record, judge the value of the status field, and dynamically override the default value of the target field. More complex logic, such as "if the order amount is >1000 and the customer type is VIP, then enjoy a 10% discount and mark it as 'priority processing,'" can also be fully translated into nested conditional statements to accurately restore the business intent.

[0053] Optionally, the HTML program can also support multi-source data fusion and null value handling. If some fields are missing in the source data (for example, the supplier code is empty), the AI-generated logic can automatically call the preset default value filling mechanism, such as "if empty, use 'DEFAULT-SUPP'". If it is necessary to synthesize a new field from multiple source fields (for example, merging "city + street" into "detailed address"), the concatenation logic can also be automatically generated to ensure the integrity of the structured data.

[0054] In this embodiment, based on mapping relationships and transformation rules, source data is converted into structured data that corresponds to the target data on the target platform. Regardless of the user's device, operating system, or browser environment when opening the HTML program, as long as the same source data is input, the output results are completely consistent, eliminating problems caused by environmental differences and greatly improving the reliability of the migration work. No manual secondary verification is required, allowing for seamless and accurate migration of structured data from the initial platform to the target platform, achieving the technical effect of effective data migration.

[0055] Step S108: Migrate the structured data from the initial platform to the target platform.

[0056] In the technical solution provided by step S108 of this application, after converting the source data into structured data that corresponds to the target data in the target platform, the structured data can be migrated from the initial platform to the target platform.

[0057] In this embodiment, the migration action can be encapsulated as an output dispatcher module within the HTML program, dynamically generating the corresponding front-end execution logic based on the output target type specified by the user during the rule configuration phase. This can include the following three flexible migration paths.

[0058] Optionally, automatic push can be achieved via the Hypertext Transfer Protocol (HTTP) API. When the target platform supports standard interfaces, the HTML program can automatically generate JavaScript code to construct JSON or form data requests that conform to the format of the target data, simulating the target platform's call behavior. For example, the cleaned receiving task data can be encapsulated and automatically include the API key entered by the user at runtime (encrypted and stored on the front end) to send a request to the interface address of the target platform (e.g., the new WMS system). The entire process requires no backend service support and is initiated entirely by the browser locally. The response results (success / failure status, error messages) can be captured and recorded in the log interface in real time, and users can retry failed items with one click.

[0059] Optionally, the system can generate and automatically upload files to a File Transfer Protocol (FTP) or local storage. If the target platform only supports file import (e.g., some older TMS or financial systems), it can automatically serialize structured data into standard CSV, TXT, or Extensible Markup Language (XML) files, and use the browser's native Binary Large Object (Blob) and related libraries to simulate file uploads to the FTP server, or provide a download link.

[0060] Optionally, mixed output and multi-target distribution are supported. In real-world business scenarios, a single migration often needs to meet the requirements of multiple downstream systems simultaneously. For example, the new WMS needs to receive API data, while the transportation system needs CSV files, and the financial system needs Excel reports. The HTML program can generate multiple output channels simultaneously. After a single conversion, JSON is automatically generated and pushed to the WMS, CSV is uploaded to the FTP server, and Excel files are saved locally, achieving one-time generation and multiple outputs.

[0061] In this embodiment of the application, after the source data is converted into structured data that corresponds to the target data in the target platform, the structured data is migrated from the initial platform to the target platform, thus ensuring the effectiveness of the data migration.

[0062] In the embodiments of this application, steps S102 to S108 above, if data migration is required, can obtain source data from the initial platform and target data from the target platform. The source data represents data to be migrated from the initial platform, and the target data represents data conforming to the specified format in the target platform. The mapping relationship and conversion rules between the source data and the target data can be determined. The mapping relationship represents the association between data elements in the source data and data elements in the target data, and the conversion rules represent the rules for migrating the source data from the initial platform to the target platform and performing conversions on the source data on the target platform. Based on the mapping relationship and conversion rules, the source data can be converted into structured data corresponding to the target data in the target platform. The structured data can then be migrated from the initial platform to the target platform. In other words, the embodiments of this application determine the mapping relationship and conversion rules between source data and target data by parsing the structural features and semantic intent of source data and target data, and generate structured data that conforms to the target data in the target platform, thereby realizing the automatic conversion and migration of source data. This overcomes the limitations of related technologies that require manual coding or complex deployment, thus solving the technical problem of ineffective data migration and achieving the technical effect of effective data migration.

[0063] The method described in this embodiment will be further described below.

[0064] As an optional embodiment, step S104, determining the mapping relationship between source data and target data, includes: matching fields and / or values ​​in the source data and target data to obtain the mapping relationship; step S104, determining the transformation rules between source data and target data, includes: determining a first transformation rule between source data and target data, and a second transformation rule between source data and target data, wherein the first transformation rule is used to represent explicit rules describing the association relationship and transformation logic between source data and target data in natural language form, and the second transformation rule is used to represent implicit transformation logic inferred by an inference model, the inference model being trained through source data samples and target data samples, the source data samples representing data samples to be migrated from the initial platform, and the target data samples representing data samples conforming to the standard form; the first transformation rule and the second transformation rule are fused to obtain the transformation rule.

[0065] In this embodiment, in the process of determining the mapping relationship and transformation rules between source data and target data, a dual-track collaborative mechanism of explicit rule guidance (corresponding to the first transformation rule) + implicit reasoning supplement (corresponding to the second transformation rule) is adopted to construct a rule generation system that is human-machine collaborative, complementary, and intelligently enhanced, which significantly improves the accuracy, completeness, and adaptability of the transformation rules.

[0066] Optionally, the above mapping relationship can be established by performing multi-dimensional matching of fields and / or values ​​in the source and target data, such as field names, field value patterns, data types, and structural features. For example, by analyzing the "order number" field in the source data and the "order_id" field in the target data, it is found that they are highly similar semantically, and their value ranges are both alphanumeric combinations (e.g., "PO-2023-001" and "IN20230001"). Through fuzzy matching algorithms, word vector similarity calculations, and field naming rule bases (e.g., "PO" corresponds to "Purchase Order", and "IN" corresponds to "Inbound"), association inferences are made, and finally, the mapping relationship "order number → order_id" is established. For value pattern matching, it is also possible to identify different formats of the same time expression, such as "May 1, 2023" and "2023-05-01", thereby assisting in establishing field-level semantic alignment.

[0067] Optionally, in constructing the aforementioned conversion rules, a distinction is made between the first and second conversion rules, forming a dual-engine structure of user intent-driven (corresponding to the first conversion rule) + AI experience supplementation (corresponding to the second conversion rule). The first conversion rule can originate from natural language descriptions actively input by users, and is an explicit rule directly expressed by business personnel based on domain knowledge and operational experience. For example, the user inputs "For records where the 'product status' is 'pending inspection,' the 'location' field will be automatically replaced with 'ZJ-01'" and "The 'total amount' field needs to be multiplied by 1.13 and rounded to the nearest integer." These first conversion rules are clear in language and have explicit intent, forming the "main framework" of the conversion logic, and possessing a high degree of controllability and interpretability.

[0068] Optionally, the second conversion rule can be automatically generated by an inference model trained by AI on a large amount of historical sample data, which is a result of implicit rule mining. This inference model can use deep learning technology to semantically model paired source-target data samples and learn the non-explicit association patterns between fields. For example, if in the past 500 migration tasks, as long as "warehouse code = WH-01" and "quantity > 50" appear in the source data, the corresponding "priority" field in the target platform has a 92% probability of being set to "high". Even if the user does not explicitly state this logic, AI can still automatically recommend "when the warehouse code is WH-01 and the quantity is greater than 50, priority = high" as a supplementary second conversion rule based on statistical regularities and contextual patterns. The second conversion rule is often hidden in the deep structure of data distribution, which is an empirical logic that is difficult for business personnel to express clearly in language but occurs frequently in practice.

[0069] Optionally, these two types of rules do not exist independently but are dynamically integrated through a fusion processing mechanism. The first transformation rule can be used as a benchmark to ensure that the user's intent is fully preserved. Subsequently, the second transformation rule is used as an enhancement signal for semantic consistency verification. If the inferred implicit rule conflicts with the explicit rule (for example, the AI ​​suggests "status = pending inspection → location = ZJ-01", but the user has explicitly specified "ZJ-02"), the first transformation rule can be adopted first, and the AI ​​suggestion can be marked as "pending confirmation" for the user to review. If the implicit rule is consistent with or complementary to the explicit rule (for example, the user did not mention the priority rule when "quantity > 50"), the second transformation rule can be automatically adopted and written into the final transformation rule set.

[0070] In this application embodiment, by synergistically integrating explicit rules (corresponding to the first transformation rule) and implicit rules (corresponding to the second transformation rule), an intelligent rule generation mechanism that respects business experience and unleashes the potential of AI is constructed, solving the pain points of incomplete rules and reliance on manual intervention in traditional methods.

[0071] As an optional embodiment, step S106, based on the mapping relationship and conversion rules, converts the source data into structured data that corresponds to the target data in the target platform, including: binding the data elements in the source data with the data elements in the target data based on the mapping relationship, and performing a serialization conversion operation on the bound source data according to the conversion rules to obtain structured data.

[0072] In this embodiment, based on the mapping relationship and conversion rules, the source data is converted into structured data that corresponds to the target data in the target platform. Through the dual mechanism of binding and serialization conversion operations, the automatic reconstruction of the format from the source data (e.g., the original semi-structured data) to the target data can be achieved, so as to ensure that the conversion process is accurate and traceable.

[0073] Optionally, based on the mapping relationship, a dynamic reference relationship can be established between fields in the source data and fields in the target data. For example, the "order number" field in the source data is mapped to the "order_id" field in the target data. The target platform does not simply copy the value, but defines a data source path for each field in the target data within the HTML program. This binding mechanism supports multiple mapping forms, including one-to-one (e.g., "product code → product_code"), one-to-many (e.g., "location A-01-02-03" split into four fields), many-to-one (e.g., "province + city + district" merged into "complete address"), and even cross-row binding (e.g., filling in the values ​​of merged cells in a table to the corresponding rows). The binding process can be automatically completed by an AI-generated JavaScript engine without manual intervention, and can intelligently handle exceptions such as missing fields, null values, and type mismatches. For example, when a field in the source data is empty, it is automatically skipped or filled with a default value.

[0074] Optionally, after binding is complete, the serialization conversion stage can begin. This stage is not a simple assignment, but rather performs semantic-level processing on each bound data item according to the conversion rules. Each conversion rule can be compiled into an independent, executable function unit, embedded in the JavaScript execution of the HTML program.

[0075] As an optional embodiment, the method further includes: verifying the conversion rules to obtain a verification result; in response to the verification result indicating that the data obtained after converting the source data conforms to structured data, extracting the structured data to obtain an extraction result, wherein the extraction result includes at least one of the following: the field structure, data type, value mode, and conversion logic of the structured data; and outputting the extraction result as a structured file, wherein the structured file is used to represent a structured description file containing a field mapping relationship table and a data processing flow.

[0076] In this embodiment, after the mapping relationship and transformation rules are constructed, a rule verification and structured description file output mechanism can be introduced to form a closed-loop process of "rule generation - verification and confirmation - file output".

[0077] Optionally, the AI ​​platform can provide a validation environment. For example, a user can upload a new, untrained source data test set and instruct the AI ​​to generate target data based on the learned transformation rules.

[0078] Optionally, after the transformation rules are validated, AI can output a structured file, which may include a detailed field mapping table and a data processing flowchart. This serves as both a confirmation document for the user and an internal blueprint for subsequent code generation.

[0079] In this embodiment, each successful migration task can be automatically transformed into a standardized, structured, executable knowledge asset, becoming a formal document library for data governance, providing a precise operational blueprint for similar platform switching and integration of old and new systems in the future.

[0080] As an optional embodiment, the method further includes: in response to the verification result that the data obtained after transforming the source data is not structured data, modifying the transformation rules, wherein the accuracy of the modified transformation rules is higher than the accuracy of the original transformation rules.

[0081] In this embodiment, if the data obtained after transforming the source data is not structured data, the iterative correction process of the transformation rules can be automatically triggered. Through human-machine collaborative feedback, the accuracy and robustness of the transformation rules can be continuously improved.

[0082] Optionally, users (or through connecting to the actual target system interface) can verify the generated results. If discrepancies exist, users can point out the problems (e.g., incorrect date format conversion). The AI ​​can analyze the reasons for the discrepancies and guide users to supplement conversion rules or correct samples, entering an iterative optimization loop until the accuracy of the conversion rules meets the requirements.

[0083] Optionally, the above correction process is not a simple rule replacement, but rather intelligent reasoning and rule generalization based on semantic understanding. The AI ​​engine can analyze error patterns. For example, if "Chinese date not converted" occurs repeatedly, it can be inferred that "the user group generally expects Chinese dates to be unified into ISO format," thereby strengthening this conversion pattern in the global rule base, not only correcting the current task but also optimizing the default behavior of similar future tasks. If it is found that "slot splitting" fails due to a change in the separator (such as from "-" to "."), the matching logic can be upgraded from exact matching to "an intelligent parser that supports multiple separators," improving the rule's generalization ability.

[0084] In this embodiment, through a closed-loop mechanism of verification-feedback-correction-evolution, data migration can be transformed from a one-off technical task into a sustainable, accumulative, and inheritable business capability, laying a solid foundation for the subsequent construction of an enterprise-level AI-driven data governance platform.

[0085] As an optional embodiment, the method further includes: generating an independently executable front-end application file based on the structured file, wherein the front-end application file represents a file that can be run directly through a browser in a serverless environment and outputs structured data to a target platform.

[0086] In this embodiment, after generating the structured file, a separate, directly executable front-end application file, i.e., an HTML file, can be automatically compiled and output based on the precisely recorded field mapping relationships, data type specifications, conversion logic, and processing flow in the structured file. The front-end application file embeds JavaScript code.

[0087] Optionally, the HTML file can include not only a user interface (such as a file upload area, a run button, and a log display box), but also deeply integrate four functional modules generated by AI: a data acquisition module, a conversion engine module, an output distribution module, and a configuration management module.

[0088] Optionally, the generation of this front-end application file breaks away from the dependencies of traditional data migration solutions on server deployment, middleware installation, permission requests, and network connectivity. The front-end application file does not depend on a Java environment, server type, or database connection pool, and can be opened with a browser. Users simply need to copy the front-end application file to a device with a modern browser to run it; no administrator privileges, plugin installation, or firewall policy configuration is required.

[0089] In this embodiment of the application, by automatically compiling structured files into independently executable front-end application files, not only is the technical problem of how to execute them solved, but the collaborative method of how to execute them securely is also reconstructed, making data migration more efficient.

[0090] As an optional implementation, the front-end application file includes a data acquisition module, a conversion engine module, an output module, a display module, and a configuration module. The data acquisition module is used to acquire source data and target data; the conversion engine module is used to generate mapping relationships and conversion rules; the output module is used to convert the source data into structured data that conforms to the target platform's specifications and send it to the target platform's interface, as well as generate a string of structured data; the display module is used to display the structured data; and the configuration module is used to configure the security of the structured data.

[0091] In this embodiment, the front-end application file can be a highly integrated, self-contained, dependency-free single-file web application. The front-end application file may include a data acquisition module, a conversion engine module, an output module, a display module, and a configuration module, which together realize a complete closed loop from data input to secure output.

[0092] Optionally, the data acquisition module can generate code in two ways, depending on the configuration. Graphical file upload interface: Generates HTML form containing elements and JavaScript code for the FileReader API to read manually uploaded Excel / CSV files. Automatic fetching logic (configurable on demand): Generates JavaScript code that simulates login and accesses legacy system report pages using the fetch API or XML HTTP Request API, and parses HTML table data using the Document Object Model Parser (DOMParser).

[0093] Optionally, the transformation engine module can be a core transformation engine module that generates JavaScript functions that implement cleaning, transformation, and judgment rules. For example, it can transform the natural language rule "split cargo location field" into specific string splitting and assignment code.

[0094] Optionally, the output module can be an output dispatcher module that can generate multiple output codes. For example, one path might use the fetch API to construct an HTTP POST request, sending the converted data in JSON format to the new WMS interface. Another path might use JavaScript to generate a CSV format string and use relevant libraries or Blob objects to simulate file uploads to an FTP server, or provide a download link.

[0095] Optionally, the configuration module can be a configuration management module that generates secure configuration processing code. For sensitive information such as API keys and server addresses, the AI ​​will generate an initialization configuration interface to guide the user through the initial setup. Password-like information can be encrypted on the front end, for example, using a cryptographic JavaScript library (CryptoJS) for symmetric encryption before being stored in the browser's local storage (localStorage). It can then be decrypted and used at runtime, avoiding hardcoding sensitive information in the code.

[0096] In addition, it can include a user interface (UI) and a logging module, which can generate HTML elements to display the program's running status, such as buttons, log text boxes, and progress bars. It can also generate logging functionality, writing the time, status, and details of key operations (such as "Started conversion," "API call successful / failed," "File saved") to an in-memory array and displaying it on the UI, or optionally using localStorage for persistent storage.

[0097] As an optional embodiment, the method further includes: acquiring and / or recording structured data according to a target time period.

[0098] In this embodiment, after the generation and output of structured data are completed, the data acquisition and processing process can be automatically triggered according to the target time period, which upgrades the one-time table export that originally relied on manual operation into an automated data hub with the ability to execute on a timed basis, be unattended, and run continuously, realizing the leap from passive response to proactive service.

[0099] Optionally, in the AI-generated HTML program, a JavaScript timer function can be automatically injected based on the target time period set by the user during the configuration phase (e.g., every 30 minutes, 2 AM daily, 9 AM Monday, etc.), and the corresponding data acquisition and transformation logic can be bound. When the user opens the HTML program in the browser for the first time, the scheduled task can be started without manual intervention. For example, in an embodiment during the transition period of the warehouse system switchover, the HTML program can be configured to automatically log in to the web interface of the old WMS system every hour, simulate user operations to retrieve the latest inbound plan Excel report, parse the structure, call the mapping rules and transformation logic already embedded in the program, clean and reorganize the data into the JSON format required by the new WMS, and push it to the target platform's interface in real time via HTTP POST. At the same time, the operation log is recorded on the local interface and persistently stored in the browser's localStorage. The entire process is fully automated, requiring no personnel to log in to the system, upload files, or click buttons.

[0100] Optionally, to ensure the stability and reliability of scheduled tasks, multiple fault-tolerance mechanisms can be built-in. For example, in the event of network interruption or target interface timeout, the HTML program can automatically retry (the number of retries and intervals can be configured). If consecutive failures occur, an alarm log will be triggered and scheduling will be paused until manual confirmation before resuming. If the source data format undergoes minor changes (e.g., column name order adjustment), intelligent adaptation can be attempted based on historical learning records, or the user will be prompted to re-verify the rules. In addition, the trigger time and execution status of scheduled tasks can be displayed in real time in the front-end graphical interface, including "next execution time," "last successful time," and "failure count statistics," allowing users to view the running status at any time and achieve visual monitoring.

[0101] In this embodiment, if data migration is required, source data from the initial platform and target data from the target platform can be obtained. The source data represents the data to be migrated from the initial platform, and the target data represents data conforming to the specified format in the target platform. The mapping relationship and transformation rules between the source and target data can be determined. The mapping relationship represents the association between data elements in the source data and data elements in the target data, and the transformation rules represent the rules for migrating the source data from the initial platform to the target platform and performing transformations on the source data on the target platform. Based on the mapping relationship and transformation rules, the source data can be converted into structured data corresponding to the target data in the target platform. The structured data can then be migrated from the initial platform to the target platform. In other words, this embodiment determines the mapping relationship and transformation rules between the source and target data by parsing the structural features and semantic intent of the source and target data, and generates structured data corresponding to the target data in the target platform based on this. This achieves automated conversion and migration of source data, overcoming the limitations of related technologies that require manual coding or complex deployment, thus solving the technical problem of ineffective data migration and achieving the technical effect of effective data migration.

[0102] The technical solutions of the embodiments of this application will be illustrated below with reference to preferred embodiments.

[0103] Currently, traditional data export operations are often performed entirely manually, either by writing conversion scripts by hand or by using graphical extraction, transformation, and loading (ETL) tools.

[0104] Completely manual data import involves operators manually transferring data between a source spreadsheet (e.g., Excel) and the target system interface via copy and paste. Format conversion requires manual identification and adjustment. Field mapping requires manual lookup and mapping. Data cleaning requires manual judgment and modification. Therefore, this process suffers from low efficiency, uncontrollable error rates, heavy reliance on personal experience, and poor flexibility and reusability.

[0105] Manually writing transformation scripts involves programmers writing data transformation scripts (e.g., using Python Pandas, SQL, or JavaScript) based on the format requirements of the source and target tables. These scripts need to explicitly handle all rules related to data extraction, type conversion, field mapping, and row / column transformations. Therefore, this approach suffers from low development efficiency, high professional barriers, and high maintenance costs.

[0106] Using ETL tools allows users to define data flow pipelines by dragging and dropping components and configuring parameters, such as setting up input, cleaning, transformation, and output nodes in Kettle or Informatica. However, the configuration remains complex, the generated code is opaque and rigid, and it cannot handle non-standard tables.

[0107] In conclusion, the relevant technologies have failed to effectively bridge the gap between "the business users' intended rules" and "executable procedures." Whether relying entirely on manual operation or utilizing scripts or complex tools, neither approach can guarantee ease of use while simultaneously providing the ability to intelligently understand data and generate lightweight, independently runnable, and easily customizable transformation programs. This has become the core bottleneck restricting the efficiency and quality improvement of data export.

[0108] To address the aforementioned issues, this application provides an intelligent table data migration system and method based on AI and HTML.

[0109] Solve the problem of automating the "rules-to-code" generation. AI automatically generates high-quality, HTML / JavaScript-based executable programs from user-described transformation rules or examples, achieving automated conversion from business requirements to directly runnable programs, completely replacing manual coding and tedious configuration.

[0110] Solve the challenge of intelligent parsing of complex tables. Utilizing deep learning-based Optical Character Recognition (OCR) and computer vision technologies, it accurately identifies complex structures in source tables, such as merged cells, multi-level headers, and misaligned rows and columns, laying a solid foundation for accurate subsequent conversion.

[0111] This solution addresses the challenge of intelligently identifying data mapping relationships. Even when users haven't explicitly specified all rules, AI leverages its understanding of data semantics to intelligently infer the mapping relationships and transformation logic between source and target fields, significantly reducing manual configuration workload.

[0112] This solution addresses the issues of poor compatibility and complex deployment inherent in traditional solutions. By generating programs based on standard web technology stacks, it ensures the solution can run in various environments, avoiding complex installation and deployment processes.

[0113] The above content achieves the following technical effects.

[0114] This represents a significant leap in development efficiency. The development and deployment cycle for table import programs has been reduced from hours / days to minutes, enabling development as description and deployment as generation. Users only need to focus on business rules, and the system can quickly generate runnable programs, supporting rapid iteration and instant deployment.

[0115] Significantly lowers the technical barrier. Through a graphical user interface, business personnel can complete complex data transformation tasks without programming knowledge. The system provides intuitive visual configuration, supports drag-and-drop operations and natural language input, making it easy for non-technical personnel to get started.

[0116] It boasts excellent compatibility and adaptability. The generated HTML / JavaScript programs are platform-independent and can run in various operating systems and browser environments, including various domestic IT innovation systems and professional systems with fewer users, solving the compatibility issues of traditional solutions in specific environments.

[0117] It provides comprehensive automated operation and maintenance capabilities. The system supports automatic detection of input sources and automatic data import, can automatically output processing results to designated locations, and generate detailed operation logs and audit records, greatly reducing the need for manual intervention.

[0118] The generated results are transparent, flexible, and reusable. The output HTML program is a complete front-end application with clear and readable code, facilitating subsequent manual adjustments and feature expansion. Each program can be saved and reused as an independent tool, supporting version management and team sharing.

[0119] Ensuring high precision in data processing. Deep learning models achieve significantly higher accuracy in recognizing complex table structures than traditional methods, effectively handling various non-standard table formats and fundamentally improving the success rate and accuracy of data conversion.

[0120] Build a continuously optimizing intelligent system. During the processing, the system continuously learns and accumulates transformation rules and mapping relationships, forming a reusable knowledge base to provide intelligent support for subsequent code generation tasks, thus enabling the continuous evolution of system capabilities.

[0121] The embodiments of this application will be further described below.

[0122] Figure 2 This is a schematic diagram of an AI-driven data docking system according to an embodiment of this application, such as... Figure 2 As shown, the AI-driven data docking system 200 may include: an AI platform 201, a generated independent HTML data docking hub program 202, and a deployment program for connecting with external systems 203.

[0123] In this embodiment, the user configures the docking rules through the AI ​​platform 201. After learning, the AI ​​platform 201 generates an independent HTML data docking hub program 202. This independent HTML data docking hub program 202 is deployed in the runtime environment and is responsible for obtaining data from the old system / data source, converting it internally, and outputting the results to the new system / target end.

[0124] AI Platform 201 can be deployed in the cloud or locally and includes a rule learning and validation engine. For example, it can perform input sample parsing, rule summarization and fusion, rule validation and iterative correction, and code generation.

[0125] The generated standalone HTML data docking hub program 202 may include a deployment environment, a data monitoring and acquisition module, core conversion engine functionality implementation, an output distributor, a graphical monitoring interface, and a logging system.

[0126] The deployment process connects to external systems (203), which may include legacy systems / data sources, new WMS systems, and file server / cloud storage locations.

[0127] Figure 3 This is a flowchart of a rule configuration and verification method according to an embodiment of this application, such as... Figure 3 As shown, the method includes the following steps.

[0128] Step S301: Upload samples and documents.

[0129] In this embodiment, the user uploads samples of source and target data (e.g., Excel reports from the old WMS, API request / response examples from the new WMS) and interface documentation. The AI ​​platform's parsing capabilities can parse this unstructured or semi-structured data, identifying fields, formats, and structures.

[0130] Step S302, rule input.

[0131] In this embodiment, the user describes the transformation and routing rules using natural language. For example, "Multiply the value of field A by tax rate B, round the result, and fill it into field C." The platform's Natural Language Processing (NLP) module converts this description into structured operation instructions.

[0132] Step S303: Rule extraction and summarization.

[0133] In this embodiment, the AI ​​engine combines the parsed data structure with the input manual rules for comprehensive analysis. For fields that are not explicitly described, the AI ​​attempts to infer their mapping relationships by comparing sample data (e.g., fuzzy matching of field names, value pattern matching), and suggests potential conversion rules to the user.

[0134] Step S304: Check if the rule verification passes.

[0135] In this embodiment, it can be determined whether the rule verification passes. If it passes, step S306 is executed; if it fails, step S305 is executed.

[0136] Step S305: Supplement the documentation / correct the rules.

[0137] In this embodiment, if rule validation fails, the document can be supplemented / the rule can be corrected.

[0138] Step S306: Generate the final rule set.

[0139] In this embodiment, if the rule verification passes, the final rule set can be generated.

[0140] Step S307: Prepare for program generation.

[0141] In this embodiment, the preparation procedure is generated after the final rule set is generated.

[0142] Figure 4 This is a flowchart of a program generation and deployment method according to an embodiment of this application, such as... Figure 4 As shown, the method includes the following steps.

[0143] Step S401: Generate design documents.

[0144] In this embodiment, after the rules are validated, the AI ​​platform first generates human-readable results, including a detailed field mapping table and a data processing flowchart. This serves as both a confirmation document for the user and the underlying blueprint for subsequent code generation.

[0145] Step S402: Generate source code.

[0146] In this embodiment, the AI's code generation engine automatically generates a complete, self-contained HTML file based on the aforementioned blueprint. This file will embed JavaScript code, and its main modules may include a data acquisition module, a core transformation engine module, an output distributor module, a user interface and logging module, and a configuration management module.

[0147] Optionally, the data acquisition module, core transformation engine module, output distributor module, user interface and log module, and configuration management module have been described in the foregoing sections and will not be repeated here.

[0148] Step S403: Download the HTML file.

[0149] In this embodiment, the user can download the generated single HTML file to a physical machine (such as an old computer or server) that acts as a "data relay station".

[0150] Step S404: Open and run the browser.

[0151] In this embodiment, after downloading the HTML file, the program will start running simply by opening the file with a modern browser (such as Chrome).

[0152] Step S405: Determine whether the initial configuration check needs to be completed.

[0153] In this embodiment, after the browser is opened and running, it can be determined whether an initial configuration check is required or not. If an initial configuration check is required, step S406 can be executed. If the initial configuration check is completed, step S407 can be executed.

[0154] Step S406: Enter configuration information.

[0155] In this embodiment, if an initial configuration check is required, configuration information can be entered.

[0156] Step S407: Enter the running state.

[0157] In this embodiment, once the initial configuration check is completed, the system can enter the running state.

[0158] In this embodiment of the application, 24 / 7 monitoring can be achieved by using the browser's "developer mode" or by keeping the browser window open.

[0159] Figure 5 This is a flowchart of a procedure and operation monitoring method according to an embodiment of this application, such as... Figure 5 As shown, the method includes the following steps.

[0160] Step S501: Start the program.

[0161] In this embodiment, the program can be started.

[0162] Step S502: Trigger listening.

[0163] In this embodiment, after the program starts, it works according to a preset triggering method.

[0164] Step S503: Determine whether the triggering method is automatic or manual.

[0165] In this embodiment, it can be determined whether the triggering method is automatic or manual. If it is automatically triggered, step S505 is executed. If it is manually triggered, step S504 is executed.

[0166] Step S504: Wait for the file to upload.

[0167] In this embodiment, if it is a manual upload mode, the system waits for user input.

[0168] Step S505: Periodically capture data.

[0169] In this embodiment, if it is in automatic crawling mode, a built-in timer (e.g., setInterval) is started to periodically execute data crawling tasks.

[0170] Step S506: Perform data conversion processing.

[0171] In this embodiment, after obtaining the source data, the conversion engine processes it according to the rules one by one.

[0172] Step S507, output distribution.

[0173] In this embodiment, after processing is complete, the dispatcher executes each output task concurrently or sequentially.

[0174] Step S508: Output whether the target is a file.

[0175] In this embodiment, it can be determined whether the output target is a file. If it is an FTP / file, then step S509 is executed. If it is an API interface, then step S510 is executed.

[0176] Step S509: Generate and transfer the file.

[0177] In this embodiment, if it is an FTP / file, then the file is generated and transferred.

[0178] Step S510: Send an HTTP request.

[0179] In this embodiment, if it is an API interface, an HTTP request can be sent.

[0180] Step S511: Record the log.

[0181] In this embodiment, a log can be recorded, and the above steps can all be reflected in real time through the UI.

[0182] Step S512: Determine whether the execution status is successful.

[0183] In this embodiment, the success or failure of the execution can be determined. If successful, step S502 is executed; if it fails, step S513 is executed. That is, the user can view the success / failure records at any time. For failed tasks, the interface provides a "Retry" button, which contains the code for re-executing the function corresponding to the task.

[0184] Step S513 provides a retry function.

[0185] In this embodiment, a retry function is provided if the execution status is successful.

[0186] In step S514, the user clicks "Retry".

[0187] In this embodiment, after the retry function is provided, the user can click to retry.

[0188] In this embodiment of the application, after completing the rule configuration and verification (corresponding to...) Figure 3 After the process in the middle, program generation and deployment can be performed (corresponding to...). Figure 4 (The process in the middle), after which program and runtime monitoring can be performed (corresponding to...) Figure 5 The process (in which the following is mentioned) can produce the following significant beneficial effects.

[0189] Significantly improve efficiency, reduce costs and time. API projects that traditionally require weeks of development and testing can be completed in just a few hours or days through rule configuration and program generation.

[0190] Technical Solution Implementation: Utilizing AI's NLP and code generation capabilities, the vast majority of manual code writing and debugging work is replaced. Business personnel directly participate in rule definition, saving developers significant communication costs associated with understanding business requirements. The generated HTML program requires no compilation and has minimal dependencies, achieving "generate and run immediately."

[0191] It resolves the cost-benefit contradiction of temporary and transitional integration. It provides a highly cost-effective solution for short-term needs such as system switching and temporary data migration, avoiding the need to write "permanent" code with high long-term maintenance costs for short-term needs.

[0192] The generated program itself is lightweight and disposable. It exists as a single HTML file, is easy to deploy, and can be directly deleted after the task is completed, leaving no residual server, database, or other maintenance burdens. The AI-generated code has a clear structure, and its core logic originates from the business rules themselves, making it easy to understand even if short-term maintenance is required.

[0193] Enhanced flexibility and autonomy for business personnel. Business personnel or administrators can quickly respond to changes in business rules (such as adjustments to field mappings or the addition of new output formats). When rules change, users can return to the AI ​​platform to modify or supplement the rule description, re-verify, and generate a new version of the program for replacement, reducing absolute dependence on the development team and enabling business departments to have a certain degree of self-service capability.

[0194] Enables non-intrusive integration with "long-tail" edge systems. It allows for low-cost integration with data acquisition modules from outdated or niche systems lacking open standard APIs, without relying on proprietary interfaces provided by the target system. AI can generate JavaScript code that simulates browser operations and parses webpage tables or text files in specific formats (e.g., fixed-width text). This "screen grabbing" or "file parsing" approach is an important complement to traditional API integration methods.

[0195] Promote enterprise knowledge accumulation and standardization. Make implicit business rules and integration logic explicit, documented, and executable. AI generates mapping rule tables and data processing flowcharts during the process. The generated HTML program itself is also a runnable "living document," clearly demonstrating how data flows and transforms. These materials provide an accurate starting point for future system upgrades, compliance audits, or similar projects.

[0196] It ensures basic security and controllability. While offering convenience, it also addresses the fundamental requirements of enterprise IT security. By guiding users to run the program locally, allowing users to input sensitive information themselves during runtime and encrypting and storing it on the front end, and supporting the use of virtual data for rule training, it effectively reduces the risk of data leakage. The program runs in a controlled internal environment, and network communication is limited to known target systems, resulting in a clear security boundary.

[0197] The following section will further illustrate this point using the example of a multi-source data intelligent docking hub during the transition period of a warehousing system.

[0198] Scenario: A large warehousing company decides to implement a brand-new WMS system to replace the old one. During the several-month transition period of switching between the old and new systems, severe data integration issues arise. Table 1 is an environmental table corresponding to different categories and models according to an embodiment of this application. As shown in Table 1, it describes the environmental information corresponding to different categories and models.

[0199] Table 1. Environmental Tables for Different Categories and Models

[0200]

[0201] System: Upstream, the ERP system still needs to send inbound plans and orders to the old WMS every day. Downstream, the new WMS needs to receive these instructions, but the data format and interface protocol between the new WMS are completely different from those of the old system.

[0202] Temporary requirement: Data from the old WMS system needs to be imported into the new system. Instructions from the upper-level ERP system need to be transmitted to the new WMS system in the form of export tables.

[0203] Traditional solutions: To address the window of opportunity during the development of a new WMS system interface, two strategies can be employed: assigning operators to manually convert data. Alternatively, the development team can write multiple complex, highly coupled interfaces for this temporary period, which is costly, time-consuming, and obsolete after the switch.

[0204] The implementation steps of this application embodiment may include the following steps.

[0205] Define the docking rules and generate the docking program. After that, the system administrator or ordinary operator performs the following operations in the commonly used AI platform.

[0206] In this embodiment, data samples are uploaded. For example, the "Inbound Plan" Excel template from the old WMS; the "Receiving Task" API interface document example from the new WMS; and the command document from the ERP system are uploaded respectively. To ensure data security, care should be taken not to upload sensitive data when identifying export rules. To avoid data leakage, virtual experimental data without real data can be used, or a large-scale AI model deployed locally without internet access can be used.

[0207] In this embodiment, the data conversion rules and routing rules between the two systems can be determined, and the rules can be verified.

[0208] Optionally, manually defined rules may include the following four rules.

[0209] Rule 1 (Format Conversion): "Convert the PO-2023-001 format document number exported from ERP via the old WMS to the IN20230001 format required by the new WMS."

[0210] Rule 2 (Field Splitting and Merging): "Split the 'Location' field in the old table (e.g., A-01-02-03) into four independent fields in the new system: 'Area, Shelf, Layer, Location'."

[0211] Rule 3 (Logical Judgment and Fill): "If the 'product status' is 'pending inspection', then 'quality inspection warehouse' will be automatically filled as the default storage location in the generated data."

[0212] Rule 4 (Data Routing): "Generate two outputs from the processed data: one is sent to the new WMS API via HTTP POST, and the other is saved as a CSV file conforming to the transport system format and automatically uploaded to the FTP server or stored in a specified location for retention."

[0213] Optionally, AI can be used to automatically identify conversion rules, enabling it to understand how data formats are converted between the old and new WMS systems, as well as between the ERP system and the new WMS. After clearly describing the data export rules, it is necessary to verify whether the AI ​​truly understands them.

[0214] Optionally, the verification method may include uploading an old system data table or ERP system data that has not been uploaded to AI, allowing AI to generate the data required for the new WMS according to the above rules. The generated data is then manually checked to ensure it meets the actual requirements. This data is then input into the new WMS to verify whether the system can correctly identify and perform normal production. If problems are found, the issues can be manually identified, or the discrepancies can be input into AI to identify the causes and then correct them.

[0215] By repeatedly applying the above verification method, it can be ensured that the AI ​​has correctly understood the logic of exporting tables or transmitting data. After successful verification, the AI ​​can be instructed to generate a mapping rule table and mapping rule diagram between data, which will be used to write the program's user manual at the end.

[0216] In this embodiment, the AI ​​processing and generation of the "temporary HTML data conversion hub (export table)" program may include the following:

[0217] The platform's AI engine analyzes and summarizes the complex rules mentioned above. Generally, these rules are integrated tasks involving multiple inputs, multiple outputs, and data transformation (and in rare cases, network communication). The AI ​​is then commanded to automatically generate a standalone HTML application. This application can embed the following logic.

[0218] Data Monitoring and Acquisition Module: Graphical Input. A graphical Excel input window can be set up, allowing users to drag and drop data or click the window to access a file selection interface. This method relies on manual file selection, offering high reliability and ease of implementation. Periodically log into the old WMS backend or listen for commands from the ERP system to capture specified data report pages and structure their table content. This method offers automated data reception, but its reliability is slightly lower, requires multiple adjustments, and may even fail due to protection measures in other systems' backends.

[0219] Core transformation engine: Executes all defined cleaning, transformation, and logical judgment rules.

[0220] Dual-channel output distributor (optional): One channel encapsulates the data as JSON and pushes it by calling the new WMS API; the other channel formats the data as a specific CSV and uploads it to a specified server or stores it in a specified local location by calling the built-in FTP client.

[0221] Graphical monitoring interface and log generation: Automatically generates logs for all operations of the program, and provides graphical real-time / historical log recording, success / failure statistics, manual retry button, etc.

[0222] Deployment and operation: Administrators can deploy this HTML file on an old computer or low-power server that is always on, open it in a browser and keep it running.

[0223] During the three-month transition period, the temporary data hub program operated automatically 24 / 7, faithfully and stably playing the role of a "translator" and "messenger," ensuring seamless and continuous operation of business during the system switchover.

[0224] After the transition period ends, the task program can be archived or stopped. The business logic (mapping rules, interface format) carried by the task program can be directly transferred or used for similar data exchange scenarios in the future.

[0225] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0226] According to an embodiment of this application, a data migration apparatus is also provided. It should be noted that this data migration apparatus can be used to execute the data migration method described in the above embodiments.

[0227] Figure 6 This is a schematic diagram of a data migration apparatus according to an embodiment of this application, such as... Figure 6 As shown, the data migration device 600 may include: an acquisition unit 602, a determination unit 604, a conversion unit 606, and a migration unit 608.

[0228] The acquisition unit 602 is used to acquire source data in the initial platform and target data in the target platform. The source data represents the data to be migrated from the initial platform, and the target data represents the data that conforms to the specification form in the target platform.

[0229] The determining unit 604 is used to determine the mapping relationship and transformation rules between source data and target data. The mapping relationship is used to represent the association between data elements in the source data and data elements in the target data. The transformation rules are used to represent the rules for migrating source data from the initial platform to the target platform and performing transformation on the source data on the target platform.

[0230] The transformation unit 606 is used to transform source data into structured data that corresponds to the target data in the target platform, based on mapping relationships and transformation rules.

[0231] Migration unit 608 is used to migrate structured data from the initial platform to the target platform.

[0232] Optionally, the determining unit 604 includes: a matching subunit for matching fields and / or values ​​in the source data and target data to obtain a mapping relationship; a determining subunit for determining a first transformation rule between the source data and the target data, and a second transformation rule between the source data and the target data, wherein the first transformation rule is used to represent an explicit rule describing the association and transformation logic between the source data and the target data in natural language, and the second transformation rule is used to represent an implicit transformation logic inferred using an inference model, the inference model being trained using source data samples and target data samples, the source data samples representing data samples to be migrated from the initial platform, and the target data samples representing data samples conforming to a standard form; and a fusion subunit for fusing the first transformation rule and the second transformation rule to obtain a transformation rule.

[0233] Optionally, the conversion unit 606 includes: a conversion subunit, used to bind data elements in the source data to data elements in the target data based on the mapping relationship, and to perform serialization conversion operation on the bound source data according to the conversion rules to obtain structured data.

[0234] Optionally, the data migration device 600 further includes: a verification unit for verifying the transformation rules and obtaining a verification result; and an extraction unit for extracting the structured data in response to the verification result indicating that the data obtained after transforming the source data conforms to structured data, obtaining an extraction result, wherein the extraction result includes at least one of the following: the field structure, data type, value mode, and transformation logic of the structured data; and outputting the extraction result as a structured file, wherein the structured file is used to represent a structured description file containing a field mapping relationship table and a data processing flow.

[0235] Optionally, the data migration device 600 further includes a correction unit, configured to correct the conversion rules in response to a verification result indicating that the data obtained after converting the source data does not conform to structured data, wherein the accuracy of the modified conversion rules is higher than the accuracy of the original conversion rules.

[0236] Optionally, the data migration apparatus 600 further includes a generation unit for generating an independently executable front-end application file based on the structured file, wherein the front-end application file represents a file that can be run directly through a browser in a serverless environment and outputs structured data to a target platform.

[0237] Optionally, the front-end application file includes a data acquisition module, a transformation engine module, an output module, a display module, and a configuration module. The data acquisition module is used to acquire source data and target data; the transformation engine module is used to generate mapping relationships and transformation rules; the output module is used to convert source data into structured data that conforms to the target platform's specifications and send it to the target platform's interface, as well as generate a string of structured data; the display module is used to display the structured data; and the configuration module is used to configure the security of the structured data.

[0238] Optionally, the data migration device 600 further includes a recording unit for acquiring and / or recording structured data according to a target time period.

[0239] In this embodiment, the acquisition unit 602 acquires source data from the initial platform and target data from the target platform. The source data represents data to be migrated from the initial platform, and the target data represents data conforming to the specified format in the target platform. The determination unit 604 determines the mapping relationship and conversion rules between the source data and the target data. The mapping relationship represents the association between data elements in the source data and data elements in the target data, and the conversion rules represent the rules for migrating the source data from the initial platform to the target platform and performing conversions on the source data on the target platform. The conversion unit 606, based on the mapping relationship and conversion rules, converts the source data into structured data corresponding to the target data in the target platform. The migration unit 608 migrates the structured data from the initial platform to the target platform, thereby solving the technical problem of ineffective data migration and achieving the technical effect of effective data migration.

[0240] According to an embodiment of this application, a computer-readable storage medium is also provided, the storage medium including a stored program, wherein the program executes the methods described in the above embodiments.

[0241] According to an embodiment of this application, a processor is also provided for running a program, wherein the program executes the methods described in the above embodiments during runtime.

[0242] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the method of the embodiments of this application when it runs.

[0243] Embodiments of this application also provide a computer program product. Optionally, in this embodiment, the computer program product may include a computer program that, when executed by a processor, implements the methods described in the embodiments of this application.

[0244] According to another aspect of the embodiments of this application, a vehicle is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of this application when it runs.

[0245] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0246] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0247] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0248] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0249] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0250] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A data migration method, characterized in that, include: Obtain source data from the initial platform and target data from the target platform, wherein the source data represents data to be migrated from the initial platform and the target data represents data conforming to the specified format in the target platform; Determine the mapping relationship and transformation rules between the source data and the target data, wherein the mapping relationship is used to represent the association relationship between data elements in the source data and data elements in the target data, and the transformation rules are used to represent the rules for migrating the source data from the initial platform to the target platform and performing transformation on the source data on the target platform; Based on the mapping relationship and the transformation rules, the source data is transformed into structured data that conforms to the target data in the target platform; The structured data is migrated from the initial platform to the target platform.

2. The method according to claim 1, characterized in that, Determining the mapping relationship between the source data and the target data includes: The mapping relationship is obtained by matching the fields and / or values ​​in the source data and the target data; Determining the transformation rules between the source data and the target data includes: determining a first transformation rule between the source data and the target data, and a second transformation rule between the source data and the target data, wherein the first transformation rule is used to represent an explicit rule describing the association and transformation logic between the source data and the target data in natural language, and the second transformation rule is used to represent an implicit transformation logic inferred using an inference model, wherein the inference model is trained using source data samples and target data samples, wherein the source data samples represent data samples to be migrated from the initial platform, and the target data samples represent data samples conforming to the specified form; The first conversion rule and the second conversion rule are merged to obtain the conversion rule.

3. The method according to claim 2, characterized in that, Based on the mapping relationship and the transformation rules, the source data is transformed into structured data that conforms to the target data in the target platform, including: Based on the mapping relationship, data elements in the source data are bound to data elements in the target data, and the bound source data is serialized according to the conversion rules to obtain the structured data.

4. The method according to claim 1, characterized in that, The method further includes: The conversion rules were verified, and the verification results were obtained. In response to the verification result that the data obtained after transforming the source data conforms to the structured data, the structured data is extracted to obtain the extraction result, wherein the extraction result includes at least one of the following: the field structure, data type, value mode and transformation logic of the structured data; The extraction results are output as a structured file, wherein the structured file is used to represent a structured description file containing a field mapping relationship table and a data processing flow.

5. The method according to claim 4, characterized in that, The method further includes: In response to the verification result that the data obtained after transforming the source data does not conform to the structured data, the transformation rules are corrected, wherein the accuracy of the modified transformation rules is higher than the accuracy of the unmodified transformation rules.

6. The method according to claim 4, characterized in that, The method further includes: Based on the structured file, an independently executable front-end application file is generated, wherein the front-end application file is used to represent a file that runs directly through a browser in a server-free environment and outputs the structured data to the target platform.

7. The method according to claim 6, characterized in that, The front-end application file includes a data acquisition module, a conversion engine module, an output module, a display module, and a configuration module. The data acquisition module acquires the source data and the target data. The conversion engine module generates the mapping relationship and the conversion rules. The output module converts the source data into structured data conforming to the target platform's specifications and sends it to the target platform's interface, as well as generating a string representing the structured data. The display module displays the structured data. The configuration module configures the security of the structured data.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: The structured data is acquired and / or recorded according to the target time period.

9. A data migration device, characterized in that, include: An acquisition unit is used to acquire source data in an initial platform and target data in a target platform, wherein the source data represents data to be migrated from the initial platform and the target data represents data that conforms to the specified format in the target platform; A determining unit is used to determine the mapping relationship and transformation rules between the source data and the target data, wherein the mapping relationship is used to represent the association relationship between data elements in the source data and data elements in the target data, and the transformation rules are used to represent the rules for migrating the source data from the initial platform to the target platform and performing transformation on the source data on the target platform; A conversion unit is used to convert the source data into structured data that corresponds to the target data in the target platform, based on the mapping relationship and the conversion rules. A migration unit is used to migrate the structured data from the initial platform to the target platform.

10. A processor, characterized in that, The processor is used to run a program, wherein the program is executed by the processor to perform the method according to any one of claims 1 to 8.

11. An electronic device, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the method according to any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the method according to any one of claims 1 to 8.

Images