A data migration method, device, equipment and medium for an intranet

Abstract

The application discloses a data migration method, device and equipment for an intranet and a medium. The method comprises the following steps: during the execution of a migration task, a model change event, a configuration execution event and an AI inference event are written in a structured format into an audit log library; self-evolution is performed based on historical events recorded in the audit log library, which comprises the following steps: repeated failure modes appearing in the audit log are identified, and corresponding optimization configurations or patch configurations are automatically generated according to the failure modes; wherein, the action of each self-evolution and the audit log on which the action is based are written into the audit log library as audit events; and the optimization configurations or patch configurations are applied to execute subsequent migration, and the execution results are written into the audit log library. Through the method, the migration between intranets can be automated, and the labor cost is greatly reduced.

Description

Technical Field

[0001] This application relates to the field of data migration technology, and in particular to a data migration method, apparatus, device, and medium for intranets. Background Technology

[0002] Data resources are widely distributed across various business systems, such as the case file database of the case management system, the population information database, the fugitive database, and the police integrated platform's call and response records. These systems are typically deployed on the public security intranet, prohibiting access to the internet, and have strict approval restrictions on the installation of third-party software, sometimes even allowing only the use of the operating system's built-in Python standard library. Furthermore, public security business systems are constantly evolving, with database table structures and field definitions frequently adjusted according to business needs.

[0003] In existing technologies, the need for cross-system data migration mainly relies on manually writing ETL scripts or deploying lightweight language models locally. However, these technologies currently have the following problems in public security scenarios: 1. Most migration scripts rely on third-party libraries such as pandas and SQLAlchemy, but the public security intranet cannot connect to the network to install these dependency packages.

[0004] 2. Migration tasks typically last for several hours or even days. If the table structure of the source or target business system changes during the execution period, the migration task will be terminated due to errors. In this case, manual intervention is required to re-analyze the mapping and rerun the task, which will prolong the migration window and increase the risk of data inconsistency.

[0005] 3. Each cross-system migration requires rewriting the script, manually copying it to each execution node, and restarting the service for it to take effect. Similar migration scenarios cannot accumulate reusable migration logic, resulting in significant duplication of effort.

[0006] 4. The process of generating field mappings or patch configurations using AI models is not visible, and cannot provide a decision-making basis acceptable to the security audit department, thus posing compliance issues.

[0007] 5. Locally deployed models are fixed versions and cannot be continuously improved through external feedback. When new business terms emerge or unforeseen changes occur in the data model, the model cannot adapt automatically and manual intervention is still required for correction. Summary of the Invention

[0008] This application provides a data migration method, apparatus, device, and medium for intranets, which addresses the technical problem that existing data migration methods cannot meet the needs of public security systems.

[0009] Technical solution According to one aspect of this application, a data migration method for an intranet is provided, comprising: During the migration task execution, model change events, configuration execution events, and AI inference basis events are written to the audit log library in a structured format; Self-evolution based on historical events recorded in the audit log library includes: identifying repeated failure patterns in the audit logs and automatically generating corresponding optimization configurations or patch configurations based on the failure patterns; wherein, each self-evolution action and the audit logs on which it is based are written as audit events into the audit log library; Apply the optimized configuration or patch configuration to perform subsequent migrations, and write the execution results to the audit log library.

[0010] Preferably, the method further includes: When the failure rate in the execution results of the current configuration exceeds the first threshold, query the configuration version history in the audit log library; Locate the configuration version that was most recently successfully executed, and calculate the difference between the current configuration and the successful configuration; Automatically roll back to the successful configuration version and trigger a compensatory migration of the affected data based on the unique identifier range of the failure records in the audit log; Write the rollback operation and compensation migration results to the audit log.

[0011] Preferably, the triggering of compensatory migration for the affected data includes: Extract a unique set of identifiers for failure records from the audit log; Generate a compensation migration configuration for the set of failure records, wherein the compensation migration configuration reuses the portion of the currently valid configuration that was not affected by the rollback; After the compensation migration is completed, the compensation results will be merged with the original successful records to form a complete migration dataset.

[0012] Preferably, identifying recurring failure patterns in the audit log includes: Failure records are extracted from the configuration execution events in the audit logs. Each failure record includes three dimensions: source field, target field, and error type. When the number of failures of a failure record with the same three dimensions exceeds the second threshold within a preset time window, a local patch configuration is generated for the source field and / or target field in that failure record. When the proportion of failure records with the same three dimensions affecting the total number of migration records exceeds the third threshold, a global configuration replacement recommendation is generated.

[0013] Preferably, after performing the subsequent migration using the optimized configuration or patch configuration, the process further includes: Calculate the new failure rate resulting from subsequent migration executions. When the new failure rate exceeds the fourth threshold, mark the corresponding optimization configuration or patch configuration as invalid evolution. A suppression rule is set for the invalid evolution, which prohibits the generation of configurations with the same feature fields and the same error type in subsequent self-evolution iterations; Write the invalid evolution markers and corresponding suppression rules into the audit log.

[0014] Preferably, the self-evolution based on historical events recorded in the audit log library further includes: For the same failure mode, generate two or more optimization configurations or patch configurations as candidate configurations. The candidate configuration is executed in shadow migration mode, which reads data but does not write it to the target system, or writes it to an isolated sandbox target table. Compare the execution results of each candidate configuration and select the candidate configuration with the highest success rate as the basis for self-evolution; The evaluation process and results performed under the shadow migration mode are written to the audit log.

[0015] Preferably, the self-evolution based on historical events recorded in the audit log library further includes: Extract successfully executed mapping relationships from audit logs and store them as a mapping knowledge graph by source field name, target field name, transformation function identifier, and usage frequency; When the source field of a new migration task has no direct match in the knowledge graph, the semantic similarity between the source field and the field names in the graph is calculated, and the existing mapping relationship with the highest similarity is recalled as the initial recommendation configuration.

[0016] A second aspect of this application provides a data migration apparatus for an intranet, comprising: The event logging module is used to write model change events, configuration execution events, and AI inference basis events into the audit log library in a structured format during the execution of migration tasks. The self-evolution module is used to perform self-evolution based on historical events recorded in the audit log library, including: identifying repeated failure patterns in the audit logs and automatically generating corresponding optimization configurations or patch configurations according to the failure patterns; wherein, each self-evolution action and the audit logs on which it is based are written as audit events into the audit log library; The subsequent migration module is used to apply the optimized configuration or patch configuration to perform subsequent migrations and write the execution results to the audit log library.

[0017] A third aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it causes the electronic device to perform the method described in the first aspect of this application.

[0018] A fourth aspect of this application provides a computer-readable storage medium for storing a computer program that, when run on a computer, causes the electronic device to perform the method described in the first aspect of this application.

[0019] Beneficial effects In this embodiment, a data migration method for an intranet is employed. During the migration task execution, model change events, configuration execution events, and AI inference basis events are written into an audit log library in a structured format. Self-evolution is performed based on historical events recorded in the audit log library, including: identifying recurring failure patterns in the audit logs and automatically generating corresponding optimized or patched configurations based on the failure patterns; wherein each self-evolution action and the audit logs it is based on are written into the audit log library as audit events; subsequent migrations are performed using the optimized or patched configurations, and the execution results are written into the audit log library.

[0020] This invention enables the system to continuously learn and improve its behavior from historical transfer tasks in environments without network access, manual annotation, or model updates. Each self-evolutionary action and its rationale are written back to the audit log, forming a closed loop and avoiding dependence on external feedback signals. Attached Figure Description

[0021] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of a data migration method for an intranet according to an embodiment of this application.

[0022] Figure 2 This is a flowchart involving the self-evolution step according to an embodiment of this application.

[0023] Figure 3 This is a flowchart involving subsequent migrations performed according to the embodiments of this application.

[0024] Figure 4 This is a schematic diagram of a data migration device for an intranet according to an embodiment of this application.

[0025] Figure 5This is a schematic diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0026] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0027] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0028] Embodiments of the present invention provide a system for data migration between intranets, such as data migration between intranets within a public security system. To accommodate potential model changes during the migration process without an external network and to ensure full traceability, the system requires the following preparatory work before executing the migration task: 1. The administrator pre-configures the connection information between the source and target system's databases and stores the connection parameters in an encrypted configuration file. Furthermore, due to the principle of least privilege required by the public security intranet environment, the migration system is only granted read-only permissions for the source system and write permissions for the target system.

[0029] 2. Create the initial migration configuration, including the following two methods: Method 1: Administrators select the fields to be migrated from the field list of the source system through a visual interface, and map them one by one to the fields of the target system. At the same time, they specify the conversion rules, such as direct mapping, date formatting, string truncation, etc., and configure them to be stored in JSON format.

[0030] Method 2: Deploy a local AI model in the system. The administrator enters a natural language description in the form interface, such as "migrate the case number, case name, and case filing date in the case table to the target archive, and convert the case filing date format to YYYY-MM-DD". This method calls the AI ​​model in the system to automatically recommend field mapping and conversion rules and generate an initial configuration for the administrator to confirm.

[0031] 3. Deploy a fixed configuration interpreter on the target execution node, load and execute the above JSON configuration through the interpreter, and call the pre-built atomic transformation function library to complete the data transformation.

[0032] 4. Initialization of the audit log database The system creates an audit log library in local storage.

[0033] 5. Configuration of Model Change Probes The system configures the probe's detection cycle and the range of table structures to be monitored. In this embodiment, the detection cycle is every 5 minutes. The probe can obtain table structure snapshots through the database's metadata query interface.

[0034] After completing the above preparations, the migration task can be executed by system scheduling or manually triggered by the administrator.

[0035] like Figure 1 The diagram shown is a flowchart of a data migration method for an intranet according to the present invention. Figure 1 As can be seen, the aforementioned data migration method for intranets includes the following steps: Step S101: During the migration task execution, write model change events, configuration execution events, and AI inference basis events into the audit log library in a structured format.

[0036] After the migration begins, the system periodically captures structural snapshots of the source and target data models and compares them with the previously captured snapshot. If a difference is found, a model change event is generated. This event includes the change type and a complete description of the structural differences before and after the change. The change type typically includes field addition / deletion / type change / renaming, etc. The model change event is written to the audit log database in structured JSON format.

[0037] After the migration task is completed, a configuration execution event will be generated. The configuration execution event is used to record the start and end time of this migration task, the number of records processed, the number of successful records, the number of failed records, and the detailed information of each failed record, such as the source field value, the expected type of the target field, and the error type of the failed record.

[0038] During the migration process, the local AI model is used to identify the semantic similarity between the source fields and the fields in the target system, and outputs the option with the highest similarity. The AI ​​reasoning basis event is used to record the content on which the AI ​​model bases its reasoning, including source field annotations, target field annotations, sample data, the algorithm identifier used for similarity calculation, the similarity scores of each candidate mapping, and the reasons for the final selection. For example, if a user describes the migration requirement in natural language: migrate the suspect's name and ID number from the case table to the target database, the system calls the local AI model to convert the natural language into field mappings, conversion rules, and other declarations, and simultaneously generates the matching fields, similarity scores, and the annotations or sample data on which it is based as AI reasoning basis events and writes them to the audit log.

[0039] The information that occurs during the migration process is written into the audit log database, which can be used for subsequent auditing and tracing.

[0040] Step S102: Self-evolution based on historical events recorded in the audit log library, including: identifying repeated failure patterns in the audit logs, and automatically generating corresponding optimization configurations or patch configurations according to the failure patterns; wherein, each self-evolution action and the audit logs on which it is based are written into the audit log library as audit events.

[0041] Specifically, failure records are extracted from the configuration execution events in the audit logs. Each failure record includes three dimensions: source field, target field, and error type. When the number of failures of failure records with the same three dimensions exceeds a second threshold within a preset time window (e.g., more than 10 failures in the most recent hour), a local patch configuration is generated for the source field and / or target field in that failure record. When the proportion of failure records affected globally by failure records with the same three dimensions exceeds a third threshold (e.g., setting the third threshold to 5%), a global configuration replacement suggestion is generated.

[0042] For example, the audit log shows that in the past hour, there were 15 type conversion errors from the source field "case_date" to the target field "occur_date", all of which were "ValueError". In this case, the system determines that the field-level migration failed and generates a local patch configuration for that field pair. If the same error type reaches 8% of the total records, it is determined to be a global migration failure, and a global configuration replacement suggestion is generated, such as changing the parsing strategy for all date fields from strict matching to multi-format trial.

[0043] Writing each self-evolutionary action and the audit log it is based on as an audit event into the audit log library can facilitate future traceability.

[0044] Step S103: Apply the optimized configuration or patch configuration to perform subsequent migrations and write the execution results to the audit log library.

[0045] Specifically, the configuration is deployed to the execution node using a pull-based approach for configuration updates, and then the remaining migration tasks continue to execute. Information such as new success / failure records and the number of processed records generated during this process is still written to the audit log database, forming a closed loop.

[0046] For the internal network of the public security system, which is physically isolated from the external network and lacks external feedback signals such as manual annotation and online model updates, the embodiments of this application rely on audit logs generated during their own execution to automatically identify failure modes, generate optimized configurations, and continuously improve subsequent migration effects. Simultaneously, every self-evolutionary decision and its basis are recorded back in the audit logs, making system behavior changes fully traceable and meeting the auditable requirements of the public security internal network. The entire process requires no human intervention and is highly automated.

[0047] In some preferred embodiments, the configuration suggestions provided by the system may not be effective in actual implementation. In such cases, adjustments should be made promptly. Therefore, in these embodiments, if the current configuration is not performing well, such as... Figure 3 As shown, after the optimized configuration or patch configuration described in step S103 is migrated, the following rollback steps are also included: Step S201: When the failure rate in the execution result of the current configuration exceeds the first threshold, query the configuration version history in the audit log library.

[0048] Step S202: Locate the configuration version that was most recently successfully executed and calculate the difference between the current configuration and the successful configuration.

[0049] Step S203: Automatically roll back to the successful configuration version, and trigger a compensation migration for the affected data based on the unique identifier range of the failure records in the audit log.

[0050] Specifically, the configuration execution module calculates the failure rate in real time after each batch execution, whereby the failure rate is calculated as the number of failed records divided by the total number of records in the batch. If the failure rate of three consecutive batches exceeds a first threshold (in this embodiment, the first threshold is 10%), a rollback process is triggered.

[0051] During the rollback process, the system first queries the audit log library for configuration execution events and evolution events to obtain the history of all configuration versions since the migration began. It then searches for the configuration version corresponding to the batch whose failure rate was below a first threshold during the last execution, using this as the target rollback version. Next, it calculates the difference between the current faulty configuration and the target rollback configuration, generating a difference report. The specific difference comparison method uses existing technologies, such as JSON Diff, which will not be elaborated upon here.

[0052] The system then replaces the current configuration with the target rollback configuration. Since the previous migration task may have processed a large amount of data, the rollback requires compensation for data that was incorrectly processed under the faulty configuration. To address this data effectively, the system assigns a unique identifier to each failure record during migration; this unique identifier can be a timestamp. By extracting the range of unique identifiers from all failure records since the faulty configuration took effect, all incorrectly processed data can be identified and compensated for during the migration.

[0053] Step S204: Write the rollback operation and compensation migration results into the audit log.

[0054] The rollback operation includes rollback time, configuration versions before and after rollback, and rollback reason. The compensation migration result includes the number of records processed, the number of successes, and the number of failures. All of the above data are written to the audit log library in the form of audit events to ensure that the entire fault recovery process is traceable.

[0055] In some optional embodiments, the compensation migration in the rollback step above adopts incremental compensation instead of a full rerun. A unique set of identifiers for failed records is extracted from the audit log, and a compensation migration configuration for this set of failed records is generated, reusing all field mappings and transformation rules in the currently valid configuration that were not affected by the rollback. Successful records that were not affected by the rollback are stored in a temporary staging area. When the compensation migration is executed, only failed records are compensated. The correctly transformed data is written to the target table and merged with previously successfully processed records to generate the final dataset. After merging, the system calculates the verification hash of the merged dataset and writes it to the audit log for subsequent verification of data integrity.

[0056] In some optional embodiments, the above merging steps also include a consistency verification mechanism. Before merging, the system calculates the data hash of the original successful record set and the hash of the compensation migration result, respectively. After merging, it calculates the hash of the complete dataset, and then writes the three hash values ​​and the merging logic to the audit log. If subsequent audits discover data inconsistencies, the correctness of the merging process can be verified by tracing these hashes and merging logic.

[0057] In some optional embodiments, since abbreviations commonly used in the public security intranet represent case numbers, migration failures often occur due to non-standard field naming. Therefore, the system calculates the similarity between field names using a local AI model, counts the number of failures in the set of failed records where the similarity exceeds a threshold, and generates batch patch configurations for all field pairs in the set of failed records when the number of failures exceeds a preset threshold.

[0058] In some preferred embodiments, such as Figure 3As shown, after performing the subsequent migration by applying the optimized configuration or patch configuration in step S103, the following steps are also included.

[0059] Step S104: Calculate the new failure rate caused by subsequent migration execution. When the new failure rate exceeds the fourth threshold, mark the corresponding optimization configuration or patch configuration as invalid evolution.

[0060] For example, the system generates and applies a patch configuration for the failure mode of the "case_date" field to the target field "occur_date" with the error type "ValueError". This patch changes the date parsing strategy to multi-format attempt. After application, the failure rate of this field decreases from the original failure probability of 15% to a new failure probability of 2%. However, the fourth threshold is set to 1%, meaning that 2% is greater than 1%, so the patch is considered an invalid evolution.

[0061] Step S105: Set a suppression rule for the invalid evolution, wherein the suppression rule prohibits the generation of configurations with the same feature fields and the same error type in subsequent self-evolution iterations.

[0062] The suppression rule corresponding to the above patch configuration is set to prohibit the generation of patches with multiple formats for subsequent failure modes from the source field "case_date" to the target field "occur_date" with the error type "ValueError". This rule is stored in the local evolution taboo list, which contains similar suppression rules. In each step S102, when automatically generating the corresponding optimization configuration or patch configuration based on the failure mode, the above suppression rule is read first, and the configuration in the suppression rule is avoided when generating the optimization or patch configuration.

[0063] Step S106: Write the invalid evolution markers and corresponding suppression rules into the audit log.

[0064] The markers for invalid evolutions include patch configuration content, new failure rates, and suppression rules. Writing these to the audit log library ensures that failed attempts during the evolution process are also traceable.

[0065] In some optional embodiments, the above suppression rules are time-limited. Specifically, each suppression rule is assigned an initial lifespan and a decay coefficient upon generation. Within the initial lifespan, the rule is fully effective, prohibiting the generation of patches with the same characteristics. After the lifespan ends, if the system does not find other effective alternatives and the original failure mode still exists, the suppressed solution is allowed to be tried with a lower priority, and an audit event for the attempt to revive is recorded. If the solution is still ineffective after the attempt, the lifespan of the rule is updated, reset, and doubled.

[0066] In some preferred embodiments, in order to select a more suitable patching strategy, such as Figure 2 As shown, in step S102, the self-evolution based on historical events recorded in the audit log database further includes: Step S1023: For the same failure mode, generate two or more optimized configurations or patch configurations as candidate configurations.

[0067] Step S1024: Execute the candidate configuration in shadow migration mode, where the shadow migration mode only reads data but does not write it to the target system, or writes it to an isolated sandbox target table.

[0068] Step S1025: Compare the execution results of each candidate configuration and select the candidate configuration with the highest success rate as the basis for self-evolution.

[0069] Step S1026: Write the evaluation process and results executed in the shadow migration mode into the audit log.

[0070] For example, in the case of a date format parsing failure, the system may generate three candidate patches: Candidate A attempts to parse using multiple formats; Candidate B first converts the source field to a string and then extracts the first 8 characters before parsing; Candidate C ignores the field and fills it with the default value "1900-01-01".

[0071] A temporary target table is created using shadow mode. 100 failure records are randomly selected from the audit log. Candidate configurations A, B, and C are applied sequentially to these records in shadow mode. The processing result is recorded after each execution. If candidate A has a success rate of 95%, candidate B 80%, and candidate C 60%, then candidate A is selected as the official patch configuration. Simultaneously, the evaluation process of shadow execution, including the execution results of each candidate, success / failure statistics, and the final selection result, is written to the audit log so that subsequent auditors can understand the basis for this evolutionary decision.

[0072] In some optional embodiments, during the shadow evaluation described above, the system extracts historical failure records with the same field and error type as the failure mode from the audit log, and performs shadow evaluation on these historical failure records simultaneously. In this way, the system can evaluate the repair effect of candidate configurations on historical data, thereby more comprehensively judging the long-term effectiveness of the configuration. In some preferred embodiments, after the evaluation is completed, an evaluation report containing a comparison of the performance of each candidate configuration on current and historical data is generated and written to the audit log. This approach avoids the candidate configuration being only applicable to the current sampled data while ignoring historical patterns, thus improving the robustness of evolutionary decision-making.

[0073] In some preferred embodiments, in order to enable the successful migration to be reused subsequently, such as Figure 2 As shown, in step S102, the self-evolution based on historical events recorded in the audit log database further includes the following steps: Step S1027: Extract the successfully executed mapping relationships from the audit logs and store them as a mapping knowledge graph by source field name, target field name, transformation function identifier, and usage frequency.

[0074] After each migration task is completed, the failure rate is checked to see if it is below a first threshold. For successful migration tasks, the AI ​​inference basis events and configuration execution events in the corresponding audit logs are scanned to extract all ultimately used field mapping relationships. Specifically, each mapping relationship records a quadruple of source field name, target field name, transformation function identifier, and usage frequency. The usage frequency is initially set to 1, and if the same mapping relationship is successfully reused in multiple tasks, the usage frequency is incremented. These quadruples are stored in a lightweight SQLite mapping table graph database to form a mapping knowledge graph.

[0075] Step S1028: When the source field of a new migration task has no direct match in the knowledge graph, calculate the semantic similarity between the source field and the field names in the graph, and recall the existing mapping relationship with the highest similarity as the initial recommendation configuration.

[0076] Specifically, when a user initiates a new migration task, the system first extracts a list of source fields to be mapped in the new task. For each source field, the system queries the knowledge graph to see if there is an identical source field name. If so, the mapping relationship is directly reused as part of the configuration. If not, the local AI model is invoked to calculate the semantic similarity between the source field and each source field name in the knowledge graph.

[0077] Specifically, the source field name, field annotation, and sample data are concatenated into text, and then input into the local AI model along with the corresponding text stored in the graph to calculate cosine similarity. Users can then choose the scheme with the higher similarity based on the similarity calculation results.

[0078] Based on the same inventive concept as the above-described method embodiments, such as... Figure 4 As shown in the illustration, this application also provides a data migration device for an intranet, comprising: The event logging module is used to write model change events, configuration execution events, and AI inference basis events into the audit log library in a structured format during the execution of migration tasks. The self-evolution module is used to perform self-evolution based on historical events recorded in the audit log library, including: identifying repeated failure patterns in the audit logs and automatically generating corresponding optimization configurations or patch configurations according to the failure patterns; wherein, each self-evolution action and the audit logs on which it is based are written as audit events into the audit log library; The subsequent migration module is used to apply the optimized configuration or patch configuration to perform subsequent migrations and write the execution results to the audit log library.

[0079] It should be noted that although several units or sub-units of the device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0080] In this embodiment, the Alibaba Cloud Tongyi Qianwen series of open-source models are used as the local AI inference engine and deployed on a server equipped with GPUs. The locally deployed Qwen model is encapsulated as a local AI inference engine service, providing a call interface to the self-evolution module in the system via HTTP API. This service runs independently on the GPU server and communicates with the main migration system via the intranet.

[0081] In this embodiment, the Qwen2.5-7B-Instruct and Qwen3-Embedding-8B models are used, which are suitable for different task types.

[0082] Specifically, the Qwen2.5-7B-Instruct model employs a Transformer decoder architecture with 28 layers, a hidden layer size of 3584, 28 attention heads, and a context length supporting 131072 tokens. After using the GPTQ 4-bit quantization scheme, its size is compressed to approximately 4GB. In the method of the above embodiment, this model is mainly used to parse the user's natural language input migration requirement description into a structured field mapping requirement; to generate a semantic understanding description of the target field based on the source field name, field annotations, and sample data during similarity matching; and to assist in judging the semantic rationality of candidate configurations during shadow pattern evaluation.

[0083] Qwen3-Embedding-8B employs a dual-encoder architecture, supporting dynamic output vector dimensions (1024 / 2560 / 4096 dimensions). It maps text to a unified semantic vector space, measuring semantic relevance through cosine similarity between vectors. The model has approximately 8 billion parameters, and after 4-bit quantization, the model file size is approximately 5GB. In the method described in the above embodiment, this model is primarily used to achieve semantic retrieval of the mapped knowledge graph. Specifically, when the source field of a new transfer task has no exact match in the knowledge graph, the model is invoked to convert the text description of the source field into a semantic embedding vector. This vector is then used to calculate cosine similarity with the embedding vectors of each field stored in the knowledge graph, and the existing mapping relationship with the highest similarity is retrieved as the initial recommendation configuration.

[0084] Based on the same inventive concept as the above method embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it enables the electronic device to implement the control method described in the above embodiments.

[0085] In one embodiment, the electronic device may be a server, and in this embodiment, the structure of the electronic device may be as follows: Figure 5 As shown, it includes a memory 2001, a communication module 2003, and one or more processors 2002.

[0086] The memory 2001 is used to store computer programs executed by the processor 2002. The memory 2001 may mainly include a program storage area and a data storage area. The program storage area may store the operating system and programs required to run instant messaging functions, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0087] Memory 2001 may be volatile memory, such as random-access memory (RAM); memory 2001 may also be non-volatile memory, such as read-only memory, flash memory, hard disk drive (HDD), or solid-state drive (SSD); or memory 2001 may be any other medium capable of carrying or storing a desired computer program having the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 2001 may be a combination of the above-mentioned memories.

[0088] Processor 2002 may include one or more central processing units (CPUs) or digital processing units, etc. Processor 2002 is used to implement the above-mentioned audio data processing method when calling computer programs stored in memory 2001.

[0089] The communication module 2003 is used to communicate with terminal devices and other servers.

[0090] This application embodiment does not limit the specific connection medium between the memory 2001, communication module 2003, and processor 2002. This application embodiment... Figure 5 The memory 2001 and the processor 2002 are connected via a bus 2004, which is in... Figure 5 The connections between other components are illustrated with arrows and are for illustrative purposes only, not as limiting information. The Bus 2004 can be divided into address bus, data bus, control bus, etc. For ease of description, Figure 5 The text uses only one arrow to describe it, but does not indicate that there is only one bus or one type of bus.

[0091] Based on the same inventive concept as the above-described method embodiments, embodiments of the present invention also provide a computer-readable storage medium for storing a computer program. When the computer program is run on a computer, it enables the electronic device to implement the control method described in the above embodiments. The computer-readable storage medium can be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0092] Based on the same inventive concept as the above-described method embodiments, embodiments of the present invention also provide a computer program product, which includes a computer program that, when run on an electronic device, causes the electronic device to perform the steps of the control methods described above according to various exemplary embodiments of this application. The program product may take the form of any combination of one or more readable media. These computer program commands can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the commands executed by the processor of the computer or other programmable data processing device generate a process for implementing... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0093] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

Claims

1. A data migration method for intranets, characterized in that, include: During the migration task execution, model change events, configuration execution events, and AI inference basis events are written to the audit log library in a structured format; Self-evolution based on historical events recorded in the audit log library includes: identifying repeated failure patterns in the audit logs and automatically generating corresponding optimization configurations or patch configurations based on the failure patterns; wherein, each self-evolution action and the audit logs on which it is based are written as audit events into the audit log library; Apply the optimized configuration or patch configuration to perform subsequent migrations, and write the execution results to the audit log library.

2. The method according to claim 1, characterized in that, The method further includes: When the failure rate in the execution results of the current configuration exceeds the first threshold, query the configuration version history in the audit log library; Locate the configuration version that was most recently successfully executed, and calculate the difference between the current configuration and the successful configuration; Automatically roll back to the successful configuration version and trigger a compensatory migration of the affected data based on the unique identifier range of the failure records in the audit log; Write the rollback operation and compensation migration results to the audit log.

3. The method according to claim 2, characterized in that, The triggering of compensation migration for affected data includes: Extract a unique set of identifiers for failure records from the audit log; Generate a compensation migration configuration for the set of failure records, wherein the compensation migration configuration reuses the portion of the currently valid configuration that was not affected by the rollback; After the compensation migration is completed, the compensation results will be merged with the original successful records to form a complete migration dataset.

4. The method according to claim 1, characterized in that, The identification of recurring failure patterns in the audit logs includes: Failure records are extracted from the configuration execution events in the audit logs. Each failure record includes three dimensions: source field, target field, and error type. When the number of failures of a failure record with the same three dimensions exceeds the second threshold within a preset time window, a local patch configuration is generated for the source field and / or target field in that failure record. When the proportion of failure records with the same three dimensions affecting the total number of migration records exceeds the third threshold, a global configuration replacement recommendation is generated.

5. The method according to claim 4, characterized in that: After applying the optimized configuration or patch configuration to perform subsequent migrations, the process also includes: Calculate the new failure rate resulting from subsequent migration executions. When the new failure rate exceeds the fourth threshold, mark the corresponding optimization configuration or patch configuration as invalid evolution. A suppression rule is set for the invalid evolution, which prohibits the generation of configurations with the same feature fields and the same error type in subsequent self-evolution iterations; Write the invalid evolution markers and corresponding suppression rules into the audit log.

6. The method according to claim 5, characterized in that, The self-evolution based on historical events recorded in the audit log library also includes: For the same failure mode, generate two or more optimization configurations or patch configurations as candidate configurations. The candidate configuration is executed in shadow migration mode, which reads data but does not write it to the target system, or writes it to an isolated sandbox target table. Compare the execution results of each candidate configuration and select the candidate configuration with the highest success rate as the basis for self-evolution; The evaluation process and results performed under the shadow migration mode are written to the audit log.

7. The method according to claim 1, characterized in that, The self-evolution based on historical events recorded in the audit log library also includes: Extract successfully executed mapping relationships from audit logs and store them as a mapping knowledge graph by source field name, target field name, transformation function identifier, and usage frequency; When the source field of a new migration task has no direct match in the knowledge graph, the semantic similarity between the source field and the field names in the graph is calculated, and the existing mapping relationship with the highest similarity is recalled as the initial recommendation configuration.

8. A data migration device for an intranet, characterized in that, include: The event logging module is used to write model change events, configuration execution events, and AI inference basis events into the audit log library in a structured format during the execution of migration tasks. The self-evolution module is used to perform self-evolution based on historical events recorded in the audit log library, including: identifying repeated failure patterns in the audit logs and automatically generating corresponding optimization configurations or patch configurations according to the failure patterns; wherein, each self-evolution action and the audit logs on which it is based are written as audit events into the audit log library; The subsequent migration module is used to apply the optimized configuration or patch configuration to perform subsequent migrations and write the execution results to the audit log library.

9. An electronic device, characterized in that, The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it causes the electronic device to perform the method described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program that, when run on a computer, causes the electronic device to implement the method described in any one of claims 1-7.

Images