Intelligent data conversion and optimization system and method, electronic equipment and storage medium

Abstract

The invention discloses an intelligent data conversion and optimization system, and relates to the field of service oriented architectures, and the system comprises a data loading and preprocessing module which is used for extracting target information and loading a first rule file; the analysis and mapping module is used for analyzing the first rule file and establishing a second rule file of the target information and the target data; the data conversion module is used for converting target information according to the rule file; the rule verification and anomaly detection module is used for verifying the target information according to the first rule file and recording anomaly information; the optimization suggestion generation module is used for generating optimization suggestions according to the target information in the conversion process; the result output module is used for outputting and storing the execution file; and the system life cycle evolution management module is used for expansibility architecture design and a rule file updating mechanism. Through the scheme, the automatic data conversion efficiency and accuracy are improved, an error detection and repair mechanism is introduced, and unnecessary rework caused by errors is reduced.

Description

Technical Field

[0001] The present application relates to the field of service-oriented architecture, and in particular to an intelligent data conversion and optimization system, an intelligent data conversion and optimization method, an electronic device, and a storage medium. Background Art

[0002] In next-generation automotive architectures, Service-Oriented Architecture (SOA) is being widely adopted in vehicle systems to enable more flexible and scalable communication. Using Ethernet as the communication medium, different services can be located on different computers or devices, exchanging data and interoperating via the SOME / IP protocol stack. Built on Ethernet, SOME / IP allows various embedded systems to communicate using Ethernet, thus realizing the concept of SOA. In this architecture, network professionals fill out a service information form containing detailed definitions of the services, and then generate a protocol stack based on this information. The generated protocol stack is integrated into the chassis and enables communication between various application layers through subscriptions and the interaction of service signals. However, due to the long communication links and the large number of developers, the development links for functions need to be highly unified to achieve correct signal interaction. Therefore, a method is needed to automatically generate communication links.

[0003] In most scenarios, especially in the generation of smart cockpit or service interface tables in the automotive industry, data formats, types, and structures are very complex, often requiring manual processing of large amounts of data and conversion operations. This manual process is not only time-consuming and labor-intensive, but also prone to errors. Summary of the Invention

[0004] The purpose of the present invention is to provide an intelligent data conversion and optimization system, which eliminates the complexity of manual configuration and greatly improves efficiency and accuracy by introducing a rule engine and an automated data conversion mechanism. Data types and structures are often inconsistent between different systems or platforms. For example, some data may need to be represented in different ways in different scenarios, such as converting boolean type data to uint8 type, etc. If manual processing is relied upon, errors or omissions are likely to occur. The present invention realizes automated and intelligent format compatibility processing by automatically identifying and processing different data types and structures through rule-driven. Errors in traditional data conversion processes are usually difficult to detect in the early stages and often require later investigation, resulting in delayed problem exposure and increased data processing costs. The present invention introduces an intelligent error detection and repair mechanism. During the data conversion process, the system automatically detects format mismatches, data type errors, or out-of-range values, and makes real-time repair suggestions or automatic repairs, reducing unnecessary rework caused by errors.

[0005] The present invention provides the following solutions:

[0006] According to one aspect of the present invention, there is provided an intelligent data conversion and optimization system, comprising:

[0007] Data loading and preprocessing module, parsing and mapping module, data conversion module, optimization suggestion generation module, result output module and system lifecycle evolution management module;

[0008] A data loading and preprocessing module is used to extract target information and load the first rule file;

[0009] A parsing and mapping module, configured to parse the first rule file and create a second rule file of target information and target data;

[0010] a data conversion module, configured to convert target information according to the second rule file;

[0011] A rule verification and anomaly detection module, configured to verify target information according to the first rule file and record anomaly information;

[0012] An optimization suggestion generation module is used to generate optimization suggestions based on target information during the conversion process;

[0013] Result output module, used to output and save execution files;

[0014] System lifecycle evolution management module, used for scalable architecture design and rule file update mechanism.

[0015] Furthermore, the target information includes:

[0016] Get the original data in the source file;

[0017] Acquire abnormal data generated by processing the original data;

[0018] Mark and normalize abnormal data in the acquired source files.

[0019] Furthermore, the rule file includes: type conversion rules, message format rules and field validation rules.

[0020] Furthermore, the data conversion module includes:

[0021] converting the target information into a target format according to the first rule file;

[0022] The target information is checked according to the first rule file. If the target information is empty, a default value is generated or an error prompt is output according to the first rule file.

[0023] Furthermore, according to the first rule file, a target table is created in the target data storage and corresponding field formats and data types are set.

[0024] Furthermore, the rule verification and anomaly detection module includes:

[0025] Perform multi-dimensional verification;

[0026] Generate detailed error description information for each error found during the verification process;

[0027] Based on the error description information, a verification report is output, which includes all the errors detected and the corresponding repair suggestions;

[0028] Automatically repair solvable errors based on preset rules.

[0029] Furthermore, multi-dimensional verification is performed, including verification of data type, data format, value range and field dependency.

[0030] According to two aspects of the present invention, there is provided an intelligent data conversion and optimization method, comprising:

[0031] Step S1, extracting target information from a source file and loading a first rule file, marking and normalizing the target information;

[0032] Step S2, creating a second rule file of target information and target data based on the first rule file;

[0033] Step S3, converting the target information according to the second rule file;

[0034] Step S4, verifying the target information according to the first rule file and recording abnormal information;

[0035] Step S5, generating optimization suggestions according to the target information during the conversion process;

[0036] Step S6, outputting and saving the execution file;

[0037] Step S7: dynamically update the rule file and provide plug-ins or modular design to expand system functions.

[0038] According to three aspects of the present invention, there is provided an electronic device, comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0039] The memory stores a computer program, which, when executed by the processor, causes the processor to execute the steps of the intelligent data conversion and optimization method.

[0040] According to four aspects of the present invention, a computer-readable storage medium is provided, which stores a computer program executable by an electronic device. When the computer program runs on the electronic device, the electronic device executes the steps of an intelligent data conversion and optimization method.

[0041] According to five aspects of the present invention, a detection platform is provided, comprising:

[0042] Electronic device for use in the steps of the intelligent data conversion and optimization method;

[0043] a processor that runs a program, and when the program runs, executes steps based on the intelligent data conversion and optimization method from data output by the electronic device;

[0044] The storage medium is used to store a program, and when the program is running, it executes the steps of the intelligent data conversion and optimization method for the data output from the electronic device.

[0045] Through the above solution, the following beneficial technical effects are achieved:

[0046] This invention solves problems such as inefficient manual processing, difficult error detection, insufficient data compatibility, lack of optimization suggestions, poor system scalability, and low efficiency in big data processing through a rule-driven intelligent data conversion system. It greatly improves the automation, intelligence, and flexibility of data processing and is suitable for automotive smart cockpits and other complex data processing scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a structural diagram of an intelligent data conversion and optimization system provided by one or more embodiments of the present invention.

[0048] Figure 2 This is a flowchart of an intelligent data conversion and optimization method provided by one or more embodiments of the present invention.

[0049] Figure 3 A structural block diagram of an electronic device according to one or more embodiments of the present invention provides an intelligent data conversion and optimization method. DETAILED DESCRIPTION

[0050] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0051] Figure 1 This is a structural diagram of an intelligent data conversion and optimization system provided by one or more embodiments of the present invention.

[0052] like Figure 1 An intelligent data conversion and optimization system is shown, comprising:

[0053] Data loading and preprocessing module, parsing and mapping module, data conversion module, optimization suggestion generation module, result output module and system lifecycle evolution management module;

[0054] A data loading and preprocessing module is used to extract target information and load the first rule file;

[0055] A parsing and mapping module, configured to parse the first rule file and create a second rule file of target information and target data;

[0056] A data conversion module, configured to convert target information according to a second rule file;

[0057] A rule verification and anomaly detection module, configured to verify target information according to the first rule file and record anomaly information;

[0058] An optimization suggestion generation module is used to generate optimization suggestions based on target information during the conversion process;

[0059] Result output module, used to output and save execution files;

[0060] System lifecycle evolution management module, used for scalable architecture design and rule file update mechanism.

[0061] The target information includes:

[0062] Get the original data in the source file;

[0063] Acquire abnormal data generated by processing the original data;

[0064] Mark and normalize abnormal data in the acquired source files.

[0065] Specifically, by marking null values ​​and malformed data discovered during the preprocessing process, null values ​​can be marked in a variety of ways. For example, in numeric data columns, specific placeholders such as -9999 or NaN can be used to represent missing values; for character data columns, "NULL" or "" can be used for marking. At the same time, by establishing a data quality report, the number and location of null values ​​in each column are recorded in detail for subsequent analysis and processing. For malformed data, it can be marked according to the business rules and data type of the data. For example, data with incorrect date format can be marked as "ERROR_DATE"; data that does not meet the range requirements in numeric types can be marked as "ERROR_VALUE". In addition, data visualization tools such as Seaborn and Matplotlib can be used to draw box plots, histograms, etc. to intuitively display the data distribution and help quickly locate data with formatting problems.

[0066] Furthermore, after marking is completed, subsequent processing methods are diverse. For null-valued data, you can choose to delete records containing null values, but this approach may result in information loss when the data volume is small or the proportion of null values ​​is high. You can also use filling methods, such as filling numeric data with the mean or median, filling character data with the mode, or intelligent filling based on the logical relationship between the data. For data with incorrect format, you can attempt to convert and repair the format. If repair is not possible, you need to determine whether to delete these records or perform special processing based on business needs to ensure data consistency and accuracy, laying a solid foundation for subsequent data analysis, modeling, and other work.

[0067] The rule file includes: type conversion rules, message format rules and field validation rules.

[0068] Specifically, rule file loading, as an important part of the data processing process, is a key step to ensure the accuracy and standardization of data processing.

[0069] Data type conversion rules are used to convert raw data into a data type that meets business requirements. In actual data processing, data types obtained from different data sources often differ. For example, a date field read from a database may be stored as a string, but subsequent data analysis requires converting it to a date type. In this case, a data type conversion rule explicitly specifies the target data type for each field, such as converting a string like "2024-01-01" to a datetime type. In Python, data type conversion can be achieved by writing custom functions or using the astype() method of the Pandas library. For example, if there is a DataFrame named data with a string column named date_column, to convert it to a date type, you can execute data['date_column']=

[0070] data['date_column'].astype('datetime64[ns]'). For complex type conversions, such as converting an enumerated string to its corresponding numeric encoding, the mapping relationships are detailed in the rule file. Developers can write the corresponding conversion logic based on these rules to ensure data type consistency and accuracy.

[0071] Message format rules primarily regulate the structure and format of data during transmission and storage. In scenarios such as network communication and inter-system data exchange, message formats must adhere to specific standards. For example, in HTTP requests and responses, messages typically consist of a request line / status line, header fields, and a message body, each with specific formatting requirements. A rule file defines in detail the message's hierarchical structure, field order, delimiters, and other aspects. For JSON-formatted messages, for example, the rule file specifies the name, data type, and nesting relationships of each field. During data processing, the JSON library in Python can be used to serialize and deserialize data according to the message format rules. When sending data, Python objects are converted to JSON strings that conform to the rules; when receiving data, JSON strings are parsed into Python objects, ensuring correct and consistent data formatting during transmission. Furthermore, for specialized message formats, such as custom binary message formats, the rule file details information such as byte order, field length, and checksum. Developers can then write corresponding parsing functions to parse and process the messages according to the rules.

[0072] Field validation rules are used to ensure data validity and integrity. In real-world applications, each field has specific value ranges, format requirements, and constraints. For example, a phone number field must conform to an 11-digit format, and an age field must be within a reasonable range (e.g., 0-150). The rule file clearly lists the validation criteria for each field, such as regular expressions for format verification and numeric ranges for value validity. In Python, the re module can be used with regular expressions to perform format validation on string fields. For numeric fields, comparison operators are used to determine whether they are within a specified range. For example, to validate a phone number field, you could use importre; pattern = r'^1[3-9]\d{9}$'; ifnot re.match(pattern, phone_number): # to handle validation failures. Furthermore, for complex validation logic, such as inter-field correlation validation (e.g., the start date must be earlier than the end date), the rule file details the validation rules and handling methods. Developers can then write corresponding validation functions based on these rules to rigorously validate each field during data processing to ensure data quality.

[0073] wherein, the target information is converted into a target format according to the first rule file;

[0074] The target information is checked according to the first rule file. If the target information is empty, a fill default value is generated or an error prompt is output according to the first rule file.

[0075] According to the first rule file, a target table is created in the target data storage and corresponding field formats and data types are set.

[0076] Specifically, the conversion process requires dynamically adjusting the target table's field formats and data types based on the source data table's specific content. Based on the conversion rules, a target table is created in the target data store and the corresponding field formats and data types are set. When processing structures, the data types and names of each member field are ensured to conform to the target format's requirements. Conversion Result: The source data is successfully converted to the target data format, and all fields meet the rule requirements.

[0077] Among them, the rule verification and anomaly detection module includes:

[0078] Perform multi-dimensional verification;

[0079] Generate detailed error description information for each error found during the verification process;

[0080] Based on the error description information, a verification report is output, which includes all the errors detected and the corresponding repair suggestions;

[0081] Automatically repair solvable errors based on preset rules.

[0082] Specifically, each field is verified one by one according to the rule file. For example:

[0083] Check whether the field and period meet the minimum or maximum value range requirements. Check whether the service name matches the server name in the source file.

[0084] Perform multi-dimensional verification, including verification of data type, data format, value range, and field dependencies.

[0085] The automatic repair process involves handling different data types (such as integers, floating-point numbers, and strings) with various constraints (such as value range, string length, and data format). First, a data type check or determination of the specific data type is performed, such as the isinstance() function. For example, if the task stack contains an element x, isinstance(x, int) can be used to determine whether it is an integer, isinstance(x, float) to determine whether it is a floating-point number, and isinstance(x, str) to determine whether it is a string.

[0086] Numeric values ​​(integers and floating-point numbers) are checked according to predefined range rules. For example, integers are in the range of 0 to 100, and floating-point numbers are in the range of 0.0 to 10.0. You can use comparison operators to check whether a value is out of range, such as ifx<0orx>100 (for integers) and ifx<0.0orx>10.0 (for floating-point numbers).

[0087] For string types, check whether their length meets preset rules. For example, if the string length is required to be between 5 and 20 characters, you can use the len(x) function to obtain the string length and then verify it with iflen(x) < 5 or len(x) > 20. You can also check the format of the string, such as whether it meets a specific regular expression pattern. For example, whether it is a valid email address format, you can use the re.match() function combined with a regular expression to verify it.

[0088] If the value is out of range, it is fixed according to the preset repair strategy. For example, for integers out of range, it is set to the nearest boundary value, such as 0 if it is less than 0 and 100 if it is greater than 100. For floating-point numbers out of range, it is also set to the boundary value, such as 0.0 if it is less than 0 and 10.0 if it is greater than 10.0.

[0089] If the string length does not meet the requirements, you can fix it by truncating or padding it. If the string is too long, truncate it to the maximum allowed length; if it is too short, add specific characters (such as spaces or specified padding characters) to the end to make it meet the minimum length requirement. For example, use string slicing operations for truncation, such as x = x[:20] (assuming the maximum length is 20), and use string concatenation operations for padding, such as x = x.ljust(5,") (assuming the minimum length is 5, use spaces for padding).

[0090] For strings that don't conform to the required format, we attempt to convert or correct them according to certain rules. For example, if the email address is formatted incorrectly, we might extract the valid parts and reassemble them into the correct format, or prompt the user to enter the correct email address before replacing it. For strings in other specific formats, we perform appropriate repairs based on the specific formatting specifications.

[0091] Specifically, the rule-driven intelligent data conversion system solves problems such as inefficient manual processing, difficult error detection, insufficient data compatibility, lack of optimization suggestions, poor system scalability, and inefficient big data processing. It greatly improves the automation, intelligence, and flexibility of data processing and is suitable for automotive smart cockpits and other complex data processing scenarios.

[0092] In another embodiment,

[0093] 1. Data reading and preprocessing,

[0094] Extract data from source files (e.g. Excel) and prepare them for subsequent conversion.

[0095] Data loading:

[0096] Extract raw data from data sources (Excel, database or other files). During preprocessing,

[0097] Empty values ​​and malformed data need to be marked for subsequent processing.

[0098] Rule file loading:

[0099] At the same time, the rule file is loaded, including data type conversion rules, message format rules, field validation rules, etc.

[0100] Identify and process null and duplicate values, and tag and normalize field names based on the identified and processed data to ensure matching with subsequent rules.

[0101] 2. Rule parsing and mapping preparation,

[0102] Parse the rule file and establish a mapping relationship between source data and target data.

[0103] Rule parsing: Parse the rule file into a computer-readable structure based on business needs, for example:

[0104] Field A must be an integer (rule: the data type of field A is int). If field B is bool, it needs to be converted to uint8.

[0105] Field mapping table:

[0106] Match the source data fields with the target data fields. For example, the data type field of the source data needs to be mapped to the Type column of the target table.

[0107] Complex situations need to be considered when parsing rules, such as structure data, array type data, etc.

[0108] Create flexible mapping tables to adapt to different source data table structures.

[0109] Mapping preparation results in the establishment of a source-to-target mapping rule, ensuring that these rules are applied in subsequent steps.

[0110] 3. Data conversion process,

[0111] Converts fields in source data to target formats based on rules.

[0112] Basic type conversion:

[0113] For each field, the data type is checked according to the rules. For example, if the source data type is bool, the rule requires conversion to uint8; if the field is empty, the rule requires filling in a default value or outputting an error message.

[0114] Array and structure processing:

[0115] If a field is defined as an array type, the maximum length of each element is filled in according to the rules, and the data type of each element is processed.

[0116] If a field is a structure, its member variables need to be processed recursively to ensure that the definition of each member complies with the rules.

[0117] During the conversion process, the field format and data type of the target table need to be dynamically adjusted according to the specific content of the source data table.

[0118] According to the conversion rules, a target table is created in the target data store and corresponding field formats and data types are set;

[0119] When processing a structure, ensure that the data type and name of each member field conform to the requirements of the target format.

[0120] The source data has been successfully converted to the target data format, and all fields have met the rule requirements.

[0121] 4. Intelligent verification and error detection,

[0122] Automatically detect errors or abnormal data during the conversion process and provide repair suggestions.

[0123] Verify each field one by one according to the rule file. For example, check whether the field and period meet the minimum or maximum value range requirements; and check whether the service name matches the server name in the source file.

[0124] When data anomalies are detected, the system will record and output a detailed error report. For example, a field format does not conform to the expected rules or fails to meet the expected rules, or a data type is undefined or out of support.

[0125] Validation is not limited to data types, but also includes complex validation such as format, value range, field dependencies, etc.

[0126] For each error, a detailed description should be provided, and suggestions for automatic repairs should be given.

[0127] Verification results: Output a report containing all errors and repair suggestions, and automatically fix some resolvable problems based on the rules.

[0128] 5. Automatic optimization and suggestion generation,

[0129] During the conversion process, the system makes optimization suggestions for certain fields based on historical data or best practices.

[0130] Historical data analysis:

[0131] Based on historical data analysis, identify the most commonly used message cycles, socket types, service ID generation methods, etc.

[0132] Optimization suggestion generation:

[0133] After the conversion is complete, the system can make optimization suggestions for certain fields based on the analysis results, such as:

[0134] The message period of a certain field is too long. It is recommended to shorten the period.

[0135] A method ID conflicts. It is recommended to adjust or reallocate the ID.

[0136] Optimization suggestions are generated based on actual usage data, and the suggestions can be adjusted by setting custom weights or conditions.

[0137] While generating the data table, the system provides a series of optimization suggestions to help users further improve the generated service table.

[0138] 6. Output and save results,

[0139] After data conversion, verification, and optimization are completed, the result file is output.

[0140] Target table generation:

[0141] Write the converted data to the target Excel table, ensuring that the format, type, and order of all fields are as expected.

[0142] Report Generation:

[0143] Outputs a detailed log file that records the rules applied during the conversion process, error detection and repair content, and final optimization suggestions.

[0144] Save file: Generate and save a new target file for subsequent use.

[0145] Ensure that the file format is correct and all data has been converted and verified correctly.

[0146] The output report should include detailed conversion process, error fixes, and optimization suggestions to facilitate subsequent analysis and adjustment.

[0147] 7. Expansion and maintenance,

[0148] Supports subsequent system expansion and rule updates.

[0149] Rule update: Supports dynamic update of rule files. The system can automatically adjust the data conversion method according to the new rules without modifying the code.

[0150] Functional extension: Allows users to expand system functionality through plug-ins or modular design to support more data source types, target formats, or new validation and optimization rules.

[0151] The system should be flexible and scalable to adapt to changing business needs or newly introduced rules and standards.

[0152] Figure 2 This is a flowchart of an intelligent data conversion and optimization method provided by one or more embodiments of the present invention.

[0153] like Figure 2 An intelligent data conversion and optimization method is shown, comprising:

[0154] Step S1, extracting target information from a source file and loading a first rule file, marking and normalizing the target information;

[0155] Step S2, creating a second rule file of target information and target data based on the first rule file;

[0156] Step S3, converting the target information according to the second rule file;

[0157] Step S4, verifying the target information according to the first rule file and recording abnormal information;

[0158] Step S5, generating optimization suggestions according to the target information during the conversion process;

[0159] Step S6, outputting and saving the execution file;

[0160] Step S7: dynamically update the rule file and provide plug-ins or modular design to expand system functions.

[0161] Wherein, in step S1, the target information is extracted and the rule file is loaded, and the target information is marked and normalized based on the rule file;

[0162] Specifically, this step solves the technical problems that the target information in the source file has confusing formats, lacks unified standards, is prone to errors and is inefficient in manual processing, and is difficult to use directly for subsequent processing.

[0163] By extracting target information and marking and normalizing it according to the first rule file, the information format is unified and the structure is clear, which improves the accuracy and consistency of the information, lays a good foundation for subsequent steps, and reduces processing errors caused by information confusion.

[0164] Wherein, the mapping relationship between the source data and the target data is obtained through the first rule file in step S2.

[0165] Specifically, the establishment of the second rule file solves the technical problem that there is a lack of clear mapping rules between target information and target data, which cannot ensure the accuracy and standardization of the conversion and may lead to data conversion errors or inconsistencies.

[0166] By establishing a second rule file, the conversion logic from target information to target data is clarified, and a standardized rule file is formed to provide clear guidance for subsequent data conversion and ensure the accuracy, consistency and repeatability of the conversion process.

[0167] Wherein, by converting the target information according to the second rule file in step S3,

[0168] Specifically, it aims to solve the technical problems that manual data conversion is inefficient, error-prone, and difficult to meet the needs of large-scale or complex data conversion.

[0169] By realizing automated data conversion, conversion efficiency is greatly improved, human errors are reduced, and target information is accurately converted into target data according to established rules, thereby improving overall processing efficiency and quality.

[0170] Among them, by verifying the target information and recording the abnormal information in step S4,

[0171] Specifically, this step solves the technical problem in traditional data processing where errors are difficult to detect in a timely manner and often require later investigation, which increases costs and affects data availability.

[0172] The technical effect achieved through this step is to detect the format matching and data type correctness of the target information in real time, promptly discover and record anomalies (such as format mismatch, value out of range), avoid erroneous data from flowing into subsequent links, reduce the cost of subsequent investigation, and improve the reliability and accuracy of data processing.

[0173] Wherein, step S5 generates optimization suggestions according to the target information during the conversion process.

[0174] Specifically, this step aims to solve technical problems that may exist in the data conversion process (such as efficiency, accuracy, etc.), which are difficult for humans to identify and improve in real time.

[0175] The technical effect achieved through this step is that the system automatically analyzes the target information, generates targeted optimization suggestions (such as adjusting conversion parameters, improving rules), improves the efficiency and quality of data processing, and enables the system to have a certain intelligent optimization capability to adapt to complex and changing needs.

[0176] Among them, step S6 outputs and saves the execution file.

[0177] Specifically, this step needs to solve the technical problem that if the processed data is not properly stored and output, it will affect the traceability and reuse of the data and make it difficult to support subsequent applications (such as analysis and sharing).

[0178] The technical effect achieved through this step is to output the processing results and save them as an executable file, ensuring that the data can be stored for a long time and called at any time, improving the availability and value of the data, and facilitating subsequent analysis, auditing, or interaction with other systems.

[0179] Among them, step S7 dynamically updates the rule file and provides plug-ins or modular design to expand system functions.

[0180] Specifically, this step aims to solve the technical problems that traditional system rules are fixed, difficult to adapt to new needs or technological changes, difficult to expand functions, and require a large amount of code modification or even redevelopment.

[0181] The technical effect achieved through this step is that the rule file is dynamically updated so that the system can quickly respond to changes in demand (such as new data types and conversion rules); plug-ins or modular design allow for the flexible addition of new functions (such as support for new data formats and processing algorithms), enhancing the adaptability, scalability and flexibility of the system, extending the system life cycle, and reducing maintenance and upgrade costs.

[0182] In summary, by introducing a rule engine and automated data conversion mechanism, the complexity of manual configuration is eliminated, and efficiency and accuracy are greatly improved. Data types and structures are often inconsistent between different systems or platforms. If manual processing is relied upon, errors or omissions are likely to occur. This system automatically identifies and processes different data types and structures through rule-driven processing, achieving automated and intelligent format compatibility processing. Errors in traditional data conversion processes are usually difficult to detect in the early stages and often require later investigation, resulting in delayed problem exposure and increased data processing costs. This system introduces an intelligent error detection and repair mechanism. During the data conversion process, the system automatically detects format mismatches, data type errors, or out-of-range values, and makes real-time repair suggestions or automatic repairs, reducing unnecessary rework caused by errors.

[0183] Figure 3 A block diagram of an electronic device structure of an intelligent data conversion and optimization method provided by one or more embodiments of the present invention

[0184] like Figure 3 As shown, the present application provides an electronic device, comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0185] A computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the intelligent data conversion and optimization method.

[0186] The present application also provides a computer-readable storage medium storing a computer program executable by an electronic device. When the computer program runs on the electronic device, the electronic device executes the steps of the intelligent data conversion and optimization method.

[0187] For simplicity of description, the method embodiments are described as a series of actions. However, those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because certain steps can be performed in other orders or simultaneously according to the embodiments of the present invention. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0188] Through the description of the above embodiments, it can be seen that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general-purpose hardware platform. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present application or certain parts of the embodiments.

[0189] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent data conversion and optimization system, characterized in that: include: Data loading and preprocessing module, parsing and mapping module, data conversion module, optimization suggestion generation module, result output module and system lifecycle evolution management module; A data loading and preprocessing module is used to extract target information and load the first rule file; A parsing and mapping module, configured to parse the first rule file and create a second rule file of target information and target data; a data conversion module, configured to convert target information according to the second rule file; A rule verification and anomaly detection module, configured to verify target information according to the first rule file and record anomaly information; An optimization suggestion generating module, configured to generate optimization suggestions according to the target information during the conversion process; Result output module, used to output and save execution files; System lifecycle evolution management module, used for scalable architecture design and rule file update mechanism.

2. The intelligent data conversion and optimization system according to claim 1, characterized in that: The target information includes: Get the original data in the source file; Acquire abnormal data generated by processing the original data; Mark and normalize abnormal data in the acquired source files.

3. The intelligent data conversion and optimization system according to claim 1, characterized in that: The rule file includes: type conversion rules, message format rules and field verification rules.

4. The intelligent data conversion and optimization system according to claim 1, characterized in that: The data conversion module includes: converting the target information into a target format according to the first rule file; The target information is checked according to the first rule file. If the target information is empty, a default value is generated or an error prompt is output according to the first rule file.

5. The intelligent data conversion and optimization system according to claim 4, characterized in that: According to the first rule file, a target table is created in the target data storage and corresponding field formats and data types are set.

6. The intelligent data conversion and optimization system according to claim 1, characterized in that: Rule verification and anomaly detection module, including: Perform multi-dimensional verification; Generate detailed error description information for each error found during the verification process; Based on the error description information, a verification report is output, which includes all the errors detected and the corresponding repair suggestions; Automatically repair solvable errors based on preset rules.

7. The intelligent data conversion and optimization system according to claim 6, characterized in that: Perform multi-dimensional verification, including verification of data type, data format, value range, and field dependencies.

8. An intelligent data conversion and optimization method, characterized by: Step S1, extracting target information from a source file and loading a first rule file, marking and normalizing the target information; Step S2, creating a second rule file of target information and target data based on the first rule file; Step S3, converting the target information according to the second rule file; Step S4, verifying the target information according to the first rule file and recording abnormal information; Step S5, generating optimization suggestions according to the target information during the conversion process; Step S6, outputting and saving the execution file; Step S7: dynamically update the rule file and provide plug-ins or modular design to expand system functions.

9. An electronic device, characterized in that: include: A processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus; A computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the intelligent data conversion and optimization method according to claim 8.

10. A computer-readable storage medium, characterized in that It stores a computer program that can be executed by an electronic device. When the computer program runs on the electronic device, the electronic device executes the steps of the intelligent data conversion and optimization method according to claim 8.

Images