Data format conversion method and device based on dynamic lazy loading, equipment and medium

Abstract

The invention discloses a data format conversion method and device based on dynamic lazy loading, equipment and a medium, and the method comprises the steps: obtaining original data and target format information in response to a data transmission request of a client; analyzing the original data to generate a data object; if format conversion needs to be carried out on the original data, searching a format conversion plug-in matched with the target format information in a plug-in library as a target conversion plug-in, and loading the target conversion plug-in into a memory; and after the loading of the target conversion plug-in is completed, enabling the target conversion plug-in to obtain the data object through a reflection calling plug-in method, and carrying out mapping and serialization processing on the data object to obtain to-be-transmitted target data. According to the method, loading of a large number of possible idle format conversion plug-ins during system initialization is effectively avoided, system resource occupation is greatly reduced, and the running efficiency and the starting speed of the system are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data format conversion method, device, equipment and medium based on dynamic lazy loading. Background Art

[0002] In the current complex environment of data communication and processing, data interaction between different systems will inevitably face the challenge of data format conversion to meet the specific requirements of the recipient. Traditional data format conversion methods often load all possible format conversion modules during the system initialization phase. This approach has significant disadvantages: first, a large number of modules that may be idle are loaded, which seriously occupies system resources, causing the system to start slowly and greatly reducing operating efficiency; second, once new data format conversion logic is needed, the entire system has to be redeployed and modified, which is inflexible and difficult to keep up with the rapidly changing business rhythm. Especially when sending data to the Kafka cluster, how to balance efficiency and flexibility, properly handle data in various formats, and meet the diverse needs of different customers has become a difficult problem that needs to be overcome in this field. Summary of the invention

[0003] In order to solve the above problems existing in the prior art, the present invention provides a data format conversion method, device, equipment and medium based on dynamic lazy loading. The technical problem to be solved by the present invention is achieved through the following technical solutions: A first aspect of the present invention provides a data format conversion method based on dynamic lazy loading, which is applied to a main engine. The method includes: Responding to a data transmission request from a client, obtaining original data and target format information; Parsing the original data to generate a data object; If the format of the original data needs to be converted, a format conversion plug-in matching the target format information is searched in the plug-in library as a target conversion plug-in, and the target conversion plug-in is loaded into the memory; After the target conversion plug-in is loaded, the plug-in method is called through reflection to enable the target conversion plug-in to obtain the data object, map and serialize the data object, and obtain the target data to be transmitted.

[0004] The method provided by the present invention, through a dynamic lazy loading mechanism, loads the corresponding conversion plug-in only when data format conversion is actually needed, effectively avoiding the loading of a large number of possibly idle format conversion plug-ins during system initialization, greatly reducing system resource usage, and significantly improving the system's operating efficiency and startup speed.

[0005] In a possible implementation, the method further includes: Identify whether the original data is valid; If the original data is valid, determining a valid data type; It is determined whether the format of the original data needs to be converted based on the valid data type and the target format information.

[0006] In a possible implementation, parsing the original data to generate a data object includes: Parse the original data to obtain DML data, DDL data and SQL data; Taking the DML data, the DDL data and the SQL data as data, the operation statement framework is encapsulated into an operation data object, the data dictionary is encapsulated into a dictionary data object, and the business data is encapsulated into a business data object.

[0007] In a possible implementation, the method further includes: When the main engine is started, all plug-in jar files in the plug-in directory are read and loaded into the Java virtual machine; each plug-in jar file corresponds to a conversion plug-in, and the plug-in jar file includes data format conversion logic, business processing methods and plug-in functions.

[0008] In a possible implementation, the method further includes: When a new customer data conversion customization requirement is added, a new plug-in project created by a developer in an integrated development environment is obtained, and the plug-in project is packaged and deployed to the plug-in directory.

[0009] In a possible implementation, the method further includes: When the plug-in library is initialized, unique identification information is assigned to each format conversion plug-in, and a mapping table between the unique identification information and the target format information is established; The step of searching the plug-in library for a format conversion plug-in that matches the target format information as the target conversion plug-in includes: Searching the mapping table for the unique identification information that has a mapping relationship with the target format information; The format conversion plug-in corresponding to the unique identification information is used as the target conversion plug-in.

[0010] In a possible implementation, the method further includes: receiving the target data sent by the target conversion plug-in after verifying the target data, and sending the target data to a target recipient; or, Receive error report information sent by the target conversion plug-in after the target data verification fails, and generate a processing solution based on the error report information.

[0011] A second aspect of the present invention provides a data format conversion device based on dynamic lazy loading, which is applied to a main engine, and the device includes: An acquisition module, used to respond to a data transmission request from a client and acquire original data and target format information; An object generation module, used for parsing the original data and generating a data object; A search loading module is used to search for a target conversion plug-in matching the target format information if the format conversion of the original data needs to be performed, and load the searched target conversion plug-in into the memory; The plug-in calling module is used to call the plug-in method through reflection after the target conversion plug-in is loaded, so that the target conversion plug-in obtains the data object, maps and serializes the data object, and obtains the target data to be transmitted.

[0012] The third aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a data format conversion method based on dynamic lazy loading provided in the first aspect of the present invention is implemented.

[0013] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for data format conversion based on dynamic lazy loading provided in the first aspect of the present invention is implemented.

[0014] For the specific description of the second to fourth aspects of the present invention and their various implementations, reference can be made to the detailed description of the first aspect and its various implementations; and for the beneficial effects of the second to fourth aspects and their various implementations, reference can be made to the analysis of the beneficial effects of the first aspect and its various implementations, which will not be repeated here.

[0015] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flowchart of a data format conversion method based on dynamic lazy loading according to an embodiment of the present invention; Figure 2 It is a structural block diagram of a data format conversion device based on dynamic lazy loading according to an embodiment of the present invention; Figure 3 It is a block diagram of the internal structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0017] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0018] An embodiment of the present invention provides a data format conversion method based on dynamic lazy loading, which can be executed by an electronic device, which can be a server or a terminal device. The server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services; the terminal device can be a smart phone, a tablet computer, a desktop computer, a wearable device, etc., but is not limited thereto. Of course, the electronic device can also be an indoor light source control device, or a photocatalytic air purifier.

[0019] Figure 1 The following is a flow chart of a data format conversion method based on dynamic lazy loading applied to a main engine provided in this embodiment. Figure 1 As shown, the main process of the method is described as follows (steps S101 to S104): Step S101, in response to a data transmission request from a client, obtaining original data and target format information; Step S102, parsing the original data to generate a data object; Step S103: if the format of the original data needs to be converted, a format conversion plug-in matching the target format information is searched in the plug-in library as a target conversion plug-in, and the target conversion plug-in is loaded into the memory; Step S104, after the target conversion plug-in is loaded, the plug-in method is called through reflection, so that the target conversion plug-in obtains the data object, maps and serializes the data object, and obtains the target data to be transmitted.

[0020] In this embodiment, the main engine can use the Kafka data engine (JDDM). As the key producer engine that sends standardized data to the Kafka cluster, the Kafka data engine (JDDM) starts listening on the designated port (Socket) and waits for the client's data transmission request. It has powerful parallel processing capabilities and can simultaneously receive standard data blocks sent by clients on different servers in the wide area network and local area network, covering DDL (data dictionary containing detailed table information such as user name, table name, table space, column name, column type, column length, column precision, etc.), DML (detailed data involving table operations such as insert, update, delete, etc.) and SQL (mainly referring to DDL change SQL, such as adding and subtracting columns from the table, modifying column types, adjusting column length and precision, and other database operation instructions) data.

[0021] When the system starts, the plug-in management module of the main engine first initializes the plug-in library and builds an efficient plug-in index to facilitate the subsequent quick search for the required target conversion plug-in. Specifically, when the plug-in library is initialized, each format conversion plug-in is assigned unique identification information (such as plug-in name, plug-in ID, etc.), and these unique identification information are associated with the plug-in function (such as supported data format), and then a mapping table between the unique identification information and the target format information is established. The mapping table can be a hash table or an index structure, with the target format information as the key (Key) and the corresponding unique identification information as the value (Value).

[0022] At that time, when the plug-in management module receives the target format information and needs to find a matching target conversion plug-in, it can search for unique identification information that has a mapping relationship with the target format information in a pre-established mapping table, and use the format conversion plug-in corresponding to the unique identification information as the target conversion plug-in. The corresponding target conversion plug-in can be quickly retrieved in the hash table directly through the target format information.

[0023] During the plugin loading phase, the main engine listens to plugin files in the specified path. When a new plugin file is detected, it loads the file and parses its supported data formats and other information, and dynamically updates the plugin index. This ensures that the plugin index is always up to date and can quickly respond to the addition or update of plugins.

[0024] During the plug-in development phase, metadata is defined for each format conversion plug-in, which includes information such as the data formats supported by the plug-in, functional descriptions, etc. When new customer data conversion customization requirements are added, obtain the new plug-in project / project created by the developer in the integrated development environment (such as Eclipse / IDEA of IDE integrated development tools), package the plug-in project, and deploy it to the plug-in directory.

[0025] It should be noted that the public code of the format conversion plug-in can be directly pulled from Git. Based on the pulled public code, developers can focus on implementing customer-specific data conversion functions. Since the public code already provides basic functions, developers can quickly implement customized requirements without having to write all the code from scratch.

[0026] When business requirements change and new data format conversion logic needs to be added, you only need to easily add the new format conversion plug-in to the plug-in library. The format conversion plug-in can be automatically loaded and put into use when needed, without the need for large-scale redeployment and modification of the entire system, greatly enhancing the system's adaptability to business changes.

[0027] In this embodiment, the plug-in directory refers to the Module directory, which is used to store files (such as Jar packages) of format conversion plug-ins. It is the core directory for the main engine to load and manage format conversion plug-ins. The plug-in directory contains multiple Jar files, each of which is an independent module. Plug-in projects are independent of each other, and the code is effectively isolated, which greatly facilitates project and version management. Different plug-ins are independent of each other and do not interfere with each other during development, maintenance and upgrade, making it easier to perform separate version control and management of each plug-in, effectively reducing the complexity of project management.

[0028] It can realize the parallel development of business needs of multiple industries. The business needs of different industries can be developed by different development teams or personnel at the same time. The corresponding plug-ins will not affect each other, which will help speed up the overall development progress of the project, meet the diverse needs of customers more quickly, and significantly improve the efficiency of project delivery. The plug-in mode is designed to meet the personalized data conversion needs of different customers and can be developed by different junior R&D personnel. By working in conjunction with JDDM (Kafka data master engine), it can effectively meet the complex and changing business needs of customers in different industries.

[0029] The plug-in mode effectively lowers the R&D threshold. Ordinary R&D personnel only need to focus on the specific business logic implementation of data conversion without having to delve into the complex architecture and operation mechanism of the entire Kafka data engine. This allows more ordinary R&D personnel to participate in project development, greatly expanding the scope of available R&D resources.

[0030] When the main engine is started, all plug-in jar files in the plug-in directory are read and loaded into the Java virtual machine (JVM); each plug-in jar file corresponds to a conversion plug-in, and the plug-in jar file includes data format conversion logic, business processing methods and plug-in functions.

[0031] It can be seen that in the main engine startup process, the clever method of dynamically loading the jar package can be seamlessly integrated into the business processing module of the JDDM data engine, minimizing the interference with the original architecture of the main engine.

[0032] In this embodiment, the main engine uses a DDL parser, a DML parser, and an SQL change parser to parse the original data sent by the client to obtain DDL data, DML data, and SQL data. With DML data, DDL data, and SQL data as data, the operation statement framework is encapsulated as an operation data object, the data dictionary is encapsulated as a dictionary data object, and the business data is encapsulated as a business data object.

[0033] Determine the table name and field name in the operation statement, and determine the related data dictionary. The data dictionary is a set of system tables in the database management system (DBMS) that is used to store the metadata of the database. Convert the data dictionary into a dictionary data object, which includes information such as the converted table structure and attributes.

[0034] An operation statement framework usually refers to a set of predefined logics used to generate and execute operation statements (such as SQL queries, API requests, etc.). It simplifies the development process and improves the readability and maintainability of the code by encapsulating common operations (such as database queries, API calls, data processing, etc.) into reusable components. The core goal of the operation statement framework is to reduce duplicate code, improve development efficiency, and support dynamic generation and execution of operations. Business data is the actual data stored by the user in the database, which is used to support the operation of business logic and applications.

[0035] In this embodiment, the main engine judges the format of the original data according to the pre-set format recognition rules, determines whether format conversion is required and the specific target data format, and indicates the direction for data processing.

[0036] Specifically, first identify whether the original data is valid and ensure that the original data complies with predefined rules, such as format, protocol, business logic, etc., to avoid processing errors or malicious data and ensure system stability; if the original data is valid, determine the valid data type, such as structured data (JSON / XML), binary protocol, free text, etc., to provide a decision basis for subsequent format conversion; then determine whether the original data needs to be format converted based on the valid data type and target format information. If the compared data type is incompatible with the target format, the format conversion is triggered.

[0037] If the original data is the DDL information of the database table structure, and the target customer requires the data format to be AVRO for data storage, the main engine will accurately determine the need for format conversion and pass key information such as the target data format (AVRO) to the format conversion plug-in. Through the four-layer logic of syntax feature recognition, metadata comparison, protocol matching, and intelligent inference, it ensures that the format conversion process is accurately triggered when the original data is DDL information and the target format is Avro.

[0038] After the target conversion plug-in obtains the data object (JAVA object) passed by the main engine, it first uses lexical analysis and syntax analysis technology to conduct in-depth analysis of the data. For data containing DDL information, it can accurately parse out key contents such as table structure and field definition; for DML data, it can accurately parse out the operation type (insert, modify, delete) and specific data values, and disassemble the data into basic elements for subsequent processing.

[0039] Based on the target data format (such as AVRO, JSON, ProtocolBuffer, TEXT, Bytes, etc.) and the special needs of customers, the plug-in carefully formulates corresponding data mapping rules. For example, the column names in the database table are cleverly mapped to the key names in JSON, and the database data types are accurately converted to the data types specified by AVRO. These mapping rules can be defined in a variety of ways such as flexible configuration files, concise annotations or built-in codes, ensuring that data can be accurately converted to the target format from the source.

[0040] When converted to JSON format, the data is assembled into key-value pairs in order according to the mapping rules to construct a standard string that conforms to the JSON syntax specification; when converted to AVRO format, AVRO's serialization tools are fully utilized and the data is serialized strictly according to AVRO's model definition.

[0041] For digital dictionaries, call the public List of the plugin project <object>dataDictionaryConversionByUserCustom(TableInfoVo tableInfoVo,List <tabblecolumnvo>nowColList,List <tablecolumnvo>cacheColList){…}; method performs relevant business processing of the data dictionary according to the conversion rules of the data dictionary.

[0042] For SQL statements, call the public List in the plugin project <object>The sqlStringConversionByUserCustom(String schemaName,String tableName,StringsqlString){…}; method performs related business processing of SQL statements according to the conversion rules of SQL statements.

[0043] For DML data (Insert / Update / Delete), call the public List in the plug-in project <object>The dmlDataConversionByUserCustom(PackageRetrunVo packageRetrunVo){…}; method performs relevant business processing of DML data according to the conversion rules of DML data.

[0044] After obtaining DML data, data dictionary, and SQL data, the format conversion plug-in performs format conversion according to customer-defined rules and sends the converted DML data, data dictionary, and SQL data to the list. <object>The data type is returned to the main engine.

[0045] This embodiment splits the data block into an operation statement framework, a data dictionary and business data, and performs format conversion respectively, which makes the conversion more flexible. In addition, the operation statement framework, data dictionary and business data are converted into data object forms, and parameters are transmitted in the form of data objects, which are packaged as a whole to facilitate data synchronization.

[0046] Preferably, after the target conversion plug-in converts the format of the original data, it will verify the target data to ensure the integrity and accuracy of the data. For example, carefully check whether the JSON data complies with the JSON syntax rules and whether the AVRO data perfectly matches the AVRO mode. If the verification is passed, the target conversion plug-in will send the target data to the data sending module of the main engine and send the target data to the target recipient; if the verification fails, the target conversion plug-in will generate an error report information and feedback to the main engine. The main engine generates a corresponding processing plan based on the error report information, for example, the main engine resends the data, notifies the administrator and other countermeasures to ensure the accuracy and reliability of data processing.

[0047] After the format conversion is completed, the main engine decides what kind of business processing to perform based on the business processing scope of the plug-in mode. Specifically, if the lazy loading mode contains a business sending module, the main engine waits for the data conversion and sending of the lazy loading module to be completed before processing the next batch of data; if the lazy loading mode does not contain a business sending module, the main engine receives the standard data after data conversion and sends the converted standard data through the business sending module.

[0048] Based on the same inventive concept, an embodiment of the present invention provides a data format conversion device based on dynamic lazy loading and applied to a main engine. Figure 2 FIG. 2 is a structural block diagram of a data format conversion device 200 based on dynamic lazy loading provided by an embodiment of the present invention. Figure 2 As shown, the data format conversion device 200 based on dynamic lazy loading mainly includes: An acquisition module 201 is used to acquire original data and target format information in response to a data transmission request from a client; The object generation module 202 is used to parse the original data and generate data objects; A search and loading module 203 is used to search for a target conversion plug-in that matches the target format information if format conversion is required for the original data, and load the searched target conversion plug-in into the memory; The plug-in calling module 204 is used to call the plug-in method through reflection after the target conversion plug-in is loaded, so that the target conversion plug-in obtains the data object, maps and serializes the data object, and obtains the target data to be transmitted.

[0049] In some optional embodiments, the data format conversion device 200 based on dynamic lazy loading further includes: The identification module (not shown in the figure) is connected to the acquisition module 201 and the search and load module 203 respectively, and is used to identify whether the original data is valid; if the original data is valid, determine the valid data type; based on the valid data type and the target format information, determine whether the format conversion of the original data is required.

[0050] In some optional embodiments, the object generation module 202 is specifically used to parse the original data, obtain DML data, DDL data and SQL data; use DML data, DDL data and SQL data as data, encapsulate the operation statement framework into an operation data object, encapsulate the data dictionary into a dictionary data object, and encapsulate the business data into a business data object.

[0051] In some optional embodiments, the data format conversion device 200 based on dynamic lazy loading further includes: The reading and loading module (not shown in the figure) is used to read all plug-in jar files in the plug-in directory when the main engine is started, and load the plug-in jar files into the Java virtual machine; each plug-in jar file corresponds to a conversion plug-in, and the plug-in jar file includes data format conversion logic, business processing methods and plug-in functions.

[0052] In some optional embodiments, the data format conversion device 200 based on dynamic lazy loading further includes: The plug-in adding module (not shown in the figure) is used to obtain the new plug-in project created by the developer in the integrated development environment when a new customer data conversion customization requirement is added, and to package and deploy the plug-in project to the plug-in directory.

[0053] In some optional embodiments, the data format conversion device 200 based on dynamic lazy loading further includes: An initialization module (not shown in the figure) is used to assign unique identification information to each format conversion plug-in when the plug-in library is initialized, and to establish a mapping table between the unique identification information and the target format information; The search loading module 203 is specifically used to search the mapping table for unique identification information that has a mapping relationship with the target format information; and use the format conversion plug-in corresponding to the unique identification information as the target conversion plug-in.

[0054] In some optional embodiments, the data format conversion device 200 based on dynamic lazy loading further includes: The verification receiving module (not shown in the figure) is used to receive the target data sent by the target conversion plug-in after the target data verification is passed, and send the target data to the target recipient; or, receive the error report information sent by the target conversion plug-in after the target data verification fails, and generate a processing plan based on the error report information.

[0055] The functional modules in the embodiments of the present invention can be integrated together to form an independent unit, for example, integrated in a processing unit, or each module can exist physically separately, or two or more modules can be integrated to form an independent unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the function is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions for an electronic device (which can be a personal computer, a server or a network device, etc.) or a processor (processor) to perform all or part of the steps of the methods of each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk.

[0056] The various variations and specific examples in the method provided in the embodiment of the present invention are also applicable to the data format conversion device based on dynamic lazy loading provided in this embodiment. Through the above detailed description of the data format conversion method based on dynamic lazy loading, those skilled in the art can clearly know the implementation method of the data format conversion device based on dynamic lazy loading in this embodiment. For the sake of brevity of the specification, it will not be described in detail here.

[0057] Figure 3 FIG. 3 is a structural block diagram of an electronic device 300 provided in an embodiment of the present invention. Figure 3 As shown, the electronic device 300 includes a memory 301 , a processor 302 , and a communication bus 303 ; the memory 301 and the processor 302 are connected via the communication bus 303 .

[0058] The memory 301 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 301 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, and instructions for implementing the data format conversion method based on dynamic lazy loading provided in the above embodiment, etc.; the data storage area may store data involved in the data format conversion method based on dynamic lazy loading provided in the above embodiment, etc.

[0059] The processor 302 may include one or more processing cores. The processor 302 executes various functions and processes data of the present application by running or executing instructions, programs, code sets or instruction sets stored in the memory 301, calling the data stored in the memory 301. The processor 302 may be at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller and a microprocessor. It is understandable that for different devices, the electronic device used to implement the above-mentioned processor 302 function may also be other, and the embodiment of the present invention is not specifically limited.

[0060] The communication bus 303 may include a path to transmit information between the above components. The communication bus 303 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The communication bus 303 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one double arrow is used in the diagram, but it does not mean that there is only one bus or one type of bus. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0061] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program that can be loaded by a processor and executes the data format conversion method based on dynamic lazy loading as provided in the above embodiment.

[0062] In this embodiment, the computer-readable storage medium may be a tangible device that holds and stores instructions used by the instruction execution device. The computer-readable storage medium may be, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination thereof. Specifically, the computer-readable storage medium may be a portable computer disk, a hard disk, a USB flash drive, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a podium random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, an optical disk, a magnetic disk, a mechanical encoding device, and any combination thereof.

[0063] The computer program in this embodiment includes a computer program for executing Figure 1 The program code of the method shown in the embodiment may include instructions corresponding to the steps of the method provided in the above embodiment. The computer program may be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded to an external computer or external storage device via a network (such as the Internet, a local area network, a wide area network and / or a wireless network). The computer program may be executed entirely on the user's computer or as an independent software package.

[0064] In the embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0065] The terms "comprises," "comprising," or any other variations thereof are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not expressly listed, or also includes elements inherent to such process, method, article, or apparatus.

[0066] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.< / object> < / object> < / object> < / tablecolumnvo> < / tabblecolumnvo> < / object>

Claims

1. A data format conversion method based on dynamic lazy loading, characterized in that: Applied to the main engine, the method comprises: Responding to a data transmission request from a client, obtaining original data and target format information; Parsing the original data to generate a data object; If the format of the original data needs to be converted, a format conversion plug-in matching the target format information is searched in the plug-in library as a target conversion plug-in, and the target conversion plug-in is loaded into the memory; After the target conversion plug-in is loaded, the plug-in method is called through reflection to enable the target conversion plug-in to obtain the data object, map and serialize the data object, and obtain the target data to be transmitted.

2. The data format conversion method based on dynamic lazy loading according to claim 1, characterized in that: The method further comprises: Identify whether the original data is valid; If the original data is valid, determining a valid data type; It is determined whether the format of the original data needs to be converted based on the valid data type and the target format information.

3. The data format conversion method based on dynamic lazy loading according to claim 1, characterized in that: The parsing of the original data to generate a data object comprises: Parse the original data to obtain DML data, DDL data and SQL data; Taking the DML data, the DDL data and the SQL data as data, the operation statement framework is encapsulated into an operation data object, the data dictionary is encapsulated into a dictionary data object, and the business data is encapsulated into a business data object.

4. The data format conversion method based on dynamic lazy loading according to claim 1, characterized in that: The method further comprises: When the main engine is started, all plug-in jar files in the plug-in directory are read and loaded into the Java virtual machine; each plug-in jar file corresponds to a conversion plug-in, and the plug-in jar file includes data format conversion logic, business processing methods and plug-in functions.

5. The data format conversion method based on dynamic lazy loading as claimed in claim 4 is characterized in that: The method further comprises: When a new customer data conversion customization requirement is added, a new plug-in project created by a developer in an integrated development environment is obtained, and the plug-in project is packaged and deployed to the plug-in directory.

6. The data format conversion method based on dynamic lazy loading according to claim 4 or 5, characterized in that: The method further comprises: When the plug-in library is initialized, unique identification information is assigned to each format conversion plug-in, and a mapping table between the unique identification information and the target format information is established; The step of searching the plug-in library for a format conversion plug-in that matches the target format information as the target conversion plug-in includes: Searching the mapping table for the unique identification information that has a mapping relationship with the target format information; The format conversion plug-in corresponding to the unique identification information is used as the target conversion plug-in.

7. The data format conversion method based on dynamic lazy loading according to any one of claims 1 to 5, characterized in that: The method further comprises: receiving the target data sent by the target conversion plug-in after verifying the target data, and sending the target data to a target recipient; or, Receive error report information sent by the target conversion plug-in after the target data verification fails, and generate a processing solution based on the error report information.

8. A data format conversion device based on dynamic lazy loading, characterized in that: Applied to the main engine, the device comprises: An acquisition module, used to respond to a data transmission request from a client and acquire original data and target format information; An object generation module, used for parsing the original data and generating a data object; A search loading module is used to search for a target conversion plug-in matching the target format information if the format conversion of the original data needs to be performed, and load the searched target conversion plug-in into the memory; The plug-in calling module is used to call the plug-in method through reflection after the target conversion plug-in is loaded, so that the target conversion plug-in obtains the data object, maps and serializes the data object, and obtains the target data to be transmitted.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the data format conversion method based on dynamic lazy loading as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data format conversion method based on dynamic lazy loading as described in any one of claims 1 to 7 is implemented.

Images