Method and device for transforming semi-structured data, electronic equipment and storage medium

Abstract

This application provides a method, apparatus, electronic device, and storage medium for converting semi-structured data. The method includes: acquiring semi-structured data; parsing fields of objects in the semi-structured data to obtain a base table; obtaining a derived table based on the base table; constructing a standard template document based on the base table and the derived table; and inputting the semi-structured data into the standard template document for conversion to obtain structured data. Implementing this application embodiment enables rapid conversion of semi-structured data, allowing it to realize its shared value, reducing the work costs of repeated communication and rereading development during data conversion, and greatly improving the conversion efficiency of semi-structured data.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a method, apparatus, electronic device, and storage medium for converting semi-structured data. Background Technology

[0002] The financial industry requires a lot of semi-structured data, and semi-structured data (such as JSON) is a lightweight data exchange format. It uses a language-independent text format to store and represent data. Its concise and clear hierarchical structure makes JSON an ideal data exchange language, easy for humans to read and write.

[0003] Traditional data warehouses typically store semi-structured data as fields in structured tables after data collection from business systems. However, its unique data representation gives it a predefined structural description similar to structured data. Semi-structured data can be stored as structured data through a structured parsing process, making it convenient for data analysts to process the data using SQL.

[0004] However, as the financial industry involves increasingly more big data platform processing, the industry typically places the semi-structured data parsing process at the end of the data processing chain. This is done by analysts as needed or by data marts when processing business data, which prevents semi-structured data from realizing its shared value as basic data. Furthermore, there is no unified requirement, design, development, and delivery documentation for the structure description and processing of semi-structured data, which increases the workload of repeated communication and development, resulting in low efficiency. Summary of the Invention

[0005] The purpose of this application is to provide a method, apparatus, electronic device, and storage medium for converting semi-structured data, which can realize the rapid conversion of semi-structured data, enable semi-structured data to play its shared value, reduce the work costs of repeated communication and rereading development in the data conversion process, and greatly improve the conversion efficiency of semi-structured data.

[0006] In a first aspect, embodiments of this application provide a method for converting semi-structured data, the method comprising:

[0007] Acquire semi-structured data;

[0008] The objects in the semi-structured data are parsed to obtain the base table;

[0009] The derived table is obtained from the base table;

[0010] A standard template document is constructed based on the base table and the derived table;

[0011] The semi-structured data is input into the standard template document for conversion to obtain structured data.

[0012] In the above implementation process, the fields of semi-structured data objects are parsed to obtain a base table, and then a derived table is obtained based on the base table. Then, a standard template document is constructed based on the base table and the derived table. This can realize the rapid transformation of semi-structured data, enabling semi-structured data to play its shared value, reducing the work costs of repeated communication and rereading development in the data transformation process, and greatly improving the transformation efficiency of semi-structured data.

[0013] Furthermore, the step of parsing the fields of the objects in the semi-structured data to obtain the base table includes:

[0014] The objects in the semi-structured data are parsed to obtain the parsed fields;

[0015] The parsed fields are added to the initial base table for horizontal expansion to obtain the base table.

[0016] In the above implementation process, the fields of objects in semi-structured data are parsed, and the parsed fields are then added to the initial base table for horizontal expansion, making the base table more complete and perfect.

[0017] Further, the step of obtaining the derived table based on the base table includes:

[0018] Obtain the initial derived table;

[0019] The objects in the initial derived table are decomposed to obtain multiple initial structured data;

[0020] The target field in the base table is added to the initial derived table based on the multiple initial structured data to obtain the derived table.

[0021] In the above implementation process, the target fields in the base table are added based on the multiple initial structured data obtained after decomposition, so that the derived tables can correspond one-to-many with each other, making the data structure in the derived tables clearer.

[0022] Further, the step of constructing a standard template document based on the base table and the derived table includes:

[0023] Construct a template design document based on the base table and the derived table;

[0024] The standard template document is obtained based on the template design document.

[0025] In the above implementation process, a template design document is built based on the base table and derived tables, which can adapt to the transformation needs of various structured data without the need to adjust the standard template document one by one.

[0026] Further, the step of obtaining the standard template document based on the template design document includes:

[0027] Obtain conversion needs;

[0028] Based on the transformation requirements, the template design document is structurally transformed to obtain standardized data;

[0029] The standard template document is obtained based on the standardized data.

[0030] In the above implementation process, the template design document is structurally transformed according to the transformation requirements, so that the standard template document can transform semi-structured data under various transformation requirements without repeatedly building the standard template document, which can improve efficiency.

[0031] Furthermore, the step of structurally transforming the template design document according to the transformation requirements to obtain standardized data includes:

[0032] Based on the first conversion requirement in the conversion requirements, the template design document is converted into code to generate code data;

[0033] Based on the second conversion requirement in the conversion requirements, the template design document is imported and converted to generate an import document;

[0034] The third transformation requirement in the transformation requirements is used to configure and transform the template design document to generate a scheduling job deployment package;

[0035] The code data, the imported documents, and the scheduled job deployment package are used as the standardized data.

[0036] In the above implementation process, the template design document is converted into code, document import, and configuration according to different conversion needs, so that the obtained standardized data can be mapped to multiple data conversion functions, effectively improving the practicality of standardized data.

[0037] Furthermore, after the step of inputting the semi-structured data into the standard template document for conversion to obtain structured data, the method further includes:

[0038] The structured data is then visualized.

[0039] During the above implementation process, structured data can be displayed one by one, making it easier to identify problems and make changes in a timely manner.

[0040] Secondly, embodiments of this application also provide a semi-structured data conversion apparatus, the apparatus comprising:

[0041] The acquisition module is used to acquire semi-structured data;

[0042] The parsing module is used to parse the fields of objects in the semi-structured data to obtain the base table;

[0043] The data acquisition module is used to obtain the derived table based on the base table;

[0044] The building module is used to build a standard template document based on the base table and the derived table;

[0045] The conversion module is used to input the semi-structured data into the standard template document for conversion to obtain structured data.

[0046] In the above implementation process, the fields of semi-structured data objects are parsed to obtain a base table, and then a derived table is obtained based on the base table. Then, a standard template document is constructed based on the base table and the derived table. This can realize the rapid transformation of semi-structured data, enabling semi-structured data to play its shared value, reducing the work costs of repeated communication and rereading development in the data transformation process, and greatly improving the transformation efficiency of semi-structured data.

[0047] Thirdly, an electronic device provided in this application includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described in any of the first aspects.

[0048] Fourthly, embodiments of this application provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method described in any of the first aspects.

[0049] Fifthly, embodiments of this application provide a computer program product that, when run on a computer, causes the computer to perform the method described in any of the first aspects.

[0050] Other features and advantages of this disclosure will be set forth in the following description, or some features and advantages may be inferred from the description or determined without doubt, or may be learned by practicing the techniques described above.

[0051] It can be implemented in accordance with the contents of the specification. The preferred embodiments of this application are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0052] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation on the range. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 A flowchart illustrating the method for converting semi-structured data provided in this application embodiment;

[0054] Figure 2 A schematic diagram of the structural composition of the semi-structured data conversion device provided in the embodiments of this application;

[0055] Figure 3 A schematic diagram of the structural composition of a semi-structured data conversion device provided in an embodiment of this application;

[0056] Figure 4 This is a schematic diagram of the structural composition of the electronic device provided in the embodiments of this application. Detailed Implementation

[0057] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0058] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0059] The specific embodiments of this application will be described in further detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate this application, but are not intended to limit the scope of this application.

[0060] Example 1

[0061] Figure 1 This is a flowchart illustrating the semi-structured data conversion method provided in this application embodiment, such as... Figure 1 As shown, the method includes:

[0062] S1, Obtain semi-structured data;

[0063] S2, parsing the fields of objects in semi-structured data to obtain the base table;

[0064] S3, Obtain the derived table based on the base table;

[0065] S4, build a standard template document based on the base table and derived tables;

[0066] S5 converts semi-structured data into structured data by inputting it into a standard template document.

[0067] In the above implementation process, the fields of semi-structured data objects are parsed to obtain a base table, and then a derived table is obtained based on the base table. Then, a standard template document is constructed based on the base table and the derived table. This can realize the rapid transformation of semi-structured data, enabling semi-structured data to play its shared value, reducing the work costs of repeated communication and rereading development in the data transformation process, and greatly improving the transformation efficiency of semi-structured data.

[0068] Semi-structured data is widely used in the data storage of business systems in the financial industry due to its flexible data representation, universal data exchange format, and ability to adapt to the database changes brought about by the continuous iteration of business systems.

[0069] To reduce the development cost and improve the reusability of semi-structured data, this application designs a semi-structured data processing method based on an offline data warehouse storage layer. Unlike conventional R&D processes, this application's standard template document integrates requirements specifications, logical model design, and physical model design into a single document. All relevant parties use the same template as a communication basis, forming a closed-loop process. This reduces manual maintenance, enhances information consistency, reduces document redundancy, improves the efficiency of semi-structured data development, and lowers the cost of secondary R&D communication due to later requirement changes.

[0070] In S1,

[0071] In semi-structured JSON data, there are two main data formats: JSON arrays and JSON objects. The cross-nesting of these two formats complicates the structured decomposition process.

[0072] Furthermore, S2 includes:

[0073] Parse the fields of objects in semi-structured data to obtain the parsed fields;

[0074] The parsed fields are added to the initial base table for horizontal expansion, resulting in the base table.

[0075] In the above implementation process, the fields of objects in semi-structured data are parsed, and the parsed fields are then added to the initial base table for horizontal expansion, making the base table more complete and perfect.

[0076] If semi-structured data does not contain nested array data structures, then the objects in the semi-structured data can be parsed at the field level, and the parsed fields can be added to the base table for horizontal expansion at the field level.

[0077] Furthermore, S3 includes:

[0078] Obtain the initial derived table;

[0079] The objects in the initial derived table are decomposed to obtain multiple initial structured data;

[0080] The target field in the base table is added to the initial derived table based on multiple initial structured data to obtain the derived table.

[0081] In the above implementation process, the target fields in the base table are added based on the multiple initial structured data obtained after decomposition, so that the derived tables can correspond one-to-many with each other, making the data structure in the derived tables clearer.

[0082] Semi-structured data contains nested array data structures. Due to the specific nature of individual objects within arrays, a new model table, or derived table, needs to be derived. Each object is broken down into a row in the new table, allowing the original row of semi-structured data to correspond to multiple rows of structured data, forming a one-to-many model design. This avoids the exponential growth of data volume caused by horizontal expansion of fields in the base table. Some fields in the base table are moved down to the derived table as needed.

[0083] Furthermore, S4 includes:

[0084] Build a template design document based on the base table and derived tables;

[0085] Obtain the standard template document from the template design document.

[0086] In the above implementation process, a template design document is built based on the base table and derived tables, which can adapt to the transformation needs of various structured data without the need to adjust the standard template document one by one.

[0087] The template design document is mainly divided into several areas:

[0088] Data producer input area: Data table structure description, detailed information, survey feedback information.

[0089] Basic table structure description information: The table structure description information of the basic table where the fields of the semi-structured data are stored.

[0090] Semi-structured data index description: A list of named nodes containing the semi-structured data to be parsed, along with sample data.

[0091] Detailed field descriptions for semi-structured data: A list of fields in the node containing the semi-structured data to be parsed, along with detailed descriptions of each field.

[0092] Data warehouse development and design area: Data warehouse developers conduct standardized naming, model theme classification design, field code value standardization, and other data warehouse mapping development based on survey feedback results.

[0093] Furthermore, the steps to obtain a standard template document from the template design document include:

[0094] Obtain conversion needs;

[0095] Based on the transformation requirements, the template design document is structurally transformed to obtain standardized data;

[0096] Standard template documents are obtained based on standardized data.

[0097] In the above implementation process, the template design document is structurally transformed according to the transformation requirements, so that the standard template document can transform semi-structured data under various transformation requirements without repeatedly building the standard template document, which can improve efficiency.

[0098] This application's standard template document defines a unified communication method for data producers, business users, and data developers, avoiding misunderstandings caused by the complexity of semi-structured data and delivering standardized documents. By adding automated operations to the template tool, the document maintenance process is standardized, reducing document formatting issues caused by human error and providing a more user-friendly experience. The Excel spreadsheet includes built-in macro editing code, enabling quick and flexible implementation of data processing functions and better adapting to template changes and iterations. Each step in the data development process is more clearly designed. The template tool combines the database logical and physical model design in the semi-structured data parsing process, making the transformation from semi-structured to structured data faster and more convenient, depending on the complexity of the semi-structured data structure.

[0099] By standardizing data, files that can be quickly generated for import into other systems can be seamlessly integrated during subsequent data development, reducing manual operations, lowering the error rate caused by manual operations, and improving development efficiency.

[0100] Furthermore, the steps of structurally transforming the template design document according to the transformation requirements to obtain standardized data include:

[0101] Based on the first conversion requirement in the conversion needs, the template design document is converted into code to generate code data;

[0102] Based on the second conversion requirement in the conversion needs, the template design document is imported and converted to generate an import document;

[0103] The third transformation requirement in the transformation requirements will be configured and transformed from the template design document to generate a scheduling job deployment package;

[0104] Code data, import documentation, and scheduled job deployment packages are treated as standardized data.

[0105] In the above implementation process, the template design document is converted into code, document import, and configuration according to different conversion needs, so that the obtained standardized data can be mapped to multiple data conversion functions, effectively improving the practicality of standardized data.

[0106] This application utilizes a template document language to unify the R&D process, including requirements, design, development, and implementation, within the data transformation process. This avoids information redundancy, maintenance difficulties, and misunderstandings among stakeholders caused by excessive documentation. VBA programming is used in the documents to achieve standardized naming, formatted code generation, and export of scheduling platform job configuration files, reducing manual operations and allowing the design process to focus more on content. Furthermore, the application enables more flexible expansion of structured data model tables during the structuring process of semi-structured data. It allows semi-structured data to be converted to structured data and stored in a unified data warehouse layer, placing the pre-processing of semi-structured data upstream in the data chain. This reduces the cost of subsequent redundant data processing and improves the reusability of data processing results.

[0107] Furthermore, after converting semi-structured data into structured data by inputting it into a standard template document, the process also includes:

[0108] Visualize structured data.

[0109] During the above implementation process, structured data can be displayed one by one, making it easier to identify problems and make changes in a timely manner.

[0110] Example 2

[0111] To execute the method corresponding to Embodiment 1 above and achieve the corresponding functions and technical effects, a semi-structured data conversion device is provided below, such as... Figure 2 As shown, the device includes:

[0112] Module 1 is used to acquire semi-structured data;

[0113] Parsing module 2 is used to parse the fields of objects in semi-structured data to obtain the base table;

[0114] Data acquisition module 3 is used to obtain derived tables based on base tables;

[0115] Module 4 is used to build a standard template document based on the base table and the derived table;

[0116] Transformation module 5 is used to transform semi-structured data into structured data by inputting it into a standard template document.

[0117] In the above implementation process, the fields of semi-structured data objects are parsed to obtain a base table, and then a derived table is obtained based on the base table. Then, a standard template document is constructed based on the base table and the derived table. This can realize the rapid transformation of semi-structured data, enabling semi-structured data to play its shared value, reducing the work costs of repeated communication and rereading development in the data transformation process, and greatly improving the transformation efficiency of semi-structured data.

[0118] Furthermore, parsing module 2 is also used for:

[0119] Parse the fields of objects in semi-structured data to obtain the parsed fields;

[0120] The parsed fields are added to the initial base table for horizontal expansion, resulting in the base table.

[0121] In the above implementation process, the fields of objects in semi-structured data are parsed, and the parsed fields are then added to the initial base table for horizontal expansion, making the base table more complete and perfect.

[0122] Furthermore, the data acquisition module 3 is also used for:

[0123] Obtain the initial derived table;

[0124] The objects in the initial derived table are decomposed to obtain multiple initial structured data;

[0125] The target field in the base table is added to the initial derived table based on multiple initial structured data to obtain the derived table.

[0126] In the above implementation process, the target fields in the base table are added based on the multiple initial structured data obtained after decomposition, so that the derived tables can correspond one-to-many with each other, making the data structure in the derived tables clearer.

[0127] Furthermore, Module 4 is also used for:

[0128] Build a template design document based on the base table and derived tables;

[0129] Obtain the standard template document from the template design document.

[0130] In the above implementation process, a template design document is built based on the base table and derived tables, which can adapt to the transformation needs of various structured data without the need to adjust the standard template document one by one.

[0131] Furthermore, Module 4 is also used for:

[0132] Obtain conversion needs;

[0133] Based on the transformation requirements, the template design document is structurally transformed to obtain standardized data;

[0134] Standard template documents are obtained based on standardized data.

[0135] In the above implementation process, the template design document is structurally transformed according to the transformation requirements, so that the standard template document can transform semi-structured data under various transformation requirements without repeatedly building the standard template document, which can improve efficiency.

[0136] Furthermore, Module 4 is also used for:

[0137] Based on the first conversion requirement in the conversion needs, the template design document is converted into code to generate code data;

[0138] Based on the second conversion requirement in the conversion needs, the template design document is imported and converted to generate an import document;

[0139] The third transformation requirement in the transformation requirements will be configured and transformed from the template design document to generate a scheduling job deployment package;

[0140] Code data, import documentation, and scheduled job deployment packages are treated as standardized data.

[0141] In the above implementation process, the template design document is converted into code, document import, and configuration according to different conversion needs, so that the obtained standardized data can be mapped to multiple data conversion functions, effectively improving the practicality of standardized data.

[0142] Furthermore, such as Figure 3 As shown, the device also includes a visualization module 6, used for:

[0143] Visualize structured data.

[0144] During the above implementation process, structured data can be displayed one by one, making it easier to identify problems and make changes in a timely manner.

[0145] The semi-structured data conversion apparatus described above can implement the method of Embodiment 1. The options in Embodiment 1 also apply to this embodiment, and will not be described in detail here.

[0146] The remaining contents of this embodiment can be referred to the contents of Embodiment 1 above, and will not be repeated in this embodiment.

[0147] Example 3

[0148] This application provides an electronic device, including a memory and a processor. The memory stores a computer program, and the processor runs the computer program to enable the electronic device to perform the semi-structured data conversion method of Embodiment 1.

[0149] Alternatively, the aforementioned electronic device may be a server.

[0150] Please see Figure 4 , Figure 4 This is a schematic diagram illustrating the structural composition of an electronic device provided in an embodiment of this application. The electronic device may include a processor 41, a communication interface 42, a memory 43, and at least one communication bus 44. The communication bus 44 is used to enable direct communication between these components. In this embodiment, the communication interface 42 is used for signaling or data communication with other node devices. The processor 41 may be an integrated circuit chip with signal processing capabilities.

[0151] The processor 41 described above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor, or the processor 41 can be any conventional processor.

[0152] The memory 43 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc. The memory 43 stores computer-readable instructions. When these computer-readable instructions are executed by the processor 41, the device can perform the aforementioned operations. Figure 1 The various steps involved in the method implementation examples.

[0153] Optionally, the electronic device may also include a storage controller and an input / output unit. The memory 43, storage controller, processor 41, peripheral interface, and input / output unit are electrically connected directly or indirectly to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other via a communication bus 44. The processor 41 is used to execute executable modules stored in the memory 43, such as software function modules or computer programs included in the device.

[0154] Input / output units are used to enable users to create tasks and set optional start periods or preset execution times for those tasks, facilitating user-server interaction. Input / output units can be, but are not limited to, a mouse and keyboard.

[0155] Understandable. Figure 4 The structure shown is for illustrative purposes only; the electronic device may also include components that are more advanced than those shown. Figure 4 The more or fewer components shown, or having the same Figure 4 The different configurations shown. Figure 4 The components shown can be implemented using hardware, software, or a combination thereof.

[0156] In addition, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the semi-structured data conversion method of Embodiment 1.

[0157] This application also provides a computer program product that, when run on a computer, causes the computer to perform the method described in the method embodiment.

[0158] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using dedicated hardware-based apparatus that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0159] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0160] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, external hard drives, ROM, RAM, magnetic disks, or optical disks.

[0161] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0162] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of protection of the claims.

[0163] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A method for converting semi-structured data, characterized in that, The method includes: Acquire semi-structured data; The objects in the semi-structured data are parsed to obtain the base table; The derived table is obtained from the base table; A standard template document is constructed based on the base table and the derived table; The semi-structured data is input into the standard template document for conversion to obtain structured data; The step of constructing a standard template document based on the base table and the derived table includes: Construct a template design document based on the base table and the derived table; Obtain the standard template document based on the template design document; The step of obtaining the standard template document based on the template design document includes: Obtain conversion needs; Based on the transformation requirements, the template design document is structurally transformed to obtain standardized data; The standard template document is obtained based on the standardized data; The step of structurally transforming the template design document according to the transformation requirements to obtain standardized data includes: Based on the first conversion requirement in the conversion requirements, the template design document is converted into code to generate code data; Based on the second conversion requirement in the conversion requirements, the template design document is imported and converted to generate an imported document; The third transformation requirement in the transformation requirements is used to configure and transform the template design document to generate a scheduling job deployment package; The code data, the imported document, and the scheduled job deployment package are used as the standardized data; The template design document contains multiple areas, specifically: the data producer's input area, basic table structure description information, semi-structured data index description, detailed field description of semi-structured data, and data warehouse development design area. It combines the database logical model and physical model design in the semi-structured data parsing and processing process according to the different levels of complexity of the semi-structured data structure.

2. The method for converting semi-structured data according to claim 1, characterized in that, The step of parsing the fields of the objects in the semi-structured data to obtain the base table includes: The objects in the semi-structured data are parsed to obtain the parsed fields; The parsed fields are added to the initial base table for horizontal expansion to obtain the base table.

3. The method for converting semi-structured data according to claim 1, characterized in that, The step of obtaining the derived table based on the base table includes: Obtain the initial derived table; The objects in the initial derived table are decomposed to obtain multiple initial structured data; The target field in the base table is added to the initial derived table based on the multiple initial structured data to obtain the derived table.

4. The method for converting semi-structured data according to claim 1, characterized in that, After the step of inputting the semi-structured data into the standard template document for conversion to obtain structured data, the method further includes: The structured data is then visualized.

5. A device for converting semi-structured data, characterized in that, The device includes: The acquisition module is used to acquire semi-structured data; The parsing module is used to parse the fields of objects in the semi-structured data to obtain the base table; The data acquisition module is used to obtain the derived table based on the base table; The building module is used to build a standard template document based on the base table and the derived table; The conversion module is used to input the semi-structured data into the standard template document for conversion to obtain structured data; The building module is also used for: Construct a template design document based on the base table and the derived table; Obtain the standard template document based on the template design document; Obtain conversion needs; Based on the transformation requirements, the template design document is structurally transformed to obtain standardized data; The standard template document is obtained based on the standardized data; Based on the first conversion requirement in the conversion requirements, the template design document is converted into code to generate code data; Based on the second conversion requirement in the conversion requirements, the template design document is imported and converted to generate an imported document; The third transformation requirement in the transformation requirements is used to configure and transform the template design document to generate a scheduling job deployment package; The code data, the imported document, and the scheduled job deployment package are used as the standardized data; The template design document contains multiple areas, specifically: the data producer's input area, basic table structure description information, semi-structured data index description, detailed field description of semi-structured data, and data warehouse development design area. It combines the database logical model and physical model design in the semi-structured data parsing and processing process according to the different levels of complexity of the semi-structured data structure.

6. An electronic device, characterized in that, The device includes a memory and a processor, the memory being used to store a computer program, and the processor running the computer program to cause the electronic device to perform a method for converting semi-structured data according to any one of claims 1 to 4.

7. A computer storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method for converting semi-structured data as described in any one of claims 1 to 4.

Images