Data analysis method and device, electronic equipment and storage medium

Abstract

The application provides a data analysis method and device, electronic equipment and storage medium; the method comprises the following steps: for each obtained string, calling a preset first enumeration class to analyze all instructions in the string; calling a preset second enumeration class to analyze all data codes in the string; performing syntax analysis on the string to determine the execution logic between each instruction and each data code; determining the data source pointed to by the data code according to the field in the data code, and determining the data structure of the data source; and enabling each instruction to call the data code according to the execution logic, loading data according to the data structure pointed to by the data code, and obtaining target data. Therefore, the target data can be loaded across data sources and data types.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of data processing, and particularly relate to a data analysis method and device, electronic equipment and storage medium. BACKGROUND

[0002] In the actual data application process, when loading data, instructions based on a statement often need to load data from different data sources, and the data types between different data sources are often different.

[0003] However, in related data analysis technology, instructions based on a statement are often difficult to load across different data sources or different data types, that is, when analyzing the statement, it is often difficult to parse the corresponding data loading method for different data types and data sources, which makes it very difficult to load data across data sources and across data types.

[0004] Therefore, there is a need for a solution that can analyze data across different data sources and different data types. SUMMARY

[0005] Therefore, the purpose of the present application is to provide a data analysis method and device, electronic equipment and storage medium.

[0006] To achieve the above purpose, the present application provides a data analysis method, comprising:

[0007] For each obtained string, call the preset first enumeration class to parse all instructions in the string, and call the preset second enumeration class to parse all data codes in the string;

[0008] Syntax analysis is performed on the string to determine the execution logic between each instruction and each data code;

[0009] According to the fields in the data codes, determine the data source pointed to by the data code and the data structure of the data source;

[0010] According to the execution logic, the instructions call the data codes, and perform data loading according to the data structure pointed to by the data codes to obtain target data.

[0011] Further, the first enumeration class and the second enumeration class are determined by the following method:

[0012] Each of the plurality of instructions is defined as a first object in the first enumeration class, and each first object is defined;

[0013] The data types of the plurality of data sources are taken as second objects in the second enumeration class, and each second object is defined.

[0014] Further, by calling the preset first enumeration class, all instructions in the string are parsed, including:

[0015] According to the expression order of each character in the string, the characters representing the instructions are determined in sequence, and the definitions of the instructions in the first enumeration class are queried.

[0016] Further, by calling the preset second enumeration class, all data codes in the string are parsed, including:

[0017] According to the expression order of each character in the string, the data codes representing the data sources are determined in sequence, and the definitions of the data sources corresponding to the data codes in the second enumeration class are queried.

[0018] Further, according to the fields in the data codes, the data sources pointed to by the data codes are determined, and the data structures of the data sources are determined, including:

[0019] For each data code, a first field in the data code pointing to the type of the data source is determined, and a second field in the data code pointing to the name of the data source is determined.

[0020] According to the first field, the definition of the second object is queried to determine that the data structure of the data source is one of the structured data and the semi-structured data.

[0021] According to the second field, the name of the data source is determined by querying a preset mapping class.

[0022] Further, before the data codes are called according to the execution logic, the method further includes:

[0023] According to the definition of the first object, the all instructions are divided into command instructions, connection instructions and conditional instructions.

[0024] According to the expression order of the instructions and the data codes in the string, the instructions and the data codes in the string are constructed into a syntax tree.

[0025] Further, according to the data structure pointed to by the data code, data loading is performed to obtain target data, including:

[0026] In response to determining that the data source pointed to by the data code is structured data, corresponding target data is loaded from the corresponding data source.

[0027] In response to determining that the data source pointed to by the data code is semi-structured data, the corresponding target data is obtained by parsing the semi-structured data.

[0028] Further, the corresponding target data is obtained by parsing the semi-structured data, including:

[0029] According to the data type of the semi-structured data, a preset parser corresponding to the data type is called;

[0030] By using the corresponding parser to parse the script file of the semi-structured data, the target data pointed to by the second field is determined;

[0031] Based on the determined target data, the target data is loaded.

[0032] Based on the same inventive concept, the present application also provides a data parsing method and device, including a lexical analysis module, a syntax analysis module, a data structure determination module, and a data loading module;

[0033] The lexical analysis module is configured to, for each obtained string, parse all instructions in the string by calling a preset first enumeration class, and parse all data codes in the string by calling a preset second enumeration class;

[0034] The syntax analysis module is configured to perform syntax analysis on the string to determine the execution logic between each instruction and each data code;

[0035] The data structure determination module is configured to determine the data source pointed to by each data code according to the field in the data code, and determine the data structure of the data source;

[0036] The data loading module is configured to cause the instructions to call the data codes according to the execution logic, perform data loading according to the data structure pointed to by the data code, and obtain target data.

[0037] Based on the same inventive concept, the present application also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the data parsing method of any one of the above.

[0038] Based on the same inventive concept, the present application also provides a non-transitory computer readable storage medium, wherein the non-transitory computer readable storage medium stores computer instructions for causing the computer to execute the above data parsing method.

[0039] As can be seen from the above, the data analysis method, device, electronic equipment and storage medium provided by the application, based on the defined first enumeration class and the second enumeration class, respectively classifies the instructions and data sources that may appear in the string, so that in the analysis process of the string, the instruction-related characters and the data source-related characters can be respectively analyzed according to the first enumeration class and the second enumeration class. At the same time, the syntax analysis added in the method can effectively analyze the execution logic between the instructions and the data codes.

[0040] Further, after considering the structured data and the semi-structured data, the data code is used to determine the specific data type pointed to by the data code, and when the data code is called, different data sources and different data types can be distinguished to achieve the loading of the target data using different ways for different data types, so that when different data sources appear in a string, and each data source is of different data types, the target data can be loaded across data sources and across data types. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the application or related art, the following will briefly introduce the drawings needed to be used in the embodiments or related art descriptions. Obviously, the drawings in the following description are only embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0042] Figure 1 The flowchart of the data analysis method of the embodiment of the application;

[0043] Figure 2 The logical schematic diagram of the abstract syntax tree of the embodiment of the application;

[0044] Figure 3 The logical schematic diagram of the data loading of the embodiment of the application;

[0045] Figure 4 The structural schematic diagram of the data analysis device of the embodiment of the application;

[0046] Figure 5 The structural schematic diagram of the electronic equipment of the embodiment of the application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the application more clear, the following will further describe the application in combination with specific embodiments and with reference to the drawings.

[0048] Unless otherwise defined, technical terms or scientific terms used in the embodiments of the present application shall have the meanings that are commonly understood by those skilled in the art to which the present application belongs. The terms "first", "second", and similar terms used in the embodiments of the present application do not denote any order, quantity, or importance, but are used to distinguish different components. The terms "include", "contain", and similar terms mean that the elements or objects before the terms encompass the elements or objects listed after the terms and equivalents thereof, and do not exclude other elements or objects. The terms "connect" and "connected" and similar terms do not limit to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms "upper", "lower", "left", "right", and the like are used only to indicate relative positional relationships, and when the absolute positions of the described objects are changed, the relative positional relationships can also be changed accordingly.

[0049] As described in the background section, the related data parsing method is also difficult to meet the needs of actual production.

[0050] The applicant found in the process of implementing the present application that the main problem of the related data parsing method is that, in the actual data application process, when loading data, the required loaded data can be stored in different types of data structures in advance or in different databases, so that the data of different data sources needs to be loaded based on the instructions of one statement, and the data types of different data sources are often not the same.

[0051] However, in the related data parsing technology, it is often difficult to load data across different data sources or across different data types for one statement, that is, when parsing the statement, it is often difficult to simultaneously parse different data structures and load them according to the requirements of different data types and data sources, which makes it very difficult to load data across data sources and across data types.

[0052] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0053] In the embodiments of the present application, the SQL (Structured Query) data management system is taken as a specific example, the SQL (Structured Query) language is taken as the language environment for implementing the method, and the required loaded data is stored in different data sources in different data structures in advance.

[0054] In the present embodiment, the SQL statement for calling data can be input in the SQL data management system, and the SQL statement can be regarded as a string.

[0055] Further, for the data stored in different data sources in different data structures as described above, when it is required to load different data sources at the same time, in a SQL statement, the requirement of calling multiple different data sources can be included, for example, in the SQL statement, the requirement of calling data from MYSQL (relational database management system), POSTGRESQL (object-relational database management system) and CLICKHOUSE (a column-oriented storage database) is included, and at the same time, the XML (Extensible Markup Language file) configuration file, JSON (JAVA language object profile file) log file and unprocessed raw data and the like need to be called.

[0056] Similarly, in the same SQL statement for calling data, the calling requirement can also include: calling data obtained by a crawler and calling data with parameters.

[0057] Reference Figure 1 The data analysis method of one embodiment of the present application includes the following steps:

[0058] Step S101, for each obtained string, the first enumeration class is called to parse all instructions in the string, and the second enumeration class is called to parse all data codes in the string.

[0059] In the embodiments of the present application, based on the language environment in which the method is implemented, a first enumeration class for enumerating each instruction and a second enumeration class for enumerating each data source can be defined first.

[0060] Specifically, an enumeration class can be established, in which each instruction is taken as an object and each instruction is defined to enumerate each instruction; each instruction can be all instructions possessed by the computer language in which the method is implemented, or a plurality of instructions selected in advance.

[0061] In the present embodiment, each object corresponding to each instruction is taken as a first object, and the enumeration class is taken as a first enumeration class.

[0062] Further, another enumeration class can be established, in which each data type related to each data source is taken as an object and each data type is defined to enumerate each data type; each data type can be all data types that need to be related, or a plurality of data types selected in advance.

[0063] In the present embodiment, each object corresponding to each data type is taken as a second object, and the enumeration class is taken as a second enumeration class.

[0064] In a specific example, the first enumeration class can be referred to as Token class, and the second enumeration class can be referred to as Databases class.

[0065] Further, in the Token class, a part or all of instructions of SQL can be defined.

[0066] For example, the instruction SELECT can be defined as a first object as shown below:

[0067] SELECT(“SELECT”)

[0068] Further, in the Databases class, a part or all of data types related to data sources to be invoked can be defined.

[0069] For example, the MYSQL type can be defined as a second object, and the data source of CSV type can be defined as another second object, and the definitions are shown as follows:

[0070] MYSQL(“MYSQL”),

[0071] CSV(“CSV”)

[0072] As can be seen, after each SQL instruction is defined, the Token class, i.e., the first enumeration class, which defines a plurality of SQL instructions, can be obtained; and after each data type related to a data source is defined, the Databases class, i.e., the second enumeration class, which defines a plurality of data types, can be obtained.

[0073] In the embodiment of the present application, based on the first enumeration class and the second enumeration class constructed above, the obtained string can be parsed, and all instructions and all data codes of the string can be obtained.

[0074] In the embodiment, for the string input by the user related to reading data, a part of characters can be considered as representing each instruction, and another part of characters can be considered as data codes in the embodiment, representing each data source related, wherein each data code represents a data source.

[0075] Further, after the string is obtained, the string can be input to a pre-set lexical analyzer to parse each instruction and each data source in the string.

[0076] Specifically, the lexical analyzer can distinguish the instructions and the data codes in the string in turn according to the expression order of each character in the string.

[0077] Further, for the distinguished instruction, the definition about the instruction can be queried from the first enumeration class constructed above.

[0078] Further, for the distinguished data code, the definition about the data source represented by the data code can be queried from the second enumeration class constructed above.

[0079] In a specific example, as shown in Figure 2 , the SQL statement as shown below is taken as a specific example of the SQL statement input by the user:

[0080] SELECT user.id, user.age, data.name FROM ClickHouse.user user LEFT JOIN JSON.data data ON user.id=data.id WHERE user.age>10 and user.type='new'

[0081] In the embodiment, after the SQL statement input by the user is obtained, the SQL statement is input to the Lexer in Figure 2 , where the Lexer is a kind of lexical analyzer and can perform lexical analysis on the SQL statement.

[0082] Further, after the analysis of the SQL statement, the following instructions can be obtained:

[0083] SELECT, FROM, LEFT JOIN, ON, =, WHERE, > and =

[0084] It can be seen that the Lexer determines the instructions in the order of the characters in the SQL statement.

[0085] Further, the definitions of the instructions can be queried from the Token class.

[0086] Further, after the analysis of the SQL statement, the following data codes are also obtained:

[0087] user.id, user.age, data.name, ClickHouse.user and JSON.data

[0088] It can be seen that the Lexer determines the data codes in the order of the characters in the SQL statement.

[0089] Further, the definitions of the data sources corresponding to the data codes can be queried from the Databases class.

[0090] Step S102: Perform syntax parsing on the string to determine the execution logic between each instruction and each data code.

[0091] In embodiments of this application, the obtained string can be input into a pre-set syntax parser.

[0092] Furthermore, by using this parser to parse the string, the execution logic between each instruction and each data code in the string can be obtained.

[0093] Specifically, the execution logic describes the syntax logic when each instruction calls the data code.

[0094] In specific examples, such as Figure 2 As shown, the SQL statement from the previous steps can be input into the Parser, which acts as a syntax parser and can perform syntax parsing on the SQL statement.

[0095] It should be noted that since the lexical analysis in step S102 and the syntax parsing in step S103 are performed using different parsers, the execution order of steps S102 and S103 can be either: step S102 is executed first and then step S103; step S103 is executed first and then step S102; or steps S102 and S103 are executed simultaneously.

[0096] Step S103: Determine the data source pointed to by each data code according to the fields in each data code, and determine the data structure of the data source.

[0097] In the embodiments of this application, based on the various data codes in the parsed string, the data source represented by the data code and the data type of the data source can be determined by the specific fields of the data code.

[0098] Specifically, for the obtained string, the data source pointed to by the data code can be determined according to the fields in each data code, and the data source can be determined as structured data or semi-structured data.

[0099] In this embodiment, the data code includes two fields: specifically, a field representing the data type of the data source, which is hereby designated as the first field; and a field representing the name of the data source, which is hereby designated as the second field.

[0100] Further, the first field in the embodiment can be a specific name of the data type as a representation of the data source, and can determine the data structure of the data source represented by each first field according to the definition of each second object in the second enumeration class.

[0101] In the embodiment, the data structure specifically includes structured data and unstructured data.

[0102] The structured data can further include data sources of multiple different data types. In a specific example, the structured data can include, for example, MYSQL (a relational database management system), POSTGRESQL (an object-relational database management system), and CLICKHOUSE (a column-oriented storage database), and the like.

[0103] Further, the unstructured data can also include data sources of multiple different data types. In a specific example, the structured data can include, for example, CSV (comma-separated value file), JSON (JAVA object notation file), and XML (extensible markup language file), and the like.

[0104] Further, based on the second field described above, the name of the data source pointed to by the second field can be determined.

[0105] Specifically, a mapping class can be pre-set, and multiple key-value pairs can be set in the mapping class. According to the name of the data source in the second field, the data source can be determined from the key-value pairs.

[0106] In a specific example, based on the SQL statement in the foregoing step, taking ClickHouse.user therein as an example, it can be seen that ClickHouse is the first field, and through the second enumeration class query, the data type represented by the first field ClickHouse can be determined.

[0107] Further, user as the second field can be determined as the name of the required data source.

[0108] From Figure 2 As can be seen, for each data code, it can also be divided into two data sources of table and column according to its second field; wherein TABLES represents a data source in the form of a table, two data codes, ClickHouse represents the specific data structure of the data source, and belongs to structured data, user represents the name of the data table; JSON represents the specific structure of the data source, and the data is semi-structured data, data represents the name of the semi-structured data.

[0109] Furthermore, FIELDS represents a data source in column form. Taking the parsed data code ClickHouse.user.age as an example, ClickHouse represents the specific data structure of the data source and belongs to the category of data; user represents the name of the data table; and age represents the column named age in the data table user.

[0110] Step S104: Instruct each instruction to call the data code according to the execution logic, load data according to the data structure pointed to by the data code, and obtain the target data.

[0111] In the embodiments of this application, for the obtained string, each instruction in it can be made to call the data code according to the above execution logic, and when the data code points to structured data, the corresponding target data can be loaded from the corresponding data source; when the data code points to semi-structured data, the corresponding target data can be obtained by parsing the semi-structured data.

[0112] First, based on the various instructions and data codes parsed above, a syntax tree can be constructed, and the syntax tree can be distinguished into command instructions for commands, connection instructions for association, and condition instructions for filtering.

[0113] Specifically, the instructions and data codes parsed in the aforementioned steps can still be arranged according to the order of the above description. In this embodiment, data sources in the form of columns can be used as one branch of the syntax tree, and data sources in the form of tables can be used as another branch of the syntax tree.

[0114] Specifically, such as Figure 2 As shown, the parsed instructions SELECT, FROM, JOIN, and WHERE are arranged according to the order of their expression in the SQL statement; the data codes are also arranged according to the order of their expression in the SQL statement. Specifically, the data sources in the form of columns: ClickHouse.user.id, ClickHouse.user.age, and JSON.logs.data.name are branches of FIELDS; and the data sources in the form of tables: ClickHouse.user and JSON.data are branches of TABLE.

[0115] Furthermore, among the parsed instructions, instructions with different functions can be distinguished, such as command, join, and conditional instructions. In specific examples, command instructions can be SELECT, FROM, WHERE, etc.; join instructions can be JOIN and LEFT JOIN, etc.; and conditional instructions can be the symbol >.

[0116] Furthermore, such asFigure 2 As shown, the characters and data codes related to the connection instructions and conditional instructions are set in their respective branches, and based on this, an AST (Abstract Syntax Tree) can be obtained.

[0117] Furthermore, based on the constructed AST, it can be input into the data selection process to load the target data.

[0118] Specifically, based on the structure of the constructed syntax tree, after it is input into the data selection process, each instruction can be executed according to the execution logic parsed by the syntax parser, and the relevant data code can be called to obtain the target data corresponding to the data code.

[0119] Furthermore, when calling data codes, different loading processes can be performed on structured data and semi-structured data according to the data types pointed to by each data code determined in the preceding steps.

[0120] Specifically, when the first field in the called data code points to the data source of structured data, the specific data source can be determined through the first field, and the corresponding target data can be loaded from the data source according to the second field and then loaded.

[0121] Furthermore, when the first field in the called data code points to the data source of the semi-structured data, the specific type of the semi-structured data can be determined through the first field, and the script file related to the target data can be determined according to the second field. After obtaining the script file indicated by the second field, it can be parsed, and the target data can be determined and loaded after parsing.

[0122] In particular, when parsing different types of semi-structured data, a pre-configured parser for that type can be used.

[0123] In this embodiment, with Figure 3 The data selection logic shown is a concrete example. It can be seen that, based on the constructed AST, when calling the data code within it, for structured data, taking ClickHouse.user as an example, the first field can be used to determine that the data code points to the ClickHouse database, and the second field can be used to determine that the data code points to the target data named user.

[0124] Furthermore, for semi-structured data, taking JSON.data as an example, the specific type of the semi-structured data can be determined as JSON through the first field.

[0125] Furthermore, from Figure 3As can be seen, the JSON parser is arranged for each semi-structured data, such as a CSV parser, a JSON parser, and an XML parser, and after determining that the semi-structured data is a JSON script file, the JSON parser can be used to parse the JSON script file.

[0126] Further, when parsing the JSON script file, the second field thereof can also be used as a label, and the value corresponding to the label can be determined after parsing.

[0127] In this embodiment, the second field data in the JSON.data pointing to the data source name can be used as a label of data, and the value corresponding to the label can be determined after parsing. The value corresponding to the label can be the target data, and thus, the target data to be loaded can be determined based thereon.

[0128] As can be seen, based on the determined requirement of simultaneously calling structured data and semi-structured data, since the specific data type is determined from the first field, different processing methods can be implemented for different data structures.

[0129] Further, for any type of structured data determined, the data source can be directly loaded. For semi-structured data, the semi-structured data can be parsed based on the first field, and the target data can be determined after parsing. Therefore, the structured data and semi-structured data in the SQL statement can be simultaneously parsed, that is, in this embodiment, the target data in the CLICKHOUSE and the target data pointed to by the JSON file can be determined at the same time, without having to separately process the target data in the CLICKHOUSE and the target data pointed to by the JSON file.

[0130] Further, based on the determined target data to be loaded, as described above, the relevant target data can be run according to the execution logic parsed by the syntax parser, for example, the JOIN operation of the connection instruction is performed first, and then the filtering of the condition instruction is performed, and the result of data loading is obtained.

[0131] As can be seen, the data parsing method of the embodiments of the present application classifies the instructions and data sources that can appear in the string based on the defined first and second enumeration classes, so that in the parsing process of the string, the instruction-related characters and the data source-related characters can be respectively parsed according to the first and second enumeration classes. Meanwhile, the syntax parsing added in the method can effectively parse the execution logic between the instructions and the data code.

[0132] Further, after considering the structured data and the semi-structured data, the data code is used to determine the specific data type pointed by the data code, and when the data code is called, different data sources and different data types can be distinguished, so that different data types are loaded in different ways, and when different data sources appear in a string and different data types appear in different data sources, the target data can be loaded across data sources and across data types.

[0133] It should be noted that the method of the embodiments of the present application can be executed by a single device, such as a computer or a server. The method of the embodiments can also be applied to a distributed scenario, and completed by multiple devices cooperating with each other. In this distributed scenario, one of the multiple devices can only execute one or more steps in the method of the embodiments of the present application, and the multiple devices can interact with each other to complete the method.

[0134] It should be noted that some embodiments of the present application have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than the order described above and still achieve desirable results. Additionally, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous.

[0135] Based on the same inventive concept, the embodiments of the present application also provide a data analysis device corresponding to the method of any of the above embodiments.

[0136] Reference Figure 4 The data analysis device comprises a lexical analysis module 401, a syntax analysis module 402, a data structure determination module 403, and a data loading module 404.

[0137] The lexical analysis module 401 is configured to, for each string obtained, parse all instructions in the string by calling a preset first enumeration class, and parse all data codes in the string by calling a preset second enumeration class.

[0138] The syntax analysis module 402 is configured to perform syntax analysis on the string to determine the execution logic between each instruction and each data code.

[0139] The data structure determination module 403 is configured to determine the data source pointed by the data code according to the field in the data code, and determine the data structure of the data source.

[0140] The data loading module 404 is configured to enable the respective instruction to call the data code according to the execution logic, and to perform data loading according to a data structure pointed to by the data code to obtain target data.

[0141] For the convenience of description, the above apparatus is described in various modules according to functions. Of course, the functions of the modules can be implemented in one or more software and / or hardware in the implementation of the embodiments of the present application.

[0142] The apparatus of the above embodiments is used to implement the corresponding data analysis method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described here again.

[0143] Based on the same inventive concept, corresponding to any of the above method embodiments, the embodiments of the present application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the data analysis method according to any one of the above embodiments.

[0144] Figure 5 A more specific hardware structure of an electronic device provided by the present embodiment is shown, which can include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040 and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030 and the communication interface 1040 are connected to each other through the bus 1050 for communication within the device.

[0145] The processor 1010 can be implemented in the form of a general-purpose CPU (Central Processing Unit, central processor), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., for executing related programs to implement the technical solutions provided by the embodiments of the present application.

[0146] The memory 1020 can be implemented in the form of a ROM (Read Only Memory, read-only memory), a RAM (Random Access Memory, random access memory), a static storage device, a dynamic storage device, etc. The memory 1020 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are saved in the memory 1020 and executed by the processor 1010.

[0147] The input / output interface 1030 is configured to connect an input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure) or externally connected to the device to provide corresponding functions. The input device can include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device can include a display, a speaker, a vibrator, an indicator light, etc.

[0148] The communication interface 1040 is configured to connect a communication module (not shown in the figure) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired manner (such as a USB, a network cable, etc.) or a wireless manner (such as a mobile network, WIFI, Bluetooth, etc.).

[0149] The bus 1050 includes a channel to transmit information between various components (such as the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040) of the device.

[0150] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in the specific implementation process, the device can also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device can also only contain the components necessary for implementing the embodiments of the present application, and does not have to contain all the components shown in the figure.

[0151] The device of the above embodiment is used to implement the corresponding data analysis method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described here again.

[0152] Based on the same inventive concept, corresponding to any of the above embodiment methods, the present application also provides a non-transitory computer readable storage medium storing computer instructions for causing the computer to execute the data analysis method according to any of the above embodiments.

[0153] The computer readable media of the embodiments can include permanent and non-permanent, removable and non-removable media, which can be implemented by any method or technology to store information. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0154] The storage medium of the above embodiments stores computer instructions for causing the computer to execute the data parsing method as described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described here.

[0155] Those skilled in the art should understand that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope (including claims) of the present application is limited to these examples; the above embodiments or technical features between different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes of the different aspects of the embodiments of the present application as described above. In order to be brief, they are not provided in detail.

[0156] In addition, in order to simplify the description and discussion, and so as not to make the embodiments of the present application difficult to understand, the well-known power / ground connections of integrated circuit (IC) chips and other components can or can not be shown in the provided drawings. In addition, the devices can be shown in the form of block diagrams in order to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented the embodiments of the present application (i.e. these details should be fully within the understanding of those skilled in the art). Where specific details (e.g. circuits) are set forth in order to describe the exemplary embodiments of the present application, it will be apparent to those skilled in the art that the embodiments of the present application can be implemented without these specific details or with variations of these specific details. Therefore, these descriptions should be considered illustrative rather than limiting.

[0157] While the present application has been described in connection with certain embodiments thereof, many modifications, substitutions, changes, and of forms will be apparent to those of ordinary skill in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) can use the embodiments discussed.

[0158] Embodiments of the present application are intended to embrace all such alterations, modifications, and variations that fall within the scope of the appended claims. Accordingly, the application is intended to be governed by the scope of the claims and their equivalents.

Claims

1. A data parsing method, characterized in that, include: For each string obtained, all instructions in the string are parsed by calling the first predefined enumeration class; all data codes in the string are parsed by calling the second predefined enumeration class. The string is parsed to determine the execution logic between each instruction and each data code; Determine the data source that the data code points to based on the fields in each data code, and determine the data structure of the data source; The instructions are instructed to call the data code according to the execution logic, and to load data according to the data structure pointed to by the data code to obtain the target data; wherein, The first enumeration class and the second enumeration class are determined by the following method: Each of the multiple instructions is treated as a first object in the first enumeration class, and each first object is defined. The data types of multiple data sources are used as the second objects in the second enumeration class, and each second object is defined. The step of determining the data source pointed to by each data code according to the fields in each data code, and determining the data structure of the data source, includes: For each data code, determine the first field in the data code that points to the type of the data source, and determine the second field in the data code that points to the name of the data source; Based on the first field, by querying the definition of the second object, it is determined that the data structure of the data source is either structured data or semi-structured data; Based on the second field, the name of the data source is determined by querying a preset mapping class; The step of loading data according to the data structure pointed to by the data code to obtain the target data includes: In response to determining that the data source pointed to by the data code is structured data, the corresponding target data is loaded from the corresponding data source; In response to determining that the data source pointed to by the data code is semi-structured data, the corresponding target data is obtained by parsing the semi-structured data.

2. The method according to claim 1, characterized in that, The step of parsing all instructions in the string by calling the preset first enumeration class includes: Based on the order of the characters in the string, the characters representing each instruction are determined sequentially, and the definitions of each instruction in the first enumeration class are retrieved.

3. The method according to claim 1, characterized in that, The step of parsing all the data codes in the string by calling the preset second enumeration class includes: According to the order of the characters in the string, the data codes representing each data source are determined sequentially, and the definition of the data source corresponding to each data code in the second enumeration class is retrieved.

4. The method according to claim 1, characterized in that, Before instructing each instruction to call the data code according to the execution logic, the method further includes: Based on the definition of the first object, all instructions are classified into command instructions, connection instructions, and conditional instructions; According to the order in which the instructions and data codes in the string are expressed, a syntax tree is constructed from the instructions and data codes in the string.

5. The method according to claim 1, characterized in that, The step of obtaining the corresponding target data by parsing the semi-structured data includes: Based on the data type of the semi-structured data, a pre-set parser corresponding to that data type is invoked; By using the corresponding parser to parse the script file of the semi-structured data, the target data pointed to by the second field is determined; Based on the determined target data, the target data is loaded.

6. A data parsing device, characterized in that, include: Lexical analysis module, syntax analysis module, data structure determination module, and data loading module; The lexical parsing module is configured to, for each acquired string, parse out all instructions in the string by calling a preset first enumeration class; and parse out all data codes in the string by calling a preset second enumeration class; wherein the first enumeration class and the second enumeration class are determined by the following method: Each of the multiple instructions is treated as a first object in the first enumeration class, and each first object is defined. The data types of multiple data sources are used as the second objects in the second enumeration class, and each second object is defined. The syntax parsing module is configured to perform syntax parsing on the string to determine the execution logic between each instruction and each data code; The data structure determination module is configured to determine the data source pointed to by each data code according to the fields in each data code, and to determine the data structure of the data source. The data loading module is configured to cause each instruction to call the data code according to the execution logic, and to load data according to the data structure pointed to by the data code to obtain the target data; The data structure determination module is further configured to, for each data code, determine a first field in the data code that points to the type of the data source, and determine a second field in the data code that points to the name of the data source; Based on the first field, by querying the definition of the second object, it is determined that the data structure of the data source is either structured data or semi-structured data; Based on the second field, the name of the data source is determined by querying a preset mapping class; The data loading module is further configured to load the corresponding target data from the corresponding data source in response to determining that the data code points to structured data. In response to determining that the data source pointed to by the data code is semi-structured data, the corresponding target data is obtained by parsing the semi-structured data.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executed by the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions for causing the computer to perform the method according to any one of claims 1 to 5.

Images