Blockchain-based Data Analysis Application Development System and Development Method

Abstract

The present invention discloses a development system and a development method for data analysis-based application programs, relating to the field of application program development. The method includes: a singleton application program module, which adopts the singleton pattern and is used for: saving preset basic configuration information, calling a log module to implement dynamic configuration of logs, and calling a configuration module to implement initialization loading of configurations; a configuration module, which is used for implementing configuration loading; a log module, which is used for: formatting logs into a preset format and parsing to obtain common information for locating problems in the development environment; a message event module, through which the data analysis results of the system are output by implementing the message publishing and subscribing functions; and a packaging and deployment module, which is used for packaging the application program. The present invention can meet the requirements for quickly developing application programs for data analysis.

Description

Technical Field

[0001] The present invention relates to the field of application development, and specifically, to a development system and a development method for data analysis applications based on blockchain. Background Art

[0002] In order to quickly develop applications for analyzing blockchain data, there are currently various Python frameworks, including those for web applications, crawlers, continuous integration, etc. However, with the explosion of big data, there are more and more scenarios for data analysis, especially for analyzing massive on-chain blockchain data. Since Python is naturally suitable for data analysis, the existing application frameworks are either heavy or incomplete, and most of them are development frameworks for web applications, making it difficult to meet the requirements for quickly developing data analysis applications for analyzing massive on-chain blockchain data. Summary of the Invention

[0003] The object of the present invention is to provide a lightweight application development system for data analysis applications, which can meet the requirements for quickly developing applications for analyzing massive blockchain data.

[0004] To achieve the above object of the invention, the present invention provides a development system for data analysis applications based on blockchain, the system comprising:

[0005] A singleton application module, which adopts the singleton pattern and is used for: saving preset basic configuration information, and for calling the log module to implement dynamic configuration of logs, and for calling the configuration module to implement initialization loading of configurations;

[0006] A configuration module, which is used for implementing configuration loading;

[0007] A log module, which is used for: formatting logs into a preset format, and for parsing and obtaining common information for locating problems in the development environment;

[0008] A message event module, and the output of the data analysis result of the system is implemented through the message publishing and subscribing functions of the message event module;

[0009] A packaging and deployment module, which is used for packaging the application.

[0010] Among them, the inventors of the present application found through analysis of the prior art that there is an urgent need for a lightweight system for data analysis applications. This system only needs to implement: 1. Loading the environment configuration required for program operation to ensure that the same configuration information can be obtained anywhere during program operation; 2. Printing and outputting logs during program operation to meet the viewing and collection requirements of logs in different environments; 3. Encapsulating message publishing and subscription functions based on Kafka using decorators; 4. Encapsulating the automatic packaging of programs based on Docker container technology. This system does not include commonly used URL routing mapping, complex middleware, template programming, admin, etc. in existing systems, as well as some modules for in-depth integration of database access in systems, avoiding system heaviness and making the system simplified and lightweight. If you need to access the database based on this system, you can directly use rich Python libraries. This system encapsulates a decorator for message publishing and subscription that is indispensable in the big data field, simplifying the development process of application programs. Based on this system, developers only need to write the core logic code for data analysis and can publish analysis results efficiently or directly store them in the database. They don't need to care about how the program runs, how to perform environment isolation, how to collect program logs to monitor the program status, and by what means to publish to the production environment. Developers only need to complete the logic of data analysis, improving the development efficiency of application programs. A lightweight system for data analysis applications in the present invention can help programmers quickly develop programs and pay more attention to the implementation of business logic.

[0011] Preferably, in this system, the configuration loading methods of the configuration module include a configuration update method and a configuration initialization method; the configuration update method is: using a new configuration object to overwrite the original configuration object, while ensuring that the attributes not present in the new configuration object maintain their initial values; the configuration initialization method is: by default, loading the configuration file of the development environment, looking for configuration information of the test environment or development environment in the environment variables. If it exists, use the configuration update method to overwrite the loaded development environment configuration, otherwise directly run the application program using the development environment configuration.

[0012] Preferably, in this system, the configuration module stores configurations using a preset dictionary.

[0013] Preferably, in this system, the preset dictionary is a Python dictionary.

[0014] Preferably, in this system, the common information includes but is not limited to: time, log level, application name, application version, request identifier, trace link ID, remote IP, request initiator, log method name, and log message.

[0015] Preferably, in this system, the log module is used to: format logs into Json format, implement a processing program based on the json format, and implement a processing program in string format for development environment debugging; use different processing programs according to configuration information in the singleton application module.

[0016] Preferably, this system integrates a python decorator encapsulated based on the open-source library confluent_kafa to implement event publishing and subscription.

[0017] Preferably, in this system, based on the message event-driven mode, decouple data storage and analysis in data analysis and publish the analysis results.

[0018] Preferably, the system integrates the use of DOCKER to package applications in one click. At the same time, through the integrated release script, it dynamically obtains configurations in different environments and injects the configurations into the environment variables of the DOCKER container at runtime.

[0019] The present invention also provides a development method for data analysis application programs. The method is based on the system for developing data analysis application programs, and the method includes:

[0020] Add a configuration file to the system and set default parameters;

[0021] Initialize the application program;

[0022] Call the singleton application module, configuration module, log module, and message event module to implement the development of the application program;

[0023] Package the developed application program into a DOCKER image and then deploy the application program;

[0024] When deploying the application program, pull the configuration in the configuration management and inject it into the environment variables in JSON format;

[0025] The configuration module of the system checks the configuration in the environment variables and updates the configuration;

[0026] The application program runs using the updated configuration.

[0027] One or more technical solutions provided by the present invention have at least the following technical effects or advantages:

[0028] The present invention can automatically load configurations in different environments, can be accessed in the program, can output diverse logs for different scenarios, can send and listen to event messages in data analysis application programs; the present invention can achieve rapid containerized deployment. The system in the present invention simply realizes a lightweight design, which can help programmers quickly develop programs and pay more attention to the implementation of business logic. Description of the Drawings

[0029] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, form a part of the present invention, and do not limit the embodiments of the present invention;

[0030] Figure 1 It is a schematic diagram of the composition of a development system for blockchain-based data analysis applications;

[0031] Figure 2 It is a schematic flowchart of a development method for data analysis applications. Detailed Embodiments

[0032] In order to more clearly understand the above objects, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the drawings and specific embodiments. It should be noted that, without conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0033] In the following description, many specific details are set forth in order to fully understand the present invention. However, the present invention may be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited by the specific embodiments disclosed below.

[0034] It should be understood that the "system", "device", "unit", and / or "module" used herein is a method for distinguishing different components, elements, parts, portions, or assemblies at different levels. However, if other words can achieve the same purpose, the said words can be replaced by other expressions.

[0035] As shown in this specification and the claims, unless the context clearly indicates otherwise, words such as "a", "an", "one", and / or "the" are not specifically singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. A method or device may also include other steps or elements.

[0036] Flowcharts are used in this specification to illustrate the operations performed by the systems according to the embodiments of this specification. It should be understood that the operations before or after do not necessarily need to be executed precisely in sequence. On the contrary, they can be executed in reverse order or simultaneously. At the same time, other operations can also be added to these processes, or one or several steps can be removed from these processes.

[0037] Embodiment 1

[0038] Please refer to Figure 1 , Figure 1It is a schematic diagram of the composition of a blockchain-based data analysis application development system. Embodiment 1 of the present invention provides a blockchain-based data analysis application development system, and the system includes:

[0039] A singleton application module, which adopts the singleton pattern. The singleton application module is used to: save preset basic configuration information, and to call the log module to implement dynamic configuration of logs, and to call the configuration module to implement initialization loading of configurations;

[0040] A configuration module, which is used to implement configuration loading;

[0041] A log module, which is used to: format the logs into a preset format, and to parse and obtain common information for locating problems in the development environment;

[0042] A message event module, and the data analysis result output of the system is implemented through the message publishing and subscribing functions of the message event module;

[0043] A packaging and deployment module, which is used to package the application.

[0044] The main technical problem to be solved by the embodiment of the present invention is to provide a lightweight system for quickly developing data analysis applications, which can solve problem A: the problem of automatic loading of different environment configurations and how to access them in the program; problem B: the problem of diverse log outputs for different scenarios; problem C: the problem of event sending and listening in data analysis applications; problem D: the problem of rapid containerized deployment.

[0045] To solve the above problems, the specific implementation solutions are as follows:

[0046] A: Implementation of the singleton APP: First, the system implements an APP class. 1. This class provides an instance variable self.config to save global configuration information; 2. At the same time, it encapsulates the instance method init_log(self) to call the log module to implement dynamic configuration of logs; 3. It encapsulates the instance method load_config(self) to call the configuration module to implement initialization loading of configurations; 4. To ensure that the APP class will not be instantiated into multiple objects, causing waste of memory resources, it is desired that there is only one instance object during the program operation. A commonly used software design pattern: the singleton pattern (SingletonPattern) is used, and the system implements the singleton object of the APP class based on the python metaclass.

[0047] B: Configuration Module: This module implements a configuration loading class that stores configurations using Python dictionaries. This class provides two class methods. 1. The configuration update class method uses a new configuration object to overwrite the old one, while ensuring that the attributes not present in the new configuration object maintain their initial values. 2. The configuration initialization class method first loads the configuration file for the development environment by default, and then looks for configuration information for the test or development environment in the environment variables. If it exists, it uses the configuration update class method to overwrite the loaded development environment configuration; otherwise, it directly runs the program using the development environment configuration.

[0048] The method is explicitly called in load_config of the singleton APP.

[0049] C: Logging Module: 1. This module implements a class that formats logs into JSON format. This class inherits from Python's logging.Formatter and parses common information for problem location, including time, log level, program name, program version, request URI, trace link ID, remote IP, request initiator, log method name, log message, etc. 2. The logging module implements a JSON-based handler for easy problem location in the production environment and a string-based handler for debugging in the development environment.

[0050] In init_log of the singleton APP, different handlers are used according to the configuration information.

[0051] D: In the system, the data analysis results are output by Kafka message publishing and subscribing. In this system, a Python decorator is integrated based on the open-source library confluent_kafka for easy use, which realizes event publishing and subscribing. Based on the message event-driven mode, it can solve the decoupling of data storage and analysis in data analysis, and at the same time publish the analysis results. Applications interested in the analysis results can subscribe to relevant messages to complete the function.

[0052] E: In the system, DOCKER is integrated to package the application with one key. At the same time, through the integrated release script, the configurations of different environments are dynamically obtained and injected into the environment variables of the DOCKER container at runtime. This configuration information is loaded into the program in the configuration module.

[0053] Example Two

[0054] Please refer to Figure 2 , Figure 2It is a flow schematic diagram of a development method for data analysis applications targeting blockchain massive data. Embodiment 2 of the present invention provides a development method for data analysis applications targeting blockchain massive data. The method is based on the above-mentioned data analysis application development system based on blockchain, and the method includes:

[0055] Add a configuration file to the system and set default parameters;

[0056] Initialize the application;

[0057] Call the singleton application module, configuration module, log module, and message event module to implement the development of the application;

[0058] Package the developed application into a DOCKER image and then deploy the application;

[0059] When deploying the application, pull the configuration in the configuration management and inject it into the environment variables in JSON format;

[0060] The configuration module of the system checks the configuration in the environment variables and updates the configuration;

[0061] The application runs using the updated configuration.

[0062] The beneficial effects of the present invention are: 1. Provide an optional system for applications targeting data analysis of blockchain massive data, which integrates the basic capabilities of program development and deployment; 2. Provide an idea for subsequent applications targeting data analysis of blockchain massive data, and more plugins can be encapsulated based on this idea to achieve richer data analysis capabilities; 3. Make the development of applications targeting data analysis of blockchain massive data more convenient, and can focus more on the realization of business capabilities.

[0063] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only an example and does not constitute a limitation to this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are suggested in this specification, so such modifications, improvements, and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

[0064] Meanwhile, this specification uses specific terms to describe the embodiments of this specification. For example, "an embodiment", "one embodiment", and / or "some embodiments" mean a certain feature, structure, or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that the "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more at different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this specification can be appropriately combined.

[0065] In addition, those skilled in the art can understand that various aspects of this specification can be described and illustrated by several patentable types or situations, including any new and useful process, machine, product, or composition of matter, or any new and useful improvement thereof. Accordingly, various aspects of this specification can be implemented entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software can be referred to as a "block", "module", "engine", "unit", "component", or "system". In addition, various aspects of this specification may be embodied as a computer product located in one or more computer-readable media, which includes computer-readable program code.

[0066] A computer storage medium may contain a propagated data signal that contains computer program code, such as on a baseband or as part of a carrier wave. This propagated signal may have various forms of manifestation, including electromagnetic form, optical form, etc., or a suitable combination thereof. A computer storage medium can be any computer-readable medium other than a computer-readable storage medium, which can be connected to an instruction execution system, apparatus, or device to implement communication, propagation, or transmission for use of the program. The program code located on the computer storage medium can be propagated through any suitable medium, including radio, cable, fiber optic cable, RF, or similar media, or any combination of the above media.

[0067] The computer program codes required for the operations of each part of this specification can be written in any one or more programming languages, including object-oriented programming languages such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, Python, etc., conventional procedural programming languages such as C language, Visual Basic, Fortran 2003, Perl, COBOL 2002, PHP, ABAP, dynamic programming languages such as Python, Ruby, and Groovy, or other programming languages. The program codes can run entirely on the user's computer, or run as an independent software package on the user's computer, or partly run on the user's computer and partly on a remote computer, or run entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any network form, such as a local area network (LAN) or a wide area network (WAN), or connected to an external computer (e.g., through the Internet), or in a cloud computing environment, or used as a service such as software as a service (SaaS).

[0068] In addition, unless clearly stated in the claims, the order of the processing elements and sequences, the use of numerical letters, or the use of other names in this specification are not used to limit the order of the processes and methods in this specification. Although some currently considered useful embodiments of the invention are discussed through various examples in the above disclosure, it should be understood that such details only serve the purpose of illustration, and the appended claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that conform to the essence and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only through a software solution, such as installing the described system on an existing server or mobile device.

[0069] Similarly, it should be noted that, in order to simplify the expression of the disclosure in this specification and thus help the understanding of one or more embodiments of the invention, in the foregoing description of the embodiments of this specification, sometimes multiple features are merged into one embodiment, drawing, or description thereof. However, this disclosure method does not mean that the features required by the object of this specification are more than those mentioned in the claims. In fact, the features of the embodiments are fewer than all the features of the individual embodiments disclosed above.

[0070] For each patent, patent application, patent application publication, and other materials cited in this specification, such as articles, books, specifications, publications, documents, etc., their entire contents are hereby incorporated by reference into this specification. This excludes the application history documents that are inconsistent with or conflict with the content of this specification, as well as the documents that limit the broadest scope of the claims of this specification (currently or subsequently appended to this specification). It should be noted that if there are any inconsistencies or conflicts between the descriptions, definitions, and / or uses of terms in the supplementary materials of this specification and the content described in this specification, the descriptions, definitions, and / or uses of terms in this specification shall prevail.

[0071] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, by way of example and not limitation, alternative configurations of the embodiments of this specification may be considered to be in accordance with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

[0072] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications to these embodiments once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed as including the preferred embodiments as well as all changes and modifications that fall within the scope of the present invention.

[0073] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these changes and modifications.

Claims

1. A blockchain-based data analysis application development system, characterized in that The system includes: A singleton application module that adopts the singleton pattern and is used to: save preset basic configuration information, call the log module to implement dynamic configuration of logs, and call the configuration module to implement initialization loading of configurations; A configuration module that is used to implement configuration loading; A log module that is used to: format logs into a preset format and parse to obtain common information for locating problems in the development environment; A message event module, and the output of the data analysis result of the system is implemented through the message publishing and subscribing functions of the message event module; A packaging and deployment module for packaging the application; The configuration loading methods of the configuration module include a configuration update method and a configuration initialization method; the configuration update method is: using a new configuration object to overwrite the original configuration object while ensuring that the attributes not present in the new configuration object maintain their initial values; the configuration initialization method is: by default, loading the configuration file in the development environment, looking for configuration information in the test environment or development environment in the environment variables, if present, using the configuration update method to overwrite the loaded development environment configuration, otherwise directly running the application using the development environment configuration.

2. The blockchain-based data analysis application development system according to claim 1, wherein The configuration module stores configurations using a preset dictionary.

3. The blockchain-based data analysis application development system according to claim 2, characterized in that The preset dictionary is a Python dictionary.

4. The blockchain-based data analysis application development system according to claim 1, characterized in that, The common information includes but is not limited to: time, log level, application name, application version, request identifier, tracing link ID, remote IP, request initiator, log method name, and log message.

5. The blockchain-based data analysis application development system according to claim 1, characterized in that, The log module is used to: format logs into Json format, implement a handler based on the json format, and a handler in string format for development environment debugging; use different handlers according to the configuration information in the singleton application module.

6. The blockchain-based data analysis application development system according to claim 1, characterized in that This system integrates a Python decorator encapsulated based on the open-source library confluent_kafa to implement event publishing and subscribing.

7. The blockchain-based data analysis application development system according to claim 6, wherein Based on the message event-driven mode, decouple data storage and analysis during data analysis and publish the analysis results.

8. The blockchain-based data analysis application development system according to claim 1, characterized in that The system integrates the use of DOCKER to package the application in one click. At the same time, through the integrated publishing script, it dynamically obtains configurations in different environments and injects the configurations into the environment variables of the DOCKER container during runtime.

9. A development method for data analysis applications, the method being based on the blockchain-based data analysis application development system according to any one of claims 1-8, characterized in that, The method includes: Adding a configuration file to the system and setting default parameters; Initializing the application; Calling the singleton application module, configuration module, log module, and message event module to implement the development of the application; Packaging the developed application into a DOCKER image and then deploying the application; When deploying the application, pulling the configuration in the configuration management and injecting it into the environment variables in JSON format; The configuration module of the system checks the configuration in the environment variables and updates the configuration; The application runs using the updated configuration.

Images