A data processing method and data processing system based on Internet of Things platform

Abstract

The present invention discloses a data processing method, data processing system, computer device and storage medium based on an Internet of Things platform, wherein the data processing method of one embodiment includes: in response to a user's first operation, selecting at least one visualization service from a configuration service library pre-set on the Internet of Things platform as a working node and forming a data processing workflow; in response to a user's second operation, configuring operators for each visualization service of the data processing workflow, the operators including common operators and customized operators defined by scripts; and in response to accessed data, processing the data according to the data processing workflow. The data processing method provided by the present invention customizes visualization services and forms a data processing workflow through a configuration service library of an Internet of Things platform based on a service orchestration engine, determines the data processing method of each visualization service by configuring operators, and processes the accessed data, and has broad application prospects.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things, and in particular to a data processing method, a data processing system, a computer device and a storage medium based on an Internet of Things platform. Background Art

[0002] In the existing technology, the data processing of the accessed data based on the IoT platform has the following problems:

[0003] 1) Customize data processing workflows based on business needs, typically using clustered, single applications with fixed data processing workflows.

[0004] 2) This results in long development cycles, high business requirement changes, and high maintenance costs;

[0005] 3) It requires high technical quality from developers and the development workload is large.

[0006] For example, in Chinese patent documents, a patent document entitled "An Event-Driven Message Interaction Method for Electric Power Internet of Things" with application number 202010519159.X introduces an event-driven message interaction method for electric power Internet of Things. First, the collaborative business process across organizations or departments in the Internet of Things management platform and edge Internet of Things agent / intelligent terminal of the electric power Internet of Things is modeled. Then, the execution semantics of the business process resource model are defined. Then, a parsing mechanism for the business model by the process execution engine is proposed. Finally, message interaction in the microservice architecture is implemented to achieve message interaction. This invention uses the Internet of Things management platform as an open platform for the Internet of Things at the PaaS layer. To a certain extent, it can solve the problem of direct mapping and interaction between large-scale, distributed sensors, electric power intelligent terminals and business data flows. However, it is a clustered single application with problems such as relatively fixed business processes, changes in business requirements, and high maintenance costs.

[0007] For example, in Chinese patent documents, a patent document entitled "Business process customization and distributed process scheduling method based on microservice architecture" with application number 202110493496.0 introduces a business process customization and distributed process scheduling method based on microservice architecture. The method implements business process customization and distributed process scheduling based on microservice architecture by building a unified microservice management platform, providing a lightweight microservice encapsulation model and framework for microservice standardization, and realizing the application infrastructure of microservices; defining interface interaction specifications between microservices and calling specifications for business processes for business applications, providing different types of business process gateways for microservices, and realizing business automation operation between distributed business processes and between microservices; providing a visual business process orchestration customization method, and unified management of business process metadata information, to realize business process customization and distributed process scheduling based on microservice architecture. The workflow engine of this invention is a centralized service with a state machine as the core, which can uniformly manage microservices, but has problems such as relatively fixed business processes, changes in business requirements and high maintenance costs. Summary of the Invention

[0008] In order to solve at least one of the above problems, the first embodiment of the present invention provides a data processing method based on an Internet of Things platform, comprising:

[0009] S20: In response to a first operation by the user, at least one visualization service is selected from a pre-set configuration service library on the Internet of Things platform as a working node and a data processing workflow is formed, wherein the Internet of Things platform is based on a service orchestration engine;

[0010] S40: in response to a second operation of the user, configuring operators for each visualization service of the data processing workflow respectively, wherein the operators include common operators and customized operators defined by scripts;

[0011] S60: In response to the received data, perform data processing on the data according to the data processing workflow.

[0012] For example, in the data processing method provided in some embodiments of the present application, S20 further includes:

[0013] In response to the user's first operation, at least one visualization service is selected as a work node on the workflow canvas of the Internet of Things platform and a data processing workflow is formed, wherein each operator of each visualization service of the data processing workflow is driven by an event based on the functional service architecture.

[0014] For example, in the data processing method provided in some embodiments of the present application, S20 further includes:

[0015] The location and execution order of the visualization services may vary.

[0016] For example, in the data processing method provided in some embodiments of the present application, S40 further includes:

[0017] The dynamic script of the customized operator is accessed through the interface of the working node corresponding to the visualization service.

[0018] For example, in the data processing method provided in some embodiments of the present application, S40 further includes:

[0019] Horizontally expand the number of operators of the visualization service based on the amount of computing power required for the business data volume, and each operator runs in parallel;

[0020] and / or

[0021] The functions are split according to the complexity of the computing power required for the business data volume, and the number of operators of the visualization service is horizontally expanded according to the split functions, and each operator runs in parallel.

[0022] For example, in the data processing method provided in some embodiments of the present application, S60 further includes: storing and cleaning the accessed data.

[0023] For example, in the data processing method provided in some embodiments of the present application, S60 further includes:

[0024] The data processing process of presenting said data on the IoT platform.

[0025] A second embodiment of the present invention provides a data processing system, comprising: a workflow configuration unit, a visualization service configuration unit, and an execution unit, wherein:

[0026] The workflow configuration unit is configured to select at least one visualization service as a work node from a pre-set configuration service library on the Internet of Things platform in response to a first operation of a user and form a data processing workflow, wherein the Internet of Things platform is based on a service orchestration engine;

[0027] The visualization service configuration unit is configured to configure operators for each visualization service of the data processing workflow in response to a second operation of the user, wherein the operators include common operators and customized operators defined by scripts;

[0028] The execution unit is configured to process the data in response to the received data according to the data processing workflow.

[0029] A third embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the method described in the first embodiment is implemented.

[0030] A fourth embodiment of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first embodiment when executing the program.

[0031] The beneficial effects of the present invention are as follows:

[0032] In response to the current existing problems, the present invention develops a data processing method, data processing system, computer equipment and storage medium based on the Internet of Things platform, and customizes visualization services and forms a data processing workflow through the configuration service library of the Internet of Things platform based on the service orchestration engine. The data processing method of each visualization service is determined by configuring common operators and customized operators defined by scripts, and data processing is performed on the accessed data. Therefore, the present invention has the characteristics of flexible configuration of data processing procedures and processing methods, and is easy to expand and modify, effectively making up for the problems existing in the existing technology and having broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0034] Figure 1 A flow chart showing a data processing method according to an embodiment of the present invention;

[0035] Figure 2 A structural block diagram of a data processing system according to an embodiment of the present invention is shown;

[0036] Figure 3 A schematic diagram illustrating a configuration of a data processing workflow according to an embodiment of the present invention is shown;

[0037] Figure 4 A schematic diagram showing presentation of data according to an embodiment of the present invention;

[0038] Figure 5 A schematic structural diagram of a computer device according to another embodiment of the present invention is shown. DETAILED DESCRIPTION

[0039] In order to more clearly illustrate the present invention, the present invention is further described below in conjunction with preferred embodiments and accompanying drawings. Similar components in the accompanying drawings are represented by the same reference numerals. It should be understood by those skilled in the art that the following detailed description is illustrative rather than restrictive and should not be used to limit the scope of protection of the present invention.

[0040] According to the problems existing in the prior art, such as Figure 1 As shown, one embodiment of the present invention provides a data processing method based on an Internet of Things platform, comprising:

[0041] S20: In response to a first operation by the user, at least one visualization service is selected from a pre-set configuration service library on the Internet of Things platform as a working node and a data processing workflow is formed, wherein the Internet of Things platform is based on a service orchestration engine;

[0042] S40: in response to a second operation of the user, configuring operators for each visualization service of the data processing workflow respectively, wherein the operators include common operators and customized operators defined by scripts;

[0043] S60: In response to the received data, perform data processing on the data according to the data processing workflow.

[0044] In this embodiment, the Internet of Things platform is a development framework based on a service orchestration engine. This development framework depends on the application scenario of the access and focuses on business modeling, assembly, deployment and management. Compared with the existing technology in which the Internet of Things platform uses a workflow engine as a development framework, the core implementation of the workflow engine is a state machine, that is, the execution process is built using centralized services.

[0045] This embodiment is based on a service orchestration engine and utilizes a configuration service library pre-set on the platform to provide users with visual configurations to form data processing workflows. That is, the visualization services in the configuration service library are used to customize business according to user needs and form an on-demand data processing flow. Among them, the included visualization services can be regarded as various working nodes of the data processing workflow. The location and execution order of each visualization service can be configured as needed and do not need to be fixed, thereby realizing flexible configuration and application of visualization services.

[0046] On the basis of flexible configuration of each visualization service, the configuration operator of each visualization service is further configured as needed, wherein the operator includes a common operator that realizes general functions and a customized operator with special functions defined by a script.

[0047] The IoT platform can process the monitoring data of the connected assets to be controlled according to the configured data processing workflow, such as storing, cleaning and converting the monitoring data, so as to realize other control functions of the IoT platform.

[0048] This embodiment customizes visualization services and forms a data processing workflow through the configuration service library of the Internet of Things platform based on the service orchestration engine. The data processing method of each visualization service is determined by configuring common operators and customized operators defined by scripts, and data processing is performed on the accessed data. This has the characteristics of flexible configuration of data processing flows and processing methods, and is easy to expand and modify, effectively making up for the problems existing in the existing technology and having broad application prospects.

[0049] In a specific example, Figure 2 The data processing system of this embodiment is shown, comprising: a workflow configuration unit, a visualization service configuration unit, and an execution unit. The workflow configuration unit is used to visually configure each visualization service in the data processing workflow according to user selections; the visualization service configuration unit is used to visually configure the operators of each visualization service according to user selections; and the execution unit is used to process the received data according to the data processing workflow. The following describes a specific configuration and data processing process:

[0050] In the first step, in response to a first operation of a user, at least one visualization service is selected from a configuration service library pre-set on the Internet of Things platform as a working node and a data processing workflow is formed.

[0051] In this embodiment, if Figure 3 The figure shows a schematic diagram of the configuration page for the data processing workflow of the IoT platform. On the left is the configuration service library, which includes multiple pre-set visualization services; on the right are the attribute configuration and operator list. The attribute configuration includes position and order; the operator list includes common operators, custom operators defined using scripts, and the working node interface for scripts used to access custom operators; and the workflow canvas is set in the middle.

[0052] Specifically, based on user visualization operations, such as dragging a selected visualization service onto the workflow canvas as a work node in the output processing workflow, the specific location and execution order of the visualization service can be configured through the property configuration on the right. In this embodiment, the "Data Storage" visualization service is selected as work node 1, the "Data Cleaning" visualization service is set as work node 2, the "First Monitoring Value Conversion" visualization service is set as work node 3, the "Second Monitoring Value Conversion" visualization service is set as work node 4, and the "Monitoring Value Analysis" visualization service is set as work node 5, thus forming a data processing workflow with five visualization services.

[0053] It is worth noting that the specific location and execution order of the visualization service in this embodiment can be configured by the user according to business needs, without the need for fixed settings, so that the data processing workflow can be flexibly modified according to business needs and changes in business needs.

[0054] In the second step, in response to the second operation of the user, operators are configured for each visualization service of the data processing workflow respectively, where the operators include common operators and customized operators defined by scripts.

[0055] In this embodiment, if Figure 3 As shown, for the visualization service of each working node, use the operator list on the right to configure the visualization.

[0056] Specifically, for the "Data Storage" visualization service for Worker 1, select a public operator 1 from the operator list that has the function of storing data in a specified storage location. The storage location can be a specific address pre-set in the public operator 1. If the storage location is different from the pre-set specific address, it can also be customized and formed into a custom operator using the Worker Node Interface of Worker 1 through the script service of the Worker Node Interface. The script of the custom operator is a dynamic script, that is, a script that can be added, edited, and deleted by the user at any time. Similarly, for the "Data Cleaning" visualization service for Worker 2, a public operator with a cleaning function can be selected from the operator list, or a custom operator customized through the Worker Node Interface of Worker 2. Similarly, the "First Monitoring Value Conversion" visualization service for Worker 3, the "Second Monitoring Value Conversion" visualization service for Worker 4, and the "Monitoring Value Analysis" visualization service for Worker 5 can all be set up according to the above steps and will not be repeated here.

[0057] In an optional embodiment, each operator of each visualization service of the data processing workflow is driven by events based on a functional service architecture.

[0058] In this embodiment, each operator of each visualization service is event-driven, such as a message queue based on MQTT. When an event occurs, the producer produces a product and sends a message to the message queue. The consumer who subscribes to the message receives the message and determines whether to consume it. If so, the consumption action is started or executed.

[0059] Specifically, each operator in this embodiment adopts the Function as a Service (FaaS) architecture, which is a software architecture with more fragmented services than microservices and has the following characteristics:

[0060] 1) Each application logic unit in FaaS can be considered a function. Developers only need to focus on how to implement this logic without considering performance optimization in advance, allowing them to focus their work on this function rather than the entire application.

[0061] 2) FaaS is stateless, naturally meeting the 12-factor state requirements of cloud-native applications. Statelessness means that data in local memory or disk cannot be used by subsequent operations. Most state must be stored externally, such as in databases and network storage.

[0062] 3) FaaS functions should start and execute quickly and have a short lifecycle. They should start and process tasks within a limited time and terminate after returning a result. They should also be terminated if their execution time exceeds a certain threshold.

[0063] 4) FaaS function startup delay is affected by many factors. If the function is implemented in JS or Python, its startup time generally does not exceed 10-100 milliseconds.

[0064] In this embodiment, each operator is event-driven based on a functional service-oriented architecture. This reduces the technical requirements for developers while enabling flexible configuration and rapid startup. Operators are independent of each other and are unaffected by and undependent on previous and subsequent data processing operations.

[0065] In an optional embodiment, in response to user operations, the number of operators of the visualization service is horizontally expanded according to the amount of computing power required for the amount of business data, and the operators run in parallel.

[0066] In this embodiment, users can horizontally scale out the operators of each visualization service based on the computing power required by the volume of service data. Specifically, for the "First Monitoring Value Conversion" visualization service on worker node 3, when the amount of monitoring data from the asset under control far exceeds the computing power of the visualization service's operators, the number of operators in this visualization service can be expanded. For example, the "First Monitoring Value Conversion" visualization service is configured with a common operator 3 for converting the voltage monitoring data of the asset under control. This common operator's computing power is 300 calculations per unit time. However, if the volume of incoming monitoring data reaches 30,000 per unit time, the existing common operators in this visualization service are unable to complete the unit conversion of the incoming data in real time. In this case, the user can expand the number of common operators 3 in the visualization service on worker node 3, for example, to 100 common operators 3. Each common operator 3 performs data conversion on the incoming monitoring data in parallel to complete the unit conversion of the incoming data. Similarly, the visualization service for "Second Monitoring Value Conversion" features a public operator 4, which users can scale horizontally based on the volume of monitoring data they access. It's worth noting that because the IoT platform's backend corresponds to a cloud service, there's no need to consider operational resources. This allows for resource expansion through operator horizontal expansion, enabling flexible resource allocation.

[0067] In another optional embodiment, in response to user operations, functional splitting is performed according to the complexity of computing power required for the business data volume, and the number of operators of the visualization service is horizontally expanded according to the split functions, and each operator runs in parallel.

[0068] In this embodiment, users can configure the operators of each visualization service to scale horizontally based on the complexity of the computing power required for the business data volume. Specifically, for the visualization service of "Monitoring Value Analysis" of work node 5, when the data analysis of the monitoring data of the connected controlled assets is a highly complex function operation, such as a matrix operation, the long time required for the matrix operation seriously affects the timeliness of the monitoring data analysis. Users can split the matrix operation into multiple parallel "multiplication and addition" operations, and quickly implement the matrix operation by executing multiple "multiplication and addition" operators in parallel; at this time, users can horizontally scale the operators of the visualization service based on the number of operators required for the split.

[0069] It is worth noting that the above two embodiments can also be used simultaneously. For example, users can horizontally expand operators according to the computing power required for the amount of business data and the complexity of the computing power. Those skilled in the art should understand that the design principle is to select the appropriate type and number of operators according to actual application requirements to achieve real-time and rapid data processing, which will not be repeated here.

[0070] The third step is to process the data according to the data processing workflow in response to the received data.

[0071] In this embodiment, based on the data processing workflow configured above, the Internet of Things platform processes the monitoring data of the connected assets to be controlled according to the data processing workflow, and can quickly and efficiently process the data and obtain the processed data, so as to realize other control functions of the Internet of Things platform.

[0072] In an optional embodiment, the data processing process of the data is presented on the Internet of Things platform.

[0073] In this embodiment, if Figure 4 As shown, through the visual interface of the Internet of Things platform, the monitoring data of the assets to be controlled are processed to obtain analysis results, and the analysis results are presented for users to view more intuitively.

[0074] At this point, the configuration process and data processing process are completed.

[0075] This embodiment customizes visualization services and forms a data processing workflow through the configuration service library of the Internet of Things platform based on the service orchestration engine. The data processing method of each visualization service is determined by configuring common operators and customized operators defined by scripts, and data processing is performed on the accessed data. This has the characteristics of flexible configuration of data processing flows and processing methods, and is easy to expand and modify, effectively making up for the problems existing in the existing technology and having broad application prospects.

[0076] Corresponding to the data processing method provided in the above-mentioned embodiments, an embodiment of the present application also provides a data processing system that applies the above-mentioned data processing method. Since the data processing system provided in the embodiment of the present application corresponds to the data processing methods provided in the above-mentioned embodiments, the previous implementation method is also applicable to the data processing system provided in this embodiment and will not be described in detail in this embodiment.

[0077] like Figure 2 As shown, an embodiment of the present application further provides a data processing system applying the above data processing method, comprising: a workflow configuration unit, a visualization service configuration unit and an execution unit, wherein:

[0078] The workflow configuration unit is configured to select at least one visualization service as a work node from a pre-set configuration service library on the Internet of Things platform in response to a first operation of a user and form a data processing workflow, wherein the Internet of Things platform is based on a service orchestration engine;

[0079] The visualization service configuration unit is configured to configure operators for each visualization service of the data processing workflow in response to a second operation of the user, wherein the operators include common operators and customized operators defined by scripts;

[0080] The execution unit is configured to process the data in response to the received data according to the data processing workflow.

[0081] This embodiment customizes visualization services and forms data processing workflows through the configuration service library of an IoT platform based on a service orchestration engine. By configuring common operators and script-defined custom operators, the data processing method for each visualization service is determined, and data processing is performed on the incoming data. This allows for flexible configuration of data processing flows and methods, and is easy to expand and modify. This effectively addresses the problems of existing technologies and has broad application prospects. The specific implementation methods are described in the previous embodiments and will not be repeated here.

[0082] Another embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements: S20: in response to a first operation of a user, selecting at least one visualization service as a working node from a configuration service library pre-set on an Internet of Things platform and forming a data processing workflow, wherein the Internet of Things platform is based on a service orchestration engine; S40: in response to a second operation of the user, configuring operators for each visualization service of the data processing workflow respectively, wherein the operators include common operators and custom operators defined by scripts; S60: in response to the accessed data, processing the data according to the data processing workflow.

[0083] In practical applications, the computer-readable storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present embodiment, the computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0084] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0085] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0086] Computer program code for performing the operations of the present invention may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0087] like Figure 5 As shown, a structural diagram of a computer device provided by another embodiment of the present invention. Figure 5 The computer device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0088] like Figure 5 As shown, computer device 12 is implemented as a general-purpose computing device. Components of computer device 12 may include, but are not limited to, one or more processors or processing units 16, system memory 28, and a bus 18 that connects various system components (including system memory 28 and processing unit 16).

[0089] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. Examples of these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0090] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0091] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be configured to read and write non-removable, non-volatile magnetic media ( Figure 5 Not shown, often called a "hard drive"). Although Figure 5 Not shown, a magnetic disk drive for reading and writing to a removable non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of various embodiments of the present invention.

[0092] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. Program modules 42 generally implement the functions and / or methods of the embodiments described herein.

[0093] The computer device 12 may also communicate with one or more external devices 14 (e.g., a keyboard, a pointing device, a display 24, etc.), one or more devices that enable a user to interact with the computer device 12, and / or any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may occur through an input / output (I / O) interface 22. Furthermore, the computer device 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 20. Figure 5 As shown, the network adapter 20 communicates with the other modules of the computer device 12 via the bus 18. Figure 5 Not shown, other hardware and / or software modules may be used in conjunction with computer device 12, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0094] The processor unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing a data processing method provided by an embodiment of the present invention.

[0095] In response to the current existing problems, the present invention develops a data processing method, data processing system, computer equipment and storage medium based on the Internet of Things platform, and customizes visualization services and forms a data processing workflow through the configuration service library of the Internet of Things platform based on the service orchestration engine. The data processing method of each visualization service is determined by configuring common operators and customized operators defined by scripts, and data processing is performed on the accessed data. Therefore, the present invention has the characteristics of flexible configuration of data processing procedures and processing methods, and is easy to expand and modify, effectively making up for the problems existing in the existing technology and having broad application prospects.

[0096] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not limitations on the implementation methods of the present invention. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the implementation methods here. All obvious changes or modifications derived from the technical solution of the present invention are still within the scope of protection of the present invention.

Claims

1. A data processing method based on an Internet of Things platform, characterized in that: include: S20: In response to a first operation by the user, at least one visualization service is selected from a pre-set configuration service library on the Internet of Things platform as a working node and a data processing workflow is formed, wherein the Internet of Things platform is based on a service orchestration engine; S40: In response to the second operation of the user, respectively configure operators for each visualization service of the data processing workflow, wherein the operators include common operators and custom operators defined by scripts, further comprising: accessing a dynamic script of the custom operator through an interface of a working node corresponding to the visualization service; Horizontally expand the number of operators of the visualization service according to the amount of computing power required for the business data volume, and each operator runs in parallel; and / or, split the functions according to the complexity of the computing power required for the business data volume and horizontally expand the number of operators of the visualization service according to the split functions, and each operator runs in parallel; S60: In response to the received data, perform data processing on the data according to the data processing workflow.

2. The data processing method according to claim 1, wherein: The S20 further includes: In response to the user's first operation, at least one visualization service is selected as a work node on the workflow canvas of the Internet of Things platform and a data processing workflow is formed, wherein each operator of each visualization service of the data processing workflow is driven by an event based on the functional service architecture.

3. The data processing method according to claim 2, characterized in that: The S20 further includes: The location and execution order of the visualization services may vary.

4. The data processing method according to claim 1, wherein: S60 further includes: storing and cleaning the accessed data.

5. The data processing method according to claim 4, characterized in that: S60 further includes: The data processing process of presenting said data on the IoT platform.

6. A data processing system, characterized in that: include: Workflow configuration unit, visualization service configuration unit and execution unit, where: The workflow configuration unit is configured to select at least one visualization service as a work node from a pre-set configuration service library on the Internet of Things platform in response to a first operation of a user and form a data processing workflow, wherein the Internet of Things platform is based on a service orchestration engine; The visualization service configuration unit is configured to configure operators for each visualization service of the data processing workflow in response to a second operation of the user, wherein the operators include common operators and custom operators defined by scripts, and further includes: accessing a dynamic script of the custom operator through an interface of a work node corresponding to the visualization service; horizontally expanding the number of operators of the visualization service according to the amount of computing power required for the business data volume, with each operator running in parallel; and / or performing functional decomposition according to the complexity of the computing power required for the business data volume and horizontally expanding the number of operators of the visualization service according to the decomposition functions, with each operator running in parallel; The execution unit is configured to process the data in response to the received data according to the data processing workflow.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 5 is implemented.

Images