Heterogeneous equipment data processing system and method based on intelligent micro-grid

Abstract

The invention relates to the technical field of intelligent gateways, and discloses a heterogeneous equipment data processing system and method based on an intelligent micro-grid, and the system comprises a template library, a data communication layer, a data analysis layer and a data conversion layer. The template library comprises a general format template, an uplink data template, a downlink data template and an application template; the data communication layer is used for establishing communication between equipment sides, and the data analysis layer calls a general format template to convert equipment operation data into a general format; the data conversion layer calls an uplink data template and an application template to convert the uplink data template and the application template into a required data format; manual intervention in the conversion process is simplified, the equipment access and data processing capability is integrally improved, and the operation stability of the intelligent micro-grid is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent gateways, and in particular to a heterogeneous device data processing system and a data processing method based on an intelligent microgrid. Background Art

[0002] At present, the microgrid system is connected to different types of equipment, such as concentrators, energy storage equipment, photovoltaic equipment, controllers, inspection instruments, smart meters, and multi-function meters. Since the equipment comes from different manufacturers, it has different communication protocols, data formats, and data structures. In this heterogeneous equipment scenario, different devices need to be remotely controlled, which leads to high complexity of data collection and data integration, and consumes a lot of resources for data integration and analysis.

[0003] Due to the lack of effective data conversion templates in the data processing process, the processing system needs to spend a lot of time to adapt and convert data when receiving different devices, and each update of the data conversion template needs to trigger the processing system to redeploy data identification and data conversion methods. This way of collecting and processing data is complex for microgrids with a large number of heterogeneous devices, and the ability to access devices is low.

[0004] In addition, even if the equipment is of the same type in a microgrid, the same function parameters may be inconsistent due to different manufacturers. For example, if they are all three-phase meters, the parameters of the opening and closing functions may be different if the manufacturers are different. This situation will also lead to deviations in the development and testing process, and ultimately lead to problems such as the inability to collect data or data conversion errors. Summary of the invention

[0005] The present invention aims to provide a heterogeneous device data processing system based on a smart microgrid, and a method for converting heterogeneous device data in a microgrid, thereby solving the problem of complex data processing for different types of devices in a microgrid.

[0006] To achieve the above object, the present invention adopts the following technical scheme: a heterogeneous device data processing system based on a smart microgrid, comprising a template library, a data communication layer, a data parsing layer and a data conversion layer; the template library comprises a general format template, an uplink data template, a downlink data template and an application template; In the uplink phase of collecting device data, the data communication layer is used to establish communication between device sides and maintain the stability and reliability of communication with the devices, playing the role of sending and receiving data; The data parsing layer receives the data packets forwarded by the data communication layer, decodes them according to the communication protocols supported by different devices, extracts the device operation data in the data packets, calls the universal format template, and converts the device operation data into the universal format by filling in the universal format template content; The data conversion layer receives the device operation data parsed by the data parsing layer, calls the uplink data template, and extracts data content that meets the requirements of the uplink data template from the device operation data; then verifies the uplink data template, and then fills the verified data into the corresponding application template according to the application template required by the third-party business system, and converts it into a data format that meets the requirements of the third-party business system; In the downlink stage where data is sent to the device, the data conversion layer receives the request instruction data for the device function from the third-party business system, calls the downlink data template that matches the corresponding device according to the device address in the request instruction data, extracts the data content that matches the downlink data template from the instruction, and then sends the downlink data template including the request instruction data to the data parsing layer; The data parsing layer calls the general format template to convert the request instruction data into a general format, obtains the protocol corresponding to the device type according to the device type information included in the request instruction data, encodes the request instruction data according to the protocol, and finally sends the encoded request instruction data to the data communication layer; The data communication layer sends the encoded request instruction data to the device side.

[0007] The beneficial effect of this heterogeneous device data processing system is that by pre-storing data general templates, uplink data templates, downlink data templates and application templates in the template library, the device data of various different interfaces and protocol formats can be converted by calling the corresponding data templates, which can ensure the consistency of data conversion for different devices. When the data content needs to be changed, it is only necessary to modify the template structure and content stored in the template library, which really increases the scalability of this system. The traditional operation process method that requires multiple sets of format conversions according to the preset equipment is changed to the method of calling templates, which simplifies the conversion process and reduces manual intervention, improves the overall equipment access and data processing capabilities, and improves the operation stability of the smart microgrid.

[0008] Furthermore, it also includes a rule base, which stores general format mapping rules, uplink data format mapping rules, downlink data mapping rules and application mapping rules, and completes data format conversion by calling mapping rules and corresponding data templates.

[0009] Furthermore, the built-in content of the general format template includes a message ID, a timestamp, a device type, an operation type, a submodule ID and a device content table.

[0010] Furthermore, the data format used by the general format template is JSON format, which is a lightweight data exchange format that is easy for people to read and write, and is also easy for machines to parse and generate, which is beneficial to the operating speed of the system.

[0011] Furthermore, the content of the uplink data template configuration includes the device type, the parameters of the device to be collected, and the ID of the associated application template.

[0012] Furthermore, the content of the downlink data template configuration includes a device ID, an instruction for requesting to read the device status, and a field mapping of confirmation parameters sent to the device.

[0013] Furthermore, the data communication layer includes a session management module, which is used to maintain the communication between the two parties, directly reply to the heartbeat message with a confirmation, and forward other types of messages to the data analysis layer for processing.

[0014] In order to cooperate with the heterogeneous device data processing system, the present solution also provides a heterogeneous device data processing method for a smart microgrid, including an uplink data processing method and a downlink data processing method; The uplink data processing method comprises the following steps: Step S1: The device establishes a communication connection with the data communication layer through a wireless network, maintains the communication between the two parties through the session management module in the data communication layer, directly replies to confirm the heartbeat packet, and forwards other types of messages to the data analysis layer for processing; Step S2: After receiving the data from the data communication layer, the data parsing layer decodes the data according to the communication protocols supported by different devices and extracts the device operation data in the data packet; Step S3: calling the general format template and the general format mapping rule, filling in the general format template content, and running the general format mapping rule script to convert the device operation data into the general format; Step S4: calling the uplink data template, and calling the uplink data mapping rule script according to the device ID, running the uplink data mapping rule script, and converting the device operation data into data content that meets the requirements of the uplink data template; Step S5: according to the requirements of the third-party business system, call the application template to convert the uplink data into a data format that meets the requirements of the third-party business system.

[0015] The downlink data processing method comprises the following steps: Step A1: The data conversion layer receives the request instruction data of the third-party business system for the device function; Step A2: according to the device address in the request instruction data, call the downlink data template that matches the corresponding device, extract the data content that matches the downlink data template from the instruction, call the downlink data mapping script from the template library according to the device type, run the downlink data mapping script, and convert the instruction data into a data format that matches the downlink data template; Step A3: Call the general format template to convert the request instruction data into a general format. Step A4: according to the device type information included in the request instruction data, obtain the protocol corresponding to the device type, and encode the request instruction data through the device protocol of the receiving data to form a data format that can be read by the device; Step A5: Send the encoded request instruction data to the data communication layer, and the data communication layer sends the encoded request instruction data to the device side.

[0016] Furthermore, after step S3, when the equipment operation data is completed in a common format, the data needs to be corrected, and incomplete data is marked as incomplete, and obvious error values ​​are marked as errors. Completing the data verification in this step helps R&D personnel to find data problems in time and correct them.

[0017] Finally, in step A5, the data communication layer verifies the validity of the encoded data. If it is valid, it is directly sent to the device side, otherwise the message is discarded. Invalid data is discarded before it is sent to avoid unnecessary waste of transmission resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a schematic diagram of the structure of a heterogeneous device data processing system of a smart microgrid in Example 1; Figure 2 This is a logic flow chart for data processing in Example 2. DETAILED DESCRIPTION

[0019] The following is further described in detail through specific implementation methods: Example 1 like Figure 1 As shown, a heterogeneous device data processing system based on a smart microgrid includes a template library 4, a rule library 5, a data communication layer 1, a data parsing layer 2 and a data conversion layer 3; the template library 4 includes a general format template, an uplink data template, a downlink data template and an application template; the rule library 5 stores a general format mapping rule, an uplink data format mapping rule, a downlink data mapping rule and an application mapping rule, and completes the data format conversion by calling the mapping rule and the corresponding data template.

[0020] In the uplink phase of collecting device data, data communication layer 1 is used to establish communication between device sides and maintain the stability and reliability of communication with devices, playing the role of sending and receiving data.

[0021] Data parsing layer 2 receives the data packets forwarded by the data communication layer, decodes them according to the communication protocols supported by different devices, extracts the device operation data in the data packets, calls the general format template, and converts the device operation data into a general format by filling in the general format template content; the built-in content of the general format template includes message ID, timestamp, device type, operation type, submodule ID and device content table. The data format used by the general format template is JSON format.

[0022] The data conversion layer 3 receives the device operation data parsed by the data parsing layer 2, calls the uplink data template, and extracts the data content that meets the requirements of the uplink data template from the device operation data; the content configured by the uplink data template includes the device type, the parameters of the device to be collected, and the ID of the associated application template.

[0023] The data conversion layer 3 verifies the uplink data template, and then converts the verified data into a data format that meets the requirements of the third-party business system by filling in the corresponding application template according to the application template required by the third-party business system.

[0024] The downlink stage when data is sent to the device side: the data conversion layer 3 receives the request instruction data for the device function from the third-party business system, calls the downlink data template that matches the corresponding device according to the device address in the request instruction data, extracts the data content that matches the downlink data template from the instruction, and then sends the downlink data template including the request instruction data to the data parsing layer 2; the content of the downlink data template configuration includes the device ID, the instruction to request to read the device status, and the field mapping of the confirmation parameters sent to the device.

[0025] The data parsing layer 2 calls the universal format template to convert the request instruction data into a universal format, obtains the protocol corresponding to the device type according to the device type information included in the request instruction data, encodes the request instruction data according to the protocol, and finally sends the encoded request instruction data to the data communication layer; the data communication layer sends the encoded request instruction data to the device side.

[0026] Embodiment 2: This implementation uses the example of connecting a smart meter to a data processing system to specifically illustrate the implementation steps of the heterogeneous device data processing method based on a smart microgrid.

[0027] Configure the corresponding data templates in the template library: General format template: After decoding or encoding the data uploaded by the meter, its data format is JSON format. Uplink data template: Configure the data type to be the meter curve. The parameters of the device to be collected include current, voltage, power, and the associated application template ID. Downlink data template: Configure the device ID required by the third-party business system to send the internal address information of the meter, the instruction to request to read the device status is to read the status of the meter relay, and the field mapping sent to the meter to confirm the parameters includes the meter number and data identifier. Application template: Configure the data structure required by the third party, including the data format and data storage method.

[0028] like Figure 2 As shown, a heterogeneous device data processing method based on a smart microgrid includes an uplink data processing method and a downlink data processing method.

[0029] The uplink data processing method comprises the following steps: Step S1: The smart meter establishes a communication connection with the data communication layer through the wireless network, maintains the communication between the two parties through the session management module in the data communication layer, directly replies to confirm the heartbeat packet, and forwards other types of messages to the data analysis layer for processing.

[0030] Step S2: The data analysis layer receives the curve data actively reported by the smart meter from the data communication layer, decodes it according to the communication protocol supported by the meter, and extracts the current, voltage, and power information of the meter in the data packet.

[0031] Step S3: Call the general format template and the general format mapping rule, fill in the general format template content, and run the general format mapping rule script to convert the device operation data into the general format JSON format.

[0032] Step S4: call the uplink data template, and call the uplink data mapping rule script according to the meter ID, run the uplink data mapping rule script, and convert the meter operation data into data content that meets the requirements of the uplink data template; in the uplink data template, the current of the smart meter is decomposed into phase A, phase B and phase C currents, and the voltages of phase A, phase B and phase C as well as active power and reactive power are also extracted respectively.

[0033] If only phases A and B have values ​​in the current data of the smart meter, and phase C is empty, phase C needs to be marked as incomplete according to the configuration. If the configured voltage threshold is 220V, but the reported voltage data is 380V, it is an obvious error and the voltage data should be marked as an error.

[0034] Step S5: according to the requirements of the third-party business system, call the application template to convert the uplink data into a data format that meets the requirements of the third-party business system.

[0035] The method for processing downlink data includes the following steps: Step A1: The data conversion layer receives the function request instruction data of the third-party business system to read the relay status of the smart meter; Step A2: According to the device address in the request instruction data, call the downlink data template matched by the smart meter, and extract the data content that matches the downlink data template from the instruction, including the meter number and data identifier. The downlink data template configures relevant fields and verifies the field type, field length and non-emptiness. Call the downlink data mapping script from the template library, run the downlink data mapping script, and convert the instruction data into a data format that matches the downlink data template.

[0036] Step A3: calling the general format template to convert the request instruction data into a general format; Step A4: according to the device type information included in the request instruction data, the protocol corresponding to the smart meter is obtained, and the request instruction data is encoded by the smart meter protocol of the received data to form a data format that can be read by the device; Step A5: Send the encoded request instruction data to the data communication layer. First, verify the validity of the session. If it is valid, send it directly to the device side. Otherwise, discard the message. The data communication layer sends the valid request instruction data to the device side.

[0037] The above is only an embodiment of the present invention, and the common knowledge such as the known specific technical solutions and / or characteristics in the solution is not described in detail here. It should be pointed out that for those skilled in the art, without departing from the technical solution of the present invention, several modifications and improvements can be made, which should also be regarded as the protection scope of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A heterogeneous device data processing system based on a smart microgrid, characterized by: It includes a template library, a data communication layer, a data parsing layer and a data conversion layer; the template library includes a general format template, an uplink data template, a downlink data template and an application template; In the uplink phase of collecting device data, the data communication layer is used to establish communication between device sides and maintain the stability and reliability of communication with the devices, playing the role of sending and receiving data; The data parsing layer receives the data packets forwarded by the data communication layer, decodes them according to the communication protocols supported by different devices, extracts the device operation data in the data packets, calls the universal format template, and converts the device operation data into the universal format by filling in the universal format template content; The data conversion layer receives the device operation data parsed by the data parsing layer, calls the uplink data template, and extracts data content that meets the requirements of the uplink data template from the device operation data; Then, the uplink data template is verified, and then, according to the application template required by the third-party business system, the verified data is filled in the corresponding application template and converted into a data format that meets the requirements of the third-party business system; In the downlink stage where data is sent to the device, the data conversion layer receives the request instruction data for the device function from the third-party business system, calls the downlink data template that matches the corresponding device according to the device address in the request instruction data, extracts the data content that matches the downlink data template from the instruction, and then sends the downlink data template including the request instruction data to the data parsing layer; The data parsing layer calls the general format template to convert the request instruction data into a general format, obtains the protocol corresponding to the device type according to the device type information included in the request instruction data, encodes the request instruction data according to the protocol, and finally sends the encoded request instruction data to the data communication layer; The data communication layer sends the encoded request instruction data to the device side.

2. According to claim 1, a heterogeneous device data processing system based on a smart microgrid is characterized by: It also includes a rule base, which stores general format mapping rules, uplink data format mapping rules, downlink data mapping rules and application mapping rules. By calling the mapping rules and corresponding data templates, the data format conversion is completed.

3. The heterogeneous device data processing system based on smart microgrid according to claim 1 is characterized by: The built-in content of the general format template includes a message ID, a timestamp, a device type, an operation type, a submodule ID and a device content table.

4. The heterogeneous device data processing system based on smart microgrid according to claim 1 is characterized by: The data format used by the general format template is JSON format.

5. The heterogeneous device data processing system based on smart microgrid according to claim 4 is characterized by: The content of the uplink data template configuration includes the device type, the parameters of the device to be collected, and the ID of the associated application template.

6. The heterogeneous device data processing system based on smart microgrid according to claim 5 is characterized by: The content of the downlink data template configuration includes a device ID, an instruction requesting to read the device status, and a field mapping of confirmation parameters sent to the device.

7. The heterogeneous device data processing system based on smart microgrid according to claim 6 is characterized by: The data communication layer includes a session management module, which is used to maintain the communication between the two parties, directly reply to the heartbeat message with confirmation, and forward other types of messages to the data analysis layer for processing.

8. A method for processing heterogeneous device data based on a smart microgrid, and a system for processing heterogeneous device data based on a smart microgrid according to claim 7, characterized in that: Including uplink data processing method and downlink data processing method; The uplink data processing method comprises the following steps: Step S1: The device establishes a communication connection with the data communication layer through a wireless network, maintains the communication between the two parties through the session management module in the data communication layer, directly replies to confirm the heartbeat packet, and forwards other types of messages to the data analysis layer for processing; Step S2: After receiving the data from the data communication layer, the data parsing layer decodes the data according to the communication protocols supported by different devices and extracts the device operation data in the data packet; Step S3: calling the general format template and the general format mapping rule, filling in the general format template content, and running the general format mapping rule script to convert the device operation data into the general format; Step S4: calling the uplink data template, and calling the uplink data mapping rule script according to the device ID, running the uplink data mapping rule script, and converting the device operation data into data content that meets the requirements of the uplink data template; Step S5: according to the requirements of the third-party business system, call the application template to convert the uplink data into a data format that meets the requirements of the third-party business system. The downlink data processing method comprises the following steps: Step A1: The data conversion layer receives the request instruction data of the third-party business system for the device function; Step A2: according to the device address in the request instruction data, call the downlink data template that matches the corresponding device, extract the data content that matches the downlink data template from the instruction, call the downlink data mapping script from the template library according to the device type, run the downlink data mapping script, and convert the instruction data into a data format that matches the downlink data template; Step A3: Call the general format template to convert the request instruction data into a general format. Step A4: according to the device type information included in the request instruction data, obtain the protocol corresponding to the device type, and encode the request instruction data through the device protocol of the receiving data to form a data format that can be read by the device; Step A5: Send the encoded request instruction data to the data communication layer, and the data communication layer sends the encoded request instruction data to the device side.

9. The method for processing heterogeneous device data based on a smart microgrid according to claim 8, characterized in that: After step S3, when the general format of the equipment operation data is completed, the data needs to be corrected, and the incomplete data is marked as incomplete, and the obviously wrong value is marked as wrong.

10. The method for processing heterogeneous device data based on a smart microgrid according to claim 9, characterized in that: In step A5, the data communication layer verifies the validity of the encoded data. If it is valid, it is directly sent to the device side, otherwise the message is discarded.

Images