A method for storing intelligent terminal measurement data based on MongoDB library

Abstract

The present invention discloses a smart terminal measurement data storage method based on MongoDB library, comprising: obtaining a JSON format data file sent by a smart terminal; parsing to obtain the deviceId key value and serviceId key value of the data in the JSON format data file; determining the d5000Id attribute value of the smart terminal device associated with the data according to the deviceId key value; determining the target set of data storage according to the serviceId key value; saving the data to a document under the determined target set, and data with the same saving date, d5000Id attribute value and deviceId key value are saved in the same document under the target set, each document is respectively preset with attribute fields: saving date Date, smart terminal device number d5000Id and business number deviceId, and recording the attribute field values ​​corresponding to the data saving date, deviceId key value and serviceId key value one by one. The present invention is based on the MongoDB library smart terminal measurement data storage method, which can be applicable to the data storage requirements when multiple smart terminal data are sent, has high data storage efficiency, and is easy to expand and query.

Description

Technical Field

[0001] The present invention relates to the technical field of data collection of intelligent terminals in power distribution networks, and in particular to a method for storing intelligent terminal measurement data based on a MongoDB library. Background Art

[0002] The original D5000 platform used to store the measurement values ​​collected by the distribution network terminals in a relational database. This method resulted in a large number of columns in the data table, and with one table per day, the amount of data was also very large, which brought a lot of inconvenience to the upper-level applications in querying the measurement data.

[0003] With the further development of intelligent power grid, Internet of Things and cloud computing, data collection for low-voltage distribution and transformation substations has become particularly important. When using smart terminals to collect measurement data for low-voltage substations, the traditional D5000 platform cannot meet the concurrency of 100,000 terminals, and the traditional relational database cannot support the data sent by so many terminals. In addition, the message sent by the data collected by the smart terminal uses the json format, which is completely different from the binary method of the traditional 101 and 104 protocols. When using the traditional method to save data, it will involve cumbersome logical conversion and consume system performance. At the same time, smart terminals may install various apps to collect various data. This uncertainty also conflicts with the storage method of the traditional platform-related device model. In particular, the traditional method requires the point meter tool to configure the corresponding relationship between the measurement and the device, but for smart terminals, the point meter tool cannot meet the requirements at all.

[0004] Glossary

[0005] MongoDB is a NoSQL database based on distributed file storage, which can store documents of different structures in the same database.

[0006] DeviceId, or the end device ID, is a unique ID generated after the property management platform sends a property model to the smart terminal. It is used to identify which property model the smart terminal is bound to.

[0007] ServiceId is used to mark whether the data belongs to telesignaling data or telemetry data.

[0008] D5000Id, the smart terminal device ID, is the unique ID of the smart terminal generated by the master station system. Summary of the invention

[0009] The purpose of the present invention is to provide a smart terminal measurement data storage method based on MongoDB library, which can meet the data storage needs when multiple smart terminal data are sent, has high data storage efficiency, and is easy to expand and query.

[0010] The technical solution adopted by the present invention is: a method for storing intelligent terminal measurement data based on MongoDB library, comprising:

[0011] Get the JSON format data file sent by the smart terminal;

[0012] Parse and obtain the deviceId key value and serviceId key value of the data in the JSON format data file;

[0013] Determine the d5000Id attribute value of the smart terminal device associated with the data according to the deviceId key value; determine the target set of data storage according to the serviceId key value; wherein the data storage set includes the telesignaling data set and the telemetry data set, both of which are preset in a pre-built MongoDB database;

[0014] The data is saved in the document under the determined target collection, and the data with the same saving date, d5000Id attribute value and deviceId key value are saved in the same document under the target collection. Each document has preset attribute fields: saving date Date, smart terminal device number d5000Id and service number deviceId, and records the attribute field values ​​corresponding to the data saving date, deviceId key value and serviceId key value.

[0015] Optionally, in each document, the data is arranged in order of storage time.

[0016] Optionally, the attribute field of each document also includes the number of data points, and the attribute value of the number of data points of each document is the number of data points that have been saved in real time for the document.

[0017] Optionally, the method further includes: pre-creating MongoDB indexes for the corresponding attribute fields d5000Id, deviceId, and Date, respectively, wherein d5000Id is a primary index, deviceId is a secondary index, and Date is a tertiary index. This can improve query efficiency during data application.

[0018] Optionally, determining the d5000Id attribute value of the smart terminal device associated with the data according to the deviceId key value includes:

[0019] According to the deviceId key value, the mapping table of the preset business number deviceId and the smart terminal device number d5000Id is queried to obtain the d5000Id attribute value corresponding to the smart terminal device associated with the data.

[0020] In a second aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for storing intelligent terminal measurement data based on the MongoDB library as described in the first aspect is implemented.

[0021] Beneficial Effects

[0022] The present invention uses MongoDB library technology to provide a flexible and scalable data storage method to solve the problem faced by multiple intelligent terminals in saving measurement data. Compared with the existing distribution network platform data storage method, it solves the storage pressure of relational databases and the performance bottleneck of queries. Only simple query statements are needed to return results, while the original relational database requires table splicing. Therefore, the present invention can be applied to the data storage needs when multiple intelligent terminals upload data, the data storage efficiency can be significantly improved, and it is easy to expand and query. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 The figure is a flow chart of an embodiment of the method for storing measurement data of an intelligent terminal of the present invention. DETAILED DESCRIPTION

[0024] The invention is further described below with reference to the accompanying drawings and specific embodiments.

[0025] Example 1

[0026] This embodiment introduces a method for storing intelligent terminal measurement data based on MongoDB library. Figure 1 The method comprises:

[0027] Get the JSON format data file sent by the smart terminal;

[0028] Parse and obtain the deviceId key value and serviceId key value of the data in the JSON format data file;

[0029] Determine the d5000Id attribute value of the smart terminal device associated with the data according to the deviceId key value; determine the target set of data storage according to the serviceId key value; wherein the data storage set includes the telesignaling data set and the telemetry data set, both of which are preset in a pre-built MongoDB database;

[0030] The data is saved in the document under the determined target collection, and the data with the same saving date, d5000Id attribute value and deviceId key value are saved in the same document under the target collection. Each document has preset attribute fields: saving date Date, smart terminal device number d5000Id and service number deviceId, and records the attribute field values ​​corresponding to the data saving date, deviceId key value and serviceId key value.

[0031] When the method of this embodiment is applied, it is first necessary to design the storage architecture and data storage model of the MongoDB database, as follows.

[0032] According to the characteristics of the JSON data sent by the smart terminal, the basic measurement data will have the key serviceId, whose value can be the English identifier of basic telemetry and telesignaling such as analog and discrete. Therefore, a collection of MongoDB database can be pre-built according to the serviceId key value of different data sent by the terminal, so that different types of data can be saved in different collections.

[0033] The following is a data format collected at eventTime and sent by the terminal in an application example. It records the serviceId key value and the deviceId key value, and includes all measurement items and measurement values ​​recorded by the terminal device, such as PhV_phsA identifies the A-phase voltage and A_phsC identifies the C-phase current, as defined in relevant standards.

[0034]

[0035] After obtaining the above JSON format data file, this embodiment can parse the serviceId key value and the deviceId key value, and according to the known binding relationship between the smart terminal device and the end device, the smart terminal device ID associated with the data, i.e., d5000Id, can be obtained. Specifically, the query and acquisition of d5000Id can be realized by presetting a mapping table between deviceId and d5000Id in the computer.

[0036] The target collection for data storage can be determined based on the serviceId key value.

[0037] In order to save space for data storage and improve the efficiency of data query at a specified time, this embodiment adopts a method of distinguishing data documents according to the saving time when designing the data storage model. The data saved on the same day is saved in an array form in a document of the collection, and the logic of data saving is set to arrange in order of saving time.

[0038] Specifically, the attribute fields of each document include the save date Date, the smart terminal device number d5000Id, the business number deviceId, and the number of data points Transaction_count. The data point number attribute value of each document is the number of data points that the document has saved in real time. The following is an example of a document in a collection, where measurements is an array, and each {*} under it saves all the measurement data sent by the terminal device at a time point. There can be multiple {*} under measurements, which are preferably sorted in order according to the save time.

[0039]

[0040] In order to facilitate data retrieval in application and improve retrieval efficiency, in this embodiment, MongoDB indexes are pre-created for the corresponding attribute fields d5000Id, deviceId and Date, respectively, where d5000Id is a primary index, deviceId is a secondary index, and Date is a tertiary index.

[0041] Compared with obtaining data from a relational database and maintaining key names, the MongoDB library data stored based on JSON data in this embodiment can more conveniently provide more flexible query results for query programs, especially for web programs, which can effectively reduce serialization operations and break through the limitations of entity classes to achieve dynamic insertion and increase or decrease of return fields.

[0042] Example 2

[0043] This embodiment introduces a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the intelligent terminal measurement data storage method based on the MongoDB library as described in Embodiment 1 is implemented.

[0044] In summary, the method of the present invention can solve the storage pressure and query performance bottleneck of relational databases. For example, when querying two days of data, only a simple query statement is needed for the new design to return the result, while the original relational database needs to splice the table. Especially when performing simple statistical queries on data, for example, to know whether the data of a certain day has lost points, the transaction_count in the storage model can be immediately queried, while the traditional relational database storage method needs to query all columns and perform conditional judgment on all columns. The present invention can also use the map-reduce function provided by mongodb to perform some simple data statistics functions, and has strong scalability.

[0045] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0046] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0047] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0048] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0049] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A method for storing intelligent terminal measurement data based on MongoDB library, characterized in that: include: Get the JSON format data file sent by the smart terminal; Parse and obtain the deviceId key value and serviceId key value of the data in the JSON format data file; Determine the d5000Id attribute value of the smart terminal device associated with the data according to the deviceId key value; determine the target set of data storage according to the serviceId key value; wherein the data storage set includes the telesignaling data set and the telemetry data set, both of which are preset in a pre-built MongoDB database; The data is saved in the document under the determined target collection, and the data with the same saving date, d5000Id attribute value and deviceId key value are saved in the same document under the target collection. Each document has preset attribute fields: saving date Date, smart terminal device number d5000Id and service number deviceId, and records the attribute field values ​​corresponding to the data saving date, deviceId key value and serviceId key value.

2. The method according to claim 1, characterized in that: In each of the documents, the data is arranged in the order of storage time.

3. The method according to claim 1, characterized in that: The attribute field of each document also includes the number of data points, and the attribute value of the number of data points of each document is the number of data points that have been saved in the document in real time.

4. The method according to claim 1, characterized in that include: MongoDB indexes are pre-created for the corresponding attribute fields d5000Id, deviceId, and Date, where d5000Id is the primary index, deviceId is the secondary index, and Date is the tertiary index.

5. The method according to claim 1, characterized in that: Determining the d5000Id attribute value of the smart terminal device associated with the data according to the deviceId key value includes: According to the deviceId key value, the mapping table of the preset business number deviceId and the smart terminal device number d5000Id is queried to obtain the d5000Id attribute value corresponding to the smart terminal device associated with the data.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the intelligent terminal measurement data storage method based on the MongoDB library as described in any one of claims 1 to 5 is implemented.

Images