Data publishing method and system and storage medium
Export data from the first MongoDB database through the MongoDB server and transmit it to the second MongoDB database at the industrial site, perform integrity checksum conversion, solve the security and efficiency problems in data transmission, and realize the secure transmission and efficient release of data. .
Patent Information
- Application Number
- CN202411135097.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-19
- Publication Date
- 2024-11-19
AI Technical Summary
There is a risk of data leakage and tampering in industrial sites. Existing methods cannot effectively solve these problems. Traditional carrying of storage devices is time-consuming and labor-intensive and inefficient.
Export data files from the first MongoDB database through a MongoDB server and transmit them to the second MongoDB database at the industrial site using the network to perform data integrity checksum conversion to ensure data security and efficiency.
It has achieved the security and efficiency improvement of data transmission, avoided the risks of data tampering and leakage, reduced the demand for artificially carried storage equipment, and improved the operating efficiency of industrial sites.
Smart Images

Figure CN118981794A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of data publishing, and more specifically to a data publishing method, system and storage medium. Background Art
[0002] At present, for the various data required for industrial field services, engineers usually carry mobile hard disks, external storage devices or USB drives to the industrial site and connect them to the on-site computer equipment for transmission. This is time-consuming and labor-intensive, has great limitations, and there is a risk of data leakage due to the loss of data storage devices.
[0003] In order to improve the above problems, it is proposed to transmit the data to cloud storage services or internal enterprise servers through the network for download and use by on-site computers. This eliminates the need for manual data storage devices to be carried to the industrial site, saving time and effort and improving the operating efficiency of the industrial site. However, this still cannot avoid the risk of data being tampered with at the industrial site. Summary of the invention
[0004] In view of the above problems, this application provides the following solutions:
[0005] A first aspect of the present application provides a data publishing method, the data publishing method comprising:
[0006] Acquire a data file to be published to the industrial site and a corresponding data publishing message; the data file includes engineering personnel publishing data exported by the first MongoDB server from the first MongoDB database;
[0007] Transmitting the data file to the industrial site to be imported into a second MongoDB database of the industrial site for storage;
[0008] Performing integrity check on the data file imported into the second MongoDB database according to the data publishing message;
[0009] If the verification is passed, the data file is converted into target data required by the industrial site.
[0010] In a possible implementation, performing integrity verification on the data file imported into the second MongoDB database according to the data publishing message includes:
[0011] Reading a task identifier for the data file from the second MongoDB database;
[0012] If it is determined that the task identifier exists in the data publishing message, reading service information associated with the data file from the second MongoDB database;
[0013] Performing an integrity check on the service information using at least the task identifier;
[0014] If the service information is complete, reading process information associated with the service information from the second MongoDB database;
[0015] The process information is at least checked for integrity using the service information.
[0016] In a possible implementation, performing integrity check on the service information by at least using the task identifier includes:
[0017] Determining whether the task identifier included in the service information is the same as the task identifier in the data publishing message;
[0018] The at least using the service information to perform integrity check on the process information includes:
[0019] It is determined whether the service identifier included in the process information is the same as the service identifier included in the service information.
[0020] In a possible implementation, the data publishing method further includes:
[0021] If the verification fails, prohibiting conversion of the data file imported into the second MongoDB database;
[0022] Wherein, the verification failure includes any of the following:
[0023] Determining that the task identifier does not exist in the data publishing message;
[0024] Determining that the task identifier exists in the data publishing message, but the task identifier included in the service information is different from the task identifier in the data publishing message;
[0025] It is determined that the task identifier exists in the data publishing message, and the task identifier included in the service information is the same as the task identifier in the data publishing message, but the service identifier included in the process information is different from the service identifier included in the service information.
[0026] In a possible implementation, converting the data file into target data required by the industrial site includes:
[0027] Reading and verifying the complete data file from the second MongoDB database;
[0028] Based on the constructed data conversion model, the data file is converted into target data in a specified communication protocol format required by the industrial site.
[0029] In a possible implementation, the construction of the data conversion model includes:
[0030] Perform extensible markup language modeling based on the specified communication protocol supported by the industrial site to obtain a corresponding data conversion model;
[0031] The step of converting the data file into target data in a specified communication protocol format required by the industrial site based on the constructed data conversion model includes:
[0032] Writing the data contained in the data file into the data conversion model to obtain a data structure supported by the specified communication protocol;
[0033] The data structure is written into an extensible markup language file to obtain target data required by the industrial site.
[0034] In a possible implementation, the transmitting the data file to the industrial site includes any of the following implementations:
[0035] Invoke a hypertext transfer protocol service of the industrial site to upload the data file to a second MongoDB server of the industrial site;
[0036] Invoke the encrypted file transfer protocol service of the industrial site, and transfer the data file to a communication server supporting the encrypted file transfer protocol service, so that the second MongoDB server of the industrial site downloads the data file from the communication server;
[0037] The second MongoDB server imports the received data file into a second MongoDB database at the industrial site for storage.
[0038] In a possible implementation, the first MongoDB server exports the data published by the engineering personnel, including:
[0039] The first MongoDB server calls the MongoDB data backup tool, and after security authentication, exports the first MongoDB database data released by the engineering staff for the industrial site into a data file in an archive format;
[0040] The second MongoDB server imports the received data file into the second MongoDB database of the industrial site for storage, including:
[0041] The second MongoDB server calls a MongoDB data recovery tool, and after security authentication, imports the data file into the second MongoDB database of the industrial site for storage.
[0042] A second aspect of the present application provides a data publishing system, the data publishing system comprising: a first MongoDB server, a first MongoDB database, and a business server deployed for an industrial site, a second MongoDB server, and a second MongoDB database, wherein:
[0043] The first MongoDB database is used to store data published by engineers;
[0044] The first MongoDB server is used to export the data published by the engineering personnel in the first MongoDB database into a data file;
[0045] The business server is used to obtain the data file to be published to the industrial site and the corresponding data distribution message, and transmit the data file to the second MongoDB server at the industrial site;
[0046] The second MongoDB server is used to import the received data file into the second MongoDB database for storage;
[0047] The business server is further used to perform integrity verification on the data file imported into the second MongoDB database based on the data publishing message, and if the verification is complete, convert the data file into target data required by the industrial site.
[0048] A third aspect of the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by a business server, the business server can implement the data publishing method as described above.
[0049] It can be seen that in the data publishing method provided by the present application, after the engineering personnel publish the data required for the industrial site to the first MongoDB database, the first MongoDB server will promptly export it as a data file and send it to the business server at the industrial site. The business server obtains the data file to be published and the corresponding data publishing message, publishes the data file to the industrial site, and imports it into the second MongoDB database of the industrial site for storage. Then, based on the data publishing message, the data file imported into the second MongoDB database is integrity checked, and after determining that the check passes, the data file is converted into the target data required by the industrial site, thereby ensuring the security of data publishing at the industrial site. Compared with the transmission method in which engineering personnel carry data storage devices to the industrial site, the data publishing efficiency is improved, and the timeliness of data publishing is ensured. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.
[0051] Figure 1 A flowchart of a data publishing method proposed in Embodiment 1 of the present application;
[0052] Figure 2 A flowchart of a data publishing method proposed in Embodiment 2 of the present application;
[0053] Figure 3 A flowchart of a data publishing method proposed in Embodiment 3 of the present application;
[0054] Figure 4 A schematic diagram of the structure of the data publishing device proposed in the embodiment of the present application;
[0055] Figure 5 This is a schematic diagram of the hardware structure of the data publishing system proposed in Example 1 of the present application;
[0056] Figure 6 The present invention is a schematic diagram of an optional hardware structure of a business server applicable to the data publishing method proposed in the embodiment of the present application. DETAILED DESCRIPTION
[0057] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation mode of the present application are only used to explain the specific embodiments of the present application and are not intended to limit the present application. The embodiments of the present application are described below in conjunction with the drawings. It is known to those of ordinary skill in the art that with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.
[0058] The terms "first", "second" etc. in the full text of the application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable in appropriate circumstances, and this is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the application. Belonging to "if" can be interpreted as "at the time of ... " or "when ... " or "in response to determining" etc., can be understood according to the context. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusion, so that the process, method, product or equipment comprising a series of units need not be limited to those units, but can include other units that are not clearly listed or inherent to these processes, methods, products or equipment.
[0059] In order to meet the requirement of industrial sites that the data in the operating environment cannot be modified, it is necessary to achieve the requirement of data release isolation. In the implementation of transmitting data to the industrial site through the network, the data required by the industrial site must be pushed to the industrial environment through data release. In order to improve the security and efficiency of data release, this application proposes a new data release method, that is, data release based on MongoDB (a database based on distributed file storage), combined with the security authentication mechanism of the MongoDB server, to improve data release security and release efficiency. The data release method of the embodiment of this application will be described in detail below in conjunction with the accompanying drawings.
[0060] Reference Figure 1 , is a flow chart of a data publishing method proposed in Embodiment 1 of the present application, which can be applied to business servers in any industrial site, such as Figure 1 As shown, the data publishing method may include but is not limited to the following steps:
[0061] Step S101, obtaining a data file to be published to the industrial site and a corresponding data publishing message; the data file includes engineering personnel publishing data exported by a first MongoDB server from a first MongoDB database;
[0062] For various data required for any industrial site operating environment, such as production data, equipment status and health data, production plan data, and engineering measurement data, relevant engineering personnel can publish them to the MongoDB database in the network for the industrial site based on actual needs. It can be the engineer's local MongoDB database or a MongoDB database specified from the MongoDB database cluster, which can be recorded as the first MongoDB database to distinguish it from the MongoDB database at the engineering site.
[0063] In order to improve the efficiency and security of data publishing, the MongoDB server that supports the security authentication mechanism can export the published data stored in the first MongoDB database into a data file such as an archive file in a timely manner. In order to improve the transmission efficiency, the data file can be a compressed file of the data to be published to the industrial site, and even an encrypted compression method can be used to encrypt and compress the exported data into an archive file. This application does not limit the implementation method of the encryption compression and how to export the archive file from the first MongoDB database.
[0064] Among them, for engineering personnel to publish data, it can be determined based on the data publishing task requirements and the service requirements for publishing it to the industrial site. After the engineering personnel publish the data, the task identifier of the corresponding data publishing task (such as task ID) and the service information used to implement industrial site services can be recorded, such as service name, service description, service ID, service IP address and service deployment method (such as single host single instance method or high availability deployment architecture method, etc.) and other information content. This application does not restrict the task identifier of each data publishing task and its service information.
[0065] Based on this, in order to facilitate the subsequent data security and integrity verification of the industrial site, after the data file is exported, the data release message can be sent to the industrial site to notify each industrial field device of the task of releasing data this time, which can be represented by a unique task identifier. Therefore, the data release message of this application is used to indicate the task status of the data released this time, and this application does not limit its content.
[0066] Step S102, transmitting the data file to the industrial site to be imported into a second MongoDB database at the industrial site for storage;
[0067] In an embodiment of the present application, the data files exported from the first MongoDB database can be transferred to the Mongo server at the industrial site through network communication in accordance with the corresponding communication protocol, which is recorded as the second MongoDB server. The second MongoDB server directly imports the received data files into the second MongoDB database at the industrial site for storage for subsequent reading. It can be seen that the data required by the industrial site does not require engineers to carry data storage devices (such as mobile hard disks or external storage devices or USB drives) storing the published data to the industrial site, which saves labor costs and improves data publishing efficiency. This application does not limit the type of network communication.
[0068] Step S103, performing integrity check on the data file imported into the second MongoDB database according to the data publishing message;
[0069] Step S104: if the verification passes, convert the data file into target data required by the industrial site.
[0070] Combined with the above description of the data release message, in order to ensure that the data required for release to the industrial site is correct and meet the business needs of the industrial site operating environment, for the data imported into the second MongoDB database of the industrial site, the data can be released based on the data release message for this data distribution task. The data integrity check is performed to determine whether the data imported into the second MongoDB database is data released by engineering personnel to avoid data tampering. Of course, the present application can also verify information such as industrial site services called based on the data release task to ensure that the data can reliably and accurately meet the actual needs of industrial site equipment and avoid the execution of tampered data posing a threat to the industrial site environment. The present application does not limit the implementation method of the integrity verification of step S103.
[0071] After the above data security and integrity check, the data file that passes the check can be converted and processed according to the data format of the data required by the industrial site, the communication protocol followed, etc. Therefore, the target data required by the industrial site can be data that meets the communication protocol followed by the communication interface of the industrial field equipment, as well as the requirements for the input / response data format, etc. This application does not limit the data type of the target data.
[0072] To summarize, in the embodiment of the present application, after the engineering personnel publish the data required for the industrial site to the first MongoDB database, the first MongoDB server will promptly export the data from the first MongoDB database as a data file and send it to the business server at the industrial site. The business server obtains the data file to be published and the corresponding data publishing message, publishes the data file to the industrial site, and imports it into the second MongoDB database of the industrial site for storage. Based on the data publishing message, an integrity check is performed on the data file imported into the second MongoDB database. After the check is confirmed to pass, the data file is converted into the target data required by the industrial site, thereby ensuring the security of data publishing at the industrial site. Compared with the transmission method in which engineering personnel carry data storage devices to the industrial site, the data publishing efficiency is improved and the timeliness of data publishing is ensured.
[0073] Reference Figure 2 , is a flow chart of a data publishing method proposed in the second embodiment of the present application. This embodiment can provide a detailed description of the integrity verification process of the MongoDB database data imported into the industrial site proposed above. For other implementation steps of the data publishing method, please refer to the description of the corresponding part of the context embodiment, which will not be described in detail in this embodiment. Figure 2 As shown, the data publishing method proposed in this embodiment may include:
[0074] Step S201, obtaining a data file to be published to the industrial site and a corresponding data publishing message; the data file includes the data published by the engineering personnel exported by the first MongoDB server from the first MongoDB database, and the data publishing message includes the task identifier of the data publishing task of the engineering personnel;
[0075] Step S202, transmitting the data file to the industrial site to be imported into a second MongoDB database at the industrial site for storage;
[0076] Regarding the implementation process of step S201 and step S202, reference may be made to the description of the corresponding parts of the above embodiment, such as the description of the corresponding parts of step S101 and step S102, which will not be repeated in this embodiment.
[0077] Step S203, reading a task identifier for the data file from the second MongoDB database;
[0078] Step S204, determine whether the task identifier exists in the data publishing message, if so, proceed to step S205; if not, proceed to step S212;
[0079] After reading the task identifier of the data file published this time, such as the task id, imported into the MongoDB database (i.e., Mongo database) of the industrial site, it can be determined whether the task identifier is the same as the task identifier in the data publishing message that has been notified to the industrial site for the data published this time. If they are the same, it means that the read task identifier exists in the data publishing message, and the MongoDB database (i.e., the second MongoDB database) of the industrial site stores the data published this time by the engineering personnel. The data has not been tampered with during network transmission, thereby realizing data publishing isolation.
[0080] Step S205, reading service information associated with the data file from the second MongoDB database;
[0081] Step S206, performing integrity check on the service information using at least the task identifier;
[0082] Following the above analysis, after determining that the MongoDB database at the industrial site stores the task identifier of the data release task, the integrity of the service information associated with the task identifier can be verified. Combined with the above description of the service information, it can be determined whether the service information associated with the data file released this time read from the MongoDB database at the industrial site contains the verified task identifier. Of course, other information content of the service information can also be checked for integrity, such as the service name, service description information, service deployment method, server IP and other information, to ensure that the data file released this time stored in the MongoDB database at the industrial site is used to implement the corresponding service at the industrial site.
[0083] Step S207, determine whether the service information is complete, if yes, proceed to step S208; if no, proceed to step S212;
[0084] Step S208: Read the process information associated with the service information from the second MongoDB database
[0085] Step S209, performing integrity check on the process information using at least the service information;
[0086] Step S210, determining whether the process information is complete, if yes, proceeding to step S211; if no, executing step S212;
[0087] According to the method described above, when the service information associated with the data file released this time stored in the MongoDB database of the industrial site is verified to be complete, the integrity of the process information executing the industrial site service can be further verified. After reading the process information associated with the verified complete service information from the MongoDB database of the industrial site, it can be determined whether the service identifier (such as service ID) in the process information exists in the verified complete service information, that is, whether the service executed by the corresponding industrial site service process is the industrial site service executed by the data released this time. If so, it means that the data imported into the MongoDB database of the industrial site is safe and complete, and subsequent operations can be performed.
[0088] Among them, in order to execute the process information of each industrial site service process, it may include process id, service id, process name, process cpu (core) and process memory (M), etc. This application does not limit the service information and process information content recorded in the MongoDB database of the industrial site, which can be determined according to the situation.
[0089] Step S209: converting the data file into target data required by the industrial site.
[0090] Step S210, prohibiting conversion of the data file.
[0091] In the integrity verification process described above, it is determined in turn that the task identifier of the data publishing task from the second MongoDB database, its associated service information, and the process information associated with the service information are all complete, indicating that the published data has not been tampered with, and the services that can be called at the industrial site and their execution processes are correct. The published data file imported into the second MongoDB database can be read and converted into the target data required by the industrial site. The implementation process can refer to the description of the corresponding part of the context and will not be described in detail in this embodiment.
[0092] However, in the integrity check process performed sequentially above, if the data release message does not contain the task identifier read from the second MongoDB database, it means that the data release verification has not passed and it may have been tampered with. In order to avoid threatening the business security of the industrial site operating environment, the data file released this time will no longer be converted and released to the industrial site environment. If the data release message contains the task identifier, but during further verification, it is determined that the task identifier contained in the service information read from the second MongoDB database is different from the task identifier in the data release message, that is, the service information is incomplete, indicating that the verification has not passed, and the data file conversion process will not be performed.
[0093] Furthermore, if it is determined that there is a task identifier in the data release message, and the task identifier contained in the service information is the same as the task identifier in the data release message, but the service identifier contained in the process information read from the second MongoDB database is different from the service identifier contained in the service information, that is, the process information is incomplete, indicating that the verification fails, and the data file conversion process is not performed.
[0094] It can be seen that in the embodiment of the present application, the data published by the engineering personnel to the first MongoDB database is exported as a data file, transmitted to the industrial site, and imported into the second MongoDB database of the industrial site. The business server will perform integrity verification on the imported data in the online industrial site environment, and sequentially check whether the task identifier of the entire data file exists in the second MongoDB database, whether the service information associated with the data file published this time exists in the second MongoDB database, and whether the process information associated with the service information exists in the second MongoDB database. If all exist, it means that the verification has passed, and the data file published this time is complete and has not been tampered with. It will be converted into the target data required by the industrial site to ensure the reliability of executing business in the industrial site environment. In this verification process, if any verification fails, the data file will no longer be converted into a format, so as to avoid the execution of wrong data in the industrial site environment, causing safety accidents or reducing business efficiency and reliability.
[0095] Reference Figure 3, is a flow chart of a data publishing method proposed in Embodiment 3 of the present application. This embodiment can describe an optional detailed implementation of the data publishing method proposed above, such as Figure 3 As shown, the data publishing method may include:
[0096] Step S301, obtaining a data file to be published to the industrial site and a corresponding data publishing message; the data file includes the data published by the engineering personnel exported by the first MongoDB server from the first MongoDB database, and the data publishing message includes the task identifier of the data publishing task of the engineering personnel;
[0097] In a possible implementation, the first MongoDB server can call the MongoDB data backup tool (i.e., the mongodump tool) to export the first MongoDB database data released by the engineering personnel for the industrial site into an archive format data file after security authentication. It can be seen that for the MongoDB database data during configuration, i.e., the data released by each engineering personnel, the mongoDump tool can be used to export the data into an archive file.
[0098] Among them, the data export command of the mongodump tool to implement the above data export operation can be defined as: mongodump --host=ip:port --username=user --password=password --authenticationDatabase=admin --db=project_database --gzip --archive= / path / to / archive.
[0099] In the data export command, the host parameter can refer to the host name (Ip) and port (port) of the first MongoDB database; the username parameter can refer to the username user for accessing the first MongoDB database; the password parameter can refer to the password for accessing the first MongoDB database, and the authenticationDatabase parameter can refer to the database for authentication of the user pointed to by the username; the db parameter can refer to the name of the database to be exported, such as the name of the first MongoDB database in this application (since a MongoDB database service allows the establishment of multiple databases, the names of each database can be configured to distinguish them); the gzip parameter can compress the archived file to improve the efficiency of data publishing; archive can indicate the path of the archived file.
[0100] According to the explanation of each parameter used to define the data export command, the host can be used to inform the user to export the archive file from the first MongoDB server with the host name of 192.168.1.10:27017 (or other IP address). When exporting, the user name and password (username parameter and password parameter) entered by the user are authenticated based on the security authentication mechanism supported by the MongoDB server, so that only the authenticated user can be allowed to export data from the first MongoDB database. Therefore, the mongodump tool executes the data export command containing the above parameter content, uses the binary data stream method to export the first MongoDB database data, stores it in archive mode, and realizes the data backup of the entire database. This data export method can obtain data files safely and quickly. For example, a 1G compressed data file can be exported within 3 seconds, avoiding the export of MongoDB database data by illegal users.
[0101] Step S302, calling the hypertext transfer protocol service of the industrial site, uploading the data file to the second MongoDB server of the industrial site, and the second MongoDB server calling the MongoDB data backup tool to import the data file into the second MongoDB database of the industrial site for storage;
[0102] Compared with engineers carrying data storage devices to the industrial site, this application proposes to transmit data files through a wireless communication network, which improves data transmission efficiency and saves labor costs. In a possible implementation, the Post interface (i.e., Post request) of the industrial site Http (Hypertext Transfer Protocol) service can be called to transmit the exported archive file to the industrial site.
[0103] In order to realize the transmission of the archive file, the Post interface can be defined as shown in Table 1 below to ensure that calling the Post interface can realize the function:
[0104] Table 1
[0105]
[0106] For the http interface defined in the manner shown in Table 1, it is made to perform services according to the definition content. As shown in Table 1, the interface description can be defined, that is, the function implemented by calling the Post interface, defining the uniform resource locator (URL), the request method for implementing the function, such as post request; request type; return type (such as json (JavaScript Object Notation, JavaScript Object Notation) format file), in addition, data types (such as string, boolean, int32, etc.), data file transmission methods, such as file stream methods, etc. can also be defined, including but not limited to the interface definition content that complies with the http communication protocol described in this embodiment.
[0107] For example, if the request parameter according to the definition in Table 1 is -d {file:FormData}, the request is successfully processed and the data file is successfully uploaded to the industrial field environment, and the return value may be:
[0108] {
[0109] "code": 200,
[0110] "message": "Success",
[0111] "data": true
[0112] }
[0113] Afterwards, the second MongoDB server calls the MongoDB data backup tool (mongorestore tool) to import the archive file into the second MongoDB database storage at the industrial site. Referring to the data export implementation process described above, the data import command executed in the data import process can be:
[0114] mongorestore --host=ip:port --username=user --password=password --authenticationDatabase=admin --db=project_datatbase --gzip --archive= / path / to / archive
[0115] In the data import command, host can define the host name (IP) and port (port) of the second MongoDB database where the data is imported, username and password can respectively define the username and password for accessing the MongoDB database, authenticationDatabase can define the database for authentication of the user pointed to by username, db can define the name of the database where the data needs to be exported, such as the name of the second MongoDB database in this application, gzip can define the decompression of the imported archive file; archive can define the path of the imported archive file. The mongorestore tool imports the data in the archive or standard input of the binary database created by mongodump into the second MongoDB database according to the data import command. This method of importing compressed files improves transmission efficiency. For example, importing 1G of compressed data can be completed in about 20 seconds.
[0116] The implementation process of publishing industrial data to the industrial site operating environment using the tools in the MongoDB tool set on the MongoDB server utilizes the security authentication mechanism of the MongoDB server, that is, the user name and password for accessing the MongoDB database are authenticated. Only after the authentication is passed, the above-mentioned operations of exporting data from the first MongoDB database and importing data into the second MongoDB database can be allowed to be executed. That is, the MongoDB data backup tool is called, and after security authentication, the first MongoDB database data published by the engineering personnel for the industrial site can be safely and reliably exported into an archive format data file; by calling the MongoDB data recovery tool, after security authentication, the data file is imported into the second MongoDB database of the industrial site for storage, thereby avoiding illegal users from accessing the MongoDB database and improving data security.
[0117] Step S303, performing integrity check on the data file imported into the second MongoDB database according to the data publishing message;
[0118] Regarding the integrity verification process in step S303, reference may be made to the description of the corresponding part of the above embodiment, and this embodiment will not be described in detail here.
[0119] Step S304: if the verification passes, read the verified data file from the second MongoDB database;
[0120] Step S305: based on the constructed data conversion model, convert the data file into target data in a specified communication protocol format required by the industrial site.
[0121] In one possible implementation, the method for constructing the data conversion model may be: based on a specified communication protocol supported by the industrial site (such as the OPC UA protocol, Open Platform Communications Unified Architecture, which is a service-based industrial communication protocol that can achieve interoperability between various systems and devices), extensible markup language (XML) modeling is performed to obtain a corresponding data conversion model for converting the data type stored in the MongoDB database into target data in a data format required by the industrial site. The present application does not impose any restrictions on the method for constructing the data conversion model.
[0122] It should be understood that this application only uses the data type of the data file stored in the MongoDB database (such as BSON, a binary serialization format used for data storage and network transmission in the MongoDB database, which can be a binary representation of JSON, representing simple data structures, associative arrays (which can be called objects or documents in the MongoDB database) and various data types in MongoDB) to convert to XML format files as an example to illustrate. In the case where the data type required at the industrial site changes, a data conversion model can be constructed to achieve the conversion between the corresponding two data formats, and this application does not give detailed examples one by one.
[0123] Based on this, for reading the data file released this time from the second MongoDB database, a data conversion model of the XML data structure defined by the OPC UA protocol is constructed. The data contained in the read data file can be written into the data conversion model, and the data structure supported by the OPC UA protocol (that is, the specified communication protocol) is obtained by conversion, that is, the XML data structure, and then it is written into the XML file to obtain the target data required by the industrial site for use by corresponding businesses in the industrial site environment.
[0124] In summary, in the embodiments of the present application, the data backup tool of the MongoDB database is directly used to automatically export the data published by the engineering personnel to the first MongoDB database as an archive file, the archive file is transferred to the industrial site using the http service post interface, and the data recovery tool of the MongoDB database is used to import the archive file into the second MongoDB database at the industrial site. After the data security and integrity are verified, the data conversion model is used to convert the data file into the OPC data required by the industrial site, which solves the problem of low data publishing security and low publishing efficiency in the method where the engineering personnel carry data storage devices to the industrial site to publish data. The data publishing method proposed in the present application takes into account both security and efficiency, greatly improves the security of industrial site services, and ensures the timeliness of data publishing.
[0125] In some other embodiments proposed in the present application, for the transmission method of data files exported from the first MongoDB database to the industrial site, the encrypted file transfer protocol service (SFTP service, that is, the file transfer protocol (File Transfer Protocol) based on the SSH (Secure Shell) protocol inherits the security features of the SSH protocol, and ensures the security of file transfer by encrypting the transmitted data. The implementation process is not described in detail in this application) of the industrial site can be called to transfer the data file to a communication server (SFTP server) that supports the encrypted file transfer protocol service, so that the second MongoDB server at the industrial site downloads the data file from the communication server and imports it into the second MongoDB database at the industrial site for storage.
[0126] It should be noted that in addition to the two specified communication protocols listed above, the Http protocol and the SFTP protocol, the implementation method of transferring the archive file exported from the local first MongoDB database to the industrial site for publishing data, other communication protocols used in the actual communication environment can be used to realize the transmission of the archive file. This application does not give detailed examples one by one.
[0127] The above describes the data publishing method provided by the embodiment of the present application, and the following describes a device for executing the above data publishing method.
[0128] Reference Figure 4 , is a schematic diagram of the structure of the data publishing device proposed in the embodiment of the present application, such as Figure 4 As shown, the data publishing device may include:
[0129] A first acquisition module 410 is used to acquire a data file to be published to the industrial site and a corresponding data publishing message; the data file includes engineering personnel publishing data exported by the first MongoDB server from the first MongoDB database;
[0130] A data transmission module 420, used for transmitting the data file to the industrial site to be imported into a second MongoDB database storage of the industrial site;
[0131] A verification module 430, configured to perform integrity verification on the data file imported into the second MongoDB database according to the data publishing message;
[0132] The data conversion module 440 is used to convert the data file into target data required by the industrial site if the verification passes.
[0133] In a possible implementation, the data transmission module 420 may include:
[0134] A task identifier reading unit, configured to read a task identifier for the data file from the second MongoDB database;
[0135] a service information reading unit, configured to read service information associated with the data file from the second MongoDB database if it is determined that the task identifier exists in the data publishing message;
[0136] A service information verification unit, configured to perform integrity verification on the service information using at least the task identifier;
[0137] a process information reading unit, configured to read process information associated with the service information from the second MongoDB database if the service information is complete;
[0138] The process information verification unit is used to perform integrity verification on the process information using at least the service information.
[0139] Optionally, the service information verification unit may include:
[0140] The first determining unit is used to determine whether the task identifier included in the service information is the same as the task identifier in the data publishing message.
[0141] The process information verification unit may include:
[0142] The second determining unit is used to determine whether the service identifier included in the process information is the same as the service identifier included in the service information.
[0143] Based on the above analysis, the above data publishing device may further include:
[0144] a conversion prohibiting module, used for prohibiting conversion of the data file imported into the second MongoDB database if the verification fails;
[0145] Wherein, the verification failure includes any of the following:
[0146] Determining that the task identifier does not exist in the data publishing message;
[0147] Determining that the task identifier exists in the data publishing message, but the task identifier included in the service information is different from the task identifier in the data publishing message;
[0148] It is determined that the task identifier exists in the data publishing message, and the task identifier included in the service information is the same as the task identifier in the data publishing message, but the service identifier included in the process information is different from the service identifier included in the service information.
[0149] In a possible implementation, the data conversion module 440 may include:
[0150] A data file reading unit, configured to read the verified complete data file from the second MongoDB database;
[0151] The conversion unit is used to convert the data file into target data in a specified communication protocol format required by the industrial site based on the constructed data conversion model.
[0152] Among them, the building blocks for building a data conversion model may include:
[0153] A modeling unit, used for performing extensible markup language modeling based on a specified communication protocol supported by the industrial site to obtain a corresponding data conversion model;
[0154] Based on this, the above conversion unit may include:
[0155] A first writing unit, used for writing the data contained in the data file into the data conversion model to obtain a data structure supported by the specified communication protocol;
[0156] The second writing unit is used to write the data structure into an extensible markup language file to obtain target data required by the industrial site.
[0157] In a possible implementation, the data transmission module 420 may include:
[0158] A first calling unit is used to call the hypertext transfer protocol service of the industrial site to upload the data file to a second MongoDB server of the industrial site; or
[0159] A second calling unit is used to call the encrypted file transfer protocol service of the industrial site, and transfer the data file to a communication server supporting the encrypted file transfer protocol service, so that the second MongoDB server of the industrial site downloads the data file from the communication server;
[0160] The second MongoDB server imports the received data file into the second MongoDB database of the industrial site for storage.
[0161] Based on this, the data export module in the first MongoDB server may include:
[0162] A first export unit is used to call a MongoDB data backup tool, and after security authentication, export the first MongoDB database data released by the engineering staff for the industrial site into a data file in an archive format;
[0163] The data import module in the second MongoDB server may include:
[0164] The first importing unit is used to call the MongoDB data recovery tool, and after security authentication, import the data file into the second MongoDB database of the industrial site for storage.
[0165] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on a business server, the business server implements any data publishing method provided in the embodiment of the present application.
[0166] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by a business server, the business server can implement any data publishing method provided in an embodiment of the present application.
[0167] The computer-readable storage medium may be any available medium that can be stored by a computer or a data storage device such as a training device or a data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).
[0168] The above introduces the data publishing method provided in the embodiment of the present application. The following will introduce in detail the system architecture applicable to the data publishing method proposed in the present application in conjunction with the accompanying drawings.
[0169] Reference Figure 5 , which is a hardware structure diagram of the data publishing system proposed in the first embodiment of the present application. Combined with the above analysis, the data publishing system may include a first MongoDB server 510, a first MongoDB database 520, and a business server 530 deployed for industrial sites, a second MongoDB server 540 and a second MongoDB database 550, wherein:
[0170] The first MongoDB database 520 may be used to store data published by engineers;
[0171] The first MongoDB server 510 may be used to export the data published by the engineering staff in the first MongoDB database 520 into a data file, such as an archive file;
[0172] The business server 530 may be used to obtain a data file to be published to the industrial site and a corresponding data publishing message, and transmit the data file to a second MongoDB server 540 at the industrial site;
[0173] The second MongoDB server 540 may be used to import the received data file into the second MongoDB database 550 for storage;
[0174] The business server 530 can also be used to publish messages based on the data, perform integrity verification on the data files imported into the second MongoDB database 550, and if the verification is complete, convert the data files into target data required by the industrial site.
[0175] Regarding the various steps of the data publishing method proposed in the present application executed by the business server 530 , reference may be made to the description of the corresponding part of the above method embodiment, and the present application embodiment will not be elaborated here.
[0176] In one possible implementation, Figure 6As shown, the service server 530 may include at least one communication element 531, at least one memory 532 and at least one processor 533. The at least one communication element 531, at least one memory 532 and at least one processor 533 may communicate with each other via a bus. The bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0177] The communication element 531 can be used to receive data files and corresponding data release messages. In practical applications, the communication element 531 may include wireless communication elements that support wireless communication methods such as wifi, 5G / 6G, Global System of Mobilecommunication (GSM), General Packet Radio Service (GPRS) and / or near-field channel methods to achieve data transmission between the business server and other devices (such as MongoDB database, other MongoDB servers or communication servers, etc.) at the industrial site, such as transmitting data files, data release messages, etc. This application does not elaborate on the structure of each wireless communication element and its communication transmission mechanism.
[0178] It can be understood that the communication element 531 can also include one or more wired communication elements that support wired communication methods, such as a general-purpose input / output (GPIO) interface, a USB interface, a universal asynchronous receiver / transmitter (UART) interface, etc., to achieve data transmission between various components within the business server. This application does not limit the type and quantity of wired communication elements, which can be determined according to the circumstances.
[0179] The memory 532 can be used to store computer instructions and data, and may include an instruction storage area and a data storage area. The data storage area may store various data, such as a data file for each data release and a corresponding data release message, and may also include various intermediate data generated during the data release process; the instruction storage area may store software units such as an operating system, an application, and computer instructions required for at least one function. In an embodiment of the present application, the memory 532 may provide the processor 533 with multiple computer instructions for implementing the data release method of the embodiment of the present application, as well as data or other software and hardware resources required during the execution of the data release method.
[0180] The processor 533, as the control center of the business server, can use various interfaces and lines to connect various parts of the entire business server, and implement the steps of the data publishing method proposed in the embodiment of the present application by loading and executing multiple computer instructions stored in the memory 533. The implementation process can refer to the description of the corresponding part of the method embodiment above. Other components of the business server (such as communication elements, one or more input components and output components, etc.) can also be called to implement corresponding functions.
[0181] In the embodiment of the present application, the memory 532 may include a volatile memory (volatile memory), such as a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as a read-only memory (ROM), a flash memory, a hard drive drive (HDD) or a solid state drive (SSD), etc. The processor 533 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), a digital signal processor (DSP), an application specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).
[0182] It should be understood that Figure 6 The structure of the service server shown in the figure does not constitute a limitation on the service server in the embodiment of the present application. In actual application, the service server may include Figure 6More or fewer components, or combinations of certain components, are not listed here. Figure 5 The structure of the data publishing system shown in FIG. 1 may also include Figure 6 More devices or combined devices shown, such as other networking devices, business execution devices in industrial field environments, etc., can be determined based on the business processing requirements of the actual industrial field environment and are not described in detail in this application.
[0183] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.
[0184] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0185] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function described in accordance with the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer instruction can be stored in a computer-readable storage medium, or transmitted from a computer-readable storage medium to another computer-readable storage medium, for example, the computer instruction can be transmitted from a website site, a computer, a training device or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device or data center.
Claims
1. A data publishing method, It is characterized in that The data publishing method comprises: Acquire a data file to be published to the industrial site and a corresponding data publishing message; the data file includes engineering personnel publishing data exported by the first MongoDB server from the first MongoDB database; Transmitting the data file to the industrial site to be imported into a second MongoDB database of the industrial site for storage; Performing integrity check on the data file imported into the second MongoDB database according to the data publishing message; If the verification is passed, the data file is converted into target data required by the industrial site.
2. The data publishing method according to claim 1, It is characterized in that The step of performing integrity verification on the data file imported into the second MongoDB database according to the data publishing message includes: Reading a task identifier for the data file from the second MongoDB database; If it is determined that the task identifier exists in the data publishing message, reading service information associated with the data file from the second MongoDB database; Performing an integrity check on the service information using at least the task identifier; If the service information is complete, reading process information associated with the service information from the second MongoDB database; The process information is at least checked for integrity using the service information.
3. The data publishing method according to claim 2, It is characterized in that The step of performing integrity check on the service information by at least using the task identifier includes: Determining whether the task identifier included in the service information is the same as the task identifier in the data publishing message; The at least using the service information to perform integrity check on the process information includes: It is determined whether the service identifier included in the process information is the same as the service identifier included in the service information.
4. The data publishing method according to claim 2, It is characterized in that The data publishing method further includes: If the verification fails, prohibiting conversion of the data file imported into the second MongoDB database; Wherein, the verification failure includes any of the following: Determining that the task identifier does not exist in the data publishing message; Determining that the task identifier exists in the data publishing message, but the task identifier included in the service information is different from the task identifier in the data publishing message; It is determined that the task identifier exists in the data publishing message, and the task identifier included in the service information is the same as the task identifier in the data publishing message, but the service identifier included in the process information is different from the service identifier included in the service information.
5. The data publishing method according to any one of claims 1 to 4, It is characterized in that The converting the data file into target data required by the industrial site comprises: Reading and verifying the complete data file from the second MongoDB database; Based on the constructed data conversion model, the data file is converted into target data in a specified communication protocol format required by the industrial site.
6. The data publishing method according to claim 5, It is characterized in that The construction of the data conversion model includes: Perform extensible markup language modeling based on the specified communication protocol supported by the industrial site to obtain a corresponding data conversion model; The step of converting the data file into target data in a specified communication protocol format required by the industrial site based on the constructed data conversion model includes: Writing the data contained in the data file into the data conversion model to obtain a data structure supported by the specified communication protocol; The data structure is written into an extensible markup language file to obtain target data required by the industrial site.
7. The data publishing method according to any one of claims 1 to 4, It is characterized in that The transmitting the data file to the industrial site includes any of the following implementations: Invoke a hypertext transfer protocol service of the industrial site to upload the data file to a second MongoDB server of the industrial site; Invoke the encrypted file transfer protocol service of the industrial site, and transfer the data file to a communication server supporting the encrypted file transfer protocol service, so that the second MongoDB server of the industrial site downloads the data file from the communication server; The second MongoDB server imports the received data file into a second MongoDB database at the industrial site for storage.
8. The data publishing method according to claim 7, It is characterized in that The first MongoDB server exports the data published by the engineering staff, including: The first MongoDB server calls the MongoDB data backup tool, and after security authentication, exports the first MongoDB database data released by the engineering staff for the industrial site into a data file in an archive format; The second MongoDB server imports the received data file into the second MongoDB database of the industrial site for storage, including: The second MongoDB server calls a MongoDB data recovery tool, and after security authentication, imports the data file into the second MongoDB database of the industrial site for storage.
9. A data publishing system, It is characterized in that The data publishing system includes: a first MongoDB server, a first MongoDB database, and a business server deployed on industrial sites, a second MongoDB server, and a second MongoDB database, wherein: The first MongoDB database is used to store data published by engineers; The first MongoDB server is used to export the data published by the engineering personnel in the first MongoDB database into a data file; The business server is used to obtain the data file to be published to the industrial site and the corresponding data distribution message, and transmit the data file to the second MongoDB server at the industrial site; The second MongoDB server is used to import the received data file into the second MongoDB database for storage; The business server is further used to perform integrity verification on the data file imported into the second MongoDB database based on the data publishing message, and if the verification is complete, convert the data file into target data required by the industrial site.
10. A computer storage medium, It is characterized in that The computer storage medium carries one or more computer programs, and when the one or more computer programs are executed by the service server, the service server can implement the data publishing method according to any one of claims 1 to 8.
