Springboot-based ship equipment data acquisition method and system

Abstract

The invention discloses a ship equipment data acquisition method and system based on Springboot, and the method comprises the following steps: S1, building different types of communication clients based on the public configuration and injection mode of Springboot; s2, adding the created different types of communication clients into a client cache queue, and waiting for starting of an external trigger instruction; s3, after a trigger instruction for a target communication client is detected, starting a message communication function of the target communication client, receiving message data in real time through the target communication client, and then storing the received message data into a cache; s4, reading the message data from the cache, and performing analysis processing on the message data through the target communication client to obtain analysis data; and S5, storing the analyzed data in different databases / tables in a classified manner. According to the invention, message communication between diversified communication protocols and equipment of different suppliers is supported, and rapid acquisition and expansion of ship equipment data can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent ships, and in particular to a ship equipment data acquisition method and system based on Springboot. Background Art

[0002] At present, ship intelligence has become an irreversible development trend in the global shipping industry. With the continuous improvement of the level of intelligence, the number of automated systems and equipment installed on ships has also increased rapidly. This change has brought unprecedented efficiency improvement and operational optimization potential to the shipping industry.

[0003] However, in this process, a significant technical challenge gradually emerged: the various automation systems and equipment on board ships usually come from different manufacturers, and each manufacturer may choose different communication protocols to meet their specific functional requirements and design concepts. These protocols have significant differences in data format, transmission method, interface standard, etc., resulting in huge technical obstacles for ships when trying to integrate data from different equipment and systems. The diversity of protocols greatly increases the complexity of data collection and management.

[0004] Due to the incompatibility of protocols between devices, ship data integration has become extremely complicated. At present, ship data acquisition and transmission devices are independent individual devices. Whenever a new ship automation system or equipment is needed during ship construction or modification, new data acquisition equipment is required, and the installation and data interaction debugging of the data acquisition device are frequently performed, which greatly reduces the assembly efficiency of the new ship automation system or equipment and is not conducive to the improvement of the overall perception ability of the ship. Summary of the invention

[0005] In order to solve the problems in the prior art such as the difficulty in data collection caused by the use of multiple different communication protocols by ship equipment, the present invention proposes a ship equipment data collection method based on Springboot, which can support multiple different communication protocols, and can realize the rapid collection of ship equipment data and the efficient expansion of different protocols. The present invention also relates to a ship equipment data collection system based on Springboot.

[0006] The present invention is achieved through the following technical solutions:

[0007] A ship equipment data collection method based on Springboot, comprising the following steps:

[0008] S1: Based on the public configuration and injection method of Springboot, different types of communication clients are created according to different communication parameters; wherein each of the different types of communication clients includes different communication parameters, communication methods, message parsing and communication status, the communication parameters include the communication protocol, the IP address and port of the external device to be collected, the communication method is a method for the communication client to communicate with external devices of multiple different communication protocols, the message parsing parses and restores messages of different communication protocols, the communication status includes several states of whether it is enabled and whether the communication is normal, and different communication states are used to indicate the current usage of the communication client;

[0009] S2: Add the created different types of communication clients to the client cache queue and wait for external trigger instructions to start;

[0010] S3: after detecting a trigger instruction for a target communication client, locating the target communication client through the client cache queue, enabling a message communication function of the target communication client according to the communication parameters, and receiving message data in real time through the target communication client, and then storing the received message data in real time into a cache;

[0011] S4: allocating a parsing thread to the target communication client, the parsing thread reading the message data from the cache in chronological order, and parsing the message data through the target communication client to obtain parsed data;

[0012] S5: The parsed data are classified and stored in different databases according to the sampling frequency levels corresponding to the actual business scenarios, and the parsed data stored in the same database are classified again according to the data source and stored in different data tables.

[0013] Preferably, step S1 adopts the factory design pattern to implement the design of different types of communication clients, including constructing an abstract communication client base class and a client factory class, wherein the abstract communication client base class constrains the basic communication properties and methods of the communication client, and the client factory class generates different subclass communication clients according to different communication parameter configurations, and the subclass communication client inherits from the abstract communication client base class, and has both the basic communication parameters and methods of the communication client and customized communication parameters, methods and message parsing, and utilizes the modularization function of springboot to develop each subclass communication client separately into a separate microservice.

[0014] Preferably, in step S1, the communication parameters include basic information of the local client, basic information of the remote client and communication connection information; the basic information of the local client includes the IP address, port, creator and creation time of the local client; the basic information of the remote client includes the IP address, port, device refresh frequency and sensor information list of the remote client; the communication connection information includes the communication connection mode and the communication protocol type; the communication connection mode includes Ethernet and serial port forms; the communication protocol types include TCP, UDP, MQTT; the sensor information list includes sensor number, measurement type, measurement range, and communication coding address;

[0015] The communication status of the communication client includes not enabled, ready to enable, communicating, communication interrupted, communication abnormal, stopping and stopped. The communication status is stored in the cache for instant reading and modification. Each communication client uses a separate thread, and the newly created communication client is in the not enabled state.

[0016] Preferably, step S2 adds the created different types of communication clients to the client cache queue, and sets the communication status to ready to be enabled, waiting for an external trigger instruction to start the communication; at the same time, the client queue is maintained in the program memory, the client is retrieved according to the IP address of the local client, and a restful interface or remote procedure call is used to receive the trigger of a remote external instruction.

[0017] Preferably, step S3 comprises the following steps:

[0018] S31: receiving a trigger instruction for a target communication client;

[0019] S32: locating the target communication client through the client cache queue;

[0020] S33: according to the communication parameters, enabling the message communication function of the target communication client so that the target communication client establishes a communication connection with the external device to be collected data;

[0021] S34: if the message communication function is successfully enabled, the message data sent by the external device to be collected is received in real time, and the message data is stored in a cache in a hash table storage structure according to the time sequence of the message data, and the communication state of the target communication client is set to normal communication;

[0022] S35: If the message communication function fails to be enabled, the state information of the target communication client in the cache is set to a communication abnormal state.

[0023] Preferably, step S4 comprises the following steps:

[0024] S41: If the state information of the target communication client in the cache is a normal communication state, assigning a message parsing thread to the target communication client;

[0025] S42: reading message data in batches from the cache in chronological order through the message parsing thread;

[0026] S43: If the message parsing thread successfully reads the message data, the message parsing method of the maintenance client is maintained according to the exclusive device information of the target communication client, and the read message data is recursively parsed;

[0027] S44: Repeat steps S42-S43 until the message data stored in the target communication client in the cache is processed and cleared;

[0028] S45: If the message parsing thread cannot read the message data from the cache, the monitoring function of the target communication client is started, and whether there is new message data in the cache is monitored regularly;

[0029] S46: Once it is detected that there is new message data in the cache, jump to step S42;

[0030] S47: If no new message data is monitored, the monitoring continues until the state information of the target communication client in the cache is changed to a stop state, and then the monitoring function of the target communication client is turned off.

[0031] Preferably, in step S5, the database includes a basic information database, a high-frequency storage database, a medium-frequency storage database, and a low-frequency storage database; the basic information database is implemented using Mysql and SQL server, and is used to store basic information of the communication client, communication parameter configuration and message protocol, and databases of different frequencies are implemented using InfluxDB and TDEngine time series databases. The low-frequency storage database is used to store data with a collection frequency of minutes corresponding to the actual business scenario, the medium-frequency storage database is used to store data with a collection frequency of seconds corresponding to the actual business scenario, and the high-frequency storage database is used to store data with a collection frequency of milliseconds or microseconds corresponding to the actual business scenario.

[0032] A ship equipment data acquisition system based on Springboot, comprising a common configuration module, a management and scheduling module, a cache module, a message parsing module and a classification storage module connected in sequence; wherein,

[0033] The public configuration module is used to maintain the public configuration of Springboot, and based on the public configuration and injection method of Springboot, different types of communication clients are created according to different communication parameters; each of the different types of communication clients includes different communication parameters, communication methods, message parsing and communication status, the communication parameters include the communication protocol, the IP address and port of the external device to be collected data, the communication method is a method for the communication client to communicate with external devices of multiple different communication protocols, the message parsing is parsed and restored for messages of different communication protocols, the communication status includes several states of whether it is enabled and whether the communication is normal, and different communication states are used to indicate the current usage of the communication client;

[0034] The management and scheduling module is used to manage the different types of communication clients created; add the different types of communication clients created to the client cache queue, waiting for the external trigger instruction to start; after detecting the trigger instruction for the target communication client, locate the target communication client through the client cache queue, enable the message communication function of the target communication client according to the communication parameters, and receive the message data in real time through the target communication client; use Springboot's RestController to receive external trigger instructions, trigger the creation, start, stop, and interruption operations of the communication client, and maintain the communication status of each communication client;

[0035] The cache module is implemented by integrating JCache, EhCache and Redis cache frameworks using springboot, and is used to store the message data received in real time into the cache;

[0036] The message parsing module is used to allocate a parsing thread to the target communication client, and the parsing thread reads the message data from the cache in chronological order, and parses and processes the message data through the target communication client to obtain parsed data;

[0037] The classification storage module is used to classify and store the parsed data into different databases according to the sampling frequency levels corresponding to the actual business scenarios, and to classify and store the parsed data stored in the same database into different data tables again according to the data source.

[0038] Preferably, the common configuration module adopts the factory design pattern to implement the design of different types of communication clients, including constructing an abstract communication client base class and a client factory class. The abstract communication client base class constrains the basic communication properties and methods of the communication client. The client factory class generates different subclass communication clients according to different communication parameter configurations. The subclass communication client inherits from the abstract communication client base class, and has both the basic communication parameters and methods of the communication client and customized communication parameters, methods and message parsing. Using the modularization function of springboot, each subclass communication client is developed separately into a separate microservice.

[0039] Preferably, the management and scheduling module enables the message communication function of the target communication client according to the communication parameters, so that the target communication client establishes a communication connection with the external device for collecting data; if the message communication function is successfully enabled, the message data sent by the external device for collecting data is received in real time, and then the cache module stores the message data in the cache in a hash table storage structure according to the time sequence of the message data, and then the management and scheduling module sets the communication status of the target communication client to normal communication; if the message communication function fails to be enabled, the management and scheduling module sets the status information of the target communication client to the abnormal communication status.

[0040] Preferably, after the status information of the target communication client is normal communication, the message parsing module allocates a message parsing thread to it, and the message parsing thread reads the message data in batches from the cache in chronological order; if the message parsing thread successfully reads the message data, the read message data is recursively parsed until the message data stored in the cache by the target communication client is processed and cleared; if the message parsing thread cannot read the message data from the cache, the monitoring function of the target communication client is started, and the cache is monitored regularly for whether there is new message data; once the cache is monitored for new message data, the recursive parsing process is continued; if no new message data is monitored, the monitoring is continued until the status information of the target communication client in the cache changes to a stopped state, and the monitoring function of the target communication client is closed.

[0041] Preferably, in the classified storage module, the database includes a basic information database, a high-frequency storage database, a medium-frequency storage database, and a low-frequency storage database; the basic information database is implemented using Mysql and SQL server, and is used to store basic information of the communication client, communication parameter configuration and message protocol, and databases of different frequencies are implemented using InfluxDB and TDEngine time series databases. The low-frequency storage database is used to store data with a collection frequency of minutes corresponding to actual business scenarios, the medium-frequency storage database is used to store data with a collection frequency of seconds corresponding to actual business scenarios, and the high-frequency storage database is used to store data with a collection frequency of milliseconds or microseconds corresponding to actual business scenarios.

[0042] The beneficial effects of the present invention are:

[0043] The present invention provides a ship equipment data collection method based on Springboot, the method comprising: based on the public configuration and injection mode of Springboot, creating different types of communication clients according to different communication parameters, thus realizing unified management, configuration and sharing of communication parameters, simplifying the configuration process, improving configuration efficiency, and effectively solving the problem that support for diversified communication protocols cannot be quickly expanded; adding different types of created communication clients to a client cache queue, waiting for an external trigger instruction to start, thus, the message communication function can be enabled in time when it is necessary to collect device data, ensuring rapid collection of device data; after detecting a trigger instruction for a target communication client, locating the target communication client through the client cache queue, enabling the message communication function of the target communication client according to the communication parameters, and The client receives the message data in real time, and then stores the received message data in the cache. In this way, the message data can be stored in real time, and the efficiency of receiving the message data will not be affected by the performance of the parsing process after receiving the message data. It is also convenient to adapt to the communication refresh frequency of different devices, especially the reception of high-frequency data, so as to achieve the effect of quickly collecting device data; the message data is read from the cache, and the message data is parsed and processed by the target communication client to obtain the parsed data. In this way, the cache can be used to decouple the message communication and the message parsing to prevent the message data from being blocked, so as to achieve the effect of quickly processing the device data; the parsed data is classified and stored in different databases. In this way, the data can be classified and stored in different databases and data tables according to classification indicators such as data source, refresh frequency, data type, etc., which increases the storage efficiency while providing convenience for the data query and retrieval functions in the database. By creating different communication clients based on the public configuration of Springboot, adding the created different communication clients to the client cache queue, locating the target communication client of the client cache queue according to the trigger instruction, enabling the message communication function of the target client according to the communication parameters, storing the message data received by the target client in real time into the cache, parsing the message data read from the cache, and storing the parsed message data in different databases according to the sampling frequency level corresponding to the actual business scenario, the problem of data collection difficulty caused by the use of multiple different communication protocols in ship equipment in the prior art can be effectively solved. The method can support multiple different communication protocols, and can realize the rapid collection of ship equipment data and the efficient expansion of different protocols.

[0044] The present invention can adopt the factory design pattern to realize the design of different types of communication clients, including constructing an abstract communication client base class and a client factory class, utilizing the modular function (microservice characteristics) of springboot, and each subclass communication client is independently developed and deployed in the form of a microservice. Each communication protocol (such as HTTP, FTP, SMTP, etc.) realizes an independent microservice, and each microservice realizes its own communication client, that is, each communication client is packaged as an independent microservice to improve data collection efficiency. The factory pattern is used to create different communication clients, and the functions of the communication client are realized by calling external microservices, that is, the factory service dynamically selects different communication client microservices according to the configuration (such as the protocol type), which can ensure the collection efficiency of external device data, and when a certain communication client is abnormal, it will not affect other normal communication clients, and ensure the independence of the communication client. This architecture is very suitable for supporting multiple communication protocols in large-scale systems, and has good scalability and flexibility. If a new communication protocol (such as WebSocket, MQTT, etc.) needs to be added, it is only necessary to add a new microservice and update the protocol selection logic in the factory service.

[0045] When there is no message information of the current communication client in the cache, the present invention uses the monitoring function of the communication client to regularly monitor whether the message information of the current communication client is stored in the cache, so that the number of accesses to the cache can be effectively reduced and the waste of system resources can be reduced.

[0046] In the present invention, the message data is stored in the cache using a hash table storage structure, so that the message data in the cache can be quickly retrieved and read.

[0047] The present invention adopts a network service interface that follows the representational state transfer architectural style (i.e., a Web service interface that follows the REST (Representational State Transfer) architectural style, referred to as a restful interface) or a remote procedure call method to receive trigger instructions. In this way, the scalability of the interface is improved, and it is easier to maintain in the long term, and security issues such as SQL injection are reduced to a certain extent.

[0048] The present invention stores the parsed message data in different databases according to the collection frequency of the message data, and stores the parsed message data in different data tables according to the source of the device providing the message data. This increases the storage efficiency while facilitating the data query and retrieval functions in the database.

[0049] The present invention also relates to a ship equipment data acquisition system based on Springboot, which corresponds to the above-mentioned ship equipment data acquisition method based on Springboot, and can be understood as a system that implements the above-mentioned ship equipment data acquisition method based on Springboot, including a public configuration module, a management and scheduling module, a cache module, a message parsing module and a classification storage module, wherein the public configuration module can uniformly manage, configure and share communication parameters in a public configuration manner, simplify the configuration process, improve the configuration efficiency, and effectively solve the problem that the support for diversified communication protocols cannot be quickly expanded; the management and scheduling module can enable the business processing function in time when the equipment data needs to be collected, ensuring The high-speed cache module provides high-speed access to communication messages. On the one hand, it stores communication messages in real time, and on the other hand, it reads messages quickly, thereby realizing the decoupling between message communication and message analysis. The storage efficiency will not be affected by the analysis performance after reading, which is convenient for adapting to the communication refresh frequency of different devices, especially the reception of high-frequency data, so as to achieve the effect of quickly collecting device data. The message analysis module can recursively parse and process the message data in the cache in batches to prevent excessive accumulation of message data in the cache. The classification storage module can classify and store data into different databases / tables according to classification indicators such as data source, refresh frequency, data type, etc., which increases storage efficiency while providing convenience for data query and retrieval functions. The modules mentioned above work together to create different communication clients based on the public configuration of Springboot, add the created different communication clients to the client cache queue, locate the target communication client in the client cache queue according to the trigger instruction, enable the message communication function of the target client according to the communication parameters, store the message data received by the target client in real time into the cache, parse the message data read from the cache, and classify and store the parsed message data in different databases. This can effectively solve the problem of data collection difficulties caused by the use of multiple different communication protocols by ship equipment in the prior art. The system can support multiple different communication protocols, and can realize the rapid collection of ship equipment data and the efficient expansion of different protocols. Moreover, the Springboot framework can be used to quickly realize the combination with other communication frameworks, improve the development efficiency of the system and the rich functional expansion capabilities; use Springboot's unified configuration and dependency injection capabilities to quickly realize the management and use of public configurations; use Springboot's natural support for the Web to easily realize the integration of IT and OT systems on ships; use Springboot to quickly implement the system's microservices, and combine containerization tools to quickly realize the distributed deployment of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1It is a flow chart of the ship equipment data collection method based on Springboot of the present invention;

[0051] Figure 2 This is a structural block diagram of the ship equipment data acquisition system based on Springboot of the present invention. DETAILED DESCRIPTION

[0052] The present invention discloses a ship equipment data collection method based on Springboot, which aims to improve the development efficiency of ship equipment data collection and improve the efficiency of ship equipment data collection, and realizes the rapid collection and expansion of ship equipment data from the perspective of software development. The method can be executed by a processor or an electronic device with processing capability, such as Figure 1 As shown, first configure the communication parameters, create different types of communication clients, then add them to the client cache queue, wait for the trigger instruction to start, then start the communication transmission, receive data in real time and store it in the cache, then take out the original message from the cache for parsing, and finally classify and store the parsed data in different databases / tables. The present invention uses the public configuration and unified injection of Springboot to create different types of communication clients according to different communication parameter configurations to support message communications between diversified communication protocols and devices from different suppliers, and uses cache to decouple message communication and message parsing to adapt to different communication refresh frequencies, especially high-frequency message communications, to prevent message communication congestion. After message parsing, the data is classified and stored in different databases / tables according to classification indicators such as data source, refresh frequency, and data type to provide convenient data query and retrieval functions. Specifically, the method includes the following steps:

[0053] S1, based on the public configuration and injection method of Springboot, different types of communication clients are created according to different communication parameters; wherein, each of the different types of communication clients includes four parts: different communication parameters, communication methods, message parsing methods and communication status. Preferably, the communication parameters include basic information of the local client, basic information of the remote client and communication connection information. Among them, the basic information of the local client includes the IP address, port, creator and creation time of the local client; the basic information of the remote client includes the IP address, port, refresh frequency and sensor information list of the remote client. The communication connection information includes the communication connection method and the communication protocol type. The communication connection method includes Ethernet and serial port forms, etc. The communication protocol types include TCP, UDP, MQTT, etc. The sensor information list includes sensor number, measurement type, measurement range, communication coding address, etc. The communication method is a method in which the communication client has a message communication with external devices of multiple different communication protocol types. Message parsing is the function of parsing and restoring different message protocols. The communication status of the communication client may include not enabled, ready to enable, communicating, communication interrupted, communication abnormal, stopping and stopping, and different states are used to indicate the current usage of the client. The communication status is stored in a high-speed cache for immediate reading and modification. Each communication client uses a separate thread, and a newly created communication client is in an inactive state.

[0054] Furthermore, step S1 may also include the following steps:

[0055] S11: Create different types of communication clients based on Springboot's common configuration and injection methods;

[0056] S12: storing the communication status of the created different types of communication clients in the cache respectively;

[0057] S13: Create a dedicated device information maintenance client for each communication client.

[0058] In the embodiment of the present application, each device that needs to collect data has a configuration information stored in the public configuration of Springboot, and the configuration information includes the communication protocol, the IP address and port of the device that needs to collect data, etc. Each configuration information in the public configuration of Springboot will create a corresponding communication client. The communication client and the device that needs to collect data are in a one-to-one relationship, that is, a communication client only collects data from the device corresponding to it. The newly created communication client is in an unactivated state (the communication client has not established a communication connection with the corresponding device that needs to collect data), and the status information of the communication client is stored in the cache. Each of the communication clients will be independently deployed in the form of microservices, so that the efficiency of device data collection can be guaranteed, and when a communication client is abnormal, it will not affect other normal communication clients, thereby ensuring the independence of the communication client.

[0059] The embodiment of the present application adopts the factory design pattern to implement the design of different types of communication clients. First, an abstract base class (abstract class) of the communication client and a factory class of the communication client (or called an abstract communication client base class and a client factory class) are constructed. Then, the factory class of the communication client generates a subclass communication client inherited from the abstract base class of the communication client according to the communication parameters. The abstract communication client base class constrains the basic communication properties and methods of the communication client. The subclass communication client inherits from the abstract communication client base class, and has both the basic communication parameters and methods of the communication client and can define its own special communication parameters, methods, and message parsing, etc. Each subclass communication client is developed separately into a separate microservice by using the modularization function of springboot.

[0060] For example, the existing devices (external devices) that need to collect data include shaft power device signals and shaft vibration devices, where the shaft power device uses the TCP protocol for communication, and the shaft vibration device uses the MQTT protocol for communication. The IP addresses, ports and other information of the shaft power device and the shaft vibration device are added to the public configuration of Springboot. The factory class of the communication client generates and creates a communication client for collecting shaft power device data and a communication client for collecting shaft vibration device data according to the two configuration information about the shaft power device and the shaft vibration device in the public configuration of Springboot. The communication client for collecting shaft power device data supports establishing a connection with the shaft power device using the TCP protocol to collect shaft power device data; the communication client for collecting shaft vibration device data supports establishing a connection with the shaft vibration device using the MQTT protocol to collect shaft vibration device data. In this way, unified management, configuration and sharing of communication parameters are achieved, the configuration process is simplified, the configuration efficiency is improved, and diversified communication protocols can be quickly expanded.

[0061] Optionally, the equipment that needs to collect data may be shaft vibration equipment, engine room monitoring alarm system, GPS, depth sounder, shaft power equipment, shipboard tilt equipment, etc.

[0062] In the embodiment of the present application, the local client and the remote client represent the abstract representation of the two communicating parties, the local client represents the communication client, and the remote client represents the device that needs to collect data. The functions of the local client mainly include connection, cancellation, communication, suspension of communication, stop of communication and management. Connection refers to the establishment of a communication connection between the local client and the device that needs to collect data; communication refers to the process of data transmission between the two parties after the connection is established; management includes the startup, shutdown, release and client operation status monitoring of the local client.

[0063] The embodiment of the present application creates its own exclusive device information maintenance client for each communication client. The device information maintenance client stores all the information about the device collected by the current communication client (taking the shaft power device signal collection scenario as an example, the information in the device information maintenance client includes IP address information, port information, communication protocol, baud rate, message parsing method, etc.). In this way, the information about the device collected by the current communication client can be updated and maintained in real time to ensure the accuracy of the device information used by the communication client.

[0064] S2, adding the created different types of communication clients to the client cache queue, waiting for the external trigger command to start.

[0065] This step adds the created different types of communication clients to the client cache queue, and sets the communication status to ready to be enabled, waiting for external trigger instructions to start communication; at the same time, the client queue is maintained in the program memory, the client is retrieved according to the IP address of the local client, and the trigger of the remote external instruction is received using the restful interface or remote procedure call method.

[0066] Furthermore, step S2 may also include the following steps:

[0067] S21: adding the created communication clients of different types to the client cache queue;

[0068] S22: setting the state information specific to different communication clients in the cache to a ready-to-use state.

[0069] The client cache queue in the embodiment of the present application is maintained in the memory of the communication client, and the specified communication client can be retrieved according to the IP address and port of the local client. After completing steps S21 and S22, the current communication client is ready to wait for the external network service interface that follows the representational state transfer architecture style (i.e., the Web service interface that follows the REST (Representational State Transfer) architecture style, referred to as the restful interface) or the remote procedure call (Remote Procedure Call, RPC) to send a trigger instruction, thereby enabling the message communication function and establishing a communication connection with the device that needs to be collected.

[0070] The embodiment of the present application adds the created communication client to the client cache queue and sets the status of the communication client to an enabled state, so that the message communication function can be enabled in time when the current communication client is needed, thereby ensuring the rapid collection of device data.

[0071] S3, after detecting a trigger instruction for a target communication client, locate the target communication client through the client cache queue, enable the message communication function of the target communication client according to the communication parameters, and receive message data in real time through the target communication client, and then store the real-time received message data in the cache.

[0072] Furthermore, step S3 may also include the following steps:

[0073] S31: receiving a trigger instruction for a target communication client;

[0074] S32: locating the target communication client through the client cache queue;

[0075] S33: according to the communication parameters, enabling the message communication function of the target communication client so that the target communication client establishes a communication connection with the external device to be collected data;

[0076] S34: if the message communication function is successfully enabled, the message data sent by the external device to be collected is received in real time, and the message data is stored in a cache in a hash table storage structure according to the time sequence of the message data, and the communication state of the target communication client is set to normal communication;

[0077] S35: If the message communication function fails to be enabled, the state information of the target communication client in the cache is set to a communication abnormal state.

[0078] Furthermore, the embodiments of the present application receive trigger instructions through a network service interface that follows the representational state transfer architectural style (i.e., a Web service interface that follows the REST (Representational State Transfer) architectural style, referred to as a restful interface) or a remote procedure call (RPC). In this way, the scalability of the interface is improved, and it is easier to maintain in the long term, and security issues such as SQL injection are reduced to a certain extent.

[0079] In the embodiment of the present application, the user sends a trigger instruction through a Web page according to the actual scenario to enable the message communication function of the target communication client. After the system detects the trigger instruction for the target communication client, it locates the target communication client through the client cache queue. After the message communication function of the target communication client is enabled, according to the target IP address, port and communication protocol type and other information in the communication parameters corresponding to the target communication client, a communication connection is established between the target communication client and the device to be collected, and then the message data sent by the device to be collected is received. If the message communication function fails to start or the communication connection between the target communication client and the device to be collected fails to be established, the status information of the target communication client in the cache is set to the abnormal communication state, and the subsequent maintenance personnel will maintain the target communication client according to the target communication client status information and log information displayed on the management page (Web page), and then re-enable the message communication function of the target communication client.

[0080] In the embodiment of the present application, the cache is used to provide high-speed access to message data. On the one hand, it can provide the ability to store message data in real time and efficiently, and on the other hand, it can provide the ability to quickly read the message data in the cache, thereby realizing the decoupling between the two processes of message communication and message parsing. In this way, the message data can be stored in real time, and the efficiency of receiving the message data will not be affected by the performance of the parsing process after receiving the message data. It is also convenient to adapt to the communication refresh frequency of different devices, especially the reception of high-frequency data, to achieve the effect of quickly collecting device data.

[0081] In the embodiment of the present application, the message data stored in the cache adopts a hash table structure, including the remote communication client IP, message time and message content. In this way, fast retrieval and reading of the message data in the cache can be achieved.

[0082] Preferably, the cache can be implemented using a cache framework or technology such as JCache, EhCache, or Redis.

[0083] S4, assigning a parsing thread to the target communication client, the parsing thread reads the message data from the cache in chronological order, and parses and processes the message data through the target communication client to obtain parsed data.

[0084] Furthermore, step S4 may also include the following steps:

[0085] S41: If the state information of the target communication client in the cache is a normal communication state, assigning a message parsing thread to the target communication client;

[0086] S42: After the message parsing thread is started, immediately read the message data in batches from the cache in chronological order;

[0087] S43: If the message parsing thread successfully reads the message data, the message parsing method of the maintenance client is maintained according to the exclusive device information of the target communication client, and the read message data is recursively parsed;

[0088] S44: Repeat steps S42-S43 until the message data stored in the target communication client in the cache is processed and cleared;

[0089] S45: If the message parsing thread cannot read the message data from the cache, that is, the cached message has been read and cleared, the monitoring function of the target communication client (or the listening mode) is started, and the cache is monitored regularly to determine whether there is new message data;

[0090] S46: Once it is detected that there is new message data in the cache, jump to step S42;

[0091] S47: If no new message data is monitored, the monitoring continues until the state information of the target communication client in the cache is changed to a stop state, and then the monitoring function of the target communication client is turned off.

[0092] When the system is initialized, the embodiment of the present application creates a parsing thread pool. When the communication client is in normal communication status, a thread is allocated to the communication client from the parsing thread pool to parse the message data; when the communication client is in stopped status, the parsing thread of the communication client is recycled to the parsing thread pool.

[0093] Furthermore, in the embodiment of the present application, the parsing thread reads the original message from the cache in chronological order and parses it, and uses Springboot's aspect-oriented programming (AOP, Aspect-Oriented Programming) method to monitor and record the parsing status of the message data in real time.

[0094] Furthermore, the number of message data that the parsing thread can read from the cache each time is mainly set according to the amount of device data collected and the size of the cache, and can be set to read 5 to 100 message data in each batch. In practice, since the amount of data collected from a certain device on a ship is known, during the communication process with the communication partner (such as the shaft power device), after the communication between the two parties has been stable for a period of time, the number of message data to be read in each batch is determined based on the amount of remaining message data in the cache corresponding to the current parsing thread.

[0095] Furthermore, the period of the scheduled monitoring is obtained according to an empirical formula, as shown in formula (1):

[0096] Cache size - collection frequency * message length * timing period ≥ c * cache size (0 <c≤1)(1)

[0097] The cache size refers to the maximum amount of data that can be stored in the cache; the acquisition frequency refers to the frequency of data acquisition for the equipment to be acquired; and the message length refers to the total length of the message data sent by the equipment to be acquired. For example, the commonly used acquisition frequency of ship equipment is in the second level, and the timing period set empirically is 0.5 seconds to 30 seconds.

[0098] The embodiment of the present application reads message data in batches from a cache and then performs parsing processing. In this way, the cache can be used to decouple message communication and message parsing, thereby preventing message data congestion and achieving the effect of quickly processing the message data of the device.

[0099] S5, classify and store the parsed data into different databases according to the sampling frequency levels corresponding to the actual business scenarios, and classify the parsed data stored in the same database into different data tables according to the data sources.

[0100] Furthermore, the database can be divided into several categories according to the business type, such as basic information database, high-frequency storage database, medium-frequency storage database, and low-frequency storage database. Among them, the basic information database is mainly used to store the basic information of the communication client, communication parameter configuration, and message protocol information. According to the actual data collection scenario of ship equipment, the data collection frequency can be divided into three categories: low frequency, medium frequency, and high frequency. The low-frequency data collection frequency can be stored in the low-frequency storage database, the medium-frequency data collection frequency can be stored in the medium-frequency storage database, and the high-frequency data collection frequency can be stored in the high-frequency storage database. Low-frequency collection generally refers to the frequency of minutes, such as collecting data of shaft vibration equipment, usually collected every 5 minutes or 10 minutes; medium-frequency collection generally refers to the frequency of seconds, which is the most widely used scenario, such as the cabin monitoring alarm system, GPS equipment, depth sounder equipment, shaft power equipment, etc., which are generally set to collect data every second or every two seconds; high-frequency collection generally refers to millisecond-level or higher frequency data collection, which is relatively rare at present, such as the data collection frequency of the tilting equipment on the ship is usually between 50 milliseconds and 500 milliseconds.

[0101] The embodiment of the present application classifies the data stored in the same database into different data tables according to the data source, and each device has its own exclusive data table. Taking the shaft power device and GPS device as an example, the shaft power device and GPS device are both medium frequency acquisition frequencies, and the collected message data are all stored in the medium frequency storage database. At this time, according to the different data source devices, the data collected from the shaft power device is stored in the shaft power data table, and the data collected from the GPS device is stored in the GPS data table.

[0102] The embodiment of the present application can classify and store data into different databases and data tables according to classification indicators such as data source, refresh frequency, and data type, thereby increasing storage efficiency and facilitating data query and retrieval functions.

[0103] Preferably, the basic database can adopt Mysql, SQL server and the like.

[0104] Preferably, databases such as the high-frequency storage database, the medium-frequency storage database, and the low-frequency storage database used to store data with different collection frequencies may adopt time series databases such as InfluxDB, TDEngine, and the like.

[0105] The Springboot framework used in the embodiment of the present application has natural support for the Web (webpage), which makes it easy to realize the integration of IT and OT systems on ships. Using this framework, microservices of the system can be quickly built, and combined with containerization tools, the distributed deployment of the system can be quickly realized, thereby supporting higher data concurrent processing capabilities.

[0106] The embodiment of the present application utilizes the public configuration of Springboot to configure corresponding communication parameters for different protocols of the device, create different types of communication clients, simplify the configuration process, improve configuration efficiency, and effectively solve the problem that support for diversified communication protocols cannot be quickly expanded; the newly created communication client is added to the client cache queue to ensure that when it is necessary to expand support for the communication protocol, rapid startup is completed and device data is quickly collected; data is received in real time and stored in the cache, message data read in batches from the cache is parsed, and the parsed message data is classified and stored in different databases and data tables. The cache can be used to decouple message communication and message parsing, and the reception, parsing and storage of message data are efficiently completed, so as to realize the rapid collection of ship equipment data.

[0107] Based on the same inventive concept, one or more embodiments of the present specification also provide a ship equipment data acquisition system based on Springboot. Since the principle of solving the problem by the ship equipment data acquisition system based on Springboot is similar to the aforementioned ship equipment data acquisition method based on Springboot, the implementation of the ship equipment data acquisition system based on Springboot can refer to the aforementioned implementation of the ship equipment data acquisition method based on Springboot, and the repeated parts will not be repeated.

[0108] Figure 2 This is a structural block diagram of a ship equipment data acquisition system based on Springboot provided in one or more embodiments of this specification. Figure 2 As shown, the ship equipment data acquisition system based on Springboot includes: a public configuration module 101, a management and scheduling module 102, a cache module 103, a message parsing module 104 and a classification storage module 105. Among them,

[0109] The public configuration module 101 is used for unified configuration and configuration sharing between modules, and maintains the public configuration of Springboot; based on the public configuration and injection method of Springboot, different types of communication clients are created according to different communication parameters, that is, the communication parameter configuration of different communication clients is quickly realized based on the default configuration template of the Springboot framework. Each of the different types of communication clients includes different communication parameters, communication methods, message parsing and communication status. The communication parameters include the communication protocol, the IP address and port of the external device to be collected data. The communication method is a method for the communication client to communicate with external devices of multiple different communication protocols. The message parsing is parsed and restored for messages of different communication protocols. The communication status includes several states of whether it is enabled and whether it is communicating normally, and different communication states are used to indicate the current usage of the communication client. The public configuration module 101 can be made into a separate microservice to provide a public configuration sharing function for other microservices.

[0110] The management and scheduling module 102 manages the different types of communication clients created; adds the different types of communication clients created to the client cache queue, waiting for the external trigger instruction to start; after detecting the trigger instruction for the target communication client, locates the target communication client through the client cache queue, enables the message communication function of the target communication client according to the communication parameters, and receives the message data in real time through the target communication client; uses Springboot's RestController to receive external trigger instructions, triggers the creation, start, stop, interruption and other operations of the communication client, and maintains the client management queue and the communication status of each communication client.

[0111] The cache module 103 is used to store the real-time received message data into the cache, providing fast access capability for real-time messages; it uses springboot to integrate other cache frameworks, such as JCache, EhCache, Redis, etc.

[0112] The message parsing module 104 is used to read and parse messages from the cache. Specifically, a parsing thread is assigned to the target communication client. The parsing thread reads message data from the cache in chronological order and parses the message data through the target communication client to obtain parsed data.

[0113] The classification storage module 105 is used to provide data classification storage capabilities, classify and store the parsed data into different databases according to the sampling frequency level corresponding to the actual business scenario, and classify and store the parsed data stored in the same database into different data tables according to the data source. Springboot can be used to integrate database frameworks such as mybatis, mybatis-plus, etc.

[0114] Furthermore, the common configuration module adopts the factory design pattern to implement the design of different types of communication clients, including constructing an abstract communication client base class and a client factory class. The abstract communication client base class constrains the basic communication properties and methods of the communication client. The client factory class generates different subclass communication clients according to different communication parameter configurations. The subclass communication client inherits from the abstract communication client base class, and has both the basic communication parameters and methods of the communication client and customized communication parameters, methods and message parsing. Using the modularization function of springboot, each subclass communication client is developed separately into a separate microservice.

[0115] Furthermore, the management and scheduling module enables the message communication function of the target communication client according to the communication parameters, so that the target communication client establishes a communication connection with the external device for collecting data; if the message communication function is successfully enabled, the message data sent by the external device for collecting data is received in real time, and then the cache module stores the message data in the cache in a hash table storage structure according to the time sequence of the message data, and then the management and scheduling module sets the communication status of the target communication client to normal communication; if the message communication function fails to be enabled, the management and scheduling module sets the status information of the target communication client to the abnormal communication status.

[0116] Furthermore, after the status information of the target communication client is normal communication, the message parsing module allocates a message parsing thread to it, and the message parsing thread reads message data in batches from the cache in chronological order; if the message parsing thread successfully reads the message data, the read message data is recursively parsed until the message data stored in the cache by the target communication client is processed and cleared; if the message parsing thread cannot read the message data from the cache, the monitoring function of the target communication client is started, and the cache is monitored regularly for whether there is new message data; once the cache is monitored for new message data, the recursive parsing process is continued; if no new message data is monitored, the monitoring is continued until the status information of the target communication client in the cache changes to a stopped state, and the monitoring function of the target communication client is closed.

[0117] Furthermore, in the classified storage module, the database includes a basic information database, a high-frequency storage database, a medium-frequency storage database, and a low-frequency storage database; the basic information database is implemented using Mysql and SQL server, and is used to store basic information of the communication client, communication parameter configuration, and message protocol; databases of different frequencies are implemented using InfluxDB and TDEngine time series databases; the low-frequency storage database is used to store data with a collection frequency of minutes corresponding to actual business scenarios; the medium-frequency storage database is used to store data with a collection frequency of seconds corresponding to actual business scenarios; and the high-frequency storage database is used to store data with a collection frequency of milliseconds or microseconds corresponding to actual business scenarios.

[0118] The ship equipment data acquisition system of the present invention is based on Springboot. The Springboot framework can be used to quickly realize the combination with other communication frameworks, thereby improving the development efficiency and rich functional expansion capabilities of the system; the unified configuration and dependency injection capabilities of Springboot are used to quickly realize the management and use of public configurations; the natural support of Springboot for Web is used to easily realize the integration of IT and OT systems on ships; Springboot is used to quickly realize the microservices of the system, and the distributed deployment of the system is quickly realized in combination with containerization tools.

[0119] It should be noted that the above-described specific implementations can enable those skilled in the art to more fully understand the invention, but do not limit the invention in any way. Therefore, although this specification has described the invention in detail with reference to the drawings and embodiments, those skilled in the art should understand that the invention can still be modified or replaced by equivalents. In short, all technical solutions and improvements that do not deviate from the spirit and scope of the invention should be included in the protection scope of the patent for the invention.

Claims

1. A ship equipment data collection method based on Springboot, characterized in that: The following steps are involved: S1: Based on the public configuration and injection method of Springboot, different types of communication clients are created according to different communication parameters; wherein each of the different types of communication clients includes different communication parameters, communication methods, message parsing and communication status, the communication parameters include the communication protocol, the IP address and port of the external device to be collected, the communication method is a method for the communication client to communicate with external devices of multiple different communication protocols, the message parsing parses and restores messages of different communication protocols, the communication status includes several states of whether it is enabled and whether the communication is normal, and different communication states are used to indicate the current usage of the communication client; S2: Add the created different types of communication clients to the client cache queue and wait for external trigger instructions to start; S3: after detecting a trigger instruction for a target communication client, locating the target communication client through the client cache queue, enabling a message communication function of the target communication client according to the communication parameters, and receiving message data in real time through the target communication client, and then storing the received message data in real time into a cache; S4: allocating a parsing thread to the target communication client, the parsing thread reading the message data from the cache in chronological order, and parsing the message data through the target communication client to obtain parsed data; S5: The parsed data are classified and stored in different databases according to the sampling frequency levels corresponding to the actual business scenarios, and the parsed data stored in the same database are classified again according to the data source and stored in different data tables.

2. The ship equipment data collection method based on Springboot according to claim 1 is characterized in that: Step S1 adopts the factory design pattern to implement the design of different types of communication clients, including constructing an abstract communication client base class and a client factory class. The abstract communication client base class constrains the basic communication properties and methods of the communication client. The client factory class generates different subclass communication clients according to different communication parameter configurations. The subclass communication client inherits from the abstract communication client base class, and has both the basic communication parameters and methods of the communication client and customized communication parameters, methods and message parsing. Using the modular function of springboot, each subclass communication client is developed separately into a separate microservice.

3. The ship equipment data collection method based on Springboot according to claim 1 or 2 is characterized in that: In step S1, the communication parameters include basic information of the local client, basic information of the remote client, and communication connection information; the basic information of the local client includes the IP address, port, creator, and creation time of the local client; the basic information of the remote client includes the IP address, port, device refresh frequency, and sensor information list of the remote client; the communication connection information includes the communication connection mode and the communication protocol type; the communication connection mode includes Ethernet and serial port; the communication protocol type includes TCP, UDP, and MQTT; the sensor information list includes sensor number, measurement type, measurement range, and communication coding address; The communication status of the communication client includes not enabled, ready to enable, communicating, communication interrupted, communication abnormal, stopping and stopped. The communication status is stored in the cache for instant reading and modification. Each communication client uses a separate thread, and the newly created communication client is in the not enabled state.

4. The ship equipment data collection method based on Springboot according to claim 3 is characterized in that: Step S2 adds the created different types of communication clients to the client cache queue, and sets the communication status to ready to be enabled, waiting for an external trigger instruction to start the communication; at the same time, the client queue is maintained in the program memory, the client is retrieved according to the IP address of the local client, and the trigger of the remote external instruction is received using the restful interface or remote procedure call method.

5. The ship equipment data collection method based on Springboot according to claim 3 is characterized in that: Step S3 includes: S31: receiving a trigger instruction for a target communication client; S32: locating the target communication client through the client cache queue; S33: according to the communication parameters, enabling the message communication function of the target communication client so that the target communication client establishes a communication connection with the external device to be collected data; S34: if the message communication function is successfully enabled, the message data sent by the external device to be collected is received in real time, and the message data is stored in a cache in a hash table storage structure according to the time sequence of the message data, and the communication state of the target communication client is set to normal communication; S35: If the message communication function fails to be enabled, the state information of the target communication client in the cache is set to a communication abnormal state.

6. The ship equipment data collection method based on Springboot according to claim 5 is characterized in that: Step S4 includes: S41: If the state information of the target communication client in the cache is a normal communication state, assigning a message parsing thread to the target communication client; S42: reading message data in batches from the cache in chronological order through the message parsing thread; S43: If the message parsing thread successfully reads the message data, the message parsing method of the maintenance client is maintained according to the exclusive device information of the target communication client, and the read message data is recursively parsed; S44: Repeat steps S42-S43 until the message data stored in the target communication client in the cache is processed and cleared; S45: If the message parsing thread cannot read the message data from the cache, the monitoring function of the target communication client is started, and whether there is new message data in the cache is monitored regularly; S46: Once it is detected that there is new message data in the cache, jump to step S42; S47: If no new message data is monitored, the monitoring continues until the state information of the target communication client in the cache is changed to a stop state, and then the monitoring function of the target communication client is turned off.

7. The ship equipment data collection method based on Springboot according to claim 1 is characterized in that: In step S5, the database includes a basic information database, a high-frequency storage database, a medium-frequency storage database, and a low-frequency storage database; The basic information database is implemented using Mysql and SQL server, and is used to store the basic information of the communication client, communication parameter configuration and message protocol. The databases of different frequencies are implemented using InfluxDB and TDEngine time series databases. The low-frequency storage database is used to store data with a collection frequency of minutes corresponding to the actual business scenarios. The medium-frequency storage database is used to store data with a collection frequency of seconds corresponding to the actual business scenarios. The high-frequency storage database is used to store data with a collection frequency of milliseconds or microseconds corresponding to the actual business scenarios.

8. A ship equipment data acquisition system based on Springboot, characterized in that: It includes a public configuration module, a management and scheduling module, a cache module, a message parsing module and a classification storage module connected in sequence; wherein, The public configuration module is used to maintain the public configuration of Springboot, and based on the public configuration and injection method of Springboot, different types of communication clients are created according to different communication parameters; each of the different types of communication clients includes different communication parameters, communication methods, message parsing and communication status, the communication parameters include the communication protocol, the IP address and port of the external device to be collected data, the communication method is a method for the communication client to communicate with external devices of multiple different communication protocols, the message parsing is parsed and restored for messages of different communication protocols, the communication status includes several states of whether it is enabled and whether the communication is normal, and different communication states are used to indicate the current usage of the communication client; The management and scheduling module is used to manage the different types of communication clients created; add the different types of communication clients created to the client cache queue, waiting for the external trigger instruction to start; after detecting the trigger instruction for the target communication client, locate the target communication client through the client cache queue, enable the message communication function of the target communication client according to the communication parameters, and receive the message data in real time through the target communication client; use Springboot's RestController to receive external trigger instructions, trigger the creation, start, stop, and interruption operations of the communication client, and maintain the communication status of each communication client; The cache module is implemented by integrating JCache, EhCache and Redis cache frameworks using springboot, and is used to store the message data received in real time into the cache; The message parsing module is used to allocate a parsing thread to the target communication client, and the parsing thread reads the message data from the cache in chronological order, and parses and processes the message data through the target communication client to obtain parsed data; The classification storage module is used to classify and store the parsed data into different databases according to the sampling frequency levels corresponding to the actual business scenarios, and to classify and store the parsed data stored in the same database into different data tables again according to the data source.

9. The ship equipment data acquisition system based on Springboot according to claim 8 is characterized in that: The common configuration module adopts the factory design pattern to implement the design of different types of communication clients, including constructing an abstract communication client base class and a client factory class. The abstract communication client base class constrains the basic communication properties and methods of the communication client. The client factory class generates different subclass communication clients according to different communication parameter configurations. The subclass communication client inherits from the abstract communication client base class, and has both the basic communication parameters and methods of the communication client and customized communication parameters, methods and message parsing. Using the modular function of springboot, each subclass communication client is developed separately into a separate microservice.

10. The ship equipment data acquisition system based on Springboot according to claim 8 or 9, characterized in that: The management and scheduling module enables the message communication function of the target communication client according to the communication parameters, so that the target communication client establishes a communication connection with the external device to be collected data. If the message communication function is successfully enabled, the message data sent by the external device to be collected data is received in real time, and then the cache module stores the message data in the cache in a hash table storage structure according to the time sequence of the message data, and then the management and scheduling module sets the communication state of the target communication client to normal communication; If the message communication function fails to be enabled, the management and scheduling module sets the status information of the target communication client to a communication abnormality state.

11. The ship equipment data acquisition system based on Springboot according to claim 10, characterized in that: After the status information of the target communication client is normal communication, the message parsing module allocates a message parsing thread to it, and the message parsing thread reads the message data in batches from the cache in chronological order; if the message parsing thread successfully reads the message data, the read message data is recursively parsed until the message data stored in the cache by the target communication client is processed and cleared; If the message parsing thread cannot read the message data from the cache, the monitoring function of the target communication client is started, and whether there is new message data in the cache is monitored regularly; once the new message data is detected in the cache, the recursive parsing process is continued; If no new message data is monitored, the monitoring continues until the state information of the target communication client in the cache changes to a stop state, and then the monitoring function of the target communication client is turned off.

12. The ship equipment data acquisition system based on Springboot according to claim 8 or 9, characterized in that: In the classification storage module, the database includes a basic information database, a high-frequency storage database, a medium-frequency storage database, and a low-frequency storage database; The basic information database is implemented using Mysql and SQL server, and is used to store the basic information of the communication client, communication parameter configuration and message protocol. The databases of different frequencies are implemented using InfluxDB and TDEngine time series databases. The low-frequency storage database is used to store data with a collection frequency of minutes corresponding to the actual business scenarios. The medium-frequency storage database is used to store data with a collection frequency of seconds corresponding to the actual business scenarios. The high-frequency storage database is used to store data with a collection frequency of milliseconds or microseconds corresponding to the actual business scenarios.

Images