A ship equipment data acquisition method and system based on Springboot

Abstract

The present invention discloses a method and system for collecting ship equipment data based on Springboot. The method includes the following steps: S1, creating different types of communication clients based on the common configuration and injection method of Springboot; S2, adding the created different types of communication clients to a client cache queue, waiting for an external trigger instruction to start; S3, after detecting a trigger instruction for a target communication client, enabling the message communication function of the target communication client, and receiving message data in real time through the target communication client, and then storing the received message data in a cache; S4, reading the message data from the cache, and parsing and processing the message data through the target communication client to obtain parsed data; S5, classifying and storing the parsed data in different databases / tables. The present invention supports message communication between various communication protocols and equipment from different suppliers, and can achieve rapid collection and expansion of ship equipment data.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent ships, and in particular to a ship equipment data acquisition method and system based on Springboot. Background Art

[0002] At present, ship intelligence has become an irreversible development trend in the global shipping industry. With the continuous improvement of the level of intelligence, the number of automated systems and equipment installed on ships has also increased rapidly. This change has brought unprecedented efficiency improvement and operational optimization potential to the shipping industry.

[0003] However, during this process, a significant technical challenge gradually emerged: the various automated systems and equipment onboard a ship often come from different manufacturers, each of which may select different communication protocols to meet its specific functional requirements and design concepts. These protocols vary significantly in data formats, transmission methods, and interface standards, resulting in significant technical obstacles when ships attempt to integrate data from different devices and systems. This diversity of protocols greatly increases the complexity of data collection and management.

[0004] Due to protocol incompatibilities between devices, ship data integration is extremely complex. Current ship data acquisition and transmission devices are all independent, individual devices. Whenever new automation systems or equipment are needed during ship construction or modification, new data acquisition devices must be added, requiring frequent installation and data exchange debugging. This significantly reduces the efficiency of installing new automation systems or equipment and hinders the improvement of the ship's overall perception capabilities. Summary of the Invention

[0005] To address existing issues such as difficulty collecting data due to the use of multiple different communication protocols by ship equipment, this paper proposes a SpringBot-based ship equipment data collection method. This method supports multiple different communication protocols, enabling rapid collection of ship equipment data and efficient expansion of different protocols. The present invention also relates to a SpringBot-based ship equipment data collection system.

[0006] The present invention is achieved through the following technical solutions:

[0007] A ship equipment data collection method based on Springboot, comprising the following steps:

[0008] S1: Based on the public configuration and injection method of Springboot, different types of communication clients are created according to different communication parameters; wherein each of the different types of communication clients includes different communication parameters, communication methods, message parsing and communication status, the communication parameters include the communication protocol, the IP address and port of the external device to collect data, the communication method is a method for the communication client to communicate with external devices of multiple different communication protocols, the message parsing parses and restores messages of different communication protocols, the communication status includes several states of whether it is enabled and whether communication is normal, and different communication states are used to indicate the current usage of the communication client;

[0009] S2: Add different types of communication clients created to the client cache queue and wait for external trigger instructions to start;

[0010] S3: After detecting a trigger instruction for a target communication client, locating the target communication client through the client cache queue, enabling a message communication function of the target communication client according to the communication parameters, and receiving message data in real time through the target communication client, and then storing the received message data in a cache;

[0011] S4: allocating a parsing thread to the target communication client, the parsing thread reading the message data from the cache in chronological order, and parsing the message data through the target communication client to obtain parsed data;

[0012] S5: The parsed data are classified and stored in different databases according to the sampling frequency levels corresponding to the actual business scenarios, and the parsed data stored in the same database are classified again according to the data source and stored in different data tables.

[0013] Preferably, step S1 adopts the factory design pattern to implement the design of different types of communication clients, including constructing an abstract communication client base class and a client factory class. The abstract communication client base class constrains the basic communication properties and methods of the communication client. The client factory class generates different subclass communication clients according to different communication parameter configurations. The subclass communication client inherits from the abstract communication client base class and has both the basic communication parameters and methods of the communication client and customized communication parameters, methods and message parsing. By utilizing the modularization function of springboot, each subclass communication client is developed separately into a separate microservice.

[0014] Preferably, in step S1, the communication parameters include basic information of the local client, basic information of the remote client, and communication connection information; the basic information of the local client includes the IP address, port, creator, and creation time of the local client; the basic information of the remote client includes the IP address, port, device refresh frequency, and sensor information list of the remote client; the communication connection information includes a communication connection mode and a communication protocol type; the communication connection mode includes Ethernet and serial port forms; the communication protocol types include TCP, UDP, and MQTT; the sensor information list includes a sensor number, a measurement type, a measurement range, and a communication coding address;

[0015] The communication status of the communication client includes not enabled, ready to enable, communicating, communication interrupted, communication abnormal, stopping and stopped. The communication status is stored in the cache for immediate reading and modification. Each communication client uses a separate thread, and the newly created communication client is in the not enabled state.

[0016] Preferably, step S2 adds the created different types of communication clients to the client cache queue, and sets the communication status to ready to be enabled, waiting for an external trigger instruction to start the communication; at the same time, the client queue is maintained in the program memory, the client is retrieved according to the IP address of the local client, and the trigger of the remote external instruction is received using a restful interface or remote procedure call.

[0017] Preferably, step S3 includes the following steps:

[0018] S31: receiving a trigger instruction for a target communication client;

[0019] S32: Locating the target communication client through the client cache queue;

[0020] S33: activating the message communication function of the target communication client according to the communication parameters, so that the target communication client establishes a communication connection with the external device for data collection;

[0021] S34: If the message communication function is successfully enabled, the message data sent by the external device to be collected is received in real time, and the message data is stored in a cache in a hash table storage structure in a time sequence of the message data, and the communication state of the target communication client is set to normal communication;

[0022] S35: If the message communication function fails to be enabled, the status information of the target communication client in the cache is set to a communication abnormality state.

[0023] Preferably, step S4 includes the following steps:

[0024] S41: If the status information of the target communication client in the cache is a normal communication status, assigning a message parsing thread to the target communication client;

[0025] S42: Reading message data in batches from the cache in chronological order through the message parsing thread;

[0026] S43: If the message parsing thread successfully reads the message data, the message parsing method of the maintenance client is maintained according to the exclusive device information of the target communication client, and the read message data is recursively parsed;

[0027] S44: Repeat steps S42-S43 until the message data stored in the cache by the target communication client is processed and cleared;

[0028] S45: If the message parsing thread cannot read the message data from the cache, the monitoring function of the target communication client is activated, and the cache is regularly monitored for new message data.

[0029] S46: Once it is detected that there is new message data in the cache, jump to step S42;

[0030] S47: If no new message data is monitored, continue monitoring until the status information of the target communication client in the cache changes to a stopped state, and then disable the monitoring function of the target communication client.

[0031] Preferably, in step S5, the database includes a basic information database, a high-frequency storage database, a medium-frequency storage database, and a low-frequency storage database; the basic information database is implemented using Mysql and SQL server, and is used to store basic information of the communication client, communication parameter configuration and message protocol, and databases of different frequencies are implemented using InfluxDB and TDEngine time series databases. The low-frequency storage database is used to store data with a collection frequency of minutes corresponding to actual business scenarios, the medium-frequency storage database is used to store data with a collection frequency of seconds corresponding to actual business scenarios, and the high-frequency storage database is used to store data with a collection frequency of milliseconds or microseconds corresponding to actual business scenarios.

[0032] A ship equipment data acquisition system based on Springboot, including a public configuration module, a management and scheduling module, a cache module, a message parsing module and a classification storage module connected in sequence; wherein,

[0033] The public configuration module is used to maintain the public configuration of Springboot, and based on the public configuration and injection method of Springboot, different types of communication clients are created according to different communication parameters; each of the different types of communication clients includes different communication parameters, communication methods, message parsing and communication status, the communication parameters include the communication protocol, the IP address and port of the external device to collect data, the communication method is a method for the communication client to communicate with external devices of multiple different communication protocols, the message parsing parses and restores messages of different communication protocols, and the communication status includes several states of whether it is enabled and whether communication is normal, and different communication states are used to indicate the current usage of the communication client;

[0034] The management and scheduling module is used to manage different types of communication clients created; add different types of communication clients created to the client cache queue, waiting for external trigger instructions to start; after detecting the trigger instruction for the target communication client, locate the target communication client through the client cache queue, enable the message communication function of the target communication client according to the communication parameters, and receive message data in real time through the target communication client; use Springboot's RestController to receive external trigger instructions, trigger the creation, start, stop, and interrupt operations of the communication client, and maintain the communication status of each communication client;

[0035] The cache module is implemented by integrating JCache, EhCache and Redis cache framework using Springboot, and is used to store the message data received in real time into the cache;

[0036] The message parsing module is used to allocate a parsing thread to the target communication client, and the parsing thread reads the message data from the cache in chronological order, and parses and processes the message data through the target communication client to obtain parsed data;

[0037] The classification storage module is used to classify and store the parsed data into different databases according to the sampling frequency levels corresponding to the actual business scenarios, and to classify the parsed data stored in the same database into different data tables according to the data sources.

[0038] Preferably, the common configuration module adopts the factory design pattern to implement the design of different types of communication clients, including constructing an abstract communication client base class and a client factory class. The abstract communication client base class constrains the basic communication properties and methods of the communication client. The client factory class generates different subclass communication clients according to different communication parameter configurations. The subclass communication client inherits from the abstract communication client base class and has both the basic communication parameters and methods of the communication client and customized communication parameters, methods and message parsing. By utilizing the modularization function of springboot, each subclass communication client is developed separately into a separate microservice.

[0039] Preferably, the management and scheduling module enables the message communication function of the target communication client according to the communication parameters, so that the target communication client establishes a communication connection with the external device to be collected data. If the message communication function is successfully enabled, the message data sent by the external device to be collected data is received in real time, and then the cache module stores the message data in the cache in a hash table storage structure according to the time sequence of the message data, and then the management and scheduling module sets the communication status of the target communication client to normal communication; if the message communication function fails to be enabled, the management and scheduling module sets the status information of the target communication client to a communication abnormality status.

[0040] Preferably, after the status information of the target communication client indicates normal communication, the message parsing module allocates a message parsing thread to it, and the message parsing thread reads message data in batches from the cache in chronological order; if the message parsing thread successfully reads the message data, the message parsing thread performs recursive parsing processing on the read message data until the message data stored in the cache by the target communication client is processed and cleared; if the message parsing thread cannot read the message data from the cache, the monitoring function of the target communication client is activated, and the cache is periodically monitored for new message data; once the cache is detected to contain new message data, the recursive parsing processing is continued; if no new message data is monitored, the monitoring is continued until the status information of the target communication client in the cache changes to a stopped state, and the monitoring function of the target communication client is disabled.

[0041] Preferably, in the classified storage module, the database includes a basic information database, a high-frequency storage database, a medium-frequency storage database, and a low-frequency storage database; the basic information database is implemented using Mysql and SQL server, and is used to store the basic information of the communication client, communication parameter configuration and message protocol, and the databases of different frequencies are implemented using InfluxDB and TDEngine time series databases. The low-frequency storage database is used to store data with a collection frequency of minutes corresponding to the actual business scenario, the medium-frequency storage database is used to store data with a collection frequency of seconds corresponding to the actual business scenario, and the high-frequency storage database is used to store data with a collection frequency of milliseconds or microseconds corresponding to the actual business scenario.

[0042] The beneficial effects of the present invention are:

[0043] The present invention provides a ship equipment data collection method based on Springboot, which includes: based on the public configuration and injection mode of Springboot, creating different types of communication clients according to different communication parameters, thus realizing unified management, configuration and sharing of communication parameters, simplifying the configuration process, improving configuration efficiency, and effectively solving the problem that support for diversified communication protocols cannot be quickly expanded; adding different types of created communication clients to the client cache queue, waiting for external trigger instructions to start, so that the message communication function can be enabled in time when device data needs to be collected, ensuring rapid collection of device data; after detecting the trigger instruction for the target communication client, locating the target communication client through the client cache queue, enabling the message communication function of the target communication client according to the communication parameters, and The client receives message data in real time, and then stores the received message data in a high-speed cache. In this way, the message data can be stored in real time, and the efficiency of receiving the message data will not be affected by the performance of the parsing process after receiving the message data. It is also convenient to adapt to the communication refresh frequency of different devices, especially the reception of high-frequency data, so as to achieve the effect of quickly collecting device data; the message data is read from the cache, and the message data is parsed and processed by the target communication client to obtain the parsed data. In this way, the cache can be used to decouple message communication and message parsing to prevent message data congestion and achieve the effect of quickly processing device data; the parsed data is classified and stored in different databases. In this way, the data can be classified and stored in different databases and data tables according to classification indicators such as data source, refresh frequency, and data type, which increases storage efficiency while providing convenience for data query and retrieval functions in the database. By creating different communication clients based on the public configuration of Springboot, adding the created different communication clients to the client cache queue, locating the target communication client of the client cache queue according to the trigger instruction, enabling the message communication function of the target client according to the communication parameters, storing the message data received by the target client in real time into the cache, parsing the message data read from the cache, and storing the parsed message data in different databases according to the sampling frequency level corresponding to the actual business scenario, the problem of data collection difficulty caused by the use of multiple different communication protocols in the existing technology can be effectively solved. The method can support multiple different communication protocols, and can realize the rapid collection of ship equipment data and the efficient expansion of different protocols.

[0044] The present invention can adopt the factory design pattern to implement the design of different types of communication clients, including constructing an abstract communication client base class and a client factory class, utilizing the modular function (microservice feature) of springboot, and each subclass communication client is independently developed and deployed in the form of a microservice. Each communication protocol (such as HTTP, FTP, SMTP, etc.) implements an independent microservice, and each microservice implements its own communication client, that is, each communication client is packaged as an independent microservice to improve data collection efficiency. The factory pattern is used to create different communication clients, and the functions of the communication client are implemented by calling external microservices. That is, the factory service dynamically selects different communication client microservices according to the configuration (such as the protocol type), which can ensure the collection efficiency of external device data, and when a certain communication client is abnormal, it will not affect other normal communication clients, thereby ensuring the independence of the communication client. This architecture is very suitable for scenarios that support multiple communication protocols in large-scale systems and has good scalability and flexibility. If a new communication protocol (such as WebSocket, MQTT, etc.) needs to be added, it is only necessary to add a new microservice and update the protocol selection logic in the factory service.

[0045] When there is no message information of the current communication client in the cache, the present invention uses the monitoring function of the communication client to regularly monitor whether the message information of the current communication client is stored in the cache. In this way, the number of cache accesses can be effectively reduced and the waste of system resources can be reduced.

[0046] In the present invention, the message data is stored in the cache using a hash table storage structure, so that the message data in the cache can be quickly retrieved and read.

[0047] The present invention adopts a network service interface that follows the representational state transfer architectural style (i.e., a Web service interface that follows the REST (Representational State Transfer) architectural style, referred to as a restful interface) or a remote procedure call method to receive trigger instructions. This improves the scalability of the interface, makes long-term maintenance easier, and reduces security issues such as SQL injection to a certain extent.

[0048] The present invention stores the parsed message data in different databases according to the collection frequency of the message data, and stores the parsed message data in different data tables according to the source of the device providing the message data. This increases storage efficiency while facilitating data query and retrieval functions in the database.

[0049] The present invention also relates to a ship equipment data acquisition system based on Springboot, which corresponds to the above-mentioned ship equipment data acquisition method based on Springboot, and can be understood as a system that implements the above-mentioned ship equipment data acquisition method based on Springboot, including a public configuration module, a management and scheduling module, a cache module, a message parsing module and a classification storage module, wherein the public configuration module can uniformly manage, configure and share communication parameters in a public configuration manner, simplifying the configuration process, improving configuration efficiency, and effectively solving the problem that support for diverse communication protocols cannot be quickly expanded; the management and scheduling module can enable business processing functions in a timely manner when it is necessary to collect equipment data, ensuring The high-speed cache module provides high-speed access to communication messages. On the one hand, it stores communication messages in real time, and on the other hand, it reads messages quickly, thereby realizing the decoupling between message communication and message parsing. The storage efficiency will not be affected by the parsing performance after reading, which is convenient for adapting to the communication refresh frequency of different devices, especially the reception of high-frequency data, to achieve the effect of quickly collecting device data. The message parsing module can recursively parse and process message data in the cache in batches to prevent excessive accumulation of message data in the cache. The classified storage module can classify and store data into different databases / tables according to classification indicators such as data source, refresh frequency, and data type, while increasing storage efficiency and providing convenience for data query and retrieval functions. The various modules mentioned above work together to create different communication clients based on the public configuration of Springboot, add the created different communication clients to the client cache queue, locate the target communication client in the client cache queue according to the trigger instruction, enable the message communication function of the target client according to the communication parameters, store the message data received by the target client in real time into the cache, parse the message data read from the cache, and classify and store the parsed message data in different databases. This can effectively solve the problem of data collection difficulties caused by the use of multiple different communication protocols by ship equipment in the existing technology. The system can support multiple different communication protocols and can achieve rapid collection of ship equipment data and efficient expansion of different protocols. Moreover, the Springboot framework can be quickly combined with other communication frameworks to improve the development efficiency of the system and enrich the functional expansion capabilities; using Springboot's unified configuration and dependency injection capabilities, it is possible to quickly manage and use public configurations; using Springboot's natural support for the Web, it is easy to achieve the integration of IT and OT systems on ships; using Springboot to quickly implement system microservices, combined with containerization tools to quickly achieve distributed deployment of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1This is a flow chart of the ship equipment data collection method based on Springboot of the present invention;

[0051] Figure 2 This is a structural block diagram of the ship equipment data acquisition system based on Springboot in the present invention. DETAILED DESCRIPTION

[0052] The present invention discloses a ship equipment data collection method based on Springboot, which aims to improve the development efficiency of ship equipment data collection and improve the efficiency of ship equipment data collection. From the perspective of software development, it realizes the rapid collection and expansion of ship equipment data. The method can be executed by a processor or an electronic device with processing capabilities, such as Figure 1 As shown, first configure the communication parameters, create different types of communication clients, then add them to the client cache queue, wait for the trigger instruction to start, then start the communication transmission, receive data in real time and store it in the cache, then take out the original message from the cache for parsing, and finally classify and store the parsed data in different databases / tables. The present invention uses the public configuration and unified injection of Springboot to create different types of communication clients according to different communication parameter configurations to support message communications between diversified communication protocols and devices from different suppliers, and uses the cache to decouple message communication and message parsing to adapt to different communication refresh frequencies, especially high-frequency message communications, to prevent message communication congestion. After message parsing, the data is classified and stored in different databases / tables according to classification indicators such as data source, refresh frequency, and data type to provide convenient data query and retrieval functions. Specifically, the method includes the following steps:

[0053] S1. Based on Spring Boot's common configuration and injection methods, different types of communication clients are created based on different communication parameters. Each type of communication client includes four components: communication parameters, communication methods, message parsing methods, and communication status. Preferably, the communication parameters include basic information about the local client, basic information about the remote client, and communication connection information. The local client basic information includes the local client's IP address, port, creator, and creation time; the remote client basic information includes the remote client's IP address, port, refresh rate, and sensor information list. The communication connection information includes the communication connection method and communication protocol type. Communication connection methods include Ethernet and serial ports. Communication protocols include TCP, UDP, and MQTT. The sensor information list includes the sensor number, measurement type, measurement range, and communication encoding address. The communication method is the method the communication client uses to communicate with external devices using various different communication protocols. Message parsing is the function of parsing and restoring messages for different protocols. The communication status of the communication client can include "Not Enabled," "Ready to Enable," "Communicating," "Communication Interrupted," "Communication Abnormal," "Stopped," and "Stopped," indicating the client's current usage. The communication status is stored in a high-speed cache for immediate reading and modification. Each communication client uses a separate thread, and a newly created communication client is in an inactive state.

[0054] Furthermore, step S1 may also include the following steps:

[0055] S11: Create different types of communication clients based on Springboot's common configuration and injection methods;

[0056] S12: storing the communication status of different types of communication clients created in the cache respectively;

[0057] S13: Create a dedicated device information maintenance client for each communication client.

[0058] In an embodiment of the present application, each device that needs to collect data has a configuration information stored in the public configuration of Springboot. The configuration information includes the communication protocol, the IP address and port of the device that needs to collect data, etc. Each configuration information in the public configuration of Springboot will create a corresponding communication client. The communication client and the device that needs to collect data have a one-to-one relationship, that is, a communication client only collects data from the device corresponding to it. The newly created communication client is in an unactivated state (the communication client has not established a communication connection with the corresponding device that needs to collect data), and the status information of the communication client is stored in a cache. Each of the communication clients will be independently deployed in the form of a microservice. In this way, the efficiency of device data collection can be guaranteed, and when a communication client is abnormal, it will not affect other normal communication clients, thereby ensuring the independence of the communication client.

[0059] The embodiment of the present application adopts the factory design pattern to implement the design of different types of communication clients. First, an abstract base class (abstract class) of the communication client and a factory class of the communication client (or referred to as an abstract communication client base class and a client factory class) are constructed. Then, the factory class of the communication client generates a subclass communication client that inherits from the abstract base class of the communication client based on the communication parameters. The abstract communication client base class constrains the basic communication properties and methods of the communication client. The subclass communication client inherits from the abstract communication client base class and has both the basic communication parameters and methods of the communication client and can define its own special communication parameters, methods, and message parsing. Utilizing the modularization function of Spring Boot, each subclass communication client is developed separately as a separate microservice.

[0060] For example, the existing devices (external devices) that need to collect data include shaft power device signals and shaft vibration devices, wherein the shaft power device uses the TCP protocol for communication, and the shaft vibration device uses the MQTT protocol for communication. The IP addresses, ports and other information of the shaft power device and the shaft vibration device are added to the public configuration of Springboot. The factory class of the communication client generates and creates a communication client for collecting shaft power device data and a communication client for collecting shaft vibration device data based on the two configuration information about the shaft power device and the shaft vibration device in the public configuration of Springboot. The communication client for collecting shaft power device data supports establishing a connection with the shaft power device using the TCP protocol to collect shaft power device data; the communication client for collecting shaft vibration device data supports establishing a connection with the shaft vibration device using the MQTT protocol to collect shaft vibration device data. In this way, unified management, configuration and sharing of communication parameters are achieved, the configuration process is simplified, configuration efficiency is improved, and diversified communication protocols can be quickly expanded.

[0061] Optionally, the equipment that needs to collect data may be shaft vibration equipment, engine room monitoring and alarm system, GPS, depth sounder, shaft power equipment, shipboard tilt equipment, etc.

[0062] In the embodiments of this application, the local client and remote client represent abstract representations of the communicating parties. The local client represents the communicating client, and the remote client represents the device from which data is to be collected. The functions of the local client primarily include connection, disconnection, communication, suspension of communication, termination of communication, and management. Connection refers to the establishment of a communication connection between the local client and the device from which data is to be collected; communication refers to the process of data transmission between the two parties after the connection is established; management includes starting, shutting down, releasing the local client, and monitoring the client's operating status.

[0063] The embodiment of the present application creates its own exclusive device information maintenance client for each communication client. The device information maintenance client stores all the information of the device collected by the current communication client (taking the shaft power device signal collection scenario as an example, the information in the device information maintenance client includes IP address information, port information, communication protocol, baud rate, message parsing method, etc.). In this way, the information of the device collected by the current communication client can be updated and maintained in real time to ensure the accuracy of the device information used by the communication client.

[0064] S2, adds the created different types of communication clients to the client cache queue and waits for external trigger instructions to start.

[0065] This step adds different types of communication clients created to the client cache queue and sets the communication status to ready to be enabled, waiting for external trigger instructions to start communication; at the same time, the client queue is maintained in the program memory, and the client is retrieved according to the IP address of the local client, and the trigger of remote external instructions is received using the restful interface or remote procedure call method.

[0066] Furthermore, step S2 may also include the following steps:

[0067] S21: adding the created communication clients of different types to the client cache queue;

[0068] S22: Setting the state information specific to each of the different communication clients in the cache to a ready-to-use state.

[0069] The client cache queue in the embodiment of the present application is maintained in the memory of the communication client, and a specific communication client can be retrieved based on the IP address and port of the local client. After completing steps S21 and S22, the current communication client is ready and waits for the external network service interface that follows the representational state transfer architectural style (i.e., a web service interface that follows the REST (Representational State Transfer) architectural style, referred to as a restful interface) or a remote procedure call (RPC) to send a trigger instruction, thereby enabling the message communication function and establishing a communication connection with the device to be collected.

[0070] The embodiment of the present application adds the created communication client to the client cache queue and sets the status of the communication client to the enabled state. In this way, the message communication function can be enabled in time when the current communication client is needed, ensuring the rapid collection of device data.

[0071] S3, after detecting a trigger instruction for the target communication client, locate the target communication client through the client cache queue, enable the message communication function of the target communication client according to the communication parameters, and receive message data in real time through the target communication client, and then store the real-time received message data in the cache.

[0072] Furthermore, step S3 may also include the following steps:

[0073] S31: receiving a trigger instruction for a target communication client;

[0074] S32: Locating the target communication client through the client cache queue;

[0075] S33: activating the message communication function of the target communication client according to the communication parameters, so that the target communication client establishes a communication connection with the external device for data collection;

[0076] S34: If the message communication function is successfully enabled, the message data sent by the external device to be collected is received in real time, and the message data is stored in a cache in a hash table storage structure in a time sequence of the message data, and the communication state of the target communication client is set to normal communication;

[0077] S35: If the message communication function fails to be enabled, the state information of the target communication client in the cache is set to a communication abnormal state.

[0078] Furthermore, the embodiments of the present application receive trigger instructions through a network service interface that follows the representational state transfer architectural style (i.e., a Web service interface that follows the REST (Representational State Transfer) architectural style, referred to as a restful interface) or a remote procedure call (RPC). This improves the scalability of the interface, makes long-term maintenance easier, and reduces security issues such as SQL injection to a certain extent.

[0079] In the embodiment of the present application, the user sends a trigger instruction through a web page according to the actual scenario to enable the message communication function of the target communication client. After detecting the trigger instruction for the target communication client, the system locates the target communication client through the client cache queue. After enabling the message communication function of the target communication client, according to the target IP address, port and communication protocol type and other information in the communication parameters corresponding to the target communication client, a communication connection is established between the target communication client and the device to be collected data, and then the message data sent by the device to be collected data is received. If the message communication function fails to start or the communication connection between the target communication client and the device to be collected data fails to be established, the status information of the target communication client in the cache is set to a communication abnormality state, and the subsequent maintenance personnel will maintain the target communication client according to the target communication client status information and log information displayed on the management page (web page), and then re-enable the message communication function of the target communication client.

[0080] In the embodiments of the present application, the cache is used to provide high-speed access to message data. This not only allows for real-time and efficient storage of message data, but also allows for rapid reading of message data from the cache, thereby decoupling the message communication and message parsing processes. This allows for real-time storage of message data without affecting the efficiency of receiving message data due to the performance of the parsing process after receiving the message data. Furthermore, this facilitates adapting to the communication refresh frequencies of different devices, particularly the reception of high-frequency data, thereby achieving rapid acquisition of device data.

[0081] In the embodiment of the present application, the message data stored in the cache adopts a hash table structure, including the remote communication client IP, message time and message content. In this way, fast retrieval and reading of the message data in the cache can be achieved.

[0082] Preferably, the cache can be implemented using a cache framework or technology such as JCache, EhCache, or Redis.

[0083] S4, assigning a parsing thread to the target communication client, the parsing thread reads the message data from the cache in chronological order, and parses and processes the message data through the target communication client to obtain parsed data.

[0084] Furthermore, step S4 may also include the following steps:

[0085] S41: If the status information of the target communication client in the cache is a normal communication status, assigning a message parsing thread to the target communication client;

[0086] S42: After the message parsing thread is started, immediately read the message data in batches from the cache in chronological order;

[0087] S43: If the message parsing thread successfully reads the message data, the message parsing method of the maintenance client is maintained according to the exclusive device information of the target communication client, and the read message data is recursively parsed;

[0088] S44: Repeat steps S42-S43 until the message data stored in the cache by the target communication client is processed and cleared;

[0089] S45: If the message parsing thread cannot read the message data from the cache, that is, the cached messages have been read and cleared, the monitoring function of the target communication client is activated (or the listening mode is activated), and the cache is periodically monitored to determine whether there is new message data.

[0090] S46: Once it is detected that there is new message data in the cache, jump to step S42;

[0091] S47: If no new message data is monitored, continue monitoring until the status information of the target communication client in the cache changes to a stopped state, and then disable the monitoring function of the target communication client.

[0092] In the embodiment of the present application, a parsing thread pool is created during system initialization. When the communication client is in normal communication status, a thread is allocated to the communication client from the parsing thread pool to parse message data. When the communication client is in stopped status, the parsing thread of the communication client is recycled to the parsing thread pool.

[0093] Furthermore, in the embodiment of the present application, the parsing thread reads the original message from the cache in chronological order and parses it, and uses Springboot's aspect-oriented programming (AOP) method to monitor and record the parsing status of the message data in real time.

[0094] Furthermore, the number of message data that a parsing thread can read from the cache at a time is primarily determined by the amount of device data collected and the size of the cache, and can be set to read 5 to 100 messages per batch. In practice, since the amount of data collected from a particular device on a ship is known, during communication with the other party (such as the shaft power device), after a period of stable communication between the two parties, the number of message data to be read per batch is determined based on the amount of message data remaining in the cache corresponding to the current parsing thread.

[0095] Furthermore, the period of timed monitoring is obtained according to an empirical formula, as shown in formula (1):

[0096] Cache size - collection frequency * message length * timing period ≥ c * cache size (0 <c≤1)(1)

[0097] The cache size refers to the maximum amount of data that can be stored in the cache; the acquisition frequency refers to the frequency at which data is collected from the device to be collected; and the message length refers to the total length of the message data sent by the device to be collected. For example, ship equipment often uses a second-level acquisition frequency, so the timing period is usually set between 0.5 seconds and 30 seconds.

[0098] The embodiment of the present application reads message data in batches from the cache and then performs parsing processing. In this way, the cache can be used to decouple message communication and message parsing, prevent message data congestion, and achieve the effect of quickly processing the message data of the device.

[0099] S5: Classify and store the parsed data into different databases according to the sampling frequency levels corresponding to the actual business scenarios, and classify the parsed data stored in the same database into different data tables according to the data sources.

[0100] Furthermore, databases can be divided into several categories based on service type, including basic information databases, high-frequency storage databases, medium-frequency storage databases, and low-frequency storage databases. The basic information database primarily stores basic information about communication clients, communication parameter configurations, and message protocol information. Based on the actual data collection scenarios for ship equipment, the data collection frequency can be categorized as low-frequency, medium-frequency, and high-frequency. Low-frequency data collection frequencies can be stored in the low-frequency storage database, medium-frequency data collection frequencies can be stored in the medium-frequency storage database, and high-frequency data collection frequencies can be stored in the high-frequency storage database. Low-frequency data collection generally refers to minute-level frequencies, such as collecting data from shaft vibration equipment, typically at a fixed interval of 5 or 10 minutes. Medium-frequency data collection generally refers to second-level frequencies, which are the most widely used. For example, engine room monitoring and alarm systems, GPS equipment, depth sounders, and shaft power equipment are typically set to collect data every second or every two seconds. High-frequency data collection generally refers to data collection frequencies of milliseconds or higher, which are currently relatively rare. For example, data from shipboard tilting equipment typically collects data at a frequency between 50 and 500 milliseconds.

[0101] In the embodiment of the present application, the data stored in the same database is further classified and stored in different data tables according to the data source. Each device has its own dedicated data table. Taking the shaft power device and GPS device as an example, the shaft power device and GPS device both use the intermediate frequency acquisition frequency, and the collected message data are all stored in the intermediate frequency storage database. At this time, depending on the different data source devices, the data collected from the shaft power device is stored in the shaft power data table, and the data collected from the GPS device is stored in the GPS data table.

[0102] The embodiment of the present application can classify and store data into different databases and data tables according to classification indicators such as data source, refresh frequency, and data type, thereby increasing storage efficiency while facilitating data query and retrieval functions.

[0103] Preferably, the basic database can adopt Mysql, SQL server, etc.

[0104] Preferably, databases such as the high-frequency storage database, the medium-frequency storage database, and the low-frequency storage database used to store data with different collection frequencies can adopt time series databases such as InfluxDB, TDEngine, etc.

[0105] The Spring Boot framework used in this application embodiment has natural support for web pages, making it easy to integrate IT and OT systems onboard ships. Using this framework, microservices can be quickly built, and combined with containerization tools, distributed deployment of the system can be quickly achieved, thereby supporting higher data concurrency processing capabilities.

[0106] The embodiment of the present application utilizes the public configuration of Springboot to configure corresponding communication parameters for different protocols of the device and create different types of communication clients, which simplifies the configuration process, improves configuration efficiency, and effectively solves the problem that support for diverse communication protocols cannot be quickly expanded; the newly created communication client is added to the client cache queue to ensure that when support for the communication protocol needs to be expanded, rapid startup is completed and device data is quickly collected; data is received in real time and stored in the cache, message data read in batches from the cache is parsed, and the parsed message data is classified and stored in different databases and data tables. The cache can be used to decouple message communication and message parsing, and the reception, parsing and storage of message data are completed efficiently, which can realize the rapid collection of ship equipment data.

[0107] Based on the same inventive concept, one or more embodiments of this specification also provide a ship equipment data acquisition system based on Springboot. Since the principle of solving the problem by the ship equipment data acquisition system based on Springboot is similar to the aforementioned ship equipment data acquisition method based on Springboot, the implementation of the ship equipment data acquisition system based on Springboot can refer to the aforementioned implementation of the ship equipment data acquisition method based on Springboot, and the repeated parts will not be repeated.

[0108] Figure 2 This is a structural block diagram of a ship equipment data acquisition system based on Springboot provided in one or more embodiments of this specification. Figure 2 As shown, the ship equipment data acquisition system based on Springboot includes: a public configuration module 101, a management and scheduling module 102, a cache module 103, a message parsing module 104 and a classification storage module 105.

[0109] The public configuration module 101 is used for unified configuration and configuration sharing between modules and maintains the public configuration of Springboot. It is based on the public configuration and injection method of Springboot to create different types of communication clients according to different communication parameters, that is, based on the default configuration template of the Springboot framework, it quickly realizes the configuration of communication parameters for different communication clients. Each of the different types of communication clients includes different communication parameters, communication methods, message parsing and communication status. The communication parameters include the communication protocol, the IP address and port of the external device to collect data. The communication method is a method for the communication client to communicate messages with external devices of multiple different communication protocols. The message parsing parses and restores messages of different communication protocols. The communication status includes several states of whether it is enabled and whether communication is normal. Different communication states are used to indicate the current usage of the communication client. The public configuration module 101 can be made into a separate microservice to provide public configuration sharing functions for other microservices.

[0110] The management and scheduling module 102 manages the different types of communication clients created; adds the different types of communication clients created to the client cache queue, waiting for external trigger instructions to start; after detecting the trigger instruction for the target communication client, locates the target communication client through the client cache queue, enables the message communication function of the target communication client according to the communication parameters, and receives message data in real time through the target communication client; uses Springboot's RestController to receive external trigger instructions, triggers the creation, start, stop, and interruption of the communication client, and maintains the client management queue and the communication status of each communication client.

[0111] The cache module 103 is used to store the real-time received message data into the cache, providing fast access to real-time messages; it uses springboot to integrate other cache frameworks such as JCache, EhCache, Redis, etc.

[0112] The message parsing module 104 is used to read and parse messages from the cache. Specifically, it allocates a parsing thread to the target communication client. The parsing thread reads message data from the cache in chronological order and parses the message data through the target communication client to obtain parsed data.

[0113] The classification storage module 105 provides data classification and storage capabilities. It classifies and stores parsed data into different databases based on the sampling frequency level corresponding to the actual business scenario. Parsed data stored in the same database is further classified and stored into different data tables based on the data source. Spring Boot can be used to integrate with database frameworks such as MyBatis and MyBatis-Plus.

[0114] Furthermore, the common configuration module adopts the factory design pattern to implement the design of different types of communication clients, including constructing an abstract communication client base class and a client factory class. The abstract communication client base class constrains the basic communication properties and methods of the communication client. The client factory class generates different subclass communication clients according to different communication parameter configurations. The subclass communication client inherits from the abstract communication client base class and has both the basic communication parameters and methods of the communication client and customized communication parameters, methods and message parsing. Utilizing the modularization function of springboot, each subclass communication client is developed separately into a separate microservice.

[0115] Furthermore, the management and scheduling module enables the message communication function of the target communication client according to the communication parameters, so that the target communication client establishes a communication connection with the external device to be collected data. If the message communication function is successfully enabled, the message data sent by the external device to be collected data is received in real time, and then the cache module stores the message data in the cache in a hash table storage structure according to the time sequence of the message data, and then the management and scheduling module sets the communication status of the target communication client to normal communication; if the message communication function fails to be enabled, the management and scheduling module sets the status information of the target communication client to a communication abnormality status.

[0116] Furthermore, after the status information of the target communication client indicates normal communication, the message parsing module assigns a message parsing thread to it, and the message parsing thread reads message data in batches from the cache in chronological order; if the message parsing thread successfully reads the message data, the read message data is recursively parsed until the message data stored in the cache by the target communication client is processed and cleared; if the message parsing thread cannot read the message data from the cache, the monitoring function of the target communication client is started, and the cache is regularly monitored for new message data; once the cache is detected to contain new message data, the recursive parsing process is continued; if no new message data is monitored, the monitoring is continued until the status information of the target communication client in the cache changes to a stopped state, and the monitoring function of the target communication client is disabled.

[0117] Furthermore, in the classified storage module, the database includes a basic information database, a high-frequency storage database, a medium-frequency storage database, and a low-frequency storage database; the basic information database is implemented using Mysql and SQL server, and is used to store basic information of the communication client, communication parameter configuration, and message protocol. Databases of different frequencies are implemented using InfluxDB and TDEngine time series databases. The low-frequency storage database is used to store data with a collection frequency of minutes corresponding to actual business scenarios, the medium-frequency storage database is used to store data with a collection frequency of seconds corresponding to actual business scenarios, and the high-frequency storage database is used to store data with a collection frequency of milliseconds or microseconds corresponding to actual business scenarios.

[0118] The ship equipment data acquisition system of the present invention is based on Springboot. The Springboot framework can be quickly combined with other communication frameworks to improve the system development efficiency and rich functional expansion capabilities; the unified configuration and dependency injection capabilities of Springboot are used to quickly manage and use public configurations; the natural support of Springboot for the Web is used to easily realize the integration of IT and OT systems on ships; Springboot is used to quickly implement the system's microservices, and containerization tools are combined to quickly realize the distributed deployment of the system.

[0119] It should be noted that the specific embodiments described above can enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Therefore, although this specification has described the present invention in detail with reference to the drawings and embodiments, those skilled in the art should understand that the present invention can still be modified or replaced with equivalents. In short, all technical solutions and improvements that do not depart from the spirit and scope of the present invention should be included in the scope of protection of the patent for the present invention.

Claims

1. A ship equipment data collection method based on Springboot, characterized in that: The following steps are involved: S1: Based on the public configuration and injection method of Springboot, different types of communication clients are created according to different communication parameters; wherein each of the different types of communication clients includes different communication parameters, communication methods, message parsing and communication status, the communication parameters include the communication protocol, the IP address and port of the external device to collect data, the communication method is a method for the communication client to communicate with external devices of multiple different communication protocols, the message parsing parses and restores messages of different communication protocols, the communication status includes several states of whether it is enabled and whether communication is normal, and different communication states are used to indicate the current usage of the communication client; S2: Add different types of communication clients created to the client cache queue and wait for external trigger instructions to start; S3: After detecting a trigger instruction for a target communication client, locating the target communication client through the client cache queue, enabling a message communication function of the target communication client according to the communication parameters, and receiving message data in real time through the target communication client, and then storing the received message data in a cache; S4: allocating a parsing thread to the target communication client, the parsing thread reading the message data from the cache in chronological order, and parsing the message data through the target communication client to obtain parsed data; S5: The parsed data are classified and stored in different databases according to the sampling frequency levels corresponding to the actual business scenarios, and the parsed data stored in the same database are classified again according to the data source and stored in different data tables.

2. The ship equipment data collection method based on Springboot according to claim 1 is characterized in that, Step S1 uses the factory design pattern to implement the design of different types of communication clients, including constructing an abstract communication client base class and a client factory class. The abstract communication client base class constrains the basic communication properties and methods of the communication client. The client factory class generates different subclass communication clients according to different communication parameter configurations. The subclass communication client inherits from the abstract communication client base class and has both the basic communication parameters and methods of the communication client and customized communication parameters, methods, and message parsing. Using the modular function of springboot, each subclass communication client is developed separately into a separate microservice.

3. The ship equipment data acquisition method based on Springboot according to claim 1 or 2, characterized in that: In step S1, the communication parameters include basic information of the local client, basic information of the remote client, and communication connection information; the basic information of the local client includes the IP address, port, creator, and creation time of the local client; the basic information of the remote client includes the IP address, port, device refresh frequency, and sensor information list of the remote client; the communication connection information includes the communication connection mode and the communication protocol type; the communication connection mode includes Ethernet and serial port; the communication protocol type includes TCP, UDP, and MQTT; the sensor information list includes the sensor number, measurement type, measurement range, and communication coding address; The communication status of the communication client includes not enabled, ready to enable, communicating, communication interrupted, communication abnormal, stopping and stopped. The communication status is stored in the cache for immediate reading and modification. Each communication client uses a separate thread, and the newly created communication client is in the not enabled state.

4. The ship equipment data collection method based on Springboot according to claim 3 is characterized in that, Step S2 adds the created different types of communication clients to the client cache queue, and sets the communication status to ready to be enabled, waiting for an external trigger instruction to start the communication; at the same time, the client queue is maintained in the program memory, the client is retrieved according to the IP address of the local client, and the trigger of the remote external instruction is received using the restful interface or remote procedure call method.

5. The ship equipment data collection method based on Springboot according to claim 3 is characterized in that: Step S3 includes: S31: receiving a trigger instruction for a target communication client; S32: Locating the target communication client through the client cache queue; S33: activating the message communication function of the target communication client according to the communication parameters, so that the target communication client establishes a communication connection with the external device for data collection; S34: If the message communication function is successfully enabled, the message data sent by the external device to be collected is received in real time, and the message data is stored in a cache in a hash table storage structure in a time sequence of the message data, and the communication state of the target communication client is set to normal communication; S35: If the message communication function fails to be enabled, the status information of the target communication client in the cache is set to a communication abnormality state.

6. The ship equipment data collection method based on Springboot according to claim 5 is characterized in that, Step S4 includes: S41: If the status information of the target communication client in the cache is a normal communication status, assigning a message parsing thread to the target communication client; S42: Reading message data in batches from the cache in chronological order through the message parsing thread; S43: If the message parsing thread successfully reads the message data, the message parsing method of the maintenance client is maintained according to the exclusive device information of the target communication client, and the read message data is recursively parsed; S44: Repeat steps S42-S43 until the message data stored in the cache by the target communication client is processed and cleared; S45: If the message parsing thread cannot read the message data from the cache, the monitoring function of the target communication client is activated, and the cache is regularly monitored for new message data. S46: Once it is detected that there is new message data in the cache, jump to step S42; S47: If no new message data is monitored, the monitoring is continued until the status information of the target communication client in the cache is changed to a stopped state, and then the monitoring function of the target communication client is turned off.

7. The ship equipment data collection method based on Springboot according to claim 1 is characterized in that: In step S5, the database includes a basic information database, a high-frequency storage database, a medium-frequency storage database, and a low-frequency storage database; The basic information database is implemented using Mysql and SQL server, and is used to store the basic information of the communication client, communication parameter configuration and message protocol. The databases of different frequencies are implemented using InfluxDB and TDEngine time series database. The low-frequency storage database is used to store data with a collection frequency of minutes corresponding to actual business scenarios. The medium-frequency storage database is used to store data with a collection frequency of seconds corresponding to actual business scenarios. The high-frequency storage database is used to store data with a collection frequency of milliseconds or microseconds corresponding to actual business scenarios.

8. A ship equipment data acquisition system based on Springboot, characterized in that: It includes a public configuration module, a management and scheduling module, a cache module, a message parsing module and a classification storage module connected in sequence; wherein, The public configuration module is used to maintain the public configuration of Springboot, and based on the public configuration and injection method of Springboot, different types of communication clients are created according to different communication parameters; each of the different types of communication clients includes different communication parameters, communication methods, message parsing and communication status, the communication parameters include the communication protocol, the IP address and port of the external device to collect data, the communication method is a method for the communication client to communicate with external devices of multiple different communication protocols, the message parsing parses and restores messages of different communication protocols, and the communication status includes several states of whether it is enabled and whether communication is normal, and different communication states are used to indicate the current usage of the communication client; The management and scheduling module is used to manage different types of communication clients created; add different types of communication clients created to the client cache queue, waiting for external trigger instructions to start; after detecting the trigger instruction for the target communication client, locate the target communication client through the client cache queue, enable the message communication function of the target communication client according to the communication parameters, and receive message data in real time through the target communication client; use Springboot's RestController to receive external trigger instructions, trigger the creation, start, stop, and interrupt operations of the communication client, and maintain the communication status of each communication client; The cache module is implemented by integrating JCache, EhCache and Redis cache framework using Springboot, and is used to store the message data received in real time into the cache; The message parsing module is used to allocate a parsing thread to the target communication client, and the parsing thread reads the message data from the cache in chronological order, and parses and processes the message data through the target communication client to obtain parsed data; The classification storage module is used to classify and store the parsed data into different databases according to the sampling frequency levels corresponding to the actual business scenarios, and to classify the parsed data stored in the same database into different data tables according to the data sources.

9. The ship equipment data acquisition system based on Springboot according to claim 8 is characterized in that: The common configuration module adopts the factory design pattern to implement the design of different types of communication clients, including constructing an abstract communication client base class and a client factory class. The abstract communication client base class constrains the basic communication properties and methods of the communication client. The client factory class generates different subclass communication clients according to different communication parameter configurations. The subclass communication clients inherit from the abstract communication client base class and have both the basic communication parameters and methods of the communication client and customized communication parameters, methods, and message parsing. Utilizing the modularization function of Spring Boot, each subclass communication client is independently developed into a separate microservice.

10. The ship equipment data acquisition system based on Springboot according to claim 8 or 9, characterized in that: The management and scheduling module enables the message communication function of the target communication client according to the communication parameters, so that the target communication client establishes a communication connection with the external device for collecting data. If the message communication function is successfully enabled, the message data sent by the external device for collecting data is received in real time, and the cache module stores the message data in a cache using a hash table storage structure in a time sequence of the message data. Then, the management and scheduling module sets the communication status of the target communication client to normal communication. If the message communication function fails to be enabled, the management and scheduling module sets the status information of the target communication client to a communication abnormal state.

11. The ship equipment data acquisition system based on Springboot according to claim 10, characterized in that: After the target communication client's status information indicates normal communication, the message parsing module assigns a message parsing thread to the target communication client, and the message parsing thread reads message data in batches from the cache in chronological order; if the message parsing thread successfully reads the message data, the read message data is recursively parsed until all message data stored by the target communication client in the cache are processed and cleared; If the message parsing thread cannot read the message data from the cache, the monitoring function of the target communication client is activated, and the cache is monitored regularly to determine whether new message data exists; once the cache is detected to contain new message data, the recursive parsing process is continued; If no new message data is monitored, the monitoring continues until the state information of the target communication client in the cache changes to a stopped state, and then the monitoring function of the target communication client is turned off.

12. The ship equipment data acquisition system based on Springboot according to claim 8 or 9, characterized in that: In the classification storage module, the database includes a basic information database, a high-frequency storage database, a medium-frequency storage database, and a low-frequency storage database; The basic information database is implemented using Mysql and SQL server, and is used to store the basic information of the communication client, communication parameter configuration and message protocol. The databases of different frequencies are implemented using InfluxDB and TDEngine time series database. The low-frequency storage database is used to store data with a collection frequency of minutes corresponding to actual business scenarios. The medium-frequency storage database is used to store data with a collection frequency of seconds corresponding to actual business scenarios. The high-frequency storage database is used to store data with a collection frequency of milliseconds or microseconds corresponding to actual business scenarios.

Images