Control method and device for surface unmanned boat cluster resources

Abstract

The present invention provides a control method and device for surface unmanned boat cluster resources, including: obtaining boat-side data collected by each unmanned boat; parsing and processing the boat-side data to obtain processed data, wherein the processed data at least includes integrated data and video data; displaying the integrated data and playing the video data according to the processed data, with a modular design, supporting the integration of general loads, mission loads, mission processing control, navigation processing control and collaborative control functions. It provides a unified open carrying environment and basic capability environment, including high-speed network, electrical interface, time system, power supply, packaging, electromagnetic compatibility environment, safety protection, data storage management, software management and callable public function modules. This design enables the integration and collaborative capabilities of surface unmanned boats to be comprehensively improved to meet future development needs.

Description

Technical Field

[0001] The present invention belongs to the field of control technology, and in particular relates to a control method and device for surface unmanned boat cluster resources. Background Art

[0002] As an unmanned surface platform with broad application prospects, unmanned surface vessels (USVs) are widely used in marine scientific research, underwater exploration, surface monitoring, navigation and transportation, etc. However, the performance and functions of USVs are limited by their intelligent control devices, which usually need to support various general payloads, mission payloads, mission processing control, navigation processing control and collaborative control functions, and at the same time need to meet the requirements of high security, reconfigurability, upgradeability, scalability, and growth.

[0003] In the past, the intelligent control equipment of surface unmanned boats was often customized, designed and manufactured for specific applications and tasks, which led to interoperability issues between systems and limited the upgrade and expansion of equipment. In addition, the lack of common interfaces and standardized designs between the hardware and software of the equipment made the integration of different payloads and control modules complicated and expensive.

[0004] In the prior art, the intelligent control equipment of surface unmanned boats has the following problems and challenges: there are interoperability issues between surface unmanned boat systems of different manufacturers, and it is difficult to achieve collaborative work of equipment from multiple manufacturers. Therefore, how to provide control methods and devices for surface unmanned boat cluster resources has become a technical problem that urgently needs to be solved in this field. Summary of the invention

[0005] The purpose of the present invention is to provide a method and device for controlling surface unmanned boat cluster resources.

[0006] According to a first aspect of the present invention, a method for controlling a cluster of unmanned surface vessels is provided, comprising:

[0007] Obtain the boat-side data collected by each unmanned boat;

[0008] Analyzing and processing the boat-side data to obtain processed data, wherein the processed data at least includes integrated data and video data;

[0009] The integrated data is displayed and the video data is played according to the processed data.

[0010] Optionally, the obtaining of boat-side data collected by each unmanned boat includes:

[0011] The collected initial boat-side data is encrypted using a preset encryption algorithm to obtain the encrypted boat-side data. The initial boat-side data is collected by a collection device on the unmanned boat. The collection device includes at least sensors, radars, optoelectronic equipment and Beidou. The initial boat-side data is used to characterize the operating status of the unmanned boat.

[0012] Optionally, the method further comprises:

[0013] The non-file data is stored in a database, and the file-type data is stored in a preset directory, wherein the file-type data at least includes processed video data, and the non-file data at least includes log files.

[0014] Optionally, the parsing and processing the boat-side data to obtain processed data includes:

[0015] Receive the video stream generated by optoelectronics and store it in NVR using RTSP;

[0016] Convert JPEG images generated by sonar and radar into video streams and transfer them to NVR via RTSP.

[0017] Optionally, displaying the integrated data and playing the video data according to the processed data includes:

[0018] Based on the processed data, real-time video playback, video playback, video storage device management and video splicing display are performed.

[0019] Optionally, the method further comprises:

[0020] When receiving service requests sent by multiple unmanned boats, the service requests are processed according to the service priority levels.

[0021] Optionally, the method further comprises:

[0022] Judge the processed data to determine whether the unmanned boat has a fault;

[0023] If the processed data exceeds a preset threshold, an alarm display is performed, and the alarm display includes at least a sound alarm and a light alarm.

[0024] Optionally, the method further comprises:

[0025] When an unmanned boat fails, the failed unmanned boat is switched.

[0026] Optionally, the method further comprises:

[0027] The boat-side data of each unmanned boat and the server data corresponding to the boat-side data are shared.

[0028] According to the second aspect of the present invention, a control device for surface unmanned boat cluster resources is provided, including a control method for surface unmanned boat cluster resources as described in any one of the first aspect of the present invention, the control device comprising: multiple boards, IO boards and a chassis, adopting a rack-type structure of a backplane bus and an IO board, each board is integrated inside the chassis, connected to the backplane bus through a connector, each board is connected via Ethernet to realize data exchange, and is connected to the payload equipment on the surface unmanned boat through a specific interface on the IO board.

[0029] The beneficial effects brought by the present invention are as follows:

[0030] It can be seen from the above scheme that the embodiment of the present invention provides a control method and device for the resources of a cluster of surface unmanned boats, which has the following beneficial effects: it has a modular design and supports the integration of functions such as general loads, mission loads, mission processing control, navigation processing control and collaborative control. It provides a unified open loading environment and basic capability environment, including high-speed networks, electrical interfaces, time systems, power supplies, packaging, electromagnetic compatibility environments, safety protection, data storage management, software management and callable common function modules. This design enables the integration and collaborative capabilities of surface unmanned boats to be comprehensively improved to meet future development needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A schematic diagram of a method for controlling a cluster of unmanned surface boats resources according to an embodiment;

[0032] Figure 2 A hardware architecture diagram of an unmanned boat intelligent control device provided according to an embodiment;

[0033] Figure 3 A schematic diagram of a "subscribe-publish" mechanism provided according to an embodiment;

[0034] Figure 4 A schematic diagram of a "request-response" mechanism provided according to an embodiment;

[0035] Figure 5 A diagram of a unified data integration service architecture provided according to an embodiment;

[0036] Figure 6 It is a structural diagram of a data table in a boat provided according to an embodiment;

[0037] Figure 7 A flow chart of data synchronization between boat and shore provided according to an embodiment;

[0038] Figure 8 A diagram for constructing structured data provided according to an embodiment;

[0039] Fig. 9A diagram for forming unstructured data according to an embodiment;

[0040] Fig.10 A data analysis flow chart provided according to an embodiment;

[0041] Fig.11 A hardware structure diagram of an unmanned boat terminal system provided according to an embodiment;

[0042] Fig.12 A one-line video splicing playback interface provided according to an embodiment;

[0043] Fig.13 A video splicing configuration interface provided according to an embodiment;

[0044] Fig.14 The figure is a structure diagram of the multimedia integrated management function provided according to an embodiment. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] According to a first aspect of the present invention, a method for controlling a cluster of unmanned surface vessels is provided, comprising:

[0047] S101, obtaining boat-side data collected by each unmanned boat;

[0048] S102, parsing and processing the boat-side data to obtain processed data, wherein the processed data at least includes integrated data and video data;

[0049] S103. Display the integrated data and play the video data according to the processed data.

[0050] like Figure 2 As shown, the unmanned boat intelligent control equipment adopts the VPX bus to build a rack structure based on the backplane bus and IO board. Each functional module is integrated inside the chassis in the form of a board, connected to the backplane bus through a connector, and uniformly packaged.

[0051] Each board is connected via Ethernet to exchange data, and is connected to the payload equipment on the surface unmanned boat through a specific interface on the IO board.

[0052] The hardware architecture of the unmanned boat intelligent control device is as follows: Figure 1 shown.

[0053] The hardware composition of the unmanned boat intelligent control equipment is shown in Table 1. The standard hardware is the inherent components, and the optional hardware is the components that can be configured or replaced according to the mission type performed by the surface unmanned boat and the installed mission payload.

[0054] The main functions of the unmanned boat intelligent control equipment are as follows:

[0055] a) Have the following basic service functions:

[0056] 1) Unified computing center: Integrate the computing functions of mission control, navigation control, and payload control on the unmanned boat.

[0057] 2) Unified high-speed network: It can complete data exchange between internal boards connected to the switch board and external devices, with an exchange rate of up to gigabit.

[0058] 3) Unified data storage: It can orderly manage and store various data such as status information, control commands, video key frames, voice key information, etc. of the operation of surface unmanned boats and their equipment, and store original files such as video and voice at the same time.

[0059] 4) Unified data export: supports fast data export and is compatible with multiple mainstream databases.

[0060] 5) Unified hardware interface: Provide a unified standard electrical interface for internal boards to complete the electrical connection between internal boards. Provide multiple standard interfaces for external devices to complete the electrical connection between the unmanned boat intelligent control device and external devices.

[0061] 6) Unified software interface: Adapt the software interface for network-connected nodes and provide message publishing, subscription and software calling services.

[0062] 7) Unified power supply and power management. It can provide power for internal boards and adopts the form of partitioned power supply. Basic equipment, navigation-related, navigation parts, and optoelectronic parts can be powered on and off independently.

[0063] 8) Provide timing signal: can provide timing signal with a frequency of 50Hz.

[0064] 9) Unified packaging: The boards installed in the unmanned boat intelligent control equipment can be uniformly packaged, with certain waterproof, anti-vibration and other capabilities, and a certain anti-disassembly capability.

[0065] 10) Safety protection:

[0066] 1. It can provide operating system security protection for internal standard boards, intercept network attacks, and provide network attack prompts;

[0067] 2. After obtaining authorization from the shore-based control center, the internal data of the unmanned boat intelligent control equipment can be destroyed;

[0068] 3. Set up a firewall to intercept various network attacks;

[0069] 4. Establish data storage access strategies to prevent illegal access to data;

[0070] 5. It has an anti-dismantling mechanism to protect the data stored in the unmanned boat's intelligent control equipment from being stolen.

[0071] 11) Fault diagnosis and fault tolerance:

[0072] 1. It has the function of collecting fault information of various mission load equipment on the boat, and realizes the fault detection function together with the single boat operation console / cluster control console;

[0073] 2. It has redundancy and backup functions. It is equipped with redundant information processing boards (navigation control 1) and information processing boards (navigation control 2). When one of the boards fails, the navigation control function of the entire boat will not be affected. When both of the above information processing boards fail, the navigation control function will be performed by the information processing board (mission control) and the original task of the information processing board (mission control) will be terminated. The switch board is backed up redundantly. When the main switch board fails, it can automatically switch to the backup switch board.

[0074] b) Provide the following public basic functions to support the execution of typical tasks of unmanned boats:

[0075] 1) The boat-side algorithm library consists of boat-side dynamic path planning, target recognition, geographic coordinate conversion calculation, etc.;

[0076] 2) Ship-side public services such as multi-source information fusion and information integration services;

[0077] 3) Sensor control services such as optoelectronic payload control services and sonar payload control services;

[0078] 4) General strategy services on the boat side, such as formation keeping strategy;

[0079] 5) Boat-side monitoring service components including boat-side equipment status monitoring service, board status management service, and application service monitoring.

[0080] c) Customized applications can be loaded through the installed functional boards or deployed functional software, and customized applications can call services that provide common basic functions.

[0081] d) According to the commands of the single boat operating console / cluster console, relevant controls can be sent to the mission payload equipment to complete the control of the payload equipment.

[0082] The embodiment of the present application includes four parts of integrated design, namely software integration design, information integration design, unified data integration, and unified video integration.

[0083] This section will introduce in detail the specific implementation methods of surface unmanned boat cluster resource integration, including software integration design, information integration design, unified data integration and unified video integration. These implementation methods are intended to provide reproducible technical solutions for patents.

[0084] Software integration design

[0085] Overview

[0086] The unmanned boat cluster system adopts a "soft integration" approach. The unmanned boat platform adopts the technical system of unmanned boat intelligent control equipment. The boat side and the shore base use shared servers to build universal and standardized shared hardware resources. The underlying layer is based on the "Xuanlong" unmanned boat operating system to provide public software basic services.

[0087] Aiming at the requirements of various mission scenarios of unmanned boat clusters and the new features brought by the use of unmanned boat intelligent control equipment on each single boat platform, by upgrading and optimizing the software services in the original unmanned boat intelligent control equipment, basic and common service software including cluster information integration service, cluster unified data service, cluster unified video service, cluster equipment management service, cluster software management service and cluster communication management service are constructed to achieve standardized access and service calls to application services, payload equipment and data, forming a basic public service environment for unmanned boat clusters based on a microservice framework. The purpose of on-demand distribution / acquisition of business service information, flexible reconstruction of task execution, and online control of system operation status / software technical status is achieved, achieving the best match between software and hardware resources and task requirements.

[0088] The human-computer interaction of the unmanned boat swarm adopts a browser-based service interaction method, which supports personalized service calls based on user roles and permissions. Users log in to the server through a browser, and the server opens and pushes related services for users to use based on the user's role and permissions, thereby realizing personalized service loading and calling for different users.

[0089] Software integration operation mechanism

[0090] Software operation mechanism

[0091] The unmanned boat cluster system uniformly formulates the granularity division of software microservices and the allocation strategy of virtual machines according to business application requirements, and provides several virtual machine resources for each business application microservice. According to the pre-configured plan, after the role login application is issued, the system starts the virtual machine and the corresponding business microservice and configures the priority, stack space and other parameters to realize the loading and operation of the business application microservice.

[0092] Software Scheduling Mechanism

[0093] According to the different criticality levels of the business, different availability strategies for microservices are formulated, including two modes: real-time hot standby and migration scheduling. For services with high criticality levels such as navigation control, multiple copies of the same service or the same service from different manufacturers can be started simultaneously on the standard boards of multiple unmanned boat intelligent control devices. Software monitoring provides fault alarms and key data synchronization support for the operation of the service. The application management service completes the master-slave election and switching to achieve dynamic succession in the event of a failure. For tasks with lower criticality levels, when a running software service fails, the application management service selects an idle virtual machine to migrate the software service, and the software service is reloaded and run on the new virtual machine.

[0094] Data sharing access mechanism

[0095] The unmanned boat cluster system uses the storage server in the infrastructure layer and the unmanned boat intelligent control equipment on the boat side as the data storage infrastructure, and builds a data access and unified management platform through the software services in the basic service layer, providing a service interface for each business application microservice to access data / logs / audio and video.

[0096] During the design of the unmanned boat cluster system, relevant rules are set for the access of each business application microservice, such as database rule allocation, log classification and content format, video compression ratio, etc. When each application microservice accesses data, it directly calls the service interface provided by the local unified data service. The application microservice implements local caching, compression / decompression of data according to the data type accessed by the application service, and accesses the storage server through the unmanned boat cluster network to achieve data access, sharing, synchronization and interaction.

[0097] Software status management mechanism

[0098] Software Technical Status Management

[0099] Since all software of the unmanned boat swarm system is deployed on shore-based servers and boat-side unmanned boat intelligent control devices, the software adopts microservices and is deployed / run in virtual machines. The physical location is more dynamic, which brings new challenges to the technical status control of application services. To this end, the application management service based on the functional service layer provides centralized configuration management and online detection functions for the software services of the unmanned boat swarm system, forming a unified and controllable version view.

[0100] Technical support mechanism

[0101] The technical status management function of software services is designed as three software components, including technical status management service (referred to as service component), technical status management agent (referred to as agent component) and technical status management display and control (referred to as display and control component). A software technical status management server is established in the shared server of the infrastructure layer, and the service component is run to save the solidified entity and version information of the software microservice, which serves as a standard software library for service deployment and update; the agent component is run on the virtual machines, shared display and control devices and dedicated devices running various application services to implement the deployment update and version check of local software; the management display and control component is run on the shared display and control device to realize the control of software deployment and the visual display of version view.

[0102] Software deployment update mechanism

[0103] After the acceptance and delivery of each business application microservice software, the system administrator will store it in the technical status management server and include it in the cluster system software library for unified management. According to the deployment plan designed for the cluster system, the unmanned boat cluster system software library extracts the microservice software and deploys it to the final location where the software service runs (unmanned boat intelligent control device / shared display console / server) through the unmanned boat cluster system network to achieve the first deployment of the software service. After the first deployment is completed, if a service software needs to be replaced and upgraded due to bug fixes / function upgrades, the system administrator will also enter the new version in the software library and then implement the software update through the unmanned boat system network.

[0104] Software version checking mechanism

[0105] When each service software is stored in the unmanned boat cluster software library, the software version feature code is automatically generated and stored together. According to the inspection strategy of the cluster system, when each device (unmanned boat intelligent control device / shared display console / server) is started, or when triggered by manual operation, the local agent component checks the actual running software service feature code and compares it with the information in the software library to confirm whether it is in a controlled state. If the status check is inconsistent, according to the design strategy of the unmanned boat cluster system, the operation is restricted locally or alarm / log recording is implemented. The cluster system maintains and displays the global software version view and inspection results.

[0106] Software running status monitoring

[0107] Software operation resource monitoring

[0108] Software operation resource monitoring is implemented by the basic service layer software, which monitors the occupancy of underlying resources such as virtual machines, standard boards of public service basic equipment, including the usage of CPU, memory, and disk, and monitors the resource occupancy of application services, including the occupancy of CPU, memory, disk, network, etc. by application services. The above monitoring results are summarized and visualized by the monitoring service.

[0109] Software running status monitoring mechanism

[0110] Software operation status monitoring is mainly achieved through the collaboration of the basic service layer and software services. The software service is responsible for monitoring its own operation status and calling the interface provided by the monitoring software to issue an alarm when a failure occurs. The basic service layer software is responsible for completing the operation status monitoring of the software service and uniformly realizing status aggregation and visual display.

[0111] System information flow monitoring mechanism

[0112] Information flow monitoring is mainly implemented by cluster information integration services and data analysis tools. In the process of information subscription / publishing by application services, information integration services collect information interaction relationships, information sending and receiving status, etc., record and alarm abnormal situations, and data analysis tools perform visual playback. In addition, data analysis tools can provide recording and playback of software service data on demand.

[0113] Equipment health status monitoring mechanism

[0114] Equipment health monitoring is implemented by the basic service layer and the application service. The status monitoring service of the basic service layer is mainly responsible for completing the health status collection of the unmanned boat cluster and the equipment on the platform (such as the equipment's power on and off time, main engine oil temperature, speed, etc.). The application service can uniformly summarize the health status and evaluate and display the health of the equipment.

[0115] Task execution status monitoring mechanism

[0116] Task execution status monitoring is implemented by application services. According to task arrangement, the task association status of each business application microservice is clarified, and relevant status is collected and summarized during task execution, providing a comparison with the task plan, forming a task execution view of the unmanned boat cluster system and performing a visual display.

[0117] Information integration design

[0118] The information interaction of the surface unmanned boat swarm system adopts a subscription-publish mechanism.

[0119] The unmanned boat cluster system places high demands on information sharing and deep integration within the cluster. Combined with the mission of the unmanned boat cluster, it provides two communication modes, "subscribe-publish" and "request-response", for different information sharing demand scenarios. It makes full use of various QoS sharing strategies such as transmission priority, reliability, and persistence, strengthens the validity check of information interaction content and interaction process, and realizes "on-demand acquisition and on-demand distribution" of information between application services, ensuring that the transmission of information in the unmanned boat cluster system is flexible and controllable.

[0120] The "subscribe-publish" information sharing mechanism is mainly used by software services to share asynchronously generated one-way data. It means that the software service calls the standard information interface of the cluster information integration service to complete the sending and receiving of information. The cluster information integration service will manage information sharing and automatically distribute information to the corresponding receiving service. The sending service and the receiving service do not need to care about details such as network addresses and ports, nor do they need to care about the source and destination of the information. The unmanned boat cluster system can configure and monitor information sending and receiving permissions, addresses, policies, etc. based on the information flow design results, to achieve microservice-level flexibility and system-level controllable effects. Specifically, Figure 3 shown.

[0121] The "request-response" information sharing mechanism is mainly used for application software to share remote service data. It means that the application software completes the sending of request information and the receiving of response information by calling the standard information interface of the cluster information integration service. The cluster information integration service manages information sharing and automatically distributes requests to the corresponding service providers and responses to the corresponding service requesters. Requesters and providers do not need to worry about details such as network addresses and ports, nor do they need to worry about the source and destination of the information. The unmanned boat cluster system can configure and monitor information sending and receiving permissions, addresses, policies, etc. based on the information flow design results, to achieve microservice-level flexibility and system-level controllable effects. Figure 4 shown.

[0122] There are various sensors and collection equipment on the unmanned boat, such as radar, optoelectronic equipment, Beidou, etc., which are used to collect various data to monitor the operation status of the boat. The collected data is used for subsequent analysis and target research. The logs generated by the operation of the equipment are used to conduct specific analysis of problems generated during the operation and restore the operation process of the boat. The data and logs generated by various devices on the boat are stored senselessly, and various data generated on the boat are synchronized to the shore-based server.

[0123] After receiving the data synchronized from the boat, the shore-based system collects and analyzes it, stores non-file data in the database, and stores file data in the specified directory.

[0124] The overall design plan is as follows Figure 5 shown.

[0125] The data generated by the unmanned boat cluster are synchronized to the shore-based SFTP server through the radio network, distinguished by boat number, and stored in the database after being parsed by the collection service, and are distinguished by boat number.

[0126] File data is stored in the drive path.

[0127] 03FA0101, 03FA0102, 03FA010~N refer to the boat numbers, representing different boats.

[0128] The SFTP server is used to receive and temporarily store various types of data uploaded by each boat.

[0129] UXDB is a dedicated database used to store parsed structured data.

[0130] The file directory is used to store parsed unstructured data.

[0131] Boat side data design

[0132] a) Data storage table design Figure 6 As shown;

[0133] b) File service system

[0134] Utilize the existing hardware storage resources on the boat side to build a GlusterFS (see the appendix for an introduction) file service system. Use different methods according to the number of hardware in the environment. When the number of machines is greater than or equal to 4, use the Distributed Replicated mode, and when the number of machines is less than 4, use the Replicated mode.

[0135] Mount the created file service system directory on each machine.

[0136] When a Docker-based capacity-based application uses a file service system, the volume method is used to map the disk to the file service system path.

[0137] c) Data synchronization between boat and shore Figure 7 As shown;

[0138] Data synchronization between the boat and the shore is completed by the data synchronization service. The synchronized data includes structured data in the database on the boat and unstructured data in the file service system.

[0139] The data synchronization service uses the SFTP protocol to synchronize data and uses the gz compression method to compress and transmit data.

[0140] Due to the limited network bandwidth resources between the boat and the shore, the default data synchronization between the boat and the shore is offline, that is, the data will not be automatically synchronized after the application is started. When needed, the synchronization is enabled by sending a synchronization command through the API. After the command is triggered, it is only valid within a single process, and it will automatically return to the closed state after the application is restarted or the hardware is restarted. If real-time synchronization is required, it is necessary to send a synchronization command through the API to enable synchronization after the application is started.

[0141] Shore data design

[0142] Structured data such as Figure 8 As shown;

[0143] b) Unstructured data such as Fig. 9 As shown;

[0144] c) Analysis of shore data

[0145] After the boat synchronizes the data to the shore SFTP server, the shore collection service downloads, parses, stores and analyzes the data. The structured data is written to UXDB for storage after parsing, and the hard disk space instruction directory for storing unstructured data is as follows:

[0146] The unstructured data storage directory structure is as follows:

[0147] Log: / share / log / boat number / home / zcgj / log / log file

[0148] Video: / share / video / boat number / home / zcgj / video / video file

[0149] Image: / share / image / boat number / home / zcgj / image / image file

[0150] Data: / share / date / boat number / home / zcgj / date / data file

[0151] Data analysis flow chart Fig.10 As shown;

[0152] Database Management

[0153] The boat-shore database is accessed based on the schema name. The fixed database name is uxdb. There are multiple different schemas under one database, and the corresponding tables are managed under the schema.

[0154] Usage Process

[0155] a) Data collection service, data synchronization service, GlusterFS file system, unified data management service and UXDB database are installed on the boat side.

[0156] b) Each host on the boat side is mounted in the directory allocated by the GlusterFS file system.

[0157] c) Set up SFTP service on the shore side and install data collection service and UXDB database.

[0158] GlusterFS Overview

[0159] The GlusterFS system is a scalable network file system. Compared with other distributed file systems, the GlusterFS system has the characteristics of high scalability, high availability, high performance, and horizontal scalability. In addition, it does not have a metadata server design, so that the entire service has no hidden dangers of single point failure.

[0160] When the client accesses GlusterFS storage, the program first reads and writes data by accessing the mount point. The cluster file system is transparent to users and programs, and they cannot tell whether the file system is local or on a remote server. The read and write operations will be handled by VFS (Virtual File Syatem), which will hand over the request to the FUSE kernel module, and FUSE will hand over the data to the GlusterFS Client through the device / dev / fuse. Finally, the GlusterFS Client calculates and sends the request or data to the GlusterFS Server through the network.

[0161] Unified Video Integration

[0162] During the execution of the mission, the unmanned boat platform will provide various types of videos, including optoelectronic videos, videos from cameras installed in multiple positions on the hull, radar videos, and sonar videos. Each of the above videos uses RTSP for data stream transmission. By uniformly controlling the transmission of various video streams, a unified video service is designed and implemented to complete the unified storage management, search and playback of multimedia information, and RTSP video stream playback.

[0163] Multimedia integrated management mainly includes two parts: unmanned boat sub-system and shore sub-system.

[0164] 6.4.1 Unmanned Boat Terminal System

[0165] a) Working process

[0166] The unmanned boat terminal system is responsible for receiving, processing, storing and forwarding multimedia information output by various loads in the unmanned boat platform. It includes:

[0167] 1) When the system starts, the unmanned boat subsystem receives multimedia data generated by the payload and processes it. It receives the video stream generated by optoelectronics and transfers it to the NVR using RTSP; it converts the JPEG images generated by sonar and radar into video streams and transfers them to the NVR using RTSP.

[0168] 2) When the shore-side subsystem requests a specified video stream from the unmanned boat subsystem, the unmanned boat subsystem indexes the specific payload, processes and converts it into the corresponding RTSP stream and forwards it to the shore-side server.

[0169] b) Composition

[0170] The unmanned boat subsystem consists of a video matrix NVR, an RTSP service board, and a boat-side video management service running in it. The RTSP service module is responsible for accessing the video of the camera, radar, optoelectronics, sonar, and NVR equipment on the unmanned boat, and providing video stream request services to the outside, and analyzing and processing internal data through the data processing module. The unmanned boat subsystem can convert the data generated by multiple loads on a single boat into a unified format RTSP code stream, and exchange data with the shore cluster through only one gigabit network. Fig.11 As shown in the figure, the types of data exchanged between the unmanned boat subsystem and each device are as follows:

[0171] 1. Camera: Get H264 / H265 data from the camera via RTSP protocol

[0172] 2. Radar: Transmitting RTP+JPEG data over the network

[0173] 3. Photoelectric tracker: Get H264 data from photoelectric through RTSP protocol

[0174] 4. Sonar: Transmit RTP+JPEG data through the network, and use fixed ports to transmit position data

[0175] 5.NVR: NVR device obtains H264 / H265 / JPEG data storage from the unmanned boat through the RTSP protocol

[0176] 1) Hardware

[0177] It includes an RTSP server card and NVR. The hardware structure diagram is as follows Fig.13 As shown:

[0178] The RTSP service board of the unmanned boat subsystem is a 6U board, which is connected to the framework in VPX mode. It receives streaming media data from cameras, radars, optoelectronics, and sonars through the framework for unified management, and can achieve the splicing of more than four camera videos; it stores streaming media data through NVR devices; and it supports sending all payload data to the shore for display in the form of RTSP video streams.

[0179] The RTSP service board provides an SDI video input interface, which can receive the original video data input by SDI and provide functions such as target detection and recognition.

[0180] 2) Video management services

[0181] The submarine terminal system provides video management services. Operators can log in to the management interface through a web browser to uniformly manage the videos of system devices.

[0182] The main console interface displays the RTSP stream push status currently received and forwarded by all sub-devices, as well as the server's running time, load and traffic status, etc.

[0183] The device management interface can display the online status of the unmanned boat video equipment in real time, and can perform addition, modification and deletion operations.

[0184] 6.4.2 Shore terminal system design

[0185] a) Working process

[0186] The shore subsystem consists of shore cluster video service, video playback service and shore NVR.

[0187] 1) The shore cluster service is responsible for obtaining RTSP stream video data from the unmanned boat through the network, and generating a playable RTMP stream and forwarding it to the display and control client;

[0188] 2) The video playback service is responsible for providing the cluster display console with functions such as real-time video playback, video playback, video storage device management, and video splicing configuration.

[0189] b) Composition

[0190] 1) NVR Hardware

[0191] NVR is responsible for:

[0192] 1. According to the application of the cluster console, request streaming to the cluster video service running on the shore side

[0193] 2. According to the requirements of the cluster console, the video is selectively stored for subsequent playback, archiving, etc.

[0194] 2) On-shore cluster video service

[0195] The shore cluster service is mainly composed of the RtspServer module and the RtspClient module. The RtspServer module is responsible for RTSP negotiation and RTP data transmission. The RtspClient provides the function of negotiating and requesting data from the specified service. The shore cluster service can receive the cluster console's request for pulling streams of different videos, and pull the RTSP stream of the corresponding load from the unmanned boat subsystem and start forwarding the corresponding load.

[0196] 3) Video playback service

[0197] Provides video stitching playback interface, playback interface, and video stitching configuration interface.

[0198] 1. Video stitching playback interface

[0199] The video stitching playback interface supports the following video display modes:

[0200] In the one-line video splicing playback interface, the "front", "back", "left" and "right" videos of the unmanned boat platform and the sonar, radar and optoelectronic signals each occupy a row. Fig.12 shown.

[0201] In the four-pane video stitching playback interface, the "front", "back", "left" and "right" videos of the unmanned boat platform are on the left side of the interface, arranged in four panes. Sonar, radar and optoelectronic signals are arranged vertically on the right side of the interface.

[0202] In the nine-pane video stitching playback interface, the left side of the interface shows the "front", "back", "left" and "right" videos of the unmanned boat platform, and the other panes are reserved for use. Sonar, radar and optoelectronic signals are arranged vertically on the right side of the interface.

[0203] 2. Video playback interface

[0204] The video playback interface is played by default in a straight line. The operator can select the start and end time of the playback. After clicking "Start playback", all panes play the video of the corresponding time period at the same time.

[0205] 3. Video stitching configuration interface

[0206] In the video configuration interface, the operator can change the basic configurations of each pane, such as the display name, border color, and border width, and can select the splicing arrangement of the panes, such as Fig.13 shown.

[0207] 6.4.3 Function and performance

[0208] a) Function

[0209] Multimedia integrated management functions such as Fig.14 shown.

[0210] 1) Configuration Management

[0211] Cluster controllers can browse the video stream list, configure the image stream videoization and video matrix splicing rules through the human-computer interaction interface.

[0212] 2) RTSP stream integration

[0213] The radar, sonar and other payload equipment on the unmanned boat platform will output a timed output image sequence, for example, one image every 1.5 seconds. The RTSP stream integration function converts the output image sequence into RTSP streaming media.

[0214] For the functional payload devices such as cameras and optoelectronic trackers carried by unmanned boat platforms that directly output RTSP video streams, the RTSP stream integration function integrates these RTSP video streams together to achieve multi-channel integration of a single RTSP video address. The cluster controller can select the video to be displayed according to the channel. This realizes a single-channel multi-RTSP video address mode and provides a unified RTSP video address format.

[0215] 3) Video Processing

[0216] This function supports the combined display of multiple videos and the overlay of video information. The cluster controller can configure the video overlay text and timestamp information through the human-computer interaction interface. For example, the word "front" needs to be marked in the upper right corner of the forward-facing camera. The unmanned boat platform has 4 cameras. This function supports the combined display of the 4 cameras in the order of "one" or "left front and right back" and output addresses or channels. When combining videos for display, the border thickness and color can be set between videos.

[0217] 4) Video storage

[0218] This function supports unified video storage and management for all managed videos. Cluster controllers can set the video storage directory structure design and storage segmentation method. Videos can be segmented by time or file size for easy storage.

[0219] 5) Video playback

[0220] Provide video playback for all stored videos. The cluster controller can set the start timestamp to locate the video information and playback, and can also set the playback rate.

[0221] 6) Display and control terminal human-computer interaction

[0222] Provides a human-computer interaction interface for video data display and control. Cluster operators can set video channels, video matrices, etc. The system displays RTSP videos or RTSP channels of different addresses according to the settings. Operators can also adjust the configuration and it will take effect automatically.

[0223] b) Performance

[0224] The main performance indicators include:

[0225] 1) The RTSP video bit rate on the boat side can be adjusted to control traffic and bandwidth, and the total bandwidth can be configured on the shore side;

[0226] 2) Video stitching requirements: Ability to automatically stitch videos according to configuration requirements;

[0227] 3) Configuration requirements: Ability to customize various functional requirements;

[0228] 4) Video playback rate supports at least 0.5x, 1x, 1.2x, 1.5x, 2x, and 3x;

[0229] 5) All programs support the domestic Kylin operating system.

[0230] Optionally, the boat-side data collected by each unmanned boat is obtained, including:

[0231] The collected initial boat-side data is encrypted using a preset encryption algorithm to obtain the encrypted boat-side data. The initial boat-side data is collected by the collection equipment on the unmanned boat. The collection equipment includes at least sensors, radars, optoelectronic equipment and Beidou. The initial boat-side data is used to characterize the operating status of the unmanned boat.

[0232] Optionally, the method further comprises:

[0233] The non-file data is stored in a database, and the file data is stored in a preset directory, wherein the file data at least includes processed video data, and the non-file data at least includes log files.

[0234] Optionally, the boat-side data is parsed and processed to obtain processed data, including:

[0235] Receive the video stream generated by optoelectronics and store it in NVR using RTSP;

[0236] Convert JPEG images generated by sonar and radar into video streams and transfer them to NVR via RTSP.

[0237] Optionally, based on the processed data, displaying the integrated data and playing the video data include:

[0238] Based on the processed data, real-time video playback, video playback, video storage device management and video splicing display are carried out.

[0239] Optionally, the method further comprises:

[0240] When receiving service requests from multiple unmanned boats, the service requests are processed according to the service priority.

[0241] Optionally, the method further comprises:

[0242] Judge the processed data to determine whether the unmanned boat has a fault;

[0243] If the processed data exceeds a preset threshold, an alarm is displayed, which includes at least a sound alarm and a light alarm.

[0244] Optionally, the method further comprises:

[0245] When an unmanned boat fails, the failed unmanned boat is switched.

[0246] Optionally, the method further comprises:

[0247] The boat-side data of each unmanned boat and the server data corresponding to the boat-side data are shared.

[0248] The embodiment of the present application proposes a method for designing the integrated resources of a cluster of surface unmanned boats, which is applied to a domestically produced intelligent control device for unmanned boats. The device has a modular design and supports the integration of functions such as general loads, mission loads, mission processing control, navigation processing control, and collaborative control. It provides a unified open loading environment and basic capability environment, including high-speed networks, electrical interfaces, timing, power supply, packaging, electromagnetic compatibility environment, safety protection, data storage management, software management, and callable common function modules. This design enables the integration and collaborative capabilities of surface unmanned boats to be comprehensively improved to meet future development needs.

[0249] In addition, the embodiment of the present application also provides a hardware architecture, based on the rack-type structure of the backplane bus and the IO board, each functional module is integrated in the chassis in the form of a board, connected to the backplane bus through a connector, and uniformly packaged. This hardware architecture supports the rapid transplantation of various mission payloads and the rapid formation of collaborative formations.

[0250] To sum up, the embodiments of the present application effectively solve the problems of customization, interoperability, system upgrade, security and data management of intelligent control equipment for surface unmanned boats through the design and implementation of domestically produced intelligent control equipment for unmanned boats, and provide new paths and possibilities for the development of the field of surface unmanned boat technology.

[0251] The surface unmanned boat swarm resource integration design of the embodiment of the present application has many beneficial effects, which will profoundly affect the development and application of surface unmanned boat swarm resource inheritance design. The following is a detailed description of the beneficial effects of this patent:

[0252] 1) Modular and universal design: The present invention adopts a modular and universal design method, so that the unmanned boat intelligent control equipment can be applied to various general loads and mission loads. This design can greatly reduce the development and integration costs of the unmanned boat system, while providing higher flexibility and configurability, so that different tasks and applications can be quickly adapted and deployed.

[0253] 2) High security and data management: The present invention focuses on security and data management, and adopts multi-level security measures, including network security, data encryption, access control and firewalls. These security features help protect the unmanned boat system from the threat of network attacks and data leakage, especially for sensitive data applications, which plays an important role in protection.

[0254] 3) Fully dynamic, reconfigurable and upgradeable: The unmanned boat intelligent control device of the present invention is fully dynamic and can adapt to rapidly changing mission requirements. In addition, it also supports hardware and software upgrades, so that the system can continuously improve performance with the advancement of technology, extend the service life of the equipment, and reduce maintenance and replacement costs.

[0255] 4) Collaborative control and formation capability: Through the plug-in method of collaborative task boards, the present invention makes it possible for unmanned boats to upgrade from a single operation mode to a collaborative mode. This collaborative capability facilitates the collaborative work and formation actions between multiple boats, improves the task execution efficiency of surface unmanned boats, reduces the impact on the overall task system, and achieves the advantage of plug-and-play.

[0256] 5) Adapt to future development needs: The present invention provides a solution that adapts to future technological development needs. Through scalable hardware configuration and software upgrades, the unmanned boat intelligent control equipment can maintain compatibility with new technologies, ensuring that the system remains competitive and sustainable in the face of new challenges and opportunities.

[0257] In summary, the intelligent control equipment for unmanned surface boats and the integration method thereof of the present invention, through the characteristics of modularity, generalization, high security, full dynamics, reconfigurability, upgradeability, scalability and growability, have brought significant technological progress and application potential to the field of unmanned surface boats, and will provide more flexible, efficient and safe solutions for the development of related fields.

[0258] According to a second aspect of the present invention, a control device for surface unmanned boat cluster resources is provided, including a control method for surface unmanned boat cluster resources according to any one of the first embodiments, the control device comprising: multiple boards, IO boards and a chassis, adopting a rack-type structure of a backplane bus and an IO board, each board is integrated inside the chassis, connected to the backplane bus through a connector, each board is connected through Ethernet to realize data exchange, and is connected to the payload equipment on the surface unmanned boat through a specific interface on the IO board.

[0259] As shown in Table 1 below, the main hardware components of the unmanned boat intelligent control equipment are as follows:

[0260]

[0261] In summary, the embodiment of the present application has a modular design and supports the integration of general payload, mission payload, mission processing control, navigation processing control and collaborative control. It provides a unified open loading environment and basic capability environment, including high-speed network, electrical interface, time system, power supply, packaging, electromagnetic compatibility environment, safety protection, data storage management, software management and callable public function modules. This design enables the integration and collaborative capabilities of surface unmanned boats to be comprehensively improved to meet future development needs.

[0262] The above are preferred embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for controlling surface unmanned boat cluster resources, characterized in that: include: Obtain the boat-side data collected by each unmanned boat; Analyzing and processing the boat-side data to obtain processed data, wherein the processed data at least includes integrated data and video data; The integrated data is displayed and the video data is played according to the processed data.

2. The method for controlling the surface unmanned boat swarm resources according to claim 1, characterized in that: The obtaining of boat-side data collected by each unmanned boat includes: The collected initial boat-side data is encrypted using a preset encryption algorithm to obtain the encrypted boat-side data. The initial boat-side data is collected by a collection device on the unmanned boat. The collection device includes at least sensors, radars, optoelectronic equipment and Beidou. The initial boat-side data is used to characterize the operating status of the unmanned boat.

3. The method for controlling the surface unmanned boat cluster resources according to claim 1, characterized in that: The method further comprises: The non-file data is stored in a database, and the file-type data is stored in a preset directory, wherein the file-type data at least includes processed video data, and the non-file data at least includes log files.

4. The method for controlling the surface unmanned boat swarm resources according to claim 1, characterized in that: The analyzing and processing the boat-side data to obtain processed data includes: Receive the video stream generated by optoelectronics and store it in NVR using RTSP; Convert JPEG images generated by sonar and radar into video streams and transfer them to NVR via RTSP.

5. The method for controlling the resources of the unmanned surface boat swarm according to claim 1, characterized in that: The step of displaying the integrated data and playing the video data according to the processed data includes: Based on the processed data, real-time video playback, video playback, video storage device management and video splicing display are performed.

6. The method for controlling the resources of the unmanned surface boat swarm according to claim 1, characterized in that: The method further comprises: When receiving service requests sent by multiple unmanned boats, the service requests are processed according to the service priority levels.

7. The method for controlling the resources of the unmanned surface boat swarm according to claim 1, characterized in that: The method further comprises: Judge the processed data to determine whether the unmanned boat has a fault; If the processed data exceeds a preset threshold, an alarm display is performed, and the alarm display includes at least a sound alarm and a light alarm.

8. The method for controlling the surface unmanned boat cluster resources according to claim 7, characterized in that: The method further comprises: When an unmanned boat fails, the failed unmanned boat is switched.

9. The method for controlling the resources of the unmanned surface boat swarm according to claim 1, characterized in that: The method further comprises: The boat-side data of each unmanned boat and the server data corresponding to the boat-side data are shared.

10. A control device for a cluster of unmanned surface boats, characterized in that: A control method for surface unmanned boat cluster resources comprising the method described in any one of claims 1-9, wherein the control device comprises: multiple boards, an IO board and a chassis, and a rack-type structure of a backplane bus and an IO board. Each board is integrated inside the chassis and connected to the backplane bus through a connector. Each board is connected through Ethernet to achieve data exchange, and is connected to the payload equipment on the surface unmanned boat through a specific interface on the IO board.

Images