Ship-shore video transmission system and method

Abstract

The invention discloses a ship-shore video transmission system and method, and relates to the technical field of ship communication. The ship-shore video transmission system comprises a user side, a ship side and a shore side, the ship end comprises a client and a self-adaptive video coding module; the shore end comprises a server; the server is used for acquiring a playing instruction of a ship-end target camera and sending a transmission instruction of a current video stream of the ship-end target camera to the client; the client is used for receiving a transmission instruction of a current video stream of a ship end target camera and calling the self-adaptive video coding module; and the adaptive video coding module is used for adaptively coding the current video stream of the ship end target camera to obtain a target video stream, and transmitting the target video stream to the shore end and the user end. Therefore, by arranging the adaptive video coding module, adaptive coding adjustment can be performed on the current video stream of the ship-end target camera, so that delay can be reduced, and stability and reliability of ship-shore video transmission can be improved.

Description

Technical Field

[0001] The present application relates to the field of ship communication technology, and in particular to a ship-to-shore video transmission system and method. Background Art

[0002] Real-time video transmission is of great significance for monitoring maritime conditions and ensuring navigation safety in scenarios such as offshore operations, ocean voyages, maritime rescue, and deep-sea aquaculture. However, due to the complex marine environment, traditional satellite communications or radio communications are often affected by signal attenuation, multipath effects, atmospheric interference, and other factors when transmitting video signals, resulting in unstable video transmission quality and large delays, which cannot meet actual needs. Summary of the invention

[0003] Purpose of the invention: An embodiment of the present application provides a ship-to-shore video transmission system, aiming to overcome the technical problems of unstable ship-to-shore video transmission and large delay in the prior art; another purpose of an embodiment of the present application is to provide a ship-to-shore video transmission method.

[0004] Technical solution: A ship-to-shore video transmission system described in an embodiment of the present application includes: a user end, a ship end and a shore end; the ship end includes a client and an adaptive video encoding module; the shore end includes a server end;

[0005] The server is used to obtain a playback instruction of the ship-side target camera, and send a transmission instruction of the current video stream of the ship-side target camera to the client;

[0006] The client is used to receive the transmission instruction of the current video stream of the ship-side target camera and call the adaptive video encoding module;

[0007] The adaptive video encoding module is used to adaptively encode the current video stream of the ship-side target camera to obtain a target video stream, and transmit the target video stream to the shore end and the user end.

[0008] In some embodiments, the adaptive video encoding module includes an adaptive video encoding unit;

[0009] The adaptive video encoding unit is used to dynamically adjust the encoding parameters of the current video stream of the ship-side target camera according to the network bandwidth situation and / or the network delay situation.

[0010] In some embodiments, the encoding parameters include at least resolution, bit rate and frame rate.

[0011] In some embodiments, the adaptive video encoding unit is also used to dynamically adjust the resolution and bit rate of the current video stream of the ship-side target camera according to the network bandwidth conditions, and / or dynamically adjust the frame rate of the current video stream of the ship-side target camera according to the network delay conditions.

[0012] In some embodiments, the adaptive video encoding unit is also used to increase the resolution and the bit rate when the network bandwidth condition meets the preset network bandwidth condition, and reduce the resolution and the bit rate when the network bandwidth condition does not meet the preset network bandwidth condition; and / or, when the network delay condition meets the preset network delay condition, increase the frame rate, and when the network delay condition does not meet the preset network delay condition, reduce the frame rate.

[0013] In some embodiments, the adaptive video encoding module further includes a video capture unit, a network transmission unit, and a control response unit;

[0014] Wherein, the video capture unit is used to capture the video stream from the ship-end target camera;

[0015] The network transmission unit is used to send the target video stream to the shore end through the network;

[0016] The control response unit is used to adjust the encoding parameters of the current video stream of the ship-side target camera according to the control instruction sent by the shore.

[0017] In some embodiments, the ship side also includes a multi-mode communication fusion and intelligent routing module; the multi-mode communication fusion and intelligent routing module is used to combine multiple communication modes, monitor the status information of the communication links of various communication modes in real time, and intelligently route to the best communication link.

[0018] In some embodiments, the multi-mode communication fusion and intelligent routing module includes a signal monitoring unit, a plurality of communication interfaces, and an intelligent routing unit;

[0019] Wherein, the multiple communication interfaces are used to provide multiple communication modes;

[0020] The signal monitoring unit is used to monitor the status information of the communication links of various communication modes in real time;

[0021] The intelligent routing unit is used for intelligently routing to the best communication link according to the status information of the communication links of various communication modes and a preset evaluation mechanism.

[0022] In some embodiments, the preset evaluation mechanism includes:

[0023] Define evaluation metrics and weights for each communication link;

[0024] Calculate the comprehensive score of each communication link according to the evaluation index and weight of each communication link;

[0025] According to the comprehensive score of each communication link, the communication link with the highest score is selected as the current best link and the actual network traffic is switched to the selected best communication link.

[0026] In some embodiments, the shore end further includes a streaming media server and a video playback service module; wherein the streaming media server is used to receive the target video stream and transmit it to the video playback service module.

[0027] In some embodiments, the streaming media server is configured with multi-encoding support and cross-protocol conversion, an asynchronous event-driven multi-threading model, and zero-copy technology.

[0028] In some embodiments, the video playback service module is provided with a multi-screen display, playback control options and a user interface.

[0029] In some embodiments, the TCP transmission protocol is used between the server and the client.

[0030] Accordingly, a ship-to-shore video transmission method described in an embodiment of the present application includes:

[0031] The server obtains a start playback instruction from the ship-side target camera, and sends a transmission instruction of the current video stream of the ship-side target camera to the client according to the start playback instruction from the ship-side target camera;

[0032] The client adaptively encodes the current video stream of the ship-side target camera according to the transmission instruction of the current video stream of the ship-side target camera to obtain the target video stream, and transmits the target video stream to the shore and the user end.

[0033] Beneficial effects: Compared with the prior art, the ship-to-shore video transmission system and method of the embodiment of the present application, the ship-to-shore video transmission system includes: a user end, a ship end and a shore end; the ship end includes a client and an adaptive video encoding module; the shore end includes a server end; the server end is used to obtain the playback instruction of the ship-end target camera, and send the transmission instruction of the current video stream of the ship-end target camera to the client end; the client end is used to receive the transmission instruction of the current video stream of the ship-end target camera, and call the adaptive video encoding module; the adaptive video encoding module is used to adaptively encode the current video stream of the ship-end target camera to obtain the target video stream, and transmit the target video stream to the shore end and the user end. It can be seen that by setting the client end, the adaptive video encoding module and the server end, the video transmission between the ship end and the shore end can be realized. And by setting the adaptive video encoding module, the current video stream of the ship-end target camera can be adaptively encoded and adjusted, thereby reducing the delay and improving the stability and reliability of the ship-to-shore video transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0035] Figure 1 It is a principle structure block diagram of a ship-to-shore video transmission system provided in an embodiment of the present application;

[0036] Figure 2 It is a principle structure block diagram of another ship-to-shore video transmission system provided in an embodiment of the present application;

[0037] Figure 3 It is a principle structure block diagram of another ship-to-shore video transmission system provided in an embodiment of the present application;

[0038] Figure 4 It is a flow chart of a ship-to-shore video transmission method provided in an embodiment of the present application.

[0039] Reference numerals:

[0040] 100-user end; 200-shore end; 300-ship end; 110-video playback module; 210-server end; 220-streaming media server; 230-video playback service module; 310-client end; 320-adaptive video encoding module; 330-multi-mode communication fusion and intelligent routing module; 321-adaptive video encoding unit; 322-video capture unit; 323-network transmission unit; 324-response control unit; 331-signal monitoring unit; 332-intelligent routing unit. DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0042] It should be understood that although the terms first, second, etc. may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another component. Therefore, the first component discussed below can be referred to as the second component without departing from the teachings of the concepts of the present application. As used herein, the term "and / or" includes any one of the associated listed items and all combinations of one or more.

[0043] Those skilled in the art will appreciate that the drawings are only schematic diagrams of example embodiments and may not be to scale. The modules or processes in the drawings are not necessarily required to implement the present application and therefore cannot be used to limit the scope of protection of the present application.

[0044] Figure 1 This is a principle structure diagram of a ship-to-shore video transmission system provided in an embodiment of the present application. Figure 1 The ship-to-shore video transmission system includes: a user end 100, a ship end 300 and a shore end 200; the ship end 300 includes a client end 310 and an adaptive video encoding module 320; the shore end 200 includes a server end 210; the server end 210 is used to obtain a playback instruction of a target camera on the ship end 300, and send a transmission instruction of a current video stream of the target camera on the ship end 300 to the client end 310; the client end 310 is used to receive a transmission instruction of a current video stream of the target camera on the ship end 300, and call the adaptive video encoding module 320; the adaptive video encoding module is used to adaptively encode the current video stream of the target camera on the ship end 300 to obtain a target video stream, and transmit the target video stream to the shore end 200 and the user end 100.

[0045] The user terminal 100 is used for providing users with access to the video playback module.

[0046] Among them, the client 310 is used to establish a communication connection with the shore 200 server 210 to report data and receive control instructions. Specifically, it includes: connection establishment, actively initiating a connection request with the shore 200 server 210 to establish a stable communication channel. Data reporting, regularly reporting the status information of the ship 300 to the shore 200, such as the equipment status, network status and other information on the ship, to help the shore 200 make corresponding decisions. Instruction reception, receiving the control instructions sent by the shore 200 and passing them to the adaptive video encoding module 320 for corresponding adjustments. Error handling, with a reconnection mechanism, automatically trying to reconnect when the connection is disconnected to ensure the continuity of data transmission. Security, supporting Basic / Digest authentication methods to ensure the security of data transmission and prevent unauthorized access. Error handling, with an error detection and recovery mechanism, can automatically try to reconnect when a connection is interrupted or other abnormal conditions occur.

[0047] The TCP transmission protocol may be used between the server 210 and the client 310. Exemplarily, the server 210 is a TCP server, and the client 310 is a TCP client.

[0048] The video stream transmitted from the ship end 300 to the shore end 200 uses the Real Time Streaming Protocol (RTSP).

[0049] In the technical solution of the embodiment of the present application, the implementation process of the ship-to-shore video transmission system is as follows: Figure 1 , assuming that a certain camera (i.e., target camera) on the ship side 300 is required to play the video, the user selects the camera on the user side to start playing, and issues a corresponding camera play instruction (i.e., ship side target camera play instruction). The server side 210 obtains the ship side 300 target camera play instruction, and sends a transmission instruction of the current video stream of the ship side 300 target camera to the client side 310. The client side 310 receives the transmission instruction of the current video stream of the ship side 300 target camera, and calls the adaptive video encoding module 320. The adaptive video encoding module 320 adaptively encodes the current video stream of the ship side 300 target camera to obtain the target video stream, and transmits the target video stream to the shore side 200 and the user side 100 for playback. In this way, video transmission between the ship side 300 and the shore side 200 can be realized, and before the ship side 300 transmits the video stream to the shore side 200, the current video stream is adaptively encoded and adjusted by calling the adaptive video encoding module 320 to reduce the delay of video transmission, improve the stability and reliability of video transmission, and thus realize high-quality and low-delay ship-to-shore video transmission.

[0050] Figure 2This is a principle structure diagram of another ship-to-shore video transmission system provided in the embodiment of the present application. Figure 2 The ship side 300 also includes a multi-mode communication fusion and intelligent routing module 330; the multi-mode communication fusion and intelligent routing module 330 is used to combine multiple communication modes, monitor the status information of the communication links of various communication modes in real time, and intelligently route to the best communication link.

[0051] Among them, the multiple communication modes can be satellite communication, 4G / 5G mobile network and other communication modes, which can be set according to actual conditions and are not specifically limited here.

[0052] Among them, the benefit of setting up multi-mode communication fusion and intelligent routing modules is that by combining multiple communication modes, the status information of the communication links of each communication mode can be monitored in real time, providing a reliable decision-making basis for intelligent routing. And intelligent routing to the best communication link can ensure the stability and efficiency of video transmission.

[0053] In some embodiments, see Figure 2 The shore end 200 also includes a streaming media server 220 and a video playback service module 230; wherein the streaming media server 220 is used to receive the target video stream and transmit it to the video playback service module 230.

[0054] The streaming media server 220 is used to receive the video stream transmitted from the ship side 300 , such as the target video stream transmitted from the adaptive video encoding module 320 of the ship side 300 , and transmit it to the video playback service module 230 .

[0055] In some embodiments, the streaming media server 220 is configured with multi-encoding support and cross-protocol conversion, an asynchronous event-driven multi-threading model, and zero-copy technology.

[0056] The streaming media server 220 can improve the performance and resource utilization of the video transmission system by providing multi-coding support and cross-protocol conversion, an asynchronous event-driven multi-threaded model, and zero-copy technology.

[0057] Among them, multi-encoding support and cross-protocol conversion adopt flexible codec management mechanism and middle-layer conversion mechanism to support multiple encoding formats and realize cross-protocol conversion. Among them, the codec management mechanism allows the system to dynamically load and unload different codec plug-ins, thereby realizing support for multiple encoding formats. The middle-layer conversion mechanism can convert input streams from different protocols into RTSP output streams. Specifically, the system first parses the protocol of the input stream, and then transcodes or directly forwards it as needed. For example, it supports common encoding formats such as H.246 / H.265 / AAC / G.711 / OPUS / MJPEG, and can forward other encoding formats without changing their protocol form, and can convert input streams from RTMP or HLS into RTSP output, increasing compatibility with various types of cameras.

[0058] Exemplary code examples for multiple encoding support and cross-protocol conversion are as follows:

[0059]

[0060]

[0061] Among them, the asynchronous event-driven multi-threaded model adopts an asynchronous event-driven mechanism, combined with a multi-threaded model, so that each connection can be efficiently processed in an independent thread, reducing the context switching overhead. Among them, the asynchronous event-driven mechanism can use at least one of an event loop, a thread pool, and I / O multiplexing to process and schedule events. Specifically, an event loop means that each worker thread has its own event loop for listening to and processing network events. A thread pool refers to using a thread pool to manage and reuse multiple worker threads, reducing the overhead of thread creation and destruction. I / O multiplexing refers to using mechanisms such as epoll (Linux) and kqueue (FreeBSD) to monitor multiple file descriptors. Once a descriptor is ready, the corresponding read and write operations can be performed.

[0062] An exemplary, asynchronous event-driven multi-threaded model code example is as follows:

[0063]

[0064]

[0065]

[0066] Among them, the zero-copy technology uses the zero-copy mechanism provided by the operating system, such as the sendfile function or the splice function, to directly transfer data from the network layer to the application layer, avoiding unnecessary memory copying and improving memory usage efficiency and data processing speed. Among them, the sendfile function refers to sending file contents from one file descriptor directly to another file descriptor without passing through the user space. The splice function refers to moving data between two pipes or between a pipe and a file descriptor, which can also achieve zero copy.

[0067] An exemplary, zero-copy code example is as follows:

[0068]

[0069]

[0070] In some embodiments, the video playback service module 230 is provided with a multi-screen display, playback control options and a user interface.

[0071] Among them, multi-screen display supports the simultaneous display of multiple video streams, which is convenient for users to monitor cameras in multiple locations. Users can choose single-screen or multi-screen mode to meet different monitoring needs.

[0072] Among them, the video playback service module 230 specifically controls functions, such as setting playback control options to provide rich playback control options, such as play, pause, fast forward, fast rewind, screenshot, full screen, etc., to facilitate users to operate the video stream.

[0073] Among them, the user interface is used to provide a friendly user interface, support web browser access and mobile device access, so that users can easily use it on a variety of devices.

[0074] In some embodiments, see Figure 2 , the user terminal 100 includes a video playing module 110 .

[0075] The video playback module 110 allows the user to choose to play videos of various cameras on the ship.

[0076] The video playback module 110 of the user end 100 uses the http(s) protocol to access the video playback service at the shore end 200, and uses websocket to implement video streaming playback.

[0077] In the technical solution of this embodiment, the implementation process of the ship-to-shore video transmission system is as follows: Figure 2, assuming that a certain camera (i.e., target camera) on the ship side 300 needs to play a video, the user selects the camera in the video playback module 110 of the user side 100 to start playing, and issues a corresponding camera playback instruction (i.e., ship side target camera playback instruction). The server side 210 obtains the playback instruction of the ship side 300 target camera, and sends a transmission instruction of the current video stream of the ship side 300 target camera to the client side 310. The client side 310 receives the transmission instruction of the current video stream of the ship side 300 target camera, and calls the adaptive video encoding module 320. The adaptive video encoding module 320 adaptively encodes the current video stream of the ship side 300 target camera to obtain the target video stream, and transmits the target video stream to the streaming media server 220 on the shore side 200. The streaming media server 220 receives the video stream transmitted from the ship side 300, such as the target video stream transmitted from the ship side 300 adaptive video encoding module 320, and transmits it to the video playback service module 230. The video playback service module 230 provides the video stream for the user side 100. Thus, video transmission between the ship end 300 and the shore end 200 can be realized, and before the ship end 300 transmits the video stream to the shore end 200, the adaptive video encoding module 320 is called to adaptively adjust the encoding of the current video stream to reduce the delay of video transmission, improve the stability and reliability of video transmission, and thus realize high-quality and low-delay ship-to-shore video transmission. By setting up the multi-mode communication fusion and intelligent routing module 330, it is possible to effectively cope with the complex communication environment at sea, realize high-quality and low-delay video transmission, and improve the stability and real-time performance of video transmission.

[0078] And by setting up the streaming media server 220, the overall performance of the system and the user experience are improved. Specifically, the high concurrent processing capability and the asynchronous event-driven multi-threaded model enable the server to maintain efficient operation in a high-concurrency environment. Low-latency response reduces context switching overhead and improves the response speed of the system. Efficient memory usage and zero-copy technology reduce unnecessary memory copy operations and reduce memory consumption. High resource utilization, through thread pool and I / O multiplexing technology, system resources are effectively utilized to improve overall performance. At the same time, other modules comprehensively apply the above technologies to significantly improve the efficiency, quality and safety of ship-shore video transmission, provide strong technical support for scenarios such as offshore operations, offshore rescue, and deep-sea aquaculture, and improve work efficiency and operational safety.

[0079] Figure 3 This is a principle structure diagram of another ship-to-shore video transmission system provided in the embodiment of the present application. Figure 3 The adaptive video encoding module 320 includes an adaptive video encoding unit 321; the adaptive video encoding unit 321 is used to dynamically adjust the encoding parameters of the current video stream of the target camera 300 on the ship side according to the network bandwidth and / or network delay.

[0080] The network bandwidth refers to the ability of the network to transmit data during video transmission, which can also be understood as the data transmission rate of the network.

[0081] Among them, network delay refers to the time delay required for video transmission from the sender to the receiver.

[0082] In the technical solution of the embodiment of the present application, by setting the adaptive video encoding unit 321, it is possible to dynamically adjust the encoding parameters of the current video stream of the target camera 300 on the ship side according to the network bandwidth and / or network delay to adapt to different network conditions.

[0083] In some embodiments, the encoding parameters include at least resolution, bit rate, frame rate and compression ratio.

[0084] In some embodiments, the adaptive video encoding unit 321 is also used to dynamically adjust the resolution and bit rate of the current video stream of the ship-side target camera according to the network bandwidth conditions, and / or dynamically adjust the frame rate of the current video stream of the ship-side target camera according to the network delay conditions.

[0085] In the technical solution of the embodiment of the present application, by setting the adaptive video encoding unit 321, it is also possible to achieve: making different dynamic adjustments according to specific network conditions to adapt to the bandwidth limitations of different communication links, thereby ensuring the quality of video transmission and improving the reliability and stability of video transmission.

[0086] In some embodiments, the adaptive video encoding unit 321 is also used to increase the resolution and bit rate when the network bandwidth situation meets the preset network bandwidth condition, and reduce the resolution and bit rate when the network bandwidth situation does not meet the preset network bandwidth condition; and / or, when the network delay situation meets the preset network delay condition, increase the frame rate, and when the network delay situation does not meet the preset network delay condition, reduce the frame rate.

[0087] The preset network bandwidth condition is: the network condition is good. Specifically, the adaptive video encoding unit 321 dynamically adjusts the resolution of the video stream according to the network bandwidth. When the network condition is poor, the resolution is reduced to reduce the amount of data transmission. When the network condition is good, the resolution is increased to provide a clearer image. And according to the network bandwidth, the bit rate of the video stream is dynamically adjusted. When the network condition is poor, the bit rate is reduced to reduce the amount of data transmission. When the network condition is good, the bit rate is increased to provide a higher image quality.

[0088] The preset network delay condition is: the network delay is low. Specifically, the adaptive video encoding unit 321 dynamically adjusts the frame rate of the video stream according to the network delay. When the network delay is high, the frame rate is reduced to reduce the amount of data transmission. When the network delay is low, the frame rate is increased to provide a smoother video experience.

[0089] In some embodiments, see Figure 3 The adaptive video encoding module 320 also includes a video capture unit 322, a network transmission unit 323 and a control response unit 324; wherein the video capture unit 322 is used to capture the video stream from the target camera on the ship side 300; the network transmission unit 323 is used to send the target video stream to the shore side 200 through the network; the control response unit 324 is used to adjust the encoding parameters of the current video stream of the target camera on the ship side 300 according to the control instruction sent by the shore side 200.

[0090] Among them, video capture captures video frames from the camera and supports multiple camera models and interfaces.

[0091] The network transmission unit 323 sends the target video stream to the streaming media server 220 on the shore side 200 through the network, and supports multiple transmission protocols, such as RTP over UDP / TCP / HTTP.

[0092] Among them, the control response unit 324 is used to adjust the encoding parameters of the current video stream of the target camera on the ship side 300 according to the control instruction sent by the shore side 200, such as reducing the resolution to adapt to poor network conditions, or increasing the resolution to improve image quality.

[0093] In the technical solution of the embodiment of the present application, the adaptive video encoding module 320 is a key technology to ensure the quality of ship-to-shore video transmission and user experience. Specifically, it includes: First, network monitoring. The server 210 regularly monitors the network bandwidth and estimates the current available bandwidth by sending test data packets and measuring the response time. The network delay is evaluated by measuring the round-trip time (RTT) of the data packet. The lower the delay, the better the real-time performance of the video transmission. Second, quality adjustment. The resolution of the video stream is dynamically adjusted according to the network bandwidth. When the network condition is poor, the resolution is reduced to reduce the amount of data transmission. When the network condition is good, the resolution is increased to provide a clearer image. According to the network bandwidth, the bit rate of the video stream is dynamically adjusted. When the network condition is poor, the bit rate is reduced to reduce the amount of data transmission. When the network condition is good, the bit rate is increased to provide higher image quality. According to the network delay, the frame rate of the video stream is dynamically adjusted. When the network delay is high, the frame rate is reduced to reduce the amount of data transmission. When the network delay is low, the frame rate is increased to provide a smoother video experience. Third, the feedback mechanism, the client 310 can send feedback information to the server 210 to report the current playback status (such as buffer status, packet loss rate, etc.), and the server 210 makes further adjustments based on these feedback information. The server 210 can automatically adjust the quality parameters of the video stream according to the preset strategy, or manually adjust according to the request of the client 310. Among them, the preset strategy can be setting the adjustment parameter range, etc., which can be set according to the actual situation, and no specific limitation is made here.

[0094] Exemplarily, in an embodiment of the present application, the adaptive video encoding module 320 includes the following steps: initialize the connection, establish a connection with the client 310, and start transmitting the initially configured video stream. Monitor the network status, regularly monitor the network bandwidth and network delay, and collect relevant data. Analyze the data, and analyze the current network status based on the collected data. Adjust the parameters, and dynamically adjust the resolution, bit rate, and frame rate of the video stream based on the analysis results. Send the adjusted video stream, and send the adjusted video stream to the client 310. Receive feedback, and receive feedback information from the client 310 to further optimize the quality of the video stream. Continuously optimize and repeat the above process to ensure that the video stream can provide reliable video transmission and playback under different network conditions.

[0095] In some embodiments, see Figure 3 The multi-mode communication fusion and intelligent routing module 330 includes a signal monitoring unit 331, multiple communication interfaces and an intelligent routing unit 332; wherein the multiple communication interfaces are used to provide multiple communication modes; the signal monitoring unit 331 is used to monitor the status information of the communication links of various communication modes in real time; the intelligent routing unit 332 is used to intelligently route to the best communication link according to the status information of the communication links of various communication modes and a preset evaluation mechanism.

[0096] The signal monitoring unit 331 and multiple communication interfaces constitute the hardware architecture of the multi-mode communication fusion technology. Figure 3 The multiple communication interfaces include communication interface 1, communication interface 2, ... communication interface n. The multiple communication interfaces may include a satellite communication interface, a 4G / 5G mobile network interface, a bridge interface, and the like.

[0097] The signal monitoring unit 331 is integrated with a high-precision signal quality monitoring sensor for collecting status information of each communication link in real time, including signal quality, bandwidth, delay, etc., to provide a decision basis for the intelligent routing algorithm.

[0098] Among them, the implementation steps of multi-mode communication fusion technology include: data acquisition, which collects the signal quality (such as signal strength, bit error rate), network bandwidth, delay time, packet loss rate, etc. of each communication mode (such as satellite communication, 4G / 5G, and bridge) in real time through the program. Data preprocessing, cleans and standardizes the collected data to ensure data consistency and availability. For example, normalize the signal strength to between 0 and 1, convert the bandwidth to Mbps, and convert the delay time to milliseconds. Data storage, use a database or data cache system (such as Redis) to store the collected data for fast access and processing.

[0099] The intelligent routing unit 332 implements intelligent routing to the best communication link through an intelligent routing algorithm. The goal of the intelligent routing algorithm is to evaluate the quality of each communication link according to predefined weights and rules. The implementation steps include:

[0100] Define evaluation indicators and weights: Signal Strength weight 0.4; Bandwidth weight 0.3; Latency weight -0.2 (negative value means the lower the latency, the better); Packet Loss Rate weight -0.1 (negative value means the lower the packet loss rate, the better); Switching Cost weight -0.05 (negative value means the lower the switching cost, the better).

[0101] Calculate the comprehensive score: Calculate the comprehensive score of each communication link according to the defined weight. For example, if the signal strength of a communication link is 0.8, the bandwidth is 50Mbps, the delay is 50ms, and the packet loss rate is 0.01%, then the comprehensive score = (0.8×0.4)+(50×0.3)+(-50×-0.2)+(-0.01×-0.1)+(switching cost×-0.05).

[0102] In some embodiments, the preset evaluation mechanism includes: defining evaluation indicators and weights for each communication link; calculating a comprehensive score for each communication link based on the evaluation indicators and weights of each communication link; selecting the communication link with the highest score as the current best link based on the comprehensive score of each communication link and switching the actual network traffic to the selected best communication link.

[0103] Among them, a dynamic traffic switching mechanism is used to select the communication link with the highest score as the current best link and switch the actual network traffic to the selected best communication link. The specific implementation steps include: selecting the best link, selecting the link with the highest comprehensive score from all communication links as the current best link, and obtaining the IP address and interface information of the selected best communication link. Obtain the current default route, check the current default route, and determine the currently used exit link. Update the routing table, delete the old default route, and if other links (such as satellite communication or bridge) are currently in use, delete the old default route; add a new default route and update the default route to the IP address of the best link device. Update the DNS settings to ensure that DNS requests are also resolved through the new link. The system's DNS settings can be updated to point to the DNS server provided by the new link. Log records, record the time of each switch, the selected link, the reason and other information for subsequent analysis and optimization. For example, select the best link and obtain the highest-scoring link device information. Assume that it is a 5G link, its IP address is 192.168.1.1, and the interface name is 5g_interface. Get the current network configuration ip route show, then update the routing table and delete the old default route: ip route del default via<old_gateway_ip> , add a new default route: ip route adddefault via 192.168.1.1. Update DNS settings sudo nano / etc / resolv.conf, #Add the DNS server nameserver 192.168.1.1 provided by the 5G device.

[0104] In some embodiments, the TCP transmission protocol is used between the server 210 and the client 310.

[0105] Among them, the control instructions of TCP transmission use encrypted transmission technology and adopt AES256-bit encryption algorithm to ensure the security of video data during transmission and prevent information leakage or malicious tampering. Specific TCP data message protocol: The Data part of TCP includes the data header and data body. The data header is fixed to 8 bytes. The 1st to 4th bytes are the instruction identifier, a predefined fixed value, such as 0x00000000 for heartbeat data, 0x00000001 for starting to play the video, and 0x00000002 for stopping to play the video. The 5th to 8th bytes are the length of the entire Data, expressed in hexadecimal and stored in little endian; the data body is a Json string encrypted by AES; it is transmitted in byte array mode.

[0106] Furthermore, TCP optimization is introduced, and the specific configuration parameters are as follows:

[0107] First, set the TCP Receive Window size (TCP Receive Window, RWIN). It is recommended that the receive window size should be set large enough to fully utilize the bandwidth of the satellite link. The initial value can be set to 64KB or larger. The specific value depends on the bandwidth of the satellite link and the application requirements.

[0108] Second, set the TCP initial congestion window size (Initial Congestion Window, CWnd). The high latency of the satellite network means that a larger initial congestion window can fill the buffer of the satellite link faster and speed up the connection establishment. The recommended initial value can be set to 10 segments or more.

[0109] Third, the TCP slow start threshold (Ssthresh) should be set relatively high to avoid congestion on the satellite link. The recommended initial value can be set to half of the receive window size or larger.

[0110] Fourth, set the TCP maximum segment size (MSS). The MSS should be set smaller to reduce the waiting time during retransmission. For satellite networks, the recommended MSS value may be around 1280 bytes, but the specific value should be adjusted according to the MTU (maximum transmission unit) of the satellite link.

[0111] Fifth, set the TCP retransmission timeout (RTO). The one-way delay of the satellite network is usually between 500ms and 1000ms, so the RTO should be set long enough to avoid unnecessary retransmissions. The initial value of RTO can be set to 3 times the round-trip time (RTT), but it should be at least greater than the round-trip time of the satellite link.

[0112] Sixth, set the TCP fast retransmission threshold. In satellite networks, since the bit error rate may be high, the fast retransmission threshold should be adjusted appropriately. Generally, setting it to 3 repeated ACKs can trigger a fast retransmission.

[0113] Seventh, use the TCP congestion control algorithm. The BBR algorithm (Bottleneck Bandwidth and Round-trip propagation time is a new congestion control algorithm proposed by Google) is recommended. It dynamically adjusts the sending rate by estimating the bottleneck bandwidth and round-trip time, reducing unnecessary congestion and improving transmission efficiency.

[0114] Eighth, using SACK and FACK. Enabling SACK and FACK in satellite networks can significantly reduce unnecessary retransmissions and improve transmission efficiency.

[0115] Figure 4 is a flow chart of a ship-to-shore video transmission method provided in an embodiment of the present application. Correspondingly, the present application also provides a ship-to-shore video transmission method, please refer to Figure 4 , the method comprises the following steps:

[0116] Step 101: The server obtains a start playback instruction from the ship-side target camera, and sends a transmission instruction of the current video stream of the ship-side target camera to the client according to the start playback instruction from the ship-side target camera.

[0117] The user selects the ship-side target camera in the video playback module to start playing and issues a corresponding play command. The shore-side Web service module receives the command and sends a transmission command of the current video stream of the ship-side target camera to the ship-side client through the shore-side service end.

[0118] Step 102: The client adaptively encodes the current video stream of the ship-side target camera according to the transmission instruction of the current video stream of the ship-side target camera to obtain the target video stream, and transmits the target video stream to the shore and the user end.

[0119] Among them, after receiving the transmission instruction of the current video stream of the ship-side target camera, the ship-side client calls the adaptive video encoding module, so that the adaptive video encoding module adaptively encodes the current video stream of the ship-side target camera to obtain the target video stream, and transmits the target video stream to the shore-side streaming media server. The streaming media server transmits the video to the video playback module, and the video playback module plays the video stream.

[0120] The technical solution of the embodiment of the present application provides a ship-to-shore video transmission method, which includes: the server obtains the start playback instruction of the ship-side target camera, and sends the transmission instruction of the current video stream of the ship-side target camera to the client according to the start playback instruction of the ship-side target camera; the client adaptively encodes the current video stream of the ship-side target camera according to the transmission instruction of the current video stream of the ship-side target camera to obtain the target video stream, and transmits the target video stream to the shore and the user. It can be seen that by setting the client, the adaptive video encoding module and the server, the video transmission between the ship and the shore can be realized. And by setting the adaptive video encoding module, the current video stream of the ship-side target camera can be adaptively encoded and adjusted, thereby reducing the delay and improving the stability and reliability of the ship-to-shore video transmission.

[0121] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0122] The ship-shore video transmission system and method provided in the embodiments of the present application are introduced in detail above, and the principles and implementation methods of the present application are explained by using specific examples. The description of the above embodiments is only used to help understand the technical solution and core idea of ​​the present application; ordinary technicians in this field should understand that: they can still modify the technical solutions recorded in the aforementioned embodiments, or replace some of the technical features therein with equivalents; and these modifications or replacements do not make the essence of the corresponding technical solution deviate from the scope of the technical solution of the embodiments of the present application.

Claims

1. A ship-to-shore video transmission system, characterized in that: include: A user end, a ship end and a shore end; the ship end includes a client and an adaptive video encoding module; the shore end includes a server end; The server is used to obtain a playback instruction of the ship-side target camera, and send a transmission instruction of the current video stream of the ship-side target camera to the client; The client is used to receive the transmission instruction of the current video stream of the ship-side target camera and call the adaptive video encoding module; The adaptive video encoding module is used to adaptively encode the current video stream of the ship-side target camera to obtain a target video stream, and transmit the target video stream to the shore end and the user end.

2. The ship-to-shore video transmission system according to claim 1, characterized in that: The adaptive video encoding module includes an adaptive video encoding unit; The adaptive video encoding unit is used to dynamically adjust the encoding parameters of the current video stream of the ship-side target camera according to the network bandwidth situation and / or the network delay situation.

3. The ship-to-shore video transmission system according to claim 2, characterized in that: The encoding parameters include at least resolution, bit rate and frame rate.

4. The ship-to-shore video transmission system according to claim 3, characterized in that: The adaptive video encoding unit is also used to dynamically adjust the resolution and bit rate of the current video stream of the ship-side target camera according to the network bandwidth situation, and / or dynamically adjust the frame rate of the current video stream of the ship-side target camera according to the network delay situation.

5. The ship-to-shore video transmission system according to claim 3, characterized in that: The adaptive video encoding unit is also used to increase the resolution and the bit rate when the network bandwidth condition meets the preset network bandwidth condition, and reduce the resolution and the bit rate when the network bandwidth condition does not meet the preset network bandwidth condition; and / or, when the network delay condition meets the preset network delay condition, increase the frame rate, and when the network delay condition does not meet the preset network delay condition, reduce the frame rate.

6. The ship-to-shore video transmission system according to claim 2, characterized in that: The adaptive video encoding module also includes a video capture unit, a network transmission unit and a control response unit; Wherein, the video capture unit is used to capture the video stream from the ship-end target camera; The network transmission unit is used to send the target video stream to the shore end through the network; The control response unit is used to adjust the encoding parameters of the current video stream of the ship-side target camera according to the control instruction sent by the shore.

7. The ship-to-shore video transmission system according to claim 1, characterized in that: The ship side also includes a multi-mode communication fusion and intelligent routing module; the multi-mode communication fusion and intelligent routing module is used to combine multiple communication modes, monitor the status information of the communication links of various communication modes in real time, and intelligently route to the best communication link.

8. The ship-to-shore video transmission system according to claim 7, characterized in that: The multi-mode communication fusion and intelligent routing module includes a signal monitoring unit, a plurality of communication interfaces and an intelligent routing unit; Wherein, the multiple communication interfaces are used to provide multiple communication modes; The signal monitoring unit is used to monitor the status information of the communication links of various communication modes in real time; The intelligent routing unit is used for intelligently routing to the best communication link according to the status information of the communication links of various communication modes and a preset evaluation mechanism.

9. The ship-to-shore video transmission system according to claim 8, characterized in that: The preset evaluation mechanism includes: Define evaluation metrics and weights for each communication link; Calculate the comprehensive score of each communication link according to the evaluation index and weight of each communication link; According to the comprehensive score of each communication link, the communication link with the highest score is selected as the current best link and the actual network traffic is switched to the selected best communication link.

10. The ship-to-shore video transmission system according to claim 1, characterized in that: The shore end also includes a streaming media server and a video playback service module; wherein the streaming media server is used to receive the target video stream and transmit it to the video playback service module.

11. The ship-to-shore video transmission system according to claim 10, characterized in that: The streaming media server is provided with multi-encoding support and cross-protocol conversion, an asynchronous event-driven multi-threading model and a zero-copy technology.

12. The ship-to-shore video transmission system according to claim 10, characterized in that: The video playback service module is provided with multi-screen display, playback control options and a user interface.

13. The ship-to-shore video transmission system according to claim 1, characterized in that: The TCP transmission protocol is used between the server and the client.

14. A ship-to-shore video transmission method, characterized in that: include: The server obtains a start playback instruction from the ship-side target camera, and sends a transmission instruction of the current video stream of the ship-side target camera to the client according to the start playback instruction from the ship-side target camera; The client adaptively encodes the current video stream of the ship-side target camera according to the transmission instruction of the current video stream of the ship-side target camera to obtain the target video stream, and transmits the target video stream to the shore and the user end.

Images