Video monitoring adaptation interface and adaptation method thereof

Abstract

The embodiment of the specification discloses a video monitoring adaptation interface and an adaptation method thereof, wherein the video monitoring adaptation interface adopts a layered architecture design of a hardware adaptation layer, a core service layer and a unified interface layer. The hardware adaptation layer respectively performs adaptation conversion according to CPU architecture and OS type of a video monitoring system and a communication protocol of a video acquisition device, and realizes unified adaptation to different hardware platforms, operating systems and video devices; the core service layer provides reliable video service processing capability; and the unified interface layer converts internal services into standardized interfaces. The application realizes 'one development, multiple deployment', reduces adaptation cost between various video monitoring systems and external systems, and improves compatibility and maintainability of a ship video monitoring system.

Description

Technical Field

[0001] Several embodiments of this specification relate to the fields of computer and video processing technology, and more specifically to an optimization of a video surveillance adapter interface. Background Technology

[0002] As the importance of maritime rights becomes increasingly prominent, shipborne video surveillance systems, serving as the "eyes" of ships to perceive the external environment and grasp the internal situation, play an irreplaceable and crucial role in navigation safety, situational awareness, target identification, combat command, and daily training management. Through various camera devices, including optoelectronic, infrared, and high-definition cameras, deployed in key parts of the ship, the system provides real-time and intuitive visual information to commanders and operators.

[0003] Currently, traditional shipboard video surveillance systems are mostly built on mature commercial hardware and software platforms. At the hardware level, they generally employ Intel x86 architecture servers, network cameras (IPCs) from specific brands, and network video recorders (NVRs). At the software level, they rely on Windows or standard Linux operating systems, and video management software developed on top of them. The interface implementation of these systems is usually quite straightforward, either using proprietary software development kits (SDKs) for specific hardware devices or based on common industry protocols such as ONVIF for device access. Integration methods often revolve around a single shipboard platform or a specific model of equipment, aiming to meet the basic monitoring needs of that specific scenario, forming a relatively closed or semi-closed system architecture.

[0004] However, the commonly used implementation methods mentioned above are insufficient to meet the modern navy's requirements for security, versatility, and reliability. Furthermore, their closed architecture makes integration with other shipboard systems difficult and results in weak information fusion capabilities. First, their core hardware and software rely on foreign technology, posing risks of supply chain disruptions and information security, which contradicts the strategic requirement of promoting comprehensive domestic production in key areas, and lacks an effective compatibility layer. Second, existing solutions have poor adaptability, failing to flexibly adapt to the significant differences in scale, network, and layout among various types of ships, resulting in extensive customized development for each ship type, leading to high costs and lengthy development cycles. Additionally, the systems typically only possess basic security capabilities and lack standardized support for ship-specific business operations. Summary of the Invention

[0005] This specification provides a video surveillance adapter interface and its adaptation method. It adopts a layered architecture design of hardware adaptation layer, core service layer and unified interface layer to achieve efficient and reliable adaptation between the video surveillance system and external systems. Moreover, the video surveillance adapter interface can be "developed once and deployed in multiple types", which reduces the adaptation cost between various video surveillance systems and external systems and improves the compatibility and maintainability of ship video surveillance systems.

[0006] The technical solution is as follows:

[0007] Firstly, this specification provides a video surveillance adapter interface configured as a middleware interface connecting a video surveillance system including multiple video acquisition devices and an external system that obtains video services; the video surveillance adapter interface adopts a layered architecture, including a hardware adapter layer, a core service layer and a unified interface layer connected sequentially from bottom to top.

[0008] The hardware adaptation layer is configured to adapt and convert according to the CPU architecture and OS type of the video surveillance system and the communication protocol of the video acquisition device, so as to obtain a unified computing interface, a unified system service interface and a unified device operation interface.

[0009] The core service layer includes a device management service module and a media forwarding service module;

[0010] The device management service module is configured to manage all connected video acquisition devices by calling a unified device operation interface and to update the access status of the video acquisition devices.

[0011] The media forwarding service module is connected to the device management service module and is configured to obtain video streams by calling the unified system service interface based on video service requests and the access status of video acquisition devices, and to distribute video streams after transcoding the video streams by calling the unified computing interface.

[0012] The unified interface layer is configured to receive video service requests and output the corresponding video streams through a standardized access interface.

[0013] As a preferred embodiment, the hardware adaptation layer includes a CPU adaptation module, an OS adaptation module, and a video acquisition device adaptation module:

[0014] The CPU adaptation module includes pre-set instruction set functions and memory allocation functions corresponding to different CPU architectures. It is configured to detect the CPU architecture of the video surveillance system and load the corresponding instruction set functions and memory allocation functions to obtain a unified computing interface.

[0015] The OS adaptation module includes pre-configured underlying service call methods and network I / O models for different OS types. It is configured to detect the OS type of the video surveillance system and load the corresponding underlying service call methods and network I / O models to obtain a unified system service interface.

[0016] The video acquisition device adaptation module includes protocol conversion plugins for different video acquisition devices. It is configured to detect the communication protocols of all connected video acquisition devices and load the corresponding protocol conversion plugins to obtain a unified device operation interface.

[0017] As a preferred embodiment, the media forwarding service module includes a load balancing unit;

[0018] The load balancing unit is configured to allocate transcoding processing tasks for video streams corresponding to video service requests based on the load conditions of each of the multiple service processes.

[0019] As a preferred embodiment, the media forwarding service module includes a dynamic output unit;

[0020] The dynamic output unit is configured to dynamically adjust the output bitrate of the transcoding processing task based on network conditions.

[0021] As a preferred embodiment, the media forwarding service module includes a disconnection reconnection unit;

[0022] The disconnection reconnection unit is configured to automatically re-establish the connection according to the exponential backoff strategy when the network connection with any video acquisition device is interrupted, until the connection is restored.

[0023] As a preferred embodiment, the video service request also includes request identity information and requested device information; the core service layer also includes a permission management module;

[0024] The permission management module is connected to the device management service module and is configured as follows:

[0025] Role-based access control and attribute-based access control policies are adopted to obtain permissions and permission priority weights for video service requests.

[0026] For the same requested device, the control of the requested device is dynamically determined based on the permission priority weight corresponding to each video service request.

[0027] As a preferred embodiment, the core service layer also includes a high availability and message middleware module;

[0028] The high availability and message middleware module is connected to the device management service module, media forwarding service module and permission management module, and is configured to synchronize status and transmit events among other modules in the core service layer.

[0029] The device management service module is deployed in a primary / backup mode, using a heartbeat mechanism and distributed locks to switch between primary and backup nodes.

[0030] As a preferred embodiment, the unified system service interface provided by the OS adaptation module specifically includes:

[0031] The thread priority setting interface is used to set the scheduling priority of video processing threads on different operating systems;

[0032] A high-precision clock acquisition interface is used to obtain monotonically increasing microsecond-level timestamps on different operating systems.

[0033] Secondly, this specification provides an adaptation method based on the content described in the first aspect of the above embodiments. The adaptation method includes the following steps:

[0034] Obtain the CPU architecture and OS type of the video surveillance system;

[0035] Load the instruction set functions and memory allocation functions corresponding to the CPU architecture;

[0036] Load the underlying service invocation method and network I / O model corresponding to the OS type;

[0037] Adaptive software is built based on the loaded instruction set functions, memory allocation functions, underlying service call methods, and network I / O models;

[0038] The video surveillance system runs adaptation software, and the adaptation software performs adaptive optimization during runtime.

[0039] Thirdly, embodiments of this specification provide an electronic device, including a processor and a memory; the processor is connected to the memory; the memory is used to store executable program code; the processor reads the executable program code stored in the memory to run a program corresponding to the executable program code, so as to perform the steps described in the second aspect of the above embodiments.

[0040] Fourthly, embodiments of this specification provide a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the steps described in the second aspect of the above embodiments.

[0041] The beneficial effects of the technical solutions provided in some embodiments of this specification include at least the following:

[0042] It solves the problem of compatibility with domestic platforms, can meet the full-stack domestic configuration from hardware to software, eliminates external supply chain and security risks, and meets the security requirements of practical applications.

[0043] Through multiple redundancy and fault tolerance mechanisms, such as hot switching between primary and backup servers, automatic service restart after service process anomalies, and automatic reconnection after network connection interruptions, the system can continue to operate stably under harsh ship environments and network fluctuations, greatly reducing the risk of system downtime and maintenance costs.

[0044] It pioneered the implementation of standardized and configurable command deprivation and priority management functions at the video surveillance middleware level, meeting users' special business needs for collaborative command and efficient decision-making, and improving the combat effectiveness of ships.

[0045] The standardized RESTful APIs and SDKs provided by the unified interface layer enable other systems on board (such as the C2 system and electronic charts) to quickly and cost-effectively access video, breaking down information silos.

[0046] The microservice architecture in the core service layer highly decouples system functions, allowing for independent development, deployment, and upgrades. When adding new features or device types, only the corresponding modules need to be extended or added, without altering the overall architecture, thus reducing the complexity of evolution. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is an architecture diagram of a video surveillance adapter interface provided in the embodiments of this specification.

[0049] Figure 2 This is a state diagram of the automatic reconnection mechanism of the disconnection reconnection unit provided in the embodiments of this specification.

[0050] Figure 3 This is a flowchart of the control deprivation function of the permission management module provided in the embodiments of this specification.

[0051] Figure 4 This is a timing diagram of the primary / backup service switching of the device management service module provided in the embodiments of this specification.

[0052] Figure 5 This is a flowchart illustrating an adaptation method provided in an embodiment of this specification.

[0053] Figure 6 This is a schematic diagram of the structure of an electronic device provided in the embodiments of this specification. Detailed Implementation

[0054] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings.

[0055] The terms "first," "second," "third," etc., in the description, claims, and accompanying drawings are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0056] The following description provides examples and does not limit the scope, applicability, or examples set forth in the claims. Changes may be made to the function and arrangement of the described elements without departing from the scope of this specification. Various processes or components may be appropriately omitted, substituted, or added to the examples. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with respect to some examples may be combined into other examples.

[0057] Terminology Explanation:

[0058] Interface adaptability refers to the ability of one system, component, or module to seamlessly and correctly collaborate with another system, component, or module through an interface. Good interface adaptability allows different parts of a system to be developed, upgraded, and replaced independently. When a module needs to be modified, as long as the external interface remains unchanged, it will not affect other modules that depend on it, improving maintainability and module reusability.

[0059] Video surveillance adapter interface 1 refers to a standardized set of software specifications, protocols, and application programming interfaces (APIs) that allow different video surveillance systems 2, devices, and third-party application software to exchange data, control commands, and call functions, thereby achieving system interconnection and centralized management. It typically includes basic functions such as device information management, device control management, video stream acquisition, recording and playback, and alarm reception and processing.

[0060] Traditional shipboard video surveillance systems are mostly built on commercial hardware and software platforms (such as Intel x86 architecture servers, Windows / Linux operating systems, and specific brand IPCs / NVRs). However, in critical areas, using foreign processors (such as Intel and AMD) and operating systems (such as Windows and Red Hat) poses potential security risks. To achieve complete domestic production of hardware and software, it is necessary to use domestic CPUs such as Shenwei, Phytium, and Loongson, and domestic operating systems such as Kylin and UnionTech UOS as the basic platform for development. However, domestic platforms have serious compatibility issues with existing video applications, drivers, and middleware developed based on foreign ecosystems, lacking an adaptation layer that can effectively shield against differences in underlying hardware and operating systems.

[0061] Moreover, surface vessels encompass a variety of types, including various ships, cargo vessels, passenger vessels, and engineering vessels, with significant differences in platform size, cabin layout, and network architecture. This diversity poses a severe challenge to the adaptability of the video surveillance system 2, requiring it to interface with the server configurations, network bandwidth, storage capacity, and other infrastructure of various ships. This necessitates extensive customization for each type of ship, resulting in long deployment cycles, high costs, and difficulty in achieving standardized, large-scale application.

[0062] Furthermore, existing shipboard video systems are mostly closed or semi-closed systems with inconsistent and non-standard interfaces. When information fusion and linkage with other systems are required, the integration is difficult, time-consuming, and costly, failing to meet the needs of efficient information flow.

[0063] Therefore, the present invention provides a video surveillance adapter interface 1 that can be adapted to domestic software and hardware platforms, adapted to multiple types of ships, and called by third-party system standards.

[0064] Reference Figure 1 As shown, Figure 1 This is an architecture diagram of a video surveillance adapter interface provided in the embodiments of this specification.

[0065] The video surveillance adapter interface 1 is configured as a middleware interface connecting the video surveillance system 2, which includes multiple video acquisition devices 21, and the external system 3 that obtains video services. The video surveillance adapter interface 1 adopts a layered architecture, including a hardware adapter layer 11, a core service layer 12, and a unified interface layer 13 connected from bottom to top.

[0066] The hardware adaptation layer 11 is configured to perform adaptation and conversion according to the CPU architecture and OS type of the video surveillance system 2 and the communication protocol of the video acquisition device 21, so as to obtain a unified computing interface, a unified system service interface and a unified device operation interface.

[0067] The core service layer 12 includes a device management service module 121 and a media forwarding service module 122;

[0068] The device management service module 121 is configured to manage all connected video acquisition devices 21 by calling the unified device operation interface and update the access status of the video acquisition devices 21;

[0069] The media forwarding service module 122 is connected to the device management service module 121 and is configured to obtain video streams by calling the unified system service interface based on video service requests and the access status of video acquisition device 21, and to distribute video streams after transcoding the video streams by calling the unified computing interface.

[0070] The unified interface layer 13 is configured to receive video service requests and output the corresponding video streams of the video service requests through a standardized access interface.

[0071] Illustratively, this embodiment of the invention provides a video surveillance adaptation interface 1. This interface, as software middleware, achieves the shielding of underlying hardware differences, standardization of core business services, and unification of external interfaces through a layered architecture design. Its basic workflow includes: a hardware adaptation layer 11 adapting to and abstracting the underlying heterogeneous platform; a core service layer 12 providing core services such as device management and media stream forwarding based on the abstracted unified interface; and a unified interface layer 13 receiving video service requests from an external system 3 and returning processing results.

[0072] Explained, the hardware adaptation layer 11 shields the differences in underlying hardware (CPU, OS, device drivers), allowing developers of the core service layer 12 to avoid writing code specific to any particular chip or operating system. The hardware adaptation layer 11 provides the upper-layer core service layer 12 with three key unified interfaces: a unified computing interface for video encoding and decoding tasks, a unified system service interface for network and process management system calls, and a unified device operation interface for device control and data acquisition. The core service layer 12's functionality is built upon the stable hardware foundation provided by the three unified interfaces of the hardware adaptation layer 11, and includes the device management service module 121 and the media forwarding service module 122. The device management service module 121 manages all connected video acquisition devices 21, maintains the device list and status information (online, offline, faulty), and handles requests for adding, deleting, modifying, and querying devices. The media forwarding service module 122 is responsible for receiving, transcoding, and distributing video streams. After receiving video service requests from the unified interface layer 13, it queries the device management service module 121 for the stream address of the target device, then establishes a network connection through the unified system service interface to obtain the video stream, and calls the unified computing interface to perform transcoding, scaling, and other processing on the video stream, ultimately generating a standardized video stream. The two modules work collaboratively, decoupled through an event-driven architecture. When the device state changes, the device management module publishes state change events, and the media forwarding module subscribes to these events and adjusts the video stream session state accordingly, achieving effective management and distribution of video resources. The unified interface layer 13 exposes standard, easily integrated APIs (Application Programming Interfaces), serving as the service access point for the entire system. It is responsible for transforming internal service capabilities into standardized external interfaces, connecting the video surveillance system 2 itself and the connected external system 3 through standardized access interfaces, receiving video service requests, and sending service functions provided by the core service layer 12. For example, external system 3 requests to watch the video from a camera through the standard API provided by the unified interface layer 13. This request is passed to the core service layer 12. The media forwarding service module 122 receives the instruction and starts to pull the stream from the target device (video acquisition device 21), transcode it, and return the transcoded video stream to external system 3.

[0073] Additionally, video acquisition device 21 includes a camera and a control pan-tilt unit. Video service requests include information such as the requested target device, video acquisition request, and pan-tilt control request. The standardized access interfaces of the unified interface layer 13 mainly include two forms:

[0074] 1. RESTful API, adhering to the REST design style, using HTTP / HTTPS protocol, and employing JSON data format. Features include:

[0075] GET / api / v1 / devices - Get a list of devices;

[0076] POST / api / v1 / devices / {id} / ptz - Control the pan / tilt unit (direction, zoom);

[0077] GET / api / v1 / live / {device_id} - Get the URL of the live video stream;

[0078] POST / api / v1 / command / override - Initiates a command deprivation request;

[0079] WS / api / v1 / events - WebSocket connection used to receive real-time events (such as deprivation notifications, device status changes).

[0080] Advantages include: stateless, easy to understand, highly versatile, and suitable for integration between most systems.

[0081] 2. The native SDK provides client libraries in languages ​​such as C++, Java, or Go. It encapsulates complex network communication, protocol serialization / deserialization, connection pool management, and automatic reconnection logic. The SDK has a built-in heartbeat keep-alive mechanism with the server. When a network interruption is detected, the SDK automatically attempts to reconnect in the background and rebuilds the session and resubscribes to events after the connection is restored, all transparent to developers, greatly improving the robustness of upper-layer applications. Advantages include: easier integration, higher performance, and more powerful functionality, suitable for deep integration scenarios with high performance and demanding requirements.

[0082] In this embodiment, the video surveillance adapter interface 1 acts as middleware, effectively shielding the heterogeneity of the underlying hardware and software and providing unified service access capabilities to the upper layer. This achieves efficient and reliable adaptation between the video surveillance system 2 and the external system 3.

[0083] In one embodiment of this specification, the hardware adaptation layer 11 includes a CPU adaptation module 111, an OS adaptation module 112, and a video acquisition device 21 adaptation module 113.

[0084] CPU adaptation module 111 includes pre-set instruction set functions and memory allocation functions corresponding to different CPU architectures. It is configured to detect the CPU architecture of video surveillance system 2 and load the corresponding instruction set functions and memory allocation functions to obtain a unified computing interface.

[0085] OS adaptation module 112 includes pre-configured underlying service call methods and network I / O models for different OS types. It is configured to detect the OS type of video surveillance system 2 and load the corresponding underlying service call methods and network I / O models to obtain a unified system service interface.

[0086] The video acquisition device 21 adapter module 113 includes protocol conversion plugins corresponding to different detection video acquisition devices 21. It is configured to detect the communication protocols of all connected video acquisition devices 21 and load the corresponding protocol conversion plugins to obtain a unified device operation interface.

[0087] To illustrate, in order for core video processing algorithms (such as decoding and analysis) to run on Intel / AMD (x86) as well as Shenwei (SW64) and Phytium (ARM) platforms, and to run them quickly, customization is required for different CPU architectures. The CPU adaptation module 111 provides a unified computational abstraction layer (CAL), encapsulating complex mathematical operations (instruction set functions), such as DCT transform and motion estimation in video encoding and decoding, and memory operations (memory allocation functions), into a unified interface. Optimized code is written for x86 / ARM / SW64 respectively, and the program automatically selects the appropriate architecture during runtime. During initialization, the CPU architecture of the current platform (such as x86, domestic Phytium ARM64, and Shenwei SW64) is automatically detected. This can be achieved by reading the CPUID instruction or system information such as / proc / cpuinfo to identify the CPU manufacturer and characteristic flags. Based on the detection results, pre-built instruction set function libraries and memory allocation functions optimized for that architecture are dynamically loaded. For example, for the Phytium ARM architecture, video encoding and decoding functions accelerated using the NEON instruction set are loaded; for the Shenwei SW64 architecture, an optimized version using its unique vector instruction set is loaded. Memory allocation functions take into account the memory architecture characteristics of different CPUs, such as supporting NUMA-aware large page memory allocation on the x86 platform, while using non-cached memory on the ARM platform to optimize the access efficiency of video frame data. Through this process, a unified computing interface independent of specific CPU architecture is obtained. Upper-layer services can call this interface to execute high-performance computing tasks without needing to worry about the differences in the underlying instruction sets, ensuring efficient and stable operation on different domestic CPUs.

[0088] To ensure stable operation of software services on Windows / Linux and seamless integration on Kylin and UnionTech UOS, it's necessary to encapsulate the underlying operating system functions (such as process creation, memory management, and network communication) into a unified interface. OS Adaptation Module 112 hides system call differences and proactively integrates the security and service management framework of domestic OSes. This module encapsulates system call differences, providing a unified System Service Interface (SSI), including native system calls (underlying service call methods) for process / thread management, memory allocation, file I / O, and network sockets, defining standardized call methods and behavioral semantics. It also encapsulates high-performance network I / O models specific to different OSes (such as epoll, IOCP, and kqueue) into a unified interface, defining a standardized asynchronous / non-blocking event handling model. For example, for process creation, on Windows, the module internally calls Win32 APIs such as CreateProcessAsUser; on Kylin, it uses system calls such as fork and execve, and integrates Kylin-specific security context setting functionality. In terms of network communication, this module will adapt to the high-performance I / O models of different operating systems, such as using overlapped I / O on Windows and using epoll or sendmmsg on Linux series operating systems.

[0089] The video acquisition device 21 is compatible with the module 113, which interfaces with various domestically produced or non-standard high-definition, infrared, thermal imaging, and laser rangefinder cameras on ships. This module defines a unified device driver model (UDDM). All device drivers are developed according to this model, providing a unified interface for device discovery, initialization, control, and streaming. It supports multiple video stream acquisition protocols, including ONVIF, GB / T28181 (national standard), RTSP, and SDK proprietary protocols. For each video device communication protocol (such as ONVIF, GB / T28181, and vendor proprietary SDKs), an independent protocol conversion plugin (compiled as a .so or .dll dynamic library) is developed. All available plugins are dynamically loaded when the module starts. When a new device is connected, each plugin attempts to communicate with the device. Plugins that are successfully recognized are used to manage the device, converting the device-specific protocol into internally unified operation commands. This results in a unified device operation interface, allowing upper-layer services to consistently call methods such as initializing the device, obtaining the video stream address, and controlling the pan / tilt unit without needing to be aware of the device's protocol details.

[0090] An interpretable, intelligent build script for hardware adaptation layer 11 can be written, including a unified computing interface, a unified system service interface, and a unified device operation interface. This script can automatically detect the current operating system and CPU. If it detects a domestic Phytium + Kylin CPU, it automatically links the Kylin-specific security library and compiles the ARM-optimized code. If it detects Intel + Windows, it links Windows libraries such as DirectX and compiles the x86-optimized code. Different software images are pre-built for different platforms (x86, ARM, SW64). When deployed on domestic servers, the image specifically designed for the domestic platform is directly pulled and run. This greatly shields environmental differences and achieves "build once, run anywhere".

[0091] In one embodiment of this specification, the media forwarding service module 122 includes a load balancing unit;

[0092] The load balancing unit is configured to distribute the transcoding processing tasks of the video stream corresponding to the video service request based on the load of each of the multiple service processes.

[0093] For illustrative purposes, the forwarding service module also features load balancing technology. When multiple media service instances coexist, the load balancing unit intelligently distributes new video stream forwarding requests to the least loaded instance based on real-time load metrics such as CPU, memory, network bandwidth utilization, and the number of video streams currently being processed, thus avoiding single-point overload.

[0094] In one embodiment of this specification, the media forwarding service module 122 includes a dynamic output unit;

[0095] The dynamic output unit is configured to dynamically adjust the output bitrate of the transcoding processing task based on network conditions.

[0096] For illustrative purposes, the forwarding service module also features adaptive transcoding bitrate technology. The dynamic output unit monitors the client's network status in real time. When a decrease in network bandwidth is detected, the unit dynamically reduces the bitrate and resolution of the output video stream (e.g., switching from 1080p to 720p) to ensure smooth video streaming; once the network recovers, the quality is automatically improved. This achieves adaptive bitrate transmission.

[0097] In one embodiment of this specification, the media forwarding service module 122 includes a disconnection reconnection unit;

[0098] The disconnection reconnection unit is configured to automatically re-establish the connection according to the exponential backoff strategy when the network connection with any video capture device 21 is interrupted, until the connection is restored.

[0099] For illustrative purposes, the forwarding service module also has a reconnection function after disconnection. (See attached reference.) Figure 2 , Figure 2 This is a state diagram of the automatic reconnection mechanism of the disconnection reconnection unit provided in the embodiments of this specification. Once network fluctuations cause the video stream to be interrupted, the service will attempt to re-establish the connection according to an exponential backoff strategy (such as intervals of 1s, 2s, 4s, 8s...) until successful, which greatly enhances the robustness of the service.

[0100] In one embodiment of this specification, the video service request further includes request identity information and requested device information; the core service layer 12 also includes an access control module 123;

[0101] The access control module 123 is connected to the device management service module 121 and is configured as follows:

[0102] Role-based access control and attribute-based access control policies are adopted to obtain permissions and permission priority weights for video service requests.

[0103] For the same requested device, the control of the requested device is dynamically determined based on the permission priority weight corresponding to each video service request.

[0104] For illustrative purposes, due to the unique usage scenario of ships, the business processes and logical rules of the video surveillance system 2 need to conform to the specific operating methods and usage habits of users, and existing technologies lack mature and standardized implementation solutions. In this embodiment, the access control module 123 can implement video command deprivation functionality. In the access control service, role-based access control (RBAC) and attribute-based access control (ABAC) are two main access control methods. Requested identity information includes ID, role, assigned combat position, etc., and the requested device information includes request time, target device, and requested operation type, etc.

[0105] For example, RBAC defines access roles as "Level 1", "Level 2", "Level 3", and "Level 4", and assigns operation permissions (such as preview, PTZ control, video playback, and device configuration) to the access roles. ABAC defines dynamic policy rules, considering environmental attributes (such as "task time" and "training time"), resource attributes (such as "critical camera parts"), and operation attributes (such as "PTZ control").

[0106] Explanatoryly, the access control module 123 internally maintains an access priority weight calculation algorithm. The weight is calculated by combining static role weights (e.g., Level 1 = 100, Level 2 = 90) and dynamic context weights (e.g., bonus weight for completing a task). (See attached...) Figure 3 , Figure 3This is a flowchart of the control deprivation function of the permission management module 123 provided in the embodiments of this specification. For the same requested device, if there are multiple concurrent operation requests, the module will compare the weights of the requesters in real time, approve the request of the one with higher weight, and deprive the current low-weight operator of control.

[0107] It's easy to understand that for certain queueable operations, low-priority requests can enter a waiting queue and be automatically executed after high-priority operations are completed.

[0108] In one embodiment of this specification, the core service layer 12 further includes a high availability and message middleware module 124;

[0109] The high availability and message middleware module 124 is connected to the device management service module 121, the media forwarding service module 122 and the access management module 123, and is configured to: synchronize status and transmit events among other modules in the core service layer 12.

[0110] The device management service module 121 is deployed in a primary / standby mode, using a heartbeat mechanism and distributed locks to switch between primary and standby nodes.

[0111] Illustratively, Redis (as a distributed cache and Pub / Sub messaging system) and / or RabbitMQ (as a reliable message queue) are used to provide state synchronization, messaging, and distributed locking for all microservice modules. For example, when the permission management module 123 revokes control from the current low-weight operator, a real-time notification can be sent to the revoked party simultaneously via the message queue. This also forms the basis for master-slave failover and cluster coordination. The master-slave device management service declares its liveness by sending heartbeat packets to a specific Redis channel. Preemption commands, state changes, and other events are broadcast via the message queue to ensure eventual consistency of the state of all service instances.

[0112] Explanatoryly, high availability guarantees at core service layer 12 are crucial for ensuring stable 24 / 7 system operation. (Combined with appendix...) Figure 4 , Figure 4This is a sequence diagram of the primary / backup service switchover for the device management service module 121 provided in the embodiments of this specification. The primary node is the active node currently providing services externally, and the backup node is a redundant node in hot standby mode. The Virtual IP (VIP) is the unified service address externally. The health status of the nodes is monitored by a heartbeat detector, and the primary and backup nodes monitor each other. The backup node continuously monitors the health status of the primary node. Once the primary node fails (heartbeat timeout), the backup node immediately takes over its IP and tasks and is promoted to the primary node. This process is a millisecond-level hot switchover, transparent to upper-layer applications. The service itself has an automatic restart mechanism. If the process crashes, it is immediately restarted by the system monitoring process (such as systemd) or the container orchestration platform (such as Kubernetes). During the process, the distributed lock manager coordinates the competition between nodes and maintains data consistency through shared storage / state synchronization channels. The device management service module 121 implements millisecond-level hot switchover failover, ensuring service continuity.

[0113] In one embodiment of this specification, the unified system service interface provided by the OS adaptation module 112 specifically includes:

[0114] The thread priority setting interface is used to set the scheduling priority of video processing threads on different operating systems;

[0115] A high-precision clock acquisition interface is used to obtain monotonically increasing microsecond-level timestamps on different operating systems.

[0116] To illustrate, the unified system service interface provided by the OS adaptation module 112 is essentially a system-level abstraction layer. Its main function is to eliminate the differences in underlying system service calls between different operating systems. Specifically, it includes a thread priority setting interface and a high-precision clock acquisition interface, ensuring real-time scheduling of video processing threads and accurate time synchronization across platforms.

[0117] Explanatoryly, it encapsulates Windows' CreateProcess, Linux's fork / exec, and the process creation methods specific to domestic operating systems, providing a unified create_video_process() interface to the upper layer to achieve unified process management; it unifies Windows' thread priority API, Linux's pthread scheduling strategy, and the real-time scheduling requirements of domestic operating systems into the set_video_thread_priority() standard interface to achieve thread scheduling normalization; it abstracts different high-precision clock sources such as Windows' QueryPerformanceCounter and Linux's clock_gettime to provide cross-platform microsecond-level time acquisition services, achieving standardized time management.

[0118] The video surveillance adapter interface 1 encapsulates the multi-platform complexity of domestically produced platforms at the lower level, and provides stable, unified, and standardized services that support various businesses at the upper level. It also incorporates a high-reliability mechanism, thereby providing core support for building a new generation of domestically produced shipborne video surveillance system.

[0119] It should be noted that although this application uses a shipborne video surveillance system 2 as a typical application scenario, its core design concepts and key technologies have universal applicability. The core issues addressed by this solution, such as hardware heterogeneity adaptation, operating system compatibility, high service availability, and standardized unified interfaces, are prevalent in video surveillance and visual perception systems under various complex environments. Therefore, this solution is also applicable to, but not limited to, smart cities and public safety, transportation hubs and industrial monitoring, energy and power infrastructure, and other fields that require the integration of heterogeneous video resources and the realization of intelligent business collaboration.

[0120] The following example, using a typical dual-machine hot standby deployment scenario, details how to construct and implement the video surveillance adapter interface of this invention.

[0121] Implementation environment preparation (hardware): Two servers based on Phytium FT-2000+ / 64 processors (Server A, Server B), Gigabit Ethernet switch, disk array.

[0122] Implementation environment preparation (software): Galaxy Kylin Advanced Server Operating System (V10), Docker Container Runtime Engine, Kubernetes (K8s) Lightweight Cluster (optional).

[0123] Step 1: Deployment of Basic Environment and High Availability Architecture

[0124] Network Configuration: Configure a virtual IP (VIP) for both servers, for example, 192.168.1.100. This VIP represents the entry point for the video interface service. Initially, the VIP is held by server A (the primary server).

[0125] Deploy message middleware: Deploy a Redis-Sentinel (sentinel mode) or RedisCluster cluster on two servers. Redis is used to store storage status, heartbeat information, and distributed locks.

[0126] Deploy a load balancer: Deploy Nginx or HAProxy as a reverse proxy and load balancer, with its upstream pointer pointing to the backend service instance. If using Kubernetes, utilize its built-in Service and Ingress mechanisms.

[0127] Step 2: Implementation of Hardware Adaptation Layer 11

[0128] CPU / OS adaptation library compilation:

[0129] In Phytium and Kylin OS environments, cross-compile FFmpeg (including H.264 / H.265 encoders) from source code.

[0130] The compilation command example is optimized for the ARM architecture:

[0131] . / configure --arch=arm64 --target-os=linux --enable-neon --enable-nonfree --enable-libx264 ...

[0132] The compiled library files are packaged into the base layer of the Docker image.

[0133] Device driver plugin development:

[0134] A driver plugin was developed for a certain type of shipborne camera. This plugin implements a unified DeviceDriver interface.

[0135] init(): Reads the device IP, username, and password from the configuration file.

[0136] getStreamUrl(): Logs into the device via the vendor's SDK and obtains the RTSP stream address.

[0137] rtsp: / / [ip] / cam / realmonitor?channel=1&subtype=0

[0138] controlPtz(): Converts standard left / right / up / down commands into PTZ control protocol strings supported by the device and sends them via the SDK.

[0139] Package the driver plugin into a standalone .so file or microservice.

[0140] Step 3: Development and Deployment of Core Service Layer 12

[0141] All core services are written in Go and packaged as Docker images for easy deployment and isolation.

[0142] Equipment Management Services

[0143] / / Start the health check in the main function

[0144] func main() {

[0145] / / ... Initialize service...

[0146] go startHeartbeat() / / Send heartbeat to Redis

[0147] go monitorMaster() / / Standby node listens for the master node's heartbeat

[0148] / / ... Start the HTTP server...

[0149] }

[0150] / / Send heartbeat

[0151] func startHeartbeat() {

[0152] / / Start the timer and send a heartbeat; throw an exception.

[0153] }

[0154] / / Listen to the master node

[0155] func monitorMaster() {

[0156] / / Currently a backup node

[0157] / / If the master node's heartbeat is lost, begin preempting the master node.

[0158] / / Ensure only one standby machine is upgraded using Redis distributed locks.

[0159] / / Execute the upgrade-oriented logic: take over VIP, load the device list, etc.

[0160] }

[0161] Implementation of permission services and deprivation logic

[0162] / / Handle PTZ control requests

[0163] func HandlePtzRequest() {

[0164] / / 1. Authenticate and obtain user context

[0165] / / 2. Calculate the priority weight of the current request.

[0166] / / 3. Query the current operator of the device and its weight.

[0167] / / 4. Deprivation of decision-making

[0168] / / Approve new request

[0169] / / Update current operator information

[0170] / / If a previous operator exists, send a deprivation notification.

[0171] / / Output: Your gimbal operation has been deprived by XXX"

[0172] / / Sent to the client of currentOperator via WebSocket or message queue

[0173] / / If a user with low privileges attempts to perform the operation, an insufficient privileges message will be displayed, indicating that a higher-priority user is currently available.

[0174] }

[0175] Step 4: Encapsulation of Unified Interface Layer 13

[0176] RESTful API design:

[0177] Use the Swagger / OpenAPI 3.0 specification to write detailed API documentation, defining all endpoints, request / response models, and error codes.

[0178] Use Go Swagger or similar tools to automatically generate API documentation web pages and server code frameworks.

[0179] SDK Development (Java SDK Example)

[0180] public class VideoSDK {

[0181] / / Initialize the SDK and establish a connection

[0182] / / 1. Authentication to obtain a token

[0183] / / 2. Connect to the WebSocket event server

[0184] connectToEventServer();

[0185] / / 3. Start the background heartbeat thread

[0186] startHeartbeatThread();

[0187] }

[0188] private void connectToEventServer() {

[0189] try {

[0190] / / Process received messages, such as deprivation notifications.

[0191] handleIncomingMessage(message);

[0192] / / Attempt to connect

[0193] } catch (Exception e) {

[0194] / / Connection failed, please reconnect

[0195] }

[0196] }

[0197] private void scheduleReconnect() {

[0198] reconnectExecutor.schedule(() -> {

[0199] log.info("Attempting to reconnect...");

[0200] connectToEventServer();

[0201] }, delay, TimeUnit.SECONDS);

[0202] delay = Math.min(delay * 2, MAX_DELAY); / / exponential backoff

[0203] }

[0204] / / Gimbal control method

[0205] public void controlPtz(){

[0206] / / Build the request, automatically retries, handle token expiration, etc.

[0207] / / ... Handling the response...

[0208] }

[0209] Step 5: System Integration and Testing

[0210] Simulation test:

[0211] Use video streaming simulators (such as VLC, GStreamer) to simulate a large number of cameras.

[0212] Write test scripts to simulate network interruption and service process killing to verify the master-slave failover and automatic restart functions.

[0213] Write test cases to simulate different user roles (captain, officer, soldier) operating the same gimbal simultaneously, and verify whether the deprivation logic and notifications are triggered correctly.

[0214] Implementation and Deployment:

[0215] Upload the completed Docker image to the ship's private image repository.

[0216] Deploy all services with a single click using Docker Compose or Kubernetes deployment files.

[0217] Use the configuration management tool (Ansible) to configure server networks, virtual IPs, and the basic environment in batches.

[0218] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0219] Please refer to the following. Figure 5 , Figure 5 This is a flowchart illustrating an adaptation method provided in an embodiment of this specification.

[0220] The adaptation method includes the following steps:

[0221] Step S102: Obtain the CPU architecture and OS type of the video surveillance system 2;

[0222] Step S104: Load the instruction set functions and memory allocation functions corresponding to the CPU architecture;

[0223] Step S106: Load the underlying service call method and network I / O model corresponding to the OS type;

[0224] Step S108: Build adaptation software based on the loaded instruction set functions, memory allocation functions, underlying service call methods, and network I / O models;

[0225] In step S110, the video surveillance system 2 runs the adaptation software, and the adaptation software performs adaptive optimization during operation.

[0226] The explanatory process begins with system environment detection and resource loading. Through intelligent instruction set scheduling, unified memory management adaptation, unified system call interfaces, and cross-platform network communication adaptation, correct code is compiled and containerized to ensure a consistent runtime environment, thus building the adaptation software. This adaptation software runs on the target video surveillance system 2. During runtime, the software also detects and adaptively optimizes information that cannot be determined during the compilation phase. For example, it uses `detect_hardware_topology()` to accurately detect the number of physical cores, logical cores, NUMA node topology, cache sizes (L1 / L2 / L3), and specific instruction set support (such as AVX-512, NEON), optimizing the utilization efficiency of computing resources, memory system access performance, operating system scheduling and I / O efficiency, and the parallelism of the video processing pipeline. This achieves or approaches the highest performance level theoretically achievable by the platform.

[0227] After running this adapter software, video surveillance system 2 can provide standardized video services to external systems through unified interface layer 13. External system 3 can send a video service request to obtain the corresponding video stream and thus acquire the video service.

[0228] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the adaptation method embodiments are basically similar to the video surveillance adaptation interface embodiments, so the description is relatively simple; relevant parts can be referred to the description of the video surveillance adaptation interface embodiments.

[0229] Please see Figure 6 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this specification.

[0230] like Figure 6 As shown, the electronic device 600 may include: at least one processor 601, at least one network interface 604, user interface 603, memory 605, and at least one communication bus 602.

[0231] The communication bus 602 can be used to realize the connection and communication of the above components.

[0232] The user interface 603 may include buttons, and the optional user interface may also include a standard wired interface or a wireless interface.

[0233] The network interface 604 may include, but is not limited to, Bluetooth modules, NFC modules, Wi-Fi modules, etc.

[0234] The processor 601 may include one or more processing cores. The processor 601 connects to various parts within the electronic device 600 using various interfaces and lines. It executes various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 605, and by calling data stored in the memory 605. Optionally, the processor 601 may be implemented using at least one hardware form selected from DSP, FPGA, and PLC. The processor 601 may integrate one or more of the following: CPU, GPU, and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 601 and may be implemented as a separate chip.

[0235] The memory 605 may include RAM or ROM. Optionally, the memory 605 may include a non-transitory computer-readable medium. The memory 605 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 605 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 605 may also be at least one storage device located remotely from the aforementioned processor 601. As a computer storage medium, the memory 605 may include an operating system, a network communication module, a user interface module, and a video surveillance adapter interface application. The processor 601 may be used to call the video surveillance adapter interface application stored in the memory 605 and execute the steps of the adaptation method mentioned in the foregoing embodiments.

[0236] This specification also provides a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform one or more steps in the above-described adaptation method embodiments. If the constituent modules of the above-described electronic device are implemented as software functional units and sold or used as independent products, they can be stored in the computer-readable storage medium.

[0237] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this specification are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0238] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks. Unless otherwise specified, the technical features of this embodiment and its implementation can be combined arbitrarily.

[0239] The embodiments described above are merely preferred embodiments of this specification and are not intended to limit the scope of this specification. Any modifications and improvements made by those skilled in the art to the technical solutions of this specification without departing from the spirit of this specification should fall within the protection scope defined by the claims of this specification.

Claims

1. A video surveillance adapter interface, characterized in that, The middleware interface is configured to connect a video surveillance system, which includes multiple video acquisition devices, to an external system that obtains video services. The video surveillance adapter interface adopts a layered architecture, including a hardware adapter layer, a core service layer, and a unified interface layer connected from bottom to top. The hardware adaptation layer is configured to adapt and convert according to the CPU architecture and OS type of the video surveillance system and the communication protocol of the video acquisition device, so as to obtain a unified computing interface, a unified system service interface and a unified device operation interface. The core service layer includes a device management service module and a media forwarding service module; The device management service module is configured to manage all connected video acquisition devices by calling a unified device operation interface and to update the access status of the video acquisition devices. The media forwarding service module is connected to the device management service module and is configured to obtain video streams by calling the unified system service interface based on video service requests and the access status of video acquisition devices, and to distribute video streams after transcoding the video streams by calling the unified computing interface. The unified interface layer is configured to receive video service requests and output the corresponding video streams through a standardized access interface.

2. The video surveillance adapter interface according to claim 1, characterized in that, The hardware adaptation layer includes a CPU adaptation module, an OS adaptation module, and a video acquisition device adaptation module: The CPU adaptation module includes pre-set instruction set functions and memory allocation functions corresponding to different CPU architectures. It is configured to detect the CPU architecture of the video surveillance system and load the corresponding instruction set functions and memory allocation functions to obtain a unified computing interface. The OS adaptation module includes pre-configured underlying service call methods and network I / O models for different OS types. It is configured to detect the OS type of the video surveillance system and load the corresponding underlying service call methods and network I / O models to obtain a unified system service interface. The video acquisition device adaptation module includes protocol conversion plugins for different video acquisition devices. It is configured to detect the communication protocols of all connected video acquisition devices and load the corresponding protocol conversion plugins to obtain a unified device operation interface.

3. The video surveillance adapter interface according to claim 1, characterized in that, The media forwarding service module includes a load balancing unit; The load balancing unit is configured to allocate transcoding processing tasks for video streams corresponding to video service requests based on the load conditions of each of the multiple service processes.

4. A video surveillance adapter interface according to claim 1, characterized in that, The media forwarding service module includes a dynamic output unit; The dynamic output unit is configured to dynamically adjust the output bitrate of the transcoding processing task based on network conditions.

5. A video surveillance adapter interface according to claim 1, characterized in that, The media forwarding service module includes a disconnection reconnection unit; The disconnection reconnection unit is configured to automatically re-establish the connection according to the exponential backoff strategy when the network connection with any video acquisition device is interrupted, until the connection is restored.

6. A video surveillance adapter interface according to claim 1, characterized in that, The video service request also includes the requester's identity information and the requested device information; the core service layer also includes a permission management module. The permission management module is connected to the device management service module and is configured as follows: Role-based access control and attribute-based access control policies are adopted to obtain permissions and permission priority weights for video service requests. For the same requested device, the control of the requested device is dynamically determined based on the permission priority weight corresponding to each video service request.

7. A video surveillance adapter interface according to claim 6, characterized in that, The core service layer also includes a high availability and message middleware module; The high availability and message middleware module is connected to the device management service module, media forwarding service module and permission management module, and is configured to: synchronize status and transmit events among other modules in the core service layer; The device management service module is deployed in a primary / backup mode, using a heartbeat mechanism and distributed locks to switch between primary and backup nodes.

8. A video surveillance adapter interface according to claim 2, characterized in that, The unified system service interface provided by the OS adaptation module specifically includes: The thread priority setting interface is used to set the scheduling priority of video processing threads on different operating systems; A high-precision clock acquisition interface is used to obtain monotonically increasing microsecond-level timestamps on different operating systems.

9. An adaptation method based on a video surveillance adapter interface according to any one of claims 1-8, characterized in that, Includes the following steps: Obtain the CPU architecture and OS type of the video surveillance system; Load the instruction set functions and memory allocation functions corresponding to the CPU architecture; Load the underlying service invocation method and network I / O model corresponding to the OS type; Adaptive software is built based on the loaded instruction set functions, memory allocation functions, underlying service call methods, and network I / O models; The video surveillance system runs adaptation software, and the adaptation software performs adaptive optimization during runtime.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in claim 9.

Images