Low-cost multi-path network video recorder system based on RK3576 platform

Abstract

The application discloses a low-cost multi-path network video recorder system based on an RK3576 platform, relates to the technical field of security monitoring, and is used for improving the intelligent detection precision and concurrent processing capacity of a low-cost multi-path network video recorder in intelligent monitoring functions. In the system, a general embedded processor built-in neural network processing unit is adopted to replace a special security chip, a high-speed serial expansion bus external bus protocol bridge chip is used to realize large-capacity local storage, a shared output buffer is used as a common data source to simultaneously drive a mobile detection path and a video storage path to run in parallel, and the mobile target detection result of the neural network processing unit is directly embedded into a key frame index of a video file in the form of an event time mark, so that the intelligent detection precision and concurrent processing capacity of the multi-path network video recorder in the intelligent monitoring function are improved at a low overall material cost.

Description

Technical Field

[0001] This application relates to the field of security monitoring technology, and in particular to a low-cost multi-channel network video recorder system based on the RK3576 platform. Background Technology

[0002] Network video recorders (NVRs) are essential devices in security monitoring systems. They primarily receive video streams from network cameras (IPCs) and perform real-time preview, recording, storage, and playback retrieval. With the rapid development of the security industry, market demand for low-cost, miniaturized NVRs with intelligent analysis capabilities is increasing, especially in price-sensitive applications such as small shops, homes, and offices. Users require reliable multi-channel video storage capabilities as well as basic intelligent monitoring functions such as motion detection and event retrieval.

[0003] Currently, mainstream NVR solutions on the market primarily employ dedicated security SoC chips for video access and storage. A typical implementation uses Hisilicon Hi3531 / Hi3536 series or security-specific processors from companies like Starchip and Novatek. These chips integrate dedicated SATA controllers and video codec hardware acceleration units, directly connecting to SATA hard drives via built-in SATA interfaces for continuous writing of recording data. In terms of intelligent analysis, Hisilicon solutions utilize integrated IVE modules or external independent NPU chips to achieve motion detection and other AI functions. Consumer-grade security SoCs from Starchip and Novatek, on the other hand, achieve basic motion detection through on-chip image processing algorithms, which have limited algorithm complexity and detection accuracy. The core characteristics of these solutions are reliance on the SATA controller natively integrated into the SoC chip for large-capacity local storage and on dedicated on-chip hardware modules or external AI chips for intelligent analysis.

[0004] However, due to their high integration and design for specific application areas, dedicated security SoC chips have a purchase price that is much higher than that of general-purpose ARM processors with equivalent computing power. In low-cost solutions, consumer-grade security SoCs selected to reduce chip costs can usually only support simple motion detection algorithms and are difficult to achieve efficient concurrent intelligent analysis of multiple video streams. This results in low-cost multi-channel network video recorders having lower intelligent detection accuracy and concurrent processing capabilities in intelligent monitoring functions. Summary of the Invention

[0005] This application provides a low-cost multi-channel network video recorder system based on the RK3576 platform, which is used to improve the intelligent detection accuracy and concurrent processing capability of the low-cost multi-channel network video recorder in intelligent monitoring functions.

[0006] Firstly, a low-cost multi-channel network video recorder system based on the RK3576 platform is provided. The system includes a main control module, a network access module, and a storage expansion module. The main control module uses an RK3576 processor, which integrates a hardware video decoder, a neural network processing unit, and a high-speed serial expansion bus interface. The network access module receives encoded video stream data from multiple network cameras within a local area network via an Ethernet interface. The hardware video decoder decodes the encoded video stream data and writes the original image data of each video channel into a shared output buffer. The original image data in the shared output buffer is simultaneously used by two parallel processing paths, where the first path is a motion detection path and the second path is a recording storage path. In the first path, the neural network processing unit sequentially selects data from multiple network cameras according to a round-robin scheduling mechanism. The first path reads raw image data from each video channel in the shared output buffer for motion detection inference and outputs the moving target detection result. In the second path, the main control module reads raw image data from each video channel in the shared output buffer, encapsulates it into segmented recording files at preset time intervals, and continuously writes it to the serial storage hard disk through the storage expansion module. The main control module is also used to generate a motion event record when the moving target detection result output from the first path indicates the presence of a moving target, and directly embeds the event timestamp corresponding to the motion event record into the keyframe index of the segmented recording file currently being written in the second path. The storage expansion module includes a bus protocol bridge chip and a serial storage hard disk. The bus protocol bridge chip is connected to the RK3576 processor through a high-speed serial expansion bus interface, and the downstream port of the bus protocol bridge chip is connected to the serial storage hard disk through a serial storage bus interface.

[0007] By adopting the above technical solutions, the multi-channel network video recorder system writes the raw image data output from the hardware video decoder into a shared output buffer. Using this buffer as a common data source, it simultaneously drives the motion detection path and the recording storage path to work in parallel, eliminating the need for motion detection and continuous recording to wait for each other, thus improving the concurrent processing capability of multiple video streams. Simultaneously, by directly embedding event timestamps into the keyframe index of the currently being written segmented recording file, subsequent retrieval and playback can accurately locate the event occurrence time without traversing independent metadata files, improving the linkage efficiency between intelligent detection results and recording data. Furthermore, by employing a bus protocol bridging chip to bridge the high-speed serial expansion bus and the serial storage bus, general-purpose ARM processors without native serial storage controllers can be expanded with large-capacity local storage capabilities at low cost, thereby improving the intelligent detection accuracy and concurrent processing capability of the low-cost multi-channel network video recorder in its intelligent monitoring function.

[0008] In conjunction with some embodiments of the first aspect, in some embodiments, the multi-channel network video recorder system further includes a multi-channel write scheduling module; the multi-channel write scheduling module maintains an independent circular write buffer for each video channel to temporarily store the video data to be written read from the shared output buffer by each video channel; the multi-channel write scheduling module is also used to package the data in each circular write buffer into write requests in chronological order and add the write requests to a unified write scheduling queue; the multi-channel write scheduling module is also used to sequentially retrieve write requests from the write scheduling queue according to a first-in-first-out strategy and submit them to the bus protocol bridge chip via a high-speed serial extended bus interface to perform the write operation.

[0009] By adopting the above technical solution, the multi-channel network video recorder system decouples the generation and writing of data for each video channel in time by maintaining an independent circular write buffer for each video channel, thus avoiding direct competition for hard disk write resources among multiple video streams. Through a unified write scheduling queue and a first-in-first-out (FIFO) strategy, the dispersed multi-channel write requests are serialized, ensuring the consistency of the write order. This enables concurrent writing of multiple video streams under a single high-speed serial expansion bus interface, preventing data loss or file corruption caused by concurrent write contention.

[0010] In conjunction with some embodiments of the first aspect, in some embodiments, the multi-channel write scheduling module is also used to temporarily store the currently to-be-written video data in the circular write buffer of the corresponding video channel when the serial storage hard disk is momentarily blocked due to file system metadata updates or head seek, without blocking the upstream data stream; the multi-channel write scheduling module is also used to resubmit the temporarily stored data to the write scheduling queue in chronological order to complete the writing after the serial storage hard disk recovers to the writable state.

[0011] By adopting the above technical solution, the multi-channel network video recorder system utilizes the independent ring-shaped write buffer of each channel to continuously receive upstream video data during the momentary blockage of the hard disk. This prevents the temporary unwriteable state of the hard disk from being transmitted upward to the video stream receiving and encoding stages. After the hard disk recovers, the cached data is written in sequence, thereby improving the integrity of the recorded data in the scenario of momentary blockage of the hard disk.

[0012] In conjunction with some embodiments of the first aspect, in some embodiments, the motion detection inference process performed by the neural network processing unit includes: reading the original image data of the current frame and the image data of the previous frame as a reference frame from a shared output buffer; performing pixel-level difference calculation on the two frames to obtain a difference image; inputting the difference image into a pre-trained lightweight convolutional neural network model, and having the neural network processing unit perform inference operations; and outputting a moving target detection result, which includes a determination of whether a moving target exists and the region coordinates of the moving target in the original image.

[0013] By adopting the above technical solution, the multi-channel network video recorder system extracts motion information from the image into a differential image by performing pixel-level difference between the current frame and the reference frame. This reduces the complexity of the input features of the neural network model, enabling the lightweight convolutional neural network model to complete the detection of moving targets with low computing power consumption. It leverages the computational efficiency of the built-in neural network processing unit of the RK3576 processor and achieves real-time motion detection of multiple video streams without relying on external AI acceleration chips.

[0014] In conjunction with some embodiments of the first aspect, in some embodiments, the multi-channel network video recorder system further includes an image acceleration processing unit; the image acceleration processing unit is used to perform color space conversion and proportional scaling processing on the original image data read from the shared output buffer before the original image data is sent to the neural network processing unit; the proportional scaling processing uses a side-padding method to keep the aspect ratio of the original image unchanged, so as to obtain a pre-processed image that meets the input size requirements of the lightweight convolutional neural network model.

[0015] By adopting the above technical solution, the multi-channel network video recorder system uses the image acceleration processing unit to complete color space conversion and proportional scaling at the hardware level, avoiding the large computational overhead caused by the CPU software to perform image preprocessing, so that CPU resources can be concentrated on tasks such as video stream management, event recording and writing, and IO scheduling.

[0016] In conjunction with some embodiments of the first aspect, in some embodiments, the multi-channel network video recorder system further includes an event retrieval module; the event retrieval module is used to read the keyframe index of the segmented video file during video playback, extract the event timestamps embedded therein, and overlay the corresponding event indicator bar on the timeline progress bar of the playback interface; the event retrieval module is also used to respond to the user's selection operation on the event indicator bar, and directly locate the corresponding recording moment in the current segmented video file and start playback according to the timeline mapping relationship in the keyframe index; the event retrieval module is also used to summarize the event timestamps from the keyframe indexes of multiple segmented video files, generate an event list arranged in reverse chronological order, and respond to the user's selection operation on the record in the event list to locate the corresponding recording start moment for playback.

[0017] By adopting the above technical solution, the multi-channel network video recorder system can directly extract embedded event timestamps from the keyframe index of segmented recording files during playback, rendering the event indicator bar without accessing a separate external database, thus improving the response speed of the playback interface. The overlay display of the event indicator bar and the timeline progress bar allows users to clearly see the time distribution of events at a glance. Both click-to-jump and event list search options meet users' needs for quick location in different usage scenarios, reducing the manual screening cost of valid recording segments.

[0018] In conjunction with some embodiments of the first aspect, in some embodiments, the bus protocol bridging chip is used to convert transaction layer data packets on the high-speed serial extended bus interface into frame information structures of the serial storage bus protocol; the main control module is also used to complete link training through the high-speed serial extended bus interface during system startup, establish a data path with the bus protocol bridging chip, and complete device enumeration and block device creation of the serial storage hard disk through the bus protocol bridging chip; the main control module is also used to re-execute device enumeration when a hot-plug event occurs on the serial storage hard disk.

[0019] By adopting the above technical solution, the multi-channel network video recorder system completes the protocol conversion between high-speed serial extended bus transaction layer data packets and serial storage bus frame information structures at the hardware level through a bus protocol bridging chip. This allows the high-speed serial extended bus controller of the general-purpose ARM processor to transparently access the serial storage hard drive without modifying the upper-layer storage driver software stack. The mechanism of automatic re-enumeration after hot-plug events improves the plug-and-play capability of the hard drive in maintenance and replacement scenarios.

[0020] In conjunction with some embodiments of the first aspect, a low-cost multi-channel network video recording method based on the RK3576 platform is provided in some embodiments, applied to a multi-channel network video recorder system. The method includes: receiving encoded video stream data from multiple network cameras within a local area network via an Ethernet interface; decoding the encoded video stream data using the hardware video decoder integrated in the RK3576 processor, and writing the original image data of each video channel obtained from the decoding into a shared output buffer; using the shared output buffer as a common data source, simultaneously driving the motion detection path and the recording storage path to work in parallel; in the motion detection path, sequentially reading the original image data of each video channel from the shared output buffer according to a polling scheduling mechanism. Image data is fed into the neural network processing unit integrated in the RK3576 processor to perform motion detection inference and output moving target detection results. In the video storage path, the raw image data of each video channel is read from the shared output buffer, packaged into segmented video files at preset time intervals, and written to the serial storage hard disk via a bus protocol bridge chip through a high-speed serial expansion bus interface. When the moving target detection results output by the motion detection path indicate the presence of a moving target, a motion event record is generated, and the event timestamp corresponding to the motion event record is directly embedded into the keyframe index of the segmented video file currently being written in the video storage path, establishing a mapping relationship between the video timeline and the event timestamp within the same file.

[0021] By adopting the above technical solution, a solution that replaces dedicated security chips with a general-purpose embedded processor's built-in neural network processing unit is used. A high-speed serial expansion bus is used to connect to an external bus protocol bridging chip to achieve large-capacity local storage. The shared output buffer serves as a common data source, driving the motion detection path and the recording storage path to run in parallel. The moving target detection results of the neural network processing unit are directly embedded into the key frame index of the recording file in the form of event time stamps. Thus, with a lower overall material cost, the intelligent detection accuracy and concurrent processing capability of the multi-channel network video recorder in intelligent monitoring functions are improved.

[0022] This application provides a computer program product containing instructions that, when run on a multi-channel network video recorder system, cause the multi-channel network video recorder system to perform the method described in the first aspect and any possible implementation thereof.

[0023] Thirdly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a multi-channel network video recorder system, cause the multi-channel network video recorder system to perform the method described in the first aspect and any possible implementation thereof.

[0024] It is understood that the computer program product provided in the second aspect and the computer storage medium provided in the third aspect are both used to execute the methods provided in the embodiments of this application. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods, and will not be repeated here.

[0025] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0026] 1. The multi-channel network video recorder system writes the raw image data output from the hardware video decoder into a shared output buffer. Using this buffer as a common data source, it simultaneously drives the motion detection path and the recording storage path to work in parallel. This eliminates the need for motion detection and continuous recording to wait for each other, improving the concurrent processing capability of multiple video streams. Furthermore, by directly embedding event timestamps into the keyframe index of the currently being written segmented recording file, subsequent retrieval and playback can accurately pinpoint the event's occurrence without traversing independent metadata files, improving the linkage efficiency between intelligent detection results and recording data. In addition, by employing a bus protocol bridging chip to bridge the high-speed serial expansion bus and the serial storage bus, general-purpose ARM processors without native serial storage controllers can be expanded with large-capacity local storage capabilities at low cost. This improves the intelligent detection accuracy and concurrent processing capability of the low-cost multi-channel network video recorder in its intelligent monitoring functions.

[0027] 2. The multi-channel network video recorder system utilizes an image acceleration processing unit to complete color space conversion and proportional scaling at the hardware level, avoiding the large computational overhead caused by image preprocessing performed by CPU software. This allows CPU resources to be concentrated on tasks such as video stream management, event recording and writing, and I / O scheduling.

[0028] 3. By adopting a solution that replaces dedicated security chips with a general-purpose embedded processor with a built-in neural network processing unit, and using a high-speed serial expansion bus to connect to an external bus protocol bridge chip to achieve large-capacity local storage, the motion detection path and the recording storage path are driven to run in parallel using a shared output buffer as a common data source. The moving target detection results of the neural network processing unit are directly embedded into the key frame index of the recording file in the form of event time stamps. Thus, with a lower overall material cost, the intelligent detection accuracy and concurrent processing capability of the multi-channel network video recorder in intelligent monitoring functions are improved. Attached Figure Description

[0029] Figure 1 This is a module architecture diagram of a multi-channel network video recorder system in an embodiment of this application.

[0030] Figure 2This is a flowchart illustrating a low-cost multi-channel network video recording method based on the RK3576 platform in an embodiment of this application.

[0031] Figure 3 This is a schematic diagram of the physical device structure of a multi-channel network video recorder system in the embodiments of this application. Detailed Implementation

[0032] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0033] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0034] This application provides a low-cost multi-channel network video recorder system based on the RK3576 platform, which is used to improve the intelligent detection accuracy and concurrent processing capability of the low-cost multi-channel network video recorder in intelligent monitoring functions.

[0035] In the specific implementation scenarios of this application, it is applicable to deployment environments in price-sensitive security scenarios such as small shops, homes, and offices that require low-cost multi-channel video recording and intelligent analysis capabilities. The multi-channel network video recorder system proposed in this application uses a general-purpose embedded processor as its core, expands large-capacity local storage through bus protocol bridging, and utilizes the processor's built-in neural network processing unit to achieve real-time motion detection of multiple video streams. The detection results are linked with the continuous recording depth to build a low-cost, integrated intelligent network video recorder.

[0036] The multi-channel network video recorder system provided in the embodiments of this application will be described below.

[0037] Please see Figure 1 This is a module architecture diagram of a multi-channel network video recorder system in an embodiment of this application.

[0038] like Figure 1As shown, the multi-channel network video recorder system includes a main control module 101, a network access module 102, a storage expansion module 103, a memory module 104, and a power management module 105. The main control module 101 uses an RK3576 processor, which integrates a hardware video decoder 1011, a neural network processing unit 1012, and a high-speed serial expansion bus interface 1013. The network access module 102 is connected to the main control module 101 and is used to access multiple network cameras within the local area network. The storage expansion module 103 is connected to the main control module 101 via the high-speed serial expansion bus interface 1013 and is used for persistent storage of large-capacity local video recording data. The memory module 104 is connected to the main control module 101 and is used for buffering video stream data and providing system operating memory. The power management module 105 provides regulated power to all modules of the system.

[0039] The main control module 101 is the core processing hub of the multi-channel network video recorder system, integrated within a single RK3576 general-purpose ARM processor chip. It coordinates all tasks, including video decoding, neural network inference, recording encapsulation, and I / O scheduling, without relying on external AI acceleration chips or dedicated security processors. The hardware video decoder 1011 is a hardware-accelerated video decoding unit integrated within the processor. It can perform hardware-accelerated decoding of various mainstream video encoding standards, decode the received encoded video stream data, and write the original image data of each video channel in a luminance / chrominance separation format to a shared output buffer located in the memory module. The neural network processing unit 1012 is a neural network inference acceleration unit integrated within the processor, dedicated to performing inference operations on deep learning models such as convolutional neural networks. It sequentially reads the original image data of each video channel from the shared output buffer using a polling scheduling mechanism, performs motion detection inference, and outputs a motion target detection result containing a determination of whether a moving target exists and the region coordinates of the moving target in the original image, without requiring an additional independent AI acceleration chip. The high-speed serial expansion bus interface 1013 is a high-speed serial peripheral expansion interface provided by the processor. It is used to connect the bus protocol bridge chip in the storage expansion module 103 to realize a high-speed data path between the processor and the serial storage hard disk.

[0040] The shared output buffer is a common memory area located in the memory module. After the hardware video decoder 1011 writes the raw image data from each video channel into this buffer, the raw image data in the shared output buffer is simultaneously referenced by two parallel processing paths. The first path is the motion detection path, where the neural network processing unit 1012 sequentially reads the raw image data from each video channel from the shared output buffer using a polling scheduling mechanism to perform motion detection inference. The second path is the video recording and storage path, where the main control module 101 reads the raw image data from each video channel from the shared output buffer, encapsulates it into segmented video files at preset time intervals, and continuously writes it to the serial storage hard disk. The two paths share the same raw image data in a zero-copy manner, are independent of each other, and do not block each other.

[0041] When the moving target detection result output by the first path indicates the presence of a moving target, the main control module 101 generates a moving event record and directly embeds the event timestamp corresponding to the moving event record into the key frame index of the segmented video file currently being written in the second path, establishing a bidirectional mapping relationship between the video timeline and the event timestamp within the same file.

[0042] The upstream port of the bus protocol bridge chip in the storage expansion module 103 is connected to the high-speed serial expansion bus controller of the RK3576 processor via the high-speed serial expansion bus interface 1013. The downstream port provides multiple serial storage bus interfaces for connecting large-capacity serial storage hard drives. The bus protocol bridge chip internally performs protocol conversion between the high-speed serial expansion bus transaction layer data packets and the serial storage bus frame information structure, enabling transparent data transmission between the processor and the serial storage hard drive. This allows general-purpose ARM processors without a native serial storage controller to expand their large-capacity local storage capabilities at low cost.

[0043] The network access module 102 connects to a gigabit Ethernet physical layer chip via the Ethernet media access controller built into the RK3576 processor, providing an adaptive Ethernet interface and supporting video stream access for real-time streaming protocols and national standard video surveillance networking protocols. It is used to simultaneously receive encoded video stream data from multiple network cameras within the local area network.

[0044] The multi-channel network video recorder system also includes a multi-channel write scheduling module, which in... Figure 1Integrated within the software layer of the main control module 101, this module manages the write scheduling of each video channel in the recording storage path. The multi-channel write scheduling module maintains an independent circular write buffer for each video channel in the memory module, used to temporarily store the video data to be written, read from the shared output buffer by each video channel. Data in each circular write buffer is packaged into write requests in chronological order and added to a unified write scheduling queue. The IO scheduling thread retrieves write requests sequentially from the write scheduling queue using a first-in, first-out strategy and submits them to the bus protocol bridge chip via the high-speed serial expansion bus interface 1013 for write operation.

[0045] The multi-channel write scheduling module also has the capability to handle momentary blocking. When the serial storage hard drive experiences momentary write blocking due to file system metadata updates or head seeks, the multi-channel write scheduling module temporarily stores the currently pending video data in the circular write buffer of the corresponding video channel, without blocking the continuous access and decoding of upstream video data streams. After the serial storage hard drive recovers its writable state, the multi-channel write scheduling module resubmits the data temporarily stored in the circular write buffer to the write scheduling queue in chronological order to complete the rewriting, thereby ensuring the integrity and continuity of recorded data in scenarios of momentary hard drive blocking.

[0046] The motion detection inference process of the neural network processing unit 1012 is as follows: It reads the original image data of the current frame and the image data of the previous frame (used as a reference frame) from the shared output buffer, performs pixel-level difference calculation on the two frames, and obtains a difference image reflecting the inter-frame motion information. The difference image is then input into a pre-trained lightweight convolutional neural network model, where the neural network processing unit 1012 performs inference operations with quantization precision. The neural network processing unit 1012 outputs the moving target detection result, which includes a determination of whether a moving target exists and the region coordinates of the moving target in the original image.

[0047] The multi-channel network video recorder system also includes an image acceleration processing unit, which is integrated within the RK3576 processor. This unit performs color space conversion and proportional scaling on the raw image data read from the shared output buffer before it is fed into the neural network processing unit 1012. The proportional scaling process uses edge padding to maintain the aspect ratio of the original image, resulting in a pre-processed image that meets the input size requirements of a lightweight convolutional neural network model before being fed into the neural network processing unit 1012 for inference operations. By performing this preprocessing at the hardware level, the image acceleration processing unit avoids the significant computational overhead associated with CPU software-based image preprocessing, allowing CPU resources to be concentrated on tasks such as video stream management, event logging, and I / O scheduling.

[0048] The multi-channel network video recorder system also includes an event retrieval module, which runs at the application layer of the main control module 101. During playback, the event retrieval module reads the keyframe indices of segmented video files, extracts the embedded event timestamps, and overlays the corresponding event indicator bars onto the timeline progress bar of the playback interface. The event retrieval module also responds to user selections of event indicator bars, directly locating the corresponding recording moment within the current segmented video file and initiating playback based on the timeline mapping relationship in the keyframe indices. Furthermore, the event retrieval module aggregates event timestamps from the keyframe indices of multiple segmented video files, generating an event list arranged in reverse chronological order. Responding to user selections of records in the event list, it automatically locates the corresponding recording start moment for playback. Since the event timestamps are directly embedded in the keyframe indices of the segmented video files during the recording writing stage, the event retrieval module can complete the rendering and navigation of event indicator bars without accessing a separate external database during playback, improving the response speed and retrieval efficiency of the playback interface.

[0049] The bus protocol bridge chip in the storage expansion module 103 also undertakes system initialization and device management responsibilities. During system startup, the main control module 101 completes link training through the high-speed serial expansion bus interface 1013, establishing a data path with the bus protocol bridge chip. It then uses the bus protocol bridge chip to complete device enumeration and block device creation for the serial storage hard drive, enabling the operating system to access the serial storage hard drive using a standard block device interface. When a hot-plug event occurs with the serial storage hard drive, the main control module 101 re-executes device enumeration to identify and mount the new hard drive, achieving plug-and-play capability in hard drive maintenance and replacement scenarios.

[0050] In some embodiments, large-capacity local storage expansion can be achieved in several ways: Optionally, a bus protocol bridge chip can be connected to an external high-speed serial expansion bus interface, and then a mechanical hard drive can be connected via a serial storage bus, suitable for scenarios with high requirements for storage capacity-to-cost ratio. Optionally, a solid-state drive can be directly connected via a high-speed serial expansion bus interface, suitable for scenarios with high requirements for read and write speeds. Optionally, a conversion chip can be connected to an external processor via a high-speed serial interface, and an external storage device can be connected via a high-speed universal serial bus, suitable for scenarios with high requirements for interface flexibility. It is understood that video recording data can also be written to an external network storage device via an Ethernet interface, which is not limited here.

[0051] The multi-channel network video recording method provided in the embodiments of this application will be described below. Please refer to... Figure 2 This is a flowchart illustrating a low-cost multi-channel network video recording method based on the RK3576 platform in an embodiment of this application.

[0052] S201: Receives encoded video stream data from multiple network cameras within the local area network via the Ethernet interface.

[0053] Among them, the Ethernet interface refers to the wired network physical interface provided by the network access module 102, used to represent a wired LAN connection port that supports multi-rate adaptive negotiation. A multi-channel network camera refers to a front-end device deployed at the monitoring site that sends captured images to the network in the form of a video stream, used to represent the network front-end sensing device that serves as the source of video data for this system. Encoded video stream data refers to the data transmitted after the network camera has encoded and compressed the raw images it captures, encapsulated using network protocols, used to represent the continuously transmitted video stream in the form of network data packets after encoding and compression. The video stream access protocol refers to the application layer protocol used to establish video stream transmission sessions over the network, including real-time streaming protocols and national standard video surveillance networking protocols, used to represent the communication specifications that regulate the establishment and transmission of video streams between network cameras and recorders.

[0054] Specifically, after the multi-channel network video recorder system is powered on, the video stream receiving service on the main control module 101 initiates video stream connection requests to each configured network camera in the local area network through the Ethernet interface provided by the network access module 102. After the connection is established, each network camera continuously pushes encoded and compressed video streams to the system according to the protocol specifications. The video stream receiving service decapsulates the received network data packets, strips off the protocol header of the network transport layer, extracts the payload carrying the video frame data, obtains the encoded video stream data encapsulated in the video encoding format, and sends it to the subsequent decoding processing stage according to the channel. When a network camera loses connection due to network fluctuations, the system's connection management component will continuously attempt to reconnect after detecting the disconnection event until the video stream is resumed, so as to ensure the overall continuity of multi-channel recording.

[0055] S202. Decode the encoded video stream data using the hardware video decoder integrated in the RK3576 processor, and write the original image data of each video channel obtained from the decoding into the shared output buffer.

[0056] The hardware video decoder 1011 refers to the dedicated video decoding engine integrated within the RK3576 processor. It represents an on-chip functional unit that uses hardware acceleration to replace software decoding for efficient decoding of encoded and compressed video frames. The raw image data refers to the uncompressed image data obtained after decoding and restoring the video frames. It represents a pixel-by-pixel image matrix stored in a luminance / chrominance separation format and can be directly used in subsequent image analysis and recording / packaging stages. The shared output buffer is a public data storage area allocated in the memory module. It represents a zero-copy shared memory area where the hardware video decoder writes the decoding results and simultaneously provides data read access to the motion detection path and the recording storage path.

[0057] Specifically, after the video stream receiving service sends the encoded video stream data from each video channel to the hardware video decoder 1011, the decoder performs decoding operations such as entropy decoding, inverse quantization, inverse transform, and inter-frame / intra-frame prediction compensation on the input encoded video frames to restore the compressed encoded frames to the original image data. After decoding, the hardware video decoder 1011 writes the original image data of each channel into the corresponding slot of the pre-allocated shared output buffer in the memory module through direct memory access, and updates the frame sequence number and timestamp information of the slot. The shared output buffer adopts a circular queue structure to manage the decoded frames of each channel. When a new frame is written to the buffer slot, the old frame data is overwritten by the new frame to ensure that the latest decoded frames of each channel are always stored in the buffer, so that subsequent parallel processing paths can read them as needed without transferring data copies between paths.

[0058] S203 uses a shared output buffer as a common data source, and simultaneously drives the motion detection path and video storage path to work in parallel.

[0059] The motion detection path refers to the data processing link aimed at detecting the presence of moving targets in a video frame. It represents the complete process of reading raw image data from a shared output buffer, inferring the result through a neural network processing unit, and then outputting the moving target detection result. The video storage path refers to the data processing link aimed at persisting video data to a storage medium. It represents the complete process of reading raw image data from a shared output buffer, encapsulating it, and then writing it to a serial hard disk via a storage expansion module.

[0060] Specifically, after system initialization, the main control module 101 simultaneously starts the processing tasks for both the motion detection path and the video storage path. Both paths use a shared output buffer as their respective source of raw image data, reading decoded frame data from each video channel in the shared output buffer through their independent logic, and then executing their respective subsequent processing operations. Neither path sends data copies to the other or waits for the other during operation; the independence of their data access is guaranteed by the concurrent read access mechanism of the shared output buffer. This parallel architecture ensures that the computational latency of motion detection inference does not affect the continuity of video storage, and the disk operation latency of video writing does not block the inference rhythm of motion detection, thus guaranteeing the efficient collaborative operation of the two paths on a single processor.

[0061] S204. In the motion detection path, the original image data of each video channel is read sequentially from the shared output buffer according to the polling scheduling mechanism, and sent to the neural network processing unit integrated by the RK3576 processor to perform motion detection inference and output the moving target detection result.

[0062] The polling scheduling mechanism refers to a scheduling strategy that submits the latest decoded frames of each video channel sequentially according to a preset channel order for inference tasks. This represents a time-division multiplexing method for concurrent service of multiple video channels on a single neural network processing unit. The moving target detection result refers to the semantic information output by the neural network processing unit 1012 after completing one inference iteration. It represents the inference output data, including the determination of whether a moving target exists and the region coordinates of the moving target in the original image.

[0063] Specifically, the motion detection path scheduling module cycles through all video channels sequentially from the first to the last, according to channel number order. When processing each video channel, the scheduling module first reads the latest decoded frame written to that channel from the shared output buffer as the current frame, and reads the decoded frame cached during the previous processing of that channel as the reference frame. It then performs pixel-by-pixel difference calculations on the luminance component data of the two frames to obtain a difference image reflecting the pixel changes between the two frames. After color space conversion and proportional scaling preprocessing by the image acceleration processing unit, the difference image, along with necessary channel configuration parameters, is submitted to the neural network processing unit 1012. The neural network processing unit 1012 calls a pre-loaded lightweight convolutional neural network model to perform forward inference. The model extracts features and performs classification regression on the difference image with quantized integer precision. After inference, it outputs the moving target detection result for that channel, including the determination of whether a moving target exists and the region coordinates of the moving target in the original image coordinate system. The output result, along with the corresponding channel number and timestamp information, is passed to the event management module in step S206 for further processing. After completing the inference for the current channel, the scheduling module switches to the next channel and repeats the above process. This cycle continues until all channels have been polled, and then the next polling cycle begins.

[0064] S205. In the video recording storage path, the original image data of each video channel is read from the shared output buffer, encapsulated into segmented video files at preset time intervals, and written to the serial storage hard disk via the high-speed serial expansion bus interface and the bus protocol bridge chip.

[0065] Segmented video files refer to video data files generated by segmenting at preset time intervals, containing complete file headers and index structures. The high-speed serial expansion bus interface 1013 refers to the high-speed serial peripheral expansion bus controller provided by the RK3576 processor, used to represent the on-chip communication path for the main control module 101 to initiate data write requests to the storage expansion module 103. The bus protocol bridge chip refers to the bridge device in the storage expansion module 103 responsible for performing frame format conversion between the high-speed serial expansion bus protocol and the serial storage bus protocol, used to represent an intermediate conversion component that converts bus transaction layer data packets from the processor side into frame information structures recognizable by the storage hard disk. The serial storage hard disk refers to a large-capacity mechanical hard disk storage medium connected through the serial storage bus interface, used to represent the physical storage device for the final persistent writing of video data.

[0066] Specifically, the encapsulation module of the video storage path continuously reads the latest raw image data from each video channel from the shared output buffer. Based on the preset time interval configured by the system, it encapsulates the video data into the recording file of the current channel in chronological order. During the encapsulation process, the encapsulation module maintains a context for each channel's current file, including file header information, a keyframe index table, and the duration information already contained in the current file. When the cumulative duration of the current file reaches the preset time interval, the encapsulation module closes the current file and creates a new recording file, writing subsequent frame data into the new file, thus achieving segmented management of the recording data. The encapsulated video data is sent to the corresponding channel's circular write buffer for temporary storage in the form of write requests. The multi-channel write scheduling module packages the data in each channel's circular write buffer into write requests in chronological order and adds them to a unified write scheduling queue. The scheduling thread retrieves write requests from the queue sequentially according to a first-in-first-out strategy and submits them to the bus protocol bridge chip through the high-speed serial extended bus interface. The bus protocol bridge chip converts the received high-speed serial extended bus transaction layer data packets into a serial storage bus frame information structure and sends them to the serial storage hard disk via the serial storage bus interface to perform disk write operations. When the serial storage hard drive experiences a momentary write blockage due to file system metadata updates or head seeks, the data to be written continues to be temporarily stored in the circular write buffer of the corresponding channel. Once the hard drive returns to a writable state, the scheduling thread resubmits the temporarily stored data for writing, thus avoiding data loss due to momentary disk blockage. When the available storage space of the serial storage hard drive falls below a preset threshold, the system automatically deletes the oldest segmented video files according to a first-in-first-out (FIFO) strategy to free up space and continues writing new video files, achieving cyclic overwriting storage of video data.

[0067] S206. When the motion detection path outputs a motion target detection result indicating the presence of a motion target, a motion event record is generated, and the event timestamp corresponding to the motion event record is directly embedded into the keyframe index of the segmented video file currently being written in the video storage path, establishing a mapping relationship between the video timeline and the event timestamp within the same file.

[0068] Specifically, the event management module continuously receives detection results from each channel of the motion detection path in step S204. When the motion target detection result of a certain channel indicates the presence of a moving target, the event management module generates a motion event record for that channel. The record includes the channel number, the event start timestamp, and the region coordinates of the moving target in the original image. This record is then written to the event index database, and the event end timestamp is dynamically maintained. That is, as subsequent polling inference continues to detect the presence of a moving target in that channel, the event end timestamp is continuously updated. When no moving target is detected for several consecutive rounds of inference, the motion event is considered to have ended, the event record is completed, and archived. While generating the motion event record, the event management module extracts the event start timestamp and notifies the encapsulation module of the video storage path. The encapsulation module then embeds the corresponding event timestamp as an extended field directly into the keyframe index of the segmented video file currently being written to by the channel. The specific embedding location is at the keyframe index entry corresponding to the event start timestamp, thereby establishing a bidirectional mapping relationship between the recording timeline and the motion event timestamp within the video file. During video playback, the playback module reads the keyframe indexes of the segmented video files, parses the embedded event timestamps, and overlays an event indicator bar on the corresponding time on the timeline progress bar of the playback interface. Responding to user selection of the event indicator bar, the module directly locates the corresponding video frame within the current file based on the position offset information in the keyframe index and initiates playback. Simultaneously, the playback module can aggregate event timestamps across multiple segmented video files and, combined with the complete event records stored in the event index database, provide the user with a list of events arranged in reverse chronological order, supporting filtering by channel and one-click location playback.

[0069] In the above embodiments, by adopting a solution that uses a general-purpose embedded processor with a built-in neural network processing unit to replace a dedicated security chip, and by using a high-speed serial expansion bus to connect an external bus protocol bridge chip to achieve large-capacity local storage, and by using a shared output buffer as a common data source to drive the motion detection path and the recording storage path to run in parallel, and by directly embedding the moving target detection results of the neural network processing unit into the key frame index of the recording file in the form of event time stamps, the intelligent detection accuracy and concurrent processing capability of the multi-channel network video recorder in intelligent monitoring function are improved with a lower overall material cost.

[0070] The multi-channel network video recorder system in the embodiments of this invention is described below from the perspective of hardware processing. Please refer to [link / reference needed]. Figure 3 This is a schematic diagram of the physical device structure of a multi-channel network video recorder system in an embodiment of this application.

[0071] It should be noted that, Figure 3The structure of the multi-channel network video recorder system shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0072] like Figure 3 As shown, the multi-channel network video recorder system includes a CPU 301, which can perform various appropriate actions and processes according to a program stored in ROM 302 or a program loaded from storage section 308 into RAM 303, such as executing the methods described in the above embodiments. RAM 303 also stores various programs and data required for system operation. CPU 301, ROM 302, and RAM 303 are interconnected via bus 304. I / O interface 305 is also connected to bus 304.

[0073] The following components are connected to I / O interface 305: input section 306 including audio input devices, push-button switches, etc.; output section 307 including liquid crystal display (LCD) and audio output devices, indicator lights, etc.; storage section 308 including hard disks, etc.; and communication section 309 including network interface cards such as LAN (Local Area Network) cards, modems, etc. Communication section 309 performs communication processing via a network such as the Internet. Drive 310 is also connected to I / O interface 305 as needed. Removable media 311, such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on drive 310 as needed so that computer programs read from them can be installed into storage section 308 as needed.

[0074] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing computer programs for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311. When the computer program is executed by CPU 301, it performs the various functions defined in the present invention.

[0075] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those shown in the drawings.

[0076] Specifically, the multi-channel network video recorder system of this embodiment includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, it implements the low-cost multi-channel network video recording method based on the RK3576 platform provided in the above embodiment.

[0077] In another aspect, the present invention also provides a computer-readable storage medium, which may be included in the multi-channel network video recorder system described in the above embodiments; or it may exist independently and not assembled into the multi-channel network video recorder system. The storage medium carries one or more computer programs that, when executed by a processor of the multi-channel network video recorder system, cause the multi-channel network video recorder system to implement the low-cost multi-channel network video recording method based on the RK3576 platform provided in the above embodiments.

[0078] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0079] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

Claims

1. A low-cost multi-channel network video recorder system based on the RK3576 platform, characterized in that, The multi-channel network video recorder system includes a main control module, a network access module, and a storage expansion module; The main control module uses an RK3576 processor, which integrates a hardware video decoder, a neural network processing unit, and a high-speed serial expansion bus interface. The network access module is used to receive encoded video stream data from multiple network cameras in the local area network via an Ethernet interface. The hardware video decoder is used to decode the encoded video stream data and write the original image data of each video channel obtained by decoding into a shared output buffer. The raw image data in the shared output buffer is simultaneously fed to two parallel processing paths, where the first path is the motion detection path and the second path is the video recording storage path. In the first path, the neural network processing unit sequentially reads the original image data of each video channel from the shared output buffer according to the polling scheduling mechanism, performs motion detection inference, and outputs the moving target detection result; In the second path, the main control module reads the original image data of each video channel from the shared output buffer, encapsulates it into segmented recording files at preset time intervals, and continuously writes it to the serial storage hard disk through the storage expansion module; The main control module is also used to generate a motion event record when the motion target detection result output by the first path indicates the presence of a motion target, and to directly embed the event time stamp corresponding to the motion event record into the key frame index of the segmented video file currently being written in the second path. The storage expansion module includes a bus protocol bridge chip and the serial storage hard disk. The bus protocol bridge chip is connected to the RK3576 processor through the high-speed serial expansion bus interface, and the downstream port of the bus protocol bridge chip is connected to the serial storage hard disk through the serial storage bus interface.

2. The multi-channel network video recorder system according to claim 1, characterized in that, The multi-channel network video recorder system also includes a multi-channel write scheduling module; The multi-channel write scheduling module maintains an independent circular write buffer for each video channel to temporarily store the video data to be written that each video channel reads from the shared output buffer; The multi-channel write scheduling module is also used to package the data in each of the circular write buffers into write requests in chronological order, and add the write requests to a unified write scheduling queue. The multi-channel write scheduling module is also used to retrieve the write requests sequentially from the write scheduling queue according to a first-in-first-out strategy, and submit them to the bus protocol bridge chip via the high-speed serial extended bus interface to perform the write operation.

3. The multi-channel network video recorder system according to claim 2, characterized in that, The multi-channel write scheduling module is also used to temporarily store the video data to be written in the circular write buffer of the corresponding video channel when the serial storage hard disk is momentarily blocked due to file system metadata updates or head seek, so as not to block the upstream data flow. The multi-channel write scheduling module is also used to resubmit the temporarily stored data to the write scheduling queue in chronological order after the serial storage hard disk is restored to a writable state to complete the writing.

4. The multi-channel network video recorder system according to claim 1, characterized in that, The process of motion detection inference performed by the neural network processing unit includes: Read the original image data of the current frame and the image data of the previous frame (which serves as a reference frame) from the shared output buffer, perform pixel-level difference calculation on the two frames, and obtain the difference image. The difference image is input into a pre-trained lightweight convolutional neural network model, and the neural network processing unit performs inference operations. The neural network processing unit outputs the moving target detection result, which includes a determination of whether a moving target exists and the region coordinates of the moving target in the original image.

5. The multi-channel network video recorder system according to claim 4, characterized in that, The multi-channel network video recorder system also includes an image acceleration processing unit; The image acceleration processing unit is used to perform color space conversion and proportional scaling processing on the original image data read from the shared output buffer before the original image data is sent to the neural network processing unit; The proportional scaling process uses edge-padding to maintain the aspect ratio of the original image, resulting in a preprocessed image that meets the input size requirements of the lightweight convolutional neural network model.

6. The multi-channel network video recorder system according to claim 1, characterized in that, The multi-channel network video recorder system also includes an event retrieval module; The event retrieval module is used to read the keyframe index of the segmented video file during video playback, extract the event time markers embedded therein, and overlay the corresponding event indicator bar on the timeline progress bar of the playback interface. The event retrieval module is also used to respond to the user's selection operation on the event indicator bar, and to directly locate the corresponding recording moment in the current segmented video file and start playback based on the time axis mapping relationship in the key frame index; The event retrieval module is also used to summarize the event timestamps from the keyframe indexes of multiple segmented video files, generate an event list arranged in reverse chronological order, and respond to the user's selection operation on the record in the event list to locate the corresponding recording start time for playback.

7. The multi-channel network video recorder system according to claim 1, characterized in that, The bus protocol bridge chip is used to convert transaction layer data packets on the high-speed serial extended bus interface into frame information structures of the serial storage bus protocol. The main control module is also used to complete link training through the high-speed serial extended bus interface when the system starts, establish a data path with the bus protocol bridge chip, and complete device enumeration and block device creation of the serial storage hard disk through the bus protocol bridge chip; The main control module is also used to re-execute device enumeration when a hot-plug event occurs on the serial storage hard disk.

8. A low-cost multi-channel network video recording method based on the RK3576 platform, characterized in that, The method, applied to a multi-channel network video recorder system, includes: Receive encoded video stream data from multiple network cameras within the local area network via Ethernet interface; The encoded video stream data is decoded using the hardware video decoder integrated in the RK3576 processor, and the original image data of each video channel obtained from the decoding is written into the shared output buffer. Using the shared output buffer as a common data source, the motion detection path and the video storage path are driven to work in parallel. In the motion detection path, the raw image data of each video channel is read sequentially from the shared output buffer according to the polling scheduling mechanism, and sent to the neural network processing unit integrated in the RK3576 processor to perform motion detection inference and output the moving target detection result. In the video storage path, the original image data of each video channel is read from the shared output buffer, encapsulated into segmented video files at preset time intervals, and written to the serial storage hard disk via the high-speed serial expansion bus interface and the bus protocol bridge chip. When the motion detection path outputs a motion target detection result indicating the presence of a moving target, a motion event record is generated, and the event timestamp corresponding to the motion event record is directly embedded into the keyframe index of the segmented video file currently being written in the video storage path, establishing a mapping relationship between the video timeline and the event timestamp within the same file.

9. A computer program product containing instructions, characterized in that, The computer program product runs on the multi-channel network video recorder system as described in any one of claims 1-7.

10. A computer-readable storage medium comprising instructions, characterized in that, The instructions are executed on a multi-channel network video recorder system as described in any one of claims 1-7.

Images