Video edge processing method and system

Abstract

The invention belongs to the technical field of video processing, and discloses a video edge processing method and system, and the method comprises the steps: obtaining video data, transmitting the video data to a memory chip, and carrying out the decoding and preprocessing of the video data; storing the decoded video data in a next memory chip, and performing deep analysis processing; and storing the analyzed and processed video data in a next memory chip area, and outputting final video data in combination with a memory mapping technology and a direct memory access technology. According to the invention, by optimizing the cooperative work flow between the NPU and the VPU, the number of times of unnecessary data copying is reduced, so that the delay and bandwidth consumption of data transmission are reduced, and the overall efficiency of the system is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of video processing, and particularly to a method and system for video edge processing. Background Art

[0002] In application scenarios with high real-time requirements such as video surveillance and intelligent security, edge devices (such as intelligent cameras, edge computing nodes, etc.) need to perform real-time processing on a large amount of video data to meet the business requirements of low latency and high throughput.

[0003] The traditional video processing process usually adopts a heterogeneous computing architecture of "CPU + GPU", which involves multiple data copy transmissions between memory and video memory. Since the traditional video processing method usually involves multiple data copies and transmissions, the processing speed is slow and the power consumption is high.

[0004] The NPU (Neural Processing Unit) is mainly used to accelerate neural network operations, while the VPU (Visual Processing Unit) focuses on video encoding, decoding and analysis. How to give full play to the synergistic advantages of the two, reduce data copying, and improve processing efficiency is an urgent problem to be solved currently.

[0005] Therefore, how to provide a method and system for video edge processing is an urgent problem to be solved currently. Summary of the Invention

[0006] Embodiments of the present invention provide a method and system for video edge processing to solve the problem that the existing video processing method usually involves multiple data copies and transmissions, resulting in slow processing speed and high power consumption. The present invention optimizes the data transmission and processing process, reduces data copying, improves video processing efficiency, reduces power consumption, and discloses a method for video edge processing.

[0007] To provide a basic understanding of some aspects of the disclosed embodiments, a simple summary is given below. This summary part is not a general review, nor is it intended to identify key / important constituent elements or delineate the protection scope of these embodiments. Its sole purpose is to present some concepts in a simple form as a prelude to the detailed description that follows.

[0008] According to the first aspect of the embodiments of the present invention, a method for video edge processing is provided.

[0009] In one embodiment, the method for video edge processing includes:

[0010] Obtain video data and transmit it into the memory chip, and decode and preprocess the video data;

[0011] Store the decoded video data in the next memory chip and perform in-depth parsing processing;

[0012] Store the parsed video data in the next memory chip area, and output the final video data by combining memory mapping technology and direct memory access technology.

[0013] In one embodiment, obtaining video data and transmitting it into a memory chip, and decoding and preprocessing the video data includes:

[0014] Obtain video data through a pre-configured acquisition terminal, and transmit the video data into the memory chip through a network interface;

[0015] Parse the video bitstream of the video data transmitted into the memory chip, extract the header information of the video frame, and perform entropy decoding on the video bitstream to recover the original quantization coefficients;

[0016] Perform inverse quantization based on the original quantization coefficients to obtain the original transform coefficients and recover the original video data;

[0017] Perform motion compensation on the video frames in the decoded video data and perform deblocking processing.

[0018] In one embodiment, the formula for performing inverse quantization on the original quantization coefficients is: transform coefficient = quantization coefficient × quantization step size.

[0019] In one embodiment, performing motion compensation on the video frames in the decoded video data and performing deblocking processing includes:

[0020] Perform motion compensation on the video frames to recover the complete video frames;

[0021] Analyze the pixel data of the video frames to find the positions where block effects occur;

[0022] Through a filter, in the area where block effects are detected, smooth the pixel values at the block boundaries to reduce block effects.

[0023] In one embodiment, storing the decoded video data in the next memory chip and performing in-depth parsing processing includes:

[0024] Store the decoded video data in the pre-negotiated next memory chip and read the decoded video data;

[0025] Perform normalization and scaling processing on the read video data, and extract features in the video data by combining a convolutional neural network;

[0026] Input the extracted features into a classifier to identify the target objects in the video data and perform post-processing on the classification results;

[0027] Perform time series analysis on consecutive video frames in the video data to obtain the change and motion information of the video frames.

[0028] In one embodiment, the features in the video data include: the edges, textures, and shapes of the video frames in the video data.

[0029] In one embodiment, performing time series analysis on consecutive video frames in the video data to obtain the change and motion information of the video frames includes:

[0030] Calculate the difference in pixel values between consecutive video frames in the video data to obtain the change information between the video frames;

[0031] Predict the motion vectors of the pixel points in the video frames through the optical flow algorithm to obtain the motion direction and speed of the pixel points;

[0032] Analyze the motion trajectory of the target object based on the motion direction and speed of the pixel points, and judge the motion mode and trend of the object;

[0033] Based on the motion mode and trend of the object, combined with the recurrent neural network, identify the behavior pattern of the target object and perform anomaly detection.

[0034] In one embodiment, based on the motion mode and trend of the object, combined with the recurrent neural network, identifying the behavior pattern of the target object and performing anomaly detection includes:

[0035] Normalize and standardize the video data, construct a recurrent neural network model and train it, and adjust the weights of the recurrent neural network model through the backpropagation algorithm;

[0036] Based on the trained recurrent neural network model, obtain the recognition result of the behavior pattern of the target object;

[0037] Use the features extracted from the video data as the basis for judging anomalies, and set a threshold in combination with the recurrent neural network model;

[0038] If the obtained recognition result is within the set threshold range, the detection is normal, and the subsequent video processing process continues;

[0039] If the obtained recognition result exceeds the threshold range, the detection is abnormal, record the abnormal information and send an alarm notification.

[0040] In one embodiment, storing the parsed and processed video data in the next memory chip area, and combining memory mapping technology and direct memory access technology to output the final video data includes:

[0041] Store the processed video data in the next memory chip area, and fuse the processed video data with the original image data to generate image data with annotation information;

[0042] Entropy encode the image data with annotation information to generate a compressed video bitstream, and encapsulate the generated video bitstream into a standard video format;

[0043] The encoded video data is output through the network interface for other devices to receive and display.

[0044] According to the second aspect of the embodiments of the present invention, a video edge processing system is provided.

[0045] In one embodiment, the video edge processing system includes:

[0046] A video acquisition and processing module, configured to obtain video data and transmit it into the memory chip, and decode and preprocess the video data;

[0047] A video reading and parsing module, configured to store the decoded video data in the next memory chip and perform in-depth parsing processing;

[0048] A video encoding and output module, configured to store the parsed video data in the next memory chip area, and output the final video data by combining memory mapping technology and direct memory access technology.

[0049] According to the third aspect of the embodiments of the present invention, a computer device is provided.

[0050] In some embodiments, the computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the above method are implemented.

[0051] According to the fourth aspect of the embodiments of the present invention, a computer-readable storage medium is provided.

[0052] In one embodiment, a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0053] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects:

[0054] (1) Reduce the number of data copies: By optimizing the collaborative work process between the NPU and the VPU, the present invention reduces the unnecessary number of data copies, thereby reducing the data transmission delay and bandwidth consumption, and improving the overall efficiency of the system.

[0055] (2) Improve video processing efficiency: The optimized collaborative work process of the present invention enables the NPU and the VPU to process video data more efficiently, reduces the processing time, and improves the real-time performance and smoothness of video processing.

[0056] (3) Power consumption reduction: By reducing data copying and optimizing the processing flow, the present invention reduces the energy consumption of the NPU and VPU, and is particularly suitable for edge devices, which usually have strict requirements for power consumption.

[0057] (4) System stability enhancement: The optimized collaborative workflow of the present invention reduces the occupation of system resources, improves the stability and reliability of the system, and reduces system crashes or lags caused by resource competition.

[0058] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present invention, and are used together with the specification to explain the principles of the present invention.

[0060] Figure 1 is a flowchart of a method for video edge processing shown according to an exemplary embodiment;

[0061] Figure 2 is a structural block diagram of a method for video edge processing shown according to an exemplary embodiment;

[0062] Figure 3 is a structural block diagram of a system for video edge processing shown according to an exemplary embodiment;

[0063] Figure 4 is a schematic structural diagram of a computer device shown according to an exemplary embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0064] The following description and the accompanying drawings fully disclose specific embodiments herein, enabling those skilled in the art to practice them. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. The scope of the embodiments herein includes the entire scope of the claims and all available equivalents of the claims. In this document, the terms "first", "second", etc. are only used to distinguish one element from another, without requiring or implying any actual relationship or order between these elements. In fact, the first element can also be called the second element, and vice versa. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a structure, device or equipment comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such structure, device or equipment. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the structure, device or equipment comprising the element. The embodiments herein are described in a progressive manner, with each embodiment highlighting the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other.

[0065] In this document, the orientation or positional relationships indicated by the terms "longitudinal", "lateral", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. are based on the orientation or positional relationships shown in the drawings, and are only for the convenience of describing this document and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation of the present invention. In the description herein, unless otherwise specified and limited, the terms "mounted", "connected", "coupled" shall be understood in a broad sense. For example, it can be a mechanical connection or an electrical connection, or can also be the communication inside two elements. It can be directly connected or indirectly connected through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to specific circumstances.

[0066] In this document, unless otherwise stated, the term "plurality" means two or more.

[0067] In this document, the character " / " indicates that the objects before and after are in an "or" relationship. For example, A / B means: A or B.

[0068] In this document, the term "and / or" is an associative relationship describing an object, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B these three relationships.

[0069] It should be understood that although the various steps in the flowchart are shown sequentially according to the indication of the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless there is a clear indication in this article, there is no strict order limit for the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the figure may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0070] Each module in the device or system of the present application can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in the processor in the computer device in hardware form or be independent of it, or can be stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0071] Without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0072] Figure 1 and Figure 2 An embodiment of a method for video edge processing according to the present invention is shown.

[0073] In this alternative embodiment, the method for video edge processing includes:

[0074] Step S101, obtaining video data and transmitting it into the memory chip, and decoding and preprocessing the video data;

[0075] Step S102, storing the decoded video data in the next memory chip and performing in-depth parsing processing;

[0076] Step S103, storing the video data after parsing processing in the area of the next memory chip and outputting the final video data by combining memory mapping technology and direct memory access technology.

[0077] In this alternative embodiment, obtaining video data and transmitting it into the memory chip, and decoding and preprocessing the video data includes: obtaining video data through a pre-configured acquisition terminal and transmitting the video data into the memory chip through a network interface; parsing the video stream of the video data transmitted into the memory chip, extracting the header information of the video frame, and performing entropy decoding on the video stream to restore the original quantization coefficients; performing inverse quantization based on the original quantization coefficients to obtain the original transform coefficients and restoring the original video data; performing motion compensation on the video frames in the decoded video data and performing deblocking effect processing.

[0078] In this alternative embodiment, the formula for inverse quantization of the original quantization coefficients is: transform coefficient = quantization coefficient × quantization step size.

[0079] In this alternative embodiment, performing motion compensation on the video frames in the decoded video data and performing deblocking processing includes: performing motion compensation on the video frames to restore the complete video frames; analyzing the pixel data of the video frames to find the positions where blocking effects occur; passing through a filter to smooth the pixel values at the block boundaries in the detected blocking effect regions to reduce the blocking effects.

[0080] In this alternative embodiment, storing the decoded video data in the next memory chip and performing in-depth parsing processing includes: storing the decoded video data in the pre-negotiated next memory chip and reading the decoded video data; performing normalization and scaling processing on the read video data, and combining a convolutional neural network to extract features in the video data; inputting the extracted features into a classifier to identify the target objects in the video data and performing post-processing on the classification results; performing time series analysis on consecutive video frames in the video data to obtain the changes and motion information of the video frames.

[0081] In this alternative embodiment, the features in the video data include: the edges, textures, and shapes of the video frames in the video data.

[0082] In this alternative embodiment, performing time series analysis on consecutive video frames in the video data to obtain the changes and motion information of the video frames includes: calculating the difference in pixel values between consecutive video frames in the video data to obtain the change information between the video frames; predicting the motion vectors of the pixel points in the video frames through an optical flow algorithm to obtain the motion directions and speeds of the pixel points; analyzing the motion trajectories of the target objects based on the motion directions and speeds of the pixel points to judge the motion patterns and trends of the objects; based on the motion patterns and trends of the objects, combining a recurrent neural network to identify the behavior patterns of the target objects and perform anomaly detection.

[0083] In this alternative embodiment, based on the motion patterns and trends of the objects, combining a recurrent neural network to identify the behavior patterns of the target objects and perform anomaly detection includes: performing normalization and standardization processing on the video data, constructing and training a recurrent neural network model, and adjusting the weights of the recurrent neural network model through the backpropagation algorithm; based on the trained recurrent neural network model, obtaining the recognition results of the behavior patterns of the target objects; using the features extracted from the video data as the basis for judging anomalies, and setting a threshold in combination with the recurrent neural network model; if the obtained recognition results are within the set threshold range, the detection is normal and the subsequent video processing process continues; if the obtained recognition results exceed the threshold range, the detection is abnormal, and the abnormal information is recorded and an alarm notification is sent.

[0084] In this alternative embodiment, the processed video data after parsing is stored in the next memory chip area, and the final video data is output by combining memory mapping technology and direct memory access technology, including: storing the processed video data in the next memory chip area, and fusing the processed video data with the original image data to generate image data with annotation information; performing entropy coding on the image data with annotation information to generate a compressed video bitstream, and encapsulating the generated video bitstream into a standard video format; the encoded video data is output through a network interface for other devices to receive and display.

[0085] Figure 3 An embodiment of a video edge processing system of the present invention is shown.

[0086] In this alternative embodiment, the video edge processing system includes:

[0087] A video acquisition and processing module 201, configured to acquire video data and transmit it into a memory chip, and decode and preprocess the video data;

[0088] A video reading and parsing module 202, configured to store the decoded video data in the next memory chip and perform in-depth parsing processing;

[0089] A video encoding and output module 203, configured to store the processed video data in the next memory chip area, and output the final video data by combining memory mapping technology and direct memory access technology.

[0090] To facilitate understanding of the above technical solution of the present invention, the above technical solution of the present invention will be further described from the perspectives of architecture and principle as follows:

[0091] 1) Data acquisition and transmission: The camera acquires video data and transmits the data to the DDR (memory chip) through Ethernet.

[0092] 2) VPU (Visual Processing Unit) decoding and preliminary processing: The VPU (Visual Processing Unit) reads the video data from the DDR (memory chip) for decoding, and stores the decoded data in a pre-negotiated DDR (memory chip) area.

[0093] Reading video data: The VPU reads the video data from the DDR. The video data is usually stored in the DDR in a compressed format (such as H.264, H.265, etc.); the VPU reads the video data from the DDR through direct memory access (DMA) technology, reducing the intervention of the CPU and improving the data transmission efficiency.

[0094] Decoding process: Bitstream parsing. The VPU first parses the video bitstream to extract the header information of the video frame, including frame type (I-frame, P-frame, B-frame), timestamp, resolution, etc.; Entropy decoding. The bitstream is entropy decoded, such as using the inverse process of Huffman coding or arithmetic coding, to recover the original quantization coefficients.

[0095] Inverse quantization of quantization coefficients. In the video decoding process, the inverse quantization of quantization coefficients is an important step in recovering the original image data. The specific process is as follows:

[0096] Obtain quantization parameters: Extract quantization parameters from the video bitstream. These parameters are used to control the size of the quantization step during the encoding process.

[0097] Read quantization matrix: Obtain the quantization matrix used during encoding. This matrix defines the quantization steps for different frequency coefficients.

[0098] Inverse quantization calculation: Perform an inverse quantization operation on the quantized coefficients. The formula is: Inverse quantization coefficient = Quantization coefficient × Quantization step. Through this calculation, the original transform coefficients are recovered.

[0099] This process performs inverse quantization on the quantization coefficients according to the quantization parameters and the quantization matrix to restore the image data.

[0100] Motion compensation: For P-frames and B-frames, perform motion compensation to recover the complete image frame.

[0101] Deblocking: Perform deblocking on the decoded image to improve the image quality.

[0102] Deblocking processing is used to improve the quality of the image after video decoding and reduce blocky artifacts. The specific process is as follows:

[0103] Block effect detection: Analyze the pixel data of the image to find the locations where block effects occur. Usually, sudden changes in pixel values are detected at block boundaries.

[0104] Filter application: Apply a filter, such as a low-pass filter, to the areas where block effects are detected to smooth the pixel values at the block boundaries and reduce block effects.

[0105] Intensity adjustment: Adjust the intensity of the filter according to the severity of the block effect to achieve a balance between removing block effects and maintaining image details.

[0106] This process reduces block effects through detection and filtering to improve the image quality.

[0107] Store the decoded data: The decoded image data is stored in the pre-negotiated DDR area. These areas are usually divided into multiple buffers for subsequent processing.

[0108] 3) NPU (Neural Processing Unit) Parsing and Processing: The NPU (Neural Processing Unit) reads data from the DDR (Memory Chip) area processed by the VPU (Visual Processing Unit), performs in-depth parsing and processing, and stores the processing results in the next DDR (Memory Chip) area.

[0109] Reading Decoded Data: The NPU reads the decoded image data from the DDR area processed by the VPU. These data are usually stored in the YUV format.

[0110] Preprocessing: Perform preprocessing operations such as normalization and scaling on the image data to make it meet the input requirements of the neural network.

[0111] Feature Extraction: Use a Convolutional Neural Network (CNN) to extract features in the image, such as edges, textures, shapes, etc.

[0112] Classification: Input the extracted features into a classifier to identify the target objects in the image.

[0113] Postprocessing: Perform postprocessing on the classification results, such as Non-Maximum Suppression (NMS), to remove duplicate detection boxes.

[0114] Time Series Analysis: Perform time series analysis on consecutive image frames to capture the changes and motion information between frames. The specific process is as follows:

[0115] Frame Difference Calculation: Calculate the difference between adjacent frames to obtain the change information between frames, which can be achieved by calculating the difference in pixel values.

[0116] Optical Flow Estimation: Use an optical flow algorithm to estimate the motion vectors of pixel points, such as the Horn-Schunck algorithm or the Lucas-Kanade algorithm, to obtain the motion direction and speed of pixel points.

[0117] Motion Trajectory Analysis: Analyze the motion trajectory of an object based on the results of optical flow estimation to judge the motion pattern and trend of the object.

[0118] Behavior Recognition: Use a Recurrent Neural Network (RNN) or a Long Short-Term Memory Network (LSTM) to recognize the behavior patterns of objects. The specific process is as follows:

[0119] Data Preprocessing: Normalize and standardize the time series data to make it meet the input requirements of the neural network.

[0120] Network Construction: Construct an RNN or LSTM network and define parameters such as the number of network layers, the number of neurons, and the activation function.

[0121] Training process: Use the labeled training data to train the network, and adjust the weights of the network through the backpropagation algorithm so that the network can accurately identify behavior patterns.

[0122] Inference process: Input the time series data to be recognized into the trained network to obtain the output of the network, that is, the recognition result of the behavior pattern of the object.

[0123] Anomaly detection: During video processing, detect and handle abnormal situations. The specific process is as follows:

[0124] Feature extraction: Extract features from video data, such as pixel values, motion vectors, etc., as the basis for judging anomalies.

[0125] Anomaly judgment: According to the set threshold or model, judge whether the extracted features exceed the normal range.

[0126] Result processing: If the detection is normal, continue with the subsequent video processing process; if the detection is abnormal, trigger the corresponding alarm or processing mechanism, such as recording abnormal information, sending alarm notifications, etc.

[0127] Store the processing results: The processing results (such as object detection frames, behavior recognition results, etc.) are stored in the next DDR area for subsequent processing.

[0128] 4) VPU (Visual Processing Unit) encoding and output: The VPU (Visual Processing Unit) reads data from the DDR (memory chip) area processed by the NPU (Neural Processing Unit), performs encoding processing, and outputs the encoded video data.

[0129] Read the processed data: The VPU reads the processed data from the DDR area processed by the NPU. These data usually include object detection frames, behavior recognition results, etc.

[0130] Encoding processing: Data fusion, fuse the processed data with the original image data to generate an image with annotation information; Entropy encoding, perform entropy encoding on the fused image data, such as using Huffman encoding or arithmetic encoding, to generate a compressed bitstream; Bitstream encapsulation, encapsulate the generated bitstream into a standard video format (such as H.264, H.265, etc.).

[0131] Output video data: The encoded video data is output through a network interface (such as Ethernet) for other devices (such as a monitoring center) to receive and display, reducing the implementation of data copying.

[0132] Memory mapping: Through memory mapping technology, the VPU and NPU can directly access the data in the DDR without the need for the data to be copied multiple times between the DDR and the processing unit; the memory mapping technology allows the VPU and NPU to share the same DDR area, reducing the overhead of data transmission.

[0133] Buffer management: Using circular buffer or double buffer technology to ensure that the data does not need to be copied multiple times during the processing; the circular buffer can be dynamically allocated and released to ensure the continuity and real-time nature of the data.

[0134] Direct Memory Access (DMA): The VPU and NPU directly access the data in the DDR through DMA technology, reducing the intervention of the CPU and improving the data transmission efficiency; the DMA technology allows data to be directly transmitted between different modules without the participation of the CPU, further reducing the number of data copies. The specific process steps are as follows:

[0135] Data acquisition and transmission: The camera captures video data and transmits the data to the DDR (memory chip) through Ethernet.

[0136] VPU decoding and preliminary processing: The VPU reads the video data from the DDR and performs decoding processing; the decoded image data is stored in a pre-negotiated DDR area.

[0137] NPU parsing and processing: The NPU reads the decoded image data from the DDR area processed by the VPU and performs in-depth parsing and processing; the processing results are stored in the next DDR area.

[0138] VPU encoding and output: The VPU reads the processed data from the DDR area processed by the NPU and performs encoding processing; the encoded video data is output through the network interface.

[0139] Through the above technical means, the present invention significantly reduces the number of copies of data between the DDR and the processing unit, improving the efficiency and real-time nature of video processing.

[0140] In one embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 4As shown. The computer device includes a processor, a memory, and a network interface connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store static information and dynamic information data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, it implements the steps in the above method embodiments.

[0141] Those skilled in the art can understand that Figure 4 the structure shown in is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0142] In addition, the present invention also provides a computer device, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, it implements the steps in the above method embodiments.

[0143] In addition, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by the processor, it implements the steps in the above method embodiments.

[0144] Those of ordinary skill in the art can understand that all or part of the processes in the above method embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above method embodiments. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided by the present invention can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0145] The present invention is not limited to the structures that have been described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for video edge processing, characterized in that, Including: Obtain video data and transfer it into the memory chip, and decode and preprocess the video data; Store the decoded video data in the next memory chip and perform in-depth parsing processing; Store the video data after parsing processing in the area of the next memory chip, and combine memory mapping technology and direct memory access technology to output the final video data.

2. The method for video edge processing according to claim 1, wherein, The obtaining video data and transferring it into the memory chip, and decoding and preprocessing the video data include: Obtain video data through a pre-configured acquisition terminal and transfer the video data into the memory chip through a network interface; Parse the video stream of the video data transferred into the memory chip, extract the header information of the video frame, and perform entropy decoding on the video stream to recover the original quantization coefficients; Perform inverse quantization based on the original quantization coefficients to obtain the original transform coefficients and recover the original video data; Perform motion compensation on the video frames in the decoded video data and perform deblocking processing.

3. The method for video edge processing according to claim 2, wherein The formula for performing inverse quantization on the original quantization coefficients is: transform coefficient = quantization coefficient × quantization step size.

4. The method for video edge processing according to claim 2, wherein The performing motion compensation on the video frames in the decoded video data and performing deblocking processing include: Perform motion compensation on the video frames to recover complete video frames; Analyze the pixel data of the video frames to find the positions where block effects occur; Through a filter, in the area where block effects are detected, smooth the pixel values at the block boundaries to reduce block effects.

5. The method for video edge processing according to claim 1, wherein The storing the decoded video data in the next memory chip and performing in-depth parsing processing includes: Store the decoded video data in the next memory chip negotiated in advance and read the decoded video data; Perform normalization and scaling processing on the read video data, and combine a convolutional neural network to extract features in the video data; Input the extracted features into a classifier to identify the target objects in the video data and perform post-processing on the classification results; Perform time series analysis on consecutive video frames in the video data to obtain the changes and motion information of the video frames.

6. The method for video edge processing according to claim 5, wherein The features in the video data include: edges, textures, and shapes of video frames in the video data.

7. The method for video edge processing according to claim 5, wherein The performing time series analysis on consecutive video frames in the video data to obtain the changes and motion information of the video frames includes: Calculate the difference in pixel values between consecutive video frames in the video data to obtain the change information between video frames; Predict the motion vectors of pixel points in the video frames through an optical flow algorithm to obtain the motion directions and speeds of the pixel points; According to the motion directions and speeds of the pixel points, analyze the motion trajectories of the target objects and judge the motion patterns and trends of the objects; Based on the motion patterns and trends of the objects, combine a recurrent neural network to identify the behavior patterns of the target objects and perform anomaly detection.

8. The method for video edge processing according to claim 7, wherein The based on the motion patterns and trends of the objects, combining a recurrent neural network to identify the behavior patterns of the target objects and perform anomaly detection includes: Perform normalization and standardization processing on the video data, construct a recurrent neural network model and train it, and adjust the weights of the recurrent neural network model through the backpropagation algorithm; Based on the trained recurrent neural network model, obtain the recognition results of the behavior patterns of the target objects; Extract features from the video data as the basis for anomaly judgment, and set a threshold in combination with a recurrent neural network model; If the obtained recognition result is within the set threshold range, the detection is normal, and the subsequent video processing flow continues; If the obtained recognition result exceeds the threshold range, the detection is abnormal, and the abnormal information is recorded and an alarm notification is sent.

9. The method for video edge processing according to claim 1, wherein Storing the parsed and processed video data in the next memory chip area, and outputting the final video data in combination with memory mapping technology and direct memory access technology includes: Storing the processed video data in the next memory chip area, and fusing the processed video data with the original image data to generate image data with annotation information; Performing entropy coding on the image data with annotation information to generate a compressed video bitstream, and encapsulating the generated video bitstream into a standard video format; The encoded video data is output through a network interface for other devices to receive and display.

10. A system for video edge processing, characterized in that, Including: A video acquisition and processing module for acquiring video data and transmitting it to the memory chip, and decoding and preprocessing the video data; A video reading and parsing module for storing the decoded video data in the next memory chip and performing in-depth parsing processing; A video encoding and output module for storing the parsed and processed video data in the next memory chip area, and outputting the final video data in combination with memory mapping technology and direct memory access technology.

Images