Intelligent vision-oriented event triggering type monitoring video coding method
Through the event-triggered monitoring video encoding method, event fragments in the monitoring video frame are identified and background and foreground data are encoded separately. The problem of redundancy in monitoring video encoding is solved by using neural networks and sparse encoding technology, and an efficient encoding process is realized.
Patent Information
- Application Number
- CN202510589690.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-08
- Publication Date
- 2025-08-01
AI Technical Summary
The existing surveillance video encoding technology has large bit overhead, resulting in redundancy problems.
The event-triggered monitoring video encoding method is used to identify event segments of the monitoring video frames, encode background and foreground data respectively, and use neural network models to decode background and foreground encoder, and combine sparse encoding and motion encoding technology to optimize the encoding process.
The bit overhead of monitoring video encoding is reduced, encoding redundancy is reduced, and encoding efficiency is improved.
Smart Images
Figure CN120416486A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular, to an event-triggered monitoring video encoding method for intelligent vision. Background Art
[0002] The data volume of the monitoring video stream is very large. However, since the monitoring video is captured by a fixed camera and the background content is relatively stable, there is a large amount of redundancy in the prior art when using the visual signal representation method to encode the monitoring video, resulting in unnecessary bit overhead during encoding. Summary of the Invention
[0003] The present invention provides an event-triggered monitoring video encoding method for intelligent vision, which is used to solve the defect of large bit overhead during the encoding of the monitoring video in the prior art, and realizes the reduction of the bit overhead of the monitoring video encoding.
[0004] The present invention provides an event-triggered monitoring video encoding method for intelligent vision, including: Obtain the monitoring video to be encoded captured by the target camera, and determine the event segment based on the video frames in the monitoring video to be encoded; Encode the video frames in the event segment to obtain an encoding result, where the encoding result includes the foreground encoding data and background encoding data of the video frames; Among them, encoding the target video frame at time T in the event segment includes: Extract the background data from the target video frame, input the background data into the background encoder for encoding, and obtain the background encoding data of the target video frame; Determine the foreground data of the target video frame based on the background data of the target video frame, and encode the foreground data to obtain the foreground encoding data of the target video frame.
[0005] According to an event-triggered monitoring video encoding method for intelligent vision provided by the present invention, the determining the foreground data of the target video frame based on the background data of the target video frame includes: Input the background encoding data of the target video frame into the background decoder to obtain the background reconstruction data of the target video frame; Determine the foreground data of the target video frame based on the target video frame and the background reconstruction data of the target video frame.
[0006] According to an event-triggered monitoring video encoding method for intelligent vision provided by the present invention, the foreground encoding data includes sparse encoding data and motion encoding data; the encoding the foreground data to obtain the foreground encoding data of the target video frame includes: Perform sparse representation on the foreground data to obtain sparse coefficients; Input the foreground data to be encoded into a sparse encoder to obtain the sparse encoded data, where the foreground data to be encoded at least includes the sparse coefficients; Perform motion estimation based on the foreground data to obtain motion information, and input the motion information into a motion encoder to obtain the motion encoded data.
[0007] According to an event-triggered monitoring video encoding method for intelligent vision provided by the present invention, the performing motion estimation based on the foreground data to obtain motion information includes: Perform motion estimation based on the foreground reconstruction data of the target video frame and the reference video frame to obtain the motion information. The reference video frame includes the monitoring video to be encoded and the video frame at time T-1 in the adjacent monitoring video, and the adjacent monitoring video is the video captured by an adjacent camera of the target camera. Among them, the foreground reconstruction data of the video frame is obtained by decoding the foreground encoded data of the video frame; The foreground data to be encoded further includes temporal neighborhood information. Before inputting the foreground data to be encoded into a sparse encoder to obtain the sparse encoded data, it includes: Input the motion encoded data into a motion decoder to obtain motion reconstruction data; Determine the temporal neighborhood information based on the motion reconstruction data and the foreground reconstruction data of the reference video frame.
[0008] The present invention also provides a video decoding method based on the above-mentioned event-triggered monitoring video encoding method for intelligent vision, including: Input the background encoded data of the target video frame into a background decoder to obtain the background reconstruction data of the target video frame; Input the sparse encoded data of the target video frame into a sparse decoder to obtain sparse reconstruction data; Input the temporal neighborhood information and the sparse encoded data of the target video frame into a sparse decoder to obtain sparse decoded data, and perform sparse reconstruction on the sparse decoded data to obtain the foreground reconstruction data of the target video frame; Obtain the decoding result of the target video frame based on the background reconstruction data and the foreground reconstruction data of the target video frame.
[0009] A video decoding method provided by the present invention, wherein the background encoder, the background decoder, the sparse encoder, the sparse decoder, the motion encoder, and the motion decoder are neural network models trained based on multiple sets of sample data, and each set of sample data includes a sample surveillance video to be encoded and a decoding result corresponding to the sample surveillance video to be encoded; During the training process, the training loss includes a first loss and a second loss. The first loss reflects the distortion degree of the video frame decoding result, and the second loss reflects the number of encoded bits.
[0010] The present invention also provides an event-triggered surveillance video encoding device for intelligent vision, including: An event recognition module, configured to obtain a surveillance video to be encoded captured by a target camera, and determine an event segment based on video frames in the surveillance video to be encoded; An encoding module, configured to encode video frames in the event segment to obtain an encoding result, where the encoding result includes foreground encoding data and background encoding data of the video frames; Among them, encoding the target video frame at time T in the event segment includes: Extracting background data from the target video frame, inputting the background data into a background encoder for encoding to obtain the background encoding data of the target video frame; Determining foreground data of the target video frame based on the background data of the target video frame, and encoding the foreground data to obtain the foreground encoding data of the target video frame.
[0011] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements any one of the above-mentioned event-triggered surveillance video encoding methods for intelligent vision and / or the video decoding method as described in any one of the above.
[0012] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements any one of the above-mentioned event-triggered surveillance video encoding methods for intelligent vision and / or the video decoding method as described in any one of the above.
[0013] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements any one of the above-mentioned event-triggered surveillance video encoding methods for intelligent vision.
[0014] The event-triggered monitoring video encoding method for intelligent vision provided by the present invention includes: obtaining the monitoring video to be encoded captured by a target camera, and determining event segments based on the video frames in the monitoring video to be encoded; encoding the video frames in the event segments to obtain an encoding result, where the encoding result includes foreground encoding data and background encoding data of the video frames; among them, encoding the target video frame at time T in the event segment includes: extracting background data from the target video frame, inputting the background data into a background encoder for encoding to obtain the background encoding data of the target video frame; determining the foreground data of the target video frame based on the background data of the target video frame, and encoding the foreground data to obtain the foreground encoding data of the target video frame. The present invention encodes the monitoring video when an event occurs to achieve event-triggered encoding, and encodes the background and foreground of the video frames respectively to encode the event segments in the monitoring video, so that different degrees of encoding processing can be performed on the background and foreground, reducing the encoding redundancy of the monitoring video and reducing the bit overhead of the monitoring video encoding. Description of the Drawings
[0015] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.
[0016] Figure 1 It is a schematic flowchart of the event-triggered monitoring video encoding method for intelligent vision provided by the present invention.
[0017] Figure 2 It is a schematic diagram of an event segment in the event-triggered monitoring video encoding method for intelligent vision provided by the present invention.
[0018] Figure 3 It is a schematic diagram of the encoding process of a target video frame in the event-triggered monitoring video encoding method for intelligent vision provided by the present invention.
[0019] Figure 4 It is a schematic diagram of the structures of an encoder and a decoder in the event-triggered monitoring video encoding method for intelligent vision provided by the present invention.
[0020] Figure 5 It is a schematic diagram of the process of sparse representation and sparse reconstruction in the event-triggered monitoring video encoding method for intelligent vision provided by the present invention.
[0021] Figure 6 It is a schematic diagram of the process of mining temporal neighborhood information in the event-triggered monitoring video encoding method for intelligent vision provided by the present invention.
[0022] Figure 7 It is a schematic structural diagram of an event-triggered monitoring video encoding device for intelligent vision provided by the present invention.
[0023] Figure 8 It is a schematic structural diagram of an electronic device provided by the present invention. Specific embodiments
[0024] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Obviously, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without making creative efforts shall fall within the protection scope of the present invention.
[0025] The following combines Figures 1-6 to describe the event-triggered monitoring video encoding method for intelligent vision provided by the present invention. As Figure 1 shown, the method includes the steps: S110. Obtain the monitoring video to be encoded captured by the target camera, and determine the event segments based on the video frames in the monitoring video to be encoded; S120. Encode the video frames in the event segments to obtain an encoding result, where the encoding result includes the foreground encoding data and background encoding data of the video frames.
[0026] Among them, the process of encoding the target video frame at time T in the time segment includes the steps: Extract the background data of the target video frame, input the background data into the background encoder for encoding, and obtain the background encoding data of the target video frame; Determine the foreground data of the target video frame based on the background data of the target video frame, and encode the foreground data to obtain the foreground encoding data of the target video frame.
[0027] The method provided by the present invention encodes the monitoring video when an event occurs, realizes event-triggered encoding, and reconstructs the background and foreground of the video frames respectively to encode the event segments in the monitoring video, so that different degrees of encoding processing can be performed on the background and foreground, reducing the encoding redundancy of the monitoring video and reducing the bit overhead of the monitoring video encoding.
[0028] As Figure 2 shown, the method provided by the present invention decomposes the monitoring video to be encoded into event segments and non-event segments. Specifically, determining the event segments based on the video frames in the monitoring video to be encoded includes: When the difference between the first video frame in the surveillance video to be encoded and the background model corresponding to the first video frame is greater than the first preset threshold, determine the first video frame as the starting frame of the event segment, and the background model corresponding to the first video frame is determined based on the non-event segments between the first video frames; When the difference between the second video frame in the surveillance video to be encoded and the first video frame is less than the second preset threshold, determine the second video frame as the ending frame of the event segment, and the time of the second video frame is later than that of the first video frame.
[0029] That is to say, the event trigger information can be defined as the difference between the current video frame and the current background model being greater than the set threshold, which is the event start time. The event result time can be defined as the difference between the current video frame and the event start time being less than the set threshold. The process of determining the event segment can be expressed by a mathematical formula as: ; ; where t represents the current time, F(t) represents the surveillance video frame at the current time, B represents the background model constructed before the current time t, which is constructed by a Gaussian mixture model; Ψ() represents the pixel domain difference; Φ() represents the feature domain difference, which can be represented by the intermediate layer features of the VGG16 network; η represents the set first preset threshold, a represents the event start time, b represents the event end time, F(a) represents the video frame at the event start time, and γ represents the set second preset threshold.
[0030] As Figure 3 shown, in the method provided by the present invention, for the target video frame at time T in the event segment, the background data and the foreground data are encoded separately to obtain the background encoded data and the foreground encoded data of the target video frame. When decoding the encoding result of the video frame, the background encoded data and the foreground encoded data are decoded separately, and after obtaining the background reconstruction data and the foreground reconstruction data, they are fused to obtain the decoding result. The following details the encoding process provided by the present invention. It should be noted that when encoding at the encoding end, decoding data is actually also used, and the acquisition method of the decoding data used when encoding at the encoding end is the same as the method of decoding at the decoding end.
[0031] The background data B of the target video frame at time T T is encoded by the background encoder to obtain the background encoded data ( Figure 3 the background bitstream in), and then reconstructed by the background decoder to obtain the background reconstruction data of the target video frame at time T .
[0032] The background encoder and the background decoder can be neural network modules. Among them, the background encoder and the background decoder can be as Figure 4 shown.Figure 4 The encoder and the entropy encoder in Figure 4 constitute the background encoder, and the decoder and the entropy decoder in
[0033] In Figure 4 the background reconstruction module of , Convk5s2 represents a convolutional operation with a kernel size of 5x5 and a stride of 2, RBs represents a residual module, TConv represents a transposed convolution, ReLU represents an activation function, Q represents a quantization operation, AE represents an arithmetic encoder, and AD represents an arithmetic decoder.
[0034] In one possible implementation, the foreground data of the target video frame can be directly obtained based on the target video frame and the background data of the target video frame. In another possible implementation, the foreground data of the target video frame can be obtained by subtracting the target video frame from the background reconstruction data of the target video frame. That is, determining the foreground data of the target video frame based on the background data of the target video frame includes: Inputting the background encoded data of the target video frame into the background decoder to obtain the background reconstruction data of the target video frame; Determining the foreground data of the target video frame based on the target video frame and the background reconstruction data of the target video frame.
[0035] After determining the foreground data, encoding the foreground data to obtain the foreground encoded data of the target video frame. The foreground encoded data includes sparse encoded data and motion encoded data. The process of obtaining the foreground encoded data of the target video frame specifically includes: Performing sparse representation on the foreground data to obtain sparse coefficients; Inputting the foreground data to be encoded into the sparse encoder to obtain sparse encoded data. The foreground data to be encoded includes at least the sparse coefficients; Performing motion estimation based on the foreground data to obtain motion information, and inputting the motion information into the motion encoder to obtain motion encoded data.
[0036] Video frame X at time T T is subtracted from the event background reconstruction data at time T to obtain the event foreground data F at time T T , and the event foreground data F at time T T is sparsely represented to obtain corresponding sparse coefficients.
[0037] Since the surveillance video may involve personal privacy, in the method provided by the present invention, during the process of encoding and decoding the foreground data, sparse representation is first performed on the foreground data. Specifically, performing sparse representation on the foreground data to obtain sparse coefficients includes: Cropping the foreground data to obtain multiple sub-blocks; Using a preset sparse dictionary, sparse representation is performed on the sub-blocks through the orthogonal matching pursuit algorithm.
[0038] In order to hide personal privacy, the foreground of the dictionary-based event (personal privacy information) is represented as sparse coefficients. That is, the spatial domain content information image is cropped into units of 256×256. Currently, sub-blocks of 16×16 size use the trained dictionary (the dictionary basis size is 16×16) and perform sparse representation through the orthogonal matching pursuit algorithm, which can be expressed by the mathematical formula: ; where M represents the original 16×16 sub-block, M’ = Dα is the encoded block after sparse reconstruction, D represents the trained dictionary, α represents the sparse coefficient, and the L0 norm represents the number of non-zero sparse coefficients. Therefore, the multi-modal signal representation effectively hides personal privacy and also reduces the representation redundancy.
[0039] In a possible implementation, the foreground data to be encoded may only include sparse coefficients. The sparse coefficients pass through a sparse encoder to obtain sparse encoded data (the sparse bitstream in Figure 3 ). When decoding the sparse encoded data, the sparse encoded data passes through a sparse decoder to obtain sparse decoded data, and then the reconstructed sparse coefficients pass through sparse reconstruction to obtain the foreground reconstruction data of the T-th moment . In this process, the sparse representation and sparse reconstruction are represented by Figure 5 . In a possible implementation, the sparse encoder and the sparse decoder can adopt Figure 4 an end-to-end neural network encoder and decoder, that is, the structures of the foreground reconstruction module and the background reconstruction module are the same. Of course, it can be understood that other codec structures can also be adopted.
[0040] In another possible implementation, the foreground data to be encoded further includes temporal neighborhood information, which reflects the information of the adjacent domains of the target video frame in time and space. Specifically, in the existing encoding methods, only the content of a single camera can be processed separately, and the correlation between adjacent cameras cannot be effectively utilized, resulting in the inability to eliminate the cross-camera information redundancy and further reducing the encoding efficiency. To address this defect, in an implementation of the method provided by the present invention, based on the videos captured by the adjacent cameras of the target camera and the video frame at the previous moment of the target video frame, temporal neighborhood information is further extracted, and this information is used to optimize the sparse encoder and the sparse decoder. Specifically, before inputting the foreground data to be encoded into the sparse encoder to obtain sparse encoded data, it includes: Inputting the motion encoded data into the motion decoder to obtain motion reconstruction data; Determine the temporal neighborhood information based on the motion reconstruction data and the foreground reconstruction data of the reference video frames.
[0041] After extracting the foreground reconstruction data at each moment in the event segment of the target surveillance video, send the foreground reconstruction data at this moment to the event foreground reconstruction cache. For the adjacent surveillance videos captured by the adjacent cameras of the target camera, the foreground reconstruction data at each moment in the event segment of the adjacent surveillance videos can be obtained in the same way and also sent to the event foreground reconstruction cache. The event foreground data F at time T T and the information in the event foreground reconstruction cache , Obtain the motion information S at the current moment T through motion estimation T , the motion information S T reflects the object motion in the event from time T - 1 to time T. The motion information passes through a motion encoder to obtain motion encoded data ( Figure 3 the motion bitstream in), and the motion encoded data passes through a motion decoder to obtain the motion reconstruction data at the current moment , as Figure 3 shown, and then the motion reconstruction data at the current moment and the information in the event foreground reconstruction cache , obtain the prior information as the temporal neighborhood information through temporal neighborhood information mining for the optimization of the sparse codec. In this process, the motion encoder and the motion decoder can use Figure 4 an end-to-end neural network encoder and decoder. The temporal neighborhood information mining can be represented by Figure 6 .
[0042] Specifically, Figure 6 in represents the foreground reconstruction data of the video frame at time t - 1 in the target surveillance video, represents the foreground reconstruction data of the video frame at time t - 1 in the adjacent surveillance video captured by the adjacent camera, represents the motion reconstruction data at time t. Warp represents transforming the input features through the motion reconstruction data , the circled + represents feature addition, and the circled C represents feature fusion.
[0043] The method provided by the present invention, by constructing a hierarchical coding framework, encodes the background information and the foreground information separately, fully explores the information redundancy in the time dimension and across cameras of the surveillance video, and realizes the efficient compression and transmission of the surveillance video.
[0044] In the method provided by the present invention, the background encoder, the background decoder, the sparse encoder, the sparse decoder, the motion encoder, and the motion decoder can all adopt an end-to-end neural network module. Before encoding the video frames in the event segment to obtain the encoding result, it includes: Training the background encoder, the background decoder, the sparse encoder, the sparse decoder, the motion encoder, and the motion decoder based on multiple groups of sample data, where each group of sample data includes the sample surveillance video to be encoded and the decoding result corresponding to the sample surveillance video to be encoded; Among them, during the training process, the training loss includes a first loss and a second loss. The first loss reflects the distortion degree of the video frame decoding result, and the second loss reflects the number of encoding bits.
[0045] The training loss can be expressed by the formula: .
[0046] Where λ represents the weight between distortion and encoding bits. Since the surveillance video is not provided for consumers to watch, further defining the distortion of the image in combination with the use of the surveillance video can improve the performance of the surveillance video encoding result in different tasks. In a possible implementation manner, the distortion is further expressed as the difference in the pixel domain and the task distortion, that is: ; Where D mse represents the pixel domain distortion, D segmentation represents the segmentation task distortion, D detection represents the detection task distortion, D analysis represents the analysis task distortion.
[0047] Next, the video decoding method provided by the present invention will be described. The decoding method includes: Inputting the background encoding data of the target video frame into the background decoder to obtain the background reconstruction data of the target video frame; Inputting the sparse encoding data of the target video frame into the sparse decoder to obtain the sparse reconstruction data; Inputting the temporal neighborhood information and the sparse encoding data of the target video frame into the sparse decoder to obtain the sparse decoding data, and performing sparse reconstruction on the sparse decoding data to obtain the foreground reconstruction data of the target video frame; Based on the background reconstruction data and the foreground reconstruction data of the target video frame, obtain the decoding result of the target video frame.
[0048] In the decoding method provided by the present invention, the foreground encoded data and the background encoded data are decoded respectively to obtain the foreground reconstructed data and the background reconstructed data. Then, the foreground-background data and the background reconstructed data are fused to obtain the decoding result. The process of decoding the foreground encoded data and the background encoded data respectively to obtain the foreground reconstructed data and the background reconstructed data has been described in the foregoing description of the encoding method and will not be elaborated herein.
[0049] The event-triggered monitoring video encoding device for intelligent vision provided by the present invention will be described below. The event-triggered monitoring video encoding device for intelligent vision described below can be correspondingly referred to the event-triggered monitoring video encoding method for intelligent vision described above. As Figure 7 shown, the event-triggered monitoring video encoding device for intelligent vision provided by the present invention includes: An event recognition module 710, configured to obtain a monitoring video to be encoded captured by a target camera, and determine an event segment based on video frames in the monitoring video to be encoded; An encoding module 720, configured to encode video frames in the event segment to obtain an encoding result, where the encoding result includes foreground encoded data and background encoded data of the video frames; Among them, encoding the target video frame at time T in the event segment includes: Extracting background data from the target video frame, inputting the background data into a background encoder for encoding to obtain background encoded data of the target video frame; Determining foreground data of the target video frame based on the background data of the target video frame, and encoding the foreground data to obtain foreground encoded data of the target video frame.
[0050] Figure 8 Illustrates a schematic physical structure diagram of an electronic device, such as Figure 8As shown in the figure, the electronic device may include: a processor 810, a communications interface 820, a memory 830, and a communication bus 840. Among them, the processor 810, the communications interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logical instructions in the memory 830 to execute an event-triggered monitoring video encoding method and / or a video decoding method for intelligent vision. The event-triggered monitoring video encoding method for intelligent vision includes: obtaining a monitoring video to be encoded captured by a target camera, and determining an event segment based on the video frames in the monitoring video to be encoded; encoding the video frames in the event segment to obtain an encoding result, where the encoding result includes foreground encoding data and background encoding data of the video frames; among them, encoding the target video frame at time T in the event segment includes: extracting background data from the target video frame, inputting the background data into a background encoder for encoding to obtain the background encoding data of the target video frame; determining the foreground data of the target video frame based on the background data of the target video frame, and encoding the foreground data to obtain the foreground encoding data of the target video frame. The video decoding method includes: inputting the background encoding data of the target video frame into a background decoder to obtain the background reconstruction data of the target video frame; inputting the sparse encoding data of the target video frame into a sparse decoder to obtain sparse reconstruction data; inputting the temporal neighborhood information and the sparse encoding data of the target video frame into a sparse decoder to obtain sparse decoding data, and performing sparse reconstruction on the sparse decoding data to obtain the foreground reconstruction data of the target video frame; obtaining the decoding result of the target video frame based on the background reconstruction data and the foreground reconstruction data of the target video frame.
[0051] In addition, when the logical instructions in the above-mentioned memory 830 are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, may be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The foregoing storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk, or an optical disc that can store program codes.
[0052] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the event-triggered monitoring video encoding method for intelligent vision provided by the above-mentioned various methods. The event-triggered monitoring video encoding method for intelligent vision includes: acquiring a monitoring video to be encoded captured by a target camera, and determining an event segment based on video frames in the monitoring video to be encoded; encoding the video frames in the event segment to obtain an encoding result, where the encoding result includes foreground encoding data and background encoding data of the video frames; wherein, encoding the target video frame at time T in the event segment includes: extracting background data from the target video frame, inputting the background data into a background encoder for encoding to obtain the background encoding data of the target video frame; determining the foreground data of the target video frame based on the background data of the target video frame, and encoding the foreground data to obtain the foreground encoding data of the target video frame. The video decoding method includes: inputting the background encoding data of the target video frame into a background decoder to obtain the background reconstruction data of the target video frame; inputting the sparse encoding data of the target video frame into a sparse decoder to obtain sparse reconstruction data; inputting the temporal neighborhood information and the sparse encoding data of the target video frame into the sparse decoder to obtain sparse decoded data, and performing sparse reconstruction on the sparse decoded data to obtain the foreground reconstruction data of the target video frame; and obtaining the decoding result of the target video frame based on the background reconstruction data and the foreground reconstruction data of the target video frame.
[0053] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the event-triggered monitoring video encoding method for intelligent vision provided by the above-mentioned various methods. The event-triggered monitoring video encoding method for intelligent vision includes: obtaining a monitoring video to be encoded captured by a target camera, and determining event segments based on video frames in the monitoring video to be encoded; encoding the video frames in the event segments to obtain an encoding result, where the encoding result includes foreground encoding data and background encoding data of the video frames; wherein, encoding the target video frame at time T in the event segment includes: extracting background data from the target video frame, inputting the background data into a background encoder for encoding to obtain the background encoding data of the target video frame; determining foreground data of the target video frame based on the background data of the target video frame, and encoding the foreground data to obtain the foreground encoding data of the target video frame. The video decoding method includes: inputting the background encoding data of the target video frame into a background decoder to obtain the background reconstruction data of the target video frame; inputting the sparse encoding data of the target video frame into a sparse decoder to obtain sparse reconstruction data; inputting the temporal neighborhood information and sparse encoding data of the target video frame into the sparse decoder to obtain sparse decoded data, and performing sparse reconstruction on the sparse decoded data to obtain the foreground reconstruction data of the target video frame; based on the background reconstruction data and foreground reconstruction data of the target video frame, obtaining the decoding result of the target video frame.
[0054] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative labor.
[0055] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0056] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than limiting it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features therein; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. An event-triggered monitoring video encoding method for intelligent vision, characterized in that, Including: Obtain the surveillance video to be encoded captured by a target camera, and determine an event segment based on video frames in the surveillance video to be encoded; Encode video frames in the event segment to obtain an encoding result, where the encoding result includes foreground encoding data and background encoding data of the video frames; Among them, encoding the target video frame at time T in the event segment includes: Extract background data from the target video frame, input the background data into a background encoder for encoding, and obtain the background encoding data of the target video frame; Determine foreground data of the target video frame based on the background data of the target video frame, and encode the foreground data to obtain the foreground encoding data of the target video frame.
2. The event-triggered monitoring video encoding method for intelligent vision according to claim 1, wherein The determining the foreground data of the target video frame based on the background data of the target video frame includes: Input the background encoding data of the target video frame into a background decoder to obtain background reconstruction data of the target video frame; Determine the foreground data of the target video frame based on the target video frame and the background reconstruction data of the target video frame.
3. The event-triggered monitoring video encoding method for intelligent vision according to claim 1, characterized in that The foreground encoding data includes sparse encoding data and motion encoding data; The encoding the foreground data to obtain the foreground encoding data of the target video frame includes: Perform sparse representation on the foreground data to obtain sparse coefficients; Input the foreground data to be encoded into a sparse encoder to obtain the sparse encoding data, where the foreground data to be encoded includes at least the sparse coefficients; Perform motion estimation based on the foreground data to obtain motion information, and input the motion information into a motion encoder to obtain the motion encoding data.
4. The event-triggered monitoring video encoding method for intelligent vision according to claim 3, wherein The performing motion estimation based on the foreground data to obtain motion information includes: Perform motion estimation based on the foreground reconstruction data of the target video frame and a reference video frame, and obtain the motion information. The reference video frame includes video frames at time T-1 in the surveillance video to be encoded and an adjacent surveillance video, and the adjacent surveillance video is a video captured by an adjacent camera of the target camera. Among them, the foreground reconstruction data of the video frame is obtained by decoding the foreground encoding data of the video frame; The foreground data to be encoded further includes temporal neighborhood information. Before inputting the foreground data to be encoded into a sparse encoder to obtain the sparse encoding data, it includes: Input the motion encoding data into a motion decoder to obtain motion reconstruction data; Determine the temporal neighborhood information based on the motion reconstruction data and the foreground reconstruction data of the reference video frame.
5. A video decoding method for the event-triggered monitoring video encoding method for intelligent vision according to claim 4, characterized in that, Including: Input the background encoding data of the target video frame into a background decoder to obtain background reconstruction data of the target video frame; Input the sparse encoding data of the target video frame into a sparse decoder to obtain sparse reconstruction data; Input the temporal neighborhood information and the sparse encoding data of the target video frame into a sparse decoder to obtain sparse decoded data, and perform sparse reconstruction on the sparse decoded data to obtain foreground reconstruction data of the target video frame; Based on the background reconstruction data and the foreground reconstruction data of the target video frame, obtain the decoding result of the target video frame.
6. The video decoding method according to claim 5, wherein The background encoder, the background decoder, the sparse encoder, the sparse decoder, the motion encoder, and the motion decoder are neural network models trained based on multiple sets of sample data. Each set of sample data includes a sample monitoring video to be encoded and the decoding result corresponding to the sample monitoring video to be encoded. During the training process, the training loss includes a first loss and a second loss. The first loss reflects the distortion degree of the video frame decoding result, and the second loss reflects the number of encoding bits.
7. An event-triggered monitoring video encoding device for intelligent vision, characterized in that, The device includes: An event recognition module, configured to obtain a monitoring video to be encoded captured by a target camera, and determine an event segment based on video frames in the monitoring video to be encoded. An encoding module, configured to encode video frames in the event segment to obtain an encoding result, where the encoding result includes foreground encoding data and background encoding data of the video frames. Among them, encoding the target video frame at time T in the event segment includes: Extracting background data from the target video frame, inputting the background data into a background encoder for encoding, and obtaining the background encoding data of the target video frame. Determining foreground data of the target video frame based on the background data of the target video frame, and encoding the foreground data to obtain the foreground encoding data of the target video frame.
8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the event-triggered monitoring video encoding method for intelligent vision according to any one of claims 1 to 4 and / or the video decoding method according to any one of claims 5 to 6.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the event-triggered monitoring video encoding method for intelligent vision according to any one of claims 1 to 4 and / or the video decoding method according to any one of claims 5 to 6.
10. A computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the event-triggered monitoring video encoding method for intelligent vision according to any one of claims 1 to 4 and / or the video decoding method according to any one of claims 5 to 6.