Event-based recorded video storage method and device, electric equipment and readable medium

Abstract

The embodiment of the invention discloses a recorded video storage method and device based on events, electric equipment and a readable medium. A specific embodiment of the method comprises the following steps: acquiring an event information set; performing space-time noise reduction processing on each event information included in the event information set to obtain an event information group sequence after noise reduction; constructing each initial video frame to obtain an initial video frame sequence; performing frame reconstruction processing on the initial recorded video frame sequence to obtain a recorded video frame sequence; performing video coding processing on the initial video recording video frame sequence to generate a video recording video for the target monitoring area; and storing the recorded video. According to the embodiment, storage of redundant video frames can be reduced, so that storage space waste is reduced.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to an event-based recorded video storage method, apparatus, electrical equipment, and readable medium. Background Art

[0002] Using video equipment to dynamically monitor the target area, record the detected dynamic events, and store the recorded videos are common technical means in the security field. At present, when using monitoring devices to record and store recorded videos, the commonly used method is: using monitoring equipment with infrared monitoring function to detect dynamic events, then recording videos for dynamic events and directly storing the obtained recorded videos.

[0003] However, when using the above method to collect and store recorded videos, the following technical problems often occur:

[0004] First, infrared monitoring has a high false-trigger rate (for example, the flow of hot air generated by the air-conditioning outdoor unit, etc.). In the event of a false-trigger, the recorded video is often redundant, that is, it does not contain dynamic events, resulting in a waste of storage space.

[0005] Second, denoising event information sets through neural network models often places complex requirements on the network structure of the neural network model, resulting in high computing resource overhead.

[0006] Third, when the collected event information is relatively sparse, the number of constructed video frames is relatively small, resulting in a low frame rate of the video and poor quality of the reconstructed video.

[0007] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0008] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0009] Some embodiments of the present disclosure propose event-based recorded video storage methods, devices, electrical equipment, and readable media to solve one or more of the technical problems mentioned in the above background technology section.

[0010] In a first aspect, some embodiments of the present disclosure provide an event-based recorded video storage method, the method comprising: obtaining an event information set, wherein the event information in the event information set is collected by a monitoring device including an event sensor for a target monitored area, the event information in the event information set includes a pixel position, a collection time, and a brightness identifier representing a brightness change, and the event information set corresponds to a collection duration; performing spatiotemporal noise reduction processing on each event information included in the event information set to obtain a noise-reduced event information group sequence; constructing each initial recorded video frame according to the noise-reduced event information group sequence to obtain an initial recorded video frame sequence; performing frame reconstruction processing on the initial recorded video frame sequence to obtain a recorded video frame sequence, wherein the number of recorded video frames included in the recorded video frame sequence is greater than the number of initial recorded video frames included in the initial recorded video frame sequence; performing video encoding processing on the initial recorded video frame sequence to generate a recorded video for the target monitored area, wherein the video duration of the recorded video is equal to the collection duration; and performing storage processing on the recorded video.

[0011] In a second aspect, some embodiments of the present disclosure provide an event-based video recording storage device, the device comprising: an acquisition unit, configured to acquire an event information set, wherein the event information in the event information set is collected by a monitoring device comprising an event sensor for a target monitoring area, the event information in the event information set comprises a pixel position, a collection time and a brightness identifier representing a brightness change, and the event information set corresponds to a collection duration; a spatiotemporal noise reduction unit, configured to perform spatiotemporal noise reduction processing on each event information included in the event information set to obtain a sequence of event information groups after noise reduction; a construction unit, configured to generate a sequence of event information groups according to the above A sequence of event information groups after noise reduction is used to construct each initial recorded video frame to obtain an initial recorded video frame sequence; a frame reconstruction unit is configured to perform frame reconstruction processing on the above-mentioned initial recorded video frame sequence to obtain a recorded video frame sequence, wherein the number of recorded video frames included in the above-mentioned recorded video frame sequence is greater than the number of initial recorded video frames included in the above-mentioned initial recorded video frame sequence; a video encoding unit is configured to perform video encoding processing on the above-mentioned initial recorded video frame sequence to generate a recorded video for the above-mentioned target monitoring area, wherein the video duration of the above-mentioned recorded video is equal to the above-mentioned acquisition duration; a storage unit is configured to store the above-mentioned recorded video.

[0012] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0013] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0014] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: the event-based video recording storage method of some embodiments of the present disclosure can reduce the storage of redundant video frames, thereby reducing the waste of storage space. Specifically, the reason for the waste of storage space is that the infrared monitoring false touch rate is high (for example, the hot air flow generated by the air conditioner outdoor unit, etc.), and the video recording collected in the case of false touch is often redundant video, that is, it does not contain dynamic events, thereby causing a waste of storage space. Based on this, the event-based video recording storage method of some embodiments of the present disclosure first obtains an event information set. Among them, the event information in the above-mentioned event information set is collected by a monitoring device including an event sensor for a target monitoring area. The event information in the above-mentioned event information set includes a pixel position, a collection time, and a brightness identifier that characterizes the brightness change. The above-mentioned event information set corresponds to a collection duration. Therefore, through the event sensor, event information can be collected when a brightness change event greater than a preset brightness threshold is detected in the monitoring area, thereby reducing redundant recording caused by false triggering. Then, each event information included in the above-mentioned event information set is subjected to spatiotemporal noise reduction processing to obtain a noise-reduced event information group sequence. Thus, the noise information in the event information set can be reduced, thereby reducing the additional computing power overhead caused by processing the noise event information. Afterwards, each initial video frame is constructed according to the above-mentioned noise-reduced event information group sequence to obtain an initial video frame sequence. Thus, each discrete event information can be converted into a continuous video frame, so that the generated video frame contains less redundant information. Next, the above-mentioned initial video frame sequence is subjected to frame reconstruction processing to obtain a video frame sequence. Among them, the number of video frames contained in the above-mentioned video frame sequence is greater than the number of initial video frames contained in the above-mentioned initial video frame sequence. Thus, the number of video frames is increased by frame reconstruction processing to fill the time gaps between dynamic events and enhance the continuity and smoothness of the video stream. Secondly, the above-mentioned initial video frame sequence is subjected to video encoding processing to generate a video for the above-mentioned target monitoring area. Among them, the video duration of the above-mentioned video is equal to the above-mentioned acquisition duration. Thus, the video encoding of the video frame sequence can obtain a complete video. Finally, the above-mentioned video is stored. Because the video frame construction and reconstruction method based on event sensors is adopted, the collection of redundant videos can be reduced and video containing less redundant background information can be generated, thereby reducing the waste of storage space. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0016] Figure 1 is a flow chart of some embodiments of the event-based recorded video storage method according to the present disclosure;

[0017] Figure 2 is a schematic diagram of the structure of some embodiments of the event-based video storage device according to the present disclosure;

[0018] Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0019] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0020] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0021] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0022] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0023] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0024] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0025] Figure 1The process 100 of some embodiments of the event-based video storage method according to the present disclosure is shown. The event-based video storage method comprises the following steps:

[0026] Step 101, obtain event information set.

[0027] In some embodiments, the execution subject (e.g., a computing device) of the event-based video recording storage method can obtain an event information set through a wired connection or a wireless connection. The execution subject can be a server. The event information in the event information set is collected by the event sensor included in the monitoring device for the target monitoring area. The monitoring device can be a monitoring camera. The event sensor can be a DVS sensor (Dynamic Vision Sensor). The event information in the event information set can represent a pixel in the target monitoring area whose brightness change exceeds a preset brightness change threshold. The event information can include a pixel position, an acquisition time, and a brightness identifier representing the brightness change. The pixel position can be the position coordinates of the corresponding pixel. The acquisition time can be a timestamp for acquiring the corresponding event information. As an example, when the brightness identifier is "1", it can represent that the brightness value of the corresponding pixel at the corresponding acquisition time is greater than the brightness value at the previous acquisition time. When the brightness identifier is "-1", it can represent that the brightness value of the corresponding pixel at the corresponding acquisition time is smaller than the brightness value at the previous acquisition time. The event information set corresponds to the acquisition duration. The above-mentioned collection duration may be the duration consumed in collecting the above-mentioned event information set.

[0028] It should be noted that the above-mentioned wireless connection methods may include but are not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0029] Step 102: Perform spatiotemporal noise reduction processing on each event information included in the event information set to obtain a noise-reduced event information group sequence.

[0030] In some embodiments, the execution subject may perform spatiotemporal noise reduction processing on each event information included in the event information set to obtain a noise-reduced event information group sequence.

[0031] In some optional implementations of some embodiments, the execution subject may perform spatiotemporal noise reduction processing on each event information included in the event information set to obtain a noise-reduced event information group sequence through the following steps:

[0032] The first step is to divide the event information set according to the collection time of the event information in the event information set to obtain various event information groups. In practice, the execution subject can divide the various event information with the same collection time included in the event information set into the same event information group to obtain various event information groups.

[0033] The second step is to perform time-sequential sorting on the above-mentioned event information groups to obtain an event information group sequence. In practice, the above-mentioned execution subject can sort the above-mentioned event information groups from the earliest to the latest according to the time sequence represented by the collection time corresponding to each event information group to obtain an event information group sequence.

[0034] The third step is to select an event information from the first event information group in the above event information group sequence as signal event information and insert it into a pre-constructed signal event information queue to update the signal event information queue. The signal event information queue may be a queue for temporarily storing event information that is determined to be non-noise. In practice, the execution subject may randomly select an event information from the first event information group in the above event information group sequence as signal event information and insert it into a pre-constructed signal event information queue.

[0035] The fourth step is to insert the selected event information as noise event information into a pre-built noise event information queue to update the noise event information queue. The length of the signal event information queue is equal to the length of the noise event information queue. The noise event information queue may be a queue for temporarily storing event information determined to be noise. Here, the queue lengths of the signal event information queue and the noise event information queue are not specifically limited.

[0036] The fifth step is to perform spatial noise reduction processing on the above event information group sequence according to the updated signal event information queue and the updated noise event information queue to obtain a noise-reduced event information group sequence.

[0037] In some optional implementations of some embodiments, the execution subject may perform spatial noise reduction processing on the event information group sequence according to the updated signal event information queue and the updated noise event information queue through the following steps to obtain a noise-reduced event information group sequence:

[0038] In the first step, for each event information group in the above event information group sequence, the following spatial noise reduction steps are performed:

[0039] The first sub-step is to determine the event information group as the event information group to be denoised.

[0040] The second sub-step is to perform the following event denoising steps based on the event information group to be denoised:

[0041] Sub-step 1: Selecting an event information to be de-noised from the event information group to be de-noised. In practice, the execution subject may randomly select an event information to be de-noised from the event information group to be de-noised.

[0042] Sub-step 2: determining the pixel position distance between the pixel position included in the selected event information to be denoised and the pixel position included in each signal event information in the signal event information queue, and obtaining each pixel position distance. As an example, the determined pixel position distance may be the Manhattan distance.

[0043] Sub-step three, in response to determining that the distance between each pixel position meets the preset neighborhood quantity condition, performing the following steps:

[0044] Sub-step one, in response to determining that the signal event information queue is full, delete the target signal event information from the head of the signal event information queue, and shift each signal event information in the signal event information queue to update the signal event information queue. Among them, the target signal event information is the signal event information first inserted into the signal event information queue. The above-mentioned preset neighborhood quantity condition is that the number of pixel position distances greater than or equal to the preset pixel distance threshold among the determined pixel position distances is greater than or equal to the preset number threshold. In practice, in response to determining that the signal event information queue is full, the above-mentioned execution entity can delete the target signal event information from the head of the signal event information queue, and shift each signal event information in the signal event information queue one position toward the head to update the signal event information queue.

[0045] Sub-step 2: inserting the selected event information to be denoised as signal event information into the tail of the signal event information queue.

[0046] Sub-step 4: in response to determining that the distances between the above-mentioned pixel positions do not satisfy the preset neighborhood quantity condition, executing the following steps:

[0047] Sub-step 1, in response to determining that the noise event information queue is full, delete the target noise event information from the noise event information queue, and shift each noise event information in the noise event information queue to update the noise event information queue. The target noise event information is the noise event information first inserted into the noise event information queue. In practice, the execution subject deletes the target noise event information from the noise event information queue in response to determining that the noise event information queue is full, and shifts each noise event information in the noise event information queue one position toward the head of the queue to update the noise event information queue.

[0048] Sub-step 2: inserting the selected event information to be denoised as noise event information into the noise event information queue.

[0049] Sub-step three: deleting the event information identical to the selected event information to be denoised from the event information group sequence to update the event information group sequence.

[0050] Sub-step five: deleting the selected event information to be subjected to noise reduction from the event information group to be subjected to noise reduction, so as to update the event information group to be subjected to noise reduction.

[0051] Sub-step six: in response to determining that the event information group to be denoised is not empty, executing the above event denoising step again.

[0052] In the second step, the updated event information group sequence is determined as the denoised event information group sequence.

[0053] The relevant content of the above-mentioned embodiment, as an inventive point of the embodiment of the present disclosure, can solve the second technical problem "denoising the event information set through a neural network model often has complex requirements on the network structure of the neural network model, resulting in a large overhead of computing resources". The factors that lead to large overhead of computing resources are often as follows: denoising the event information set through a neural network model often has complex requirements on the network structure of the neural network model. If the above factors are solved, the effect of reducing the consumption of computing resources can be achieved. In order to achieve this effect, the present disclosure takes into account the spatiotemporal characteristics of event information, and denoises the event information set through the pixel position and acquisition time included in each event information. Because the noise event information is usually isolated from the signal event information (the spatial distance is far), the event information set can be denoised by constructing a queue, thereby avoiding the complex calculations brought about by the use of the neural network model, and further reducing the consumption of computing resources.

[0054] Step 103, constructing each recorded video frame image according to the noise-reduced event information group sequence to obtain a recorded video frame sequence.

[0055] In some embodiments, the execution subject may construct each recorded video frame image according to the noise-reduced event information group sequence to obtain a recorded video frame sequence.

[0056] In some optional implementations of some embodiments, the execution subject may construct each initial video frame according to the noise-reduced event information group sequence to obtain an initial video frame sequence through the following steps:

[0057] The first step is to cluster the various noise reduction event information in the above noise reduction event information group sequence to obtain a set of noise reduction event information clusters. In practice, the above execution entity can cluster the various noise reduction event information in the above noise reduction event information group sequence by a preset clustering algorithm to obtain a set of noise reduction event information clusters. In this way, various noise reduction event information with similar spatiotemporal characteristics can be divided into the same cluster. As an example, the above preset clustering algorithm can be a DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm.

[0058] The second step is to determine the time window size corresponding to each denoised event information cluster in the above denoised event information cluster set. In practice, for each denoised event information cluster in the above denoised event information cluster set, in the first step, the above execution entity can sort the various acquisition times included in the denoised event information in the above denoised event information cluster to obtain an acquisition time sequence. The second step is to determine the time difference between the time represented by the first acquisition time and the time represented by the last acquisition time in the above acquisition time sequence as the time window size.

[0059] The third step is to determine the average value of the determined time window sizes as the target time window size. In practice, the above execution entity may determine the average value of the determined time window sizes as the target time window size.

[0060] The fourth step is to perform window division processing on the above-mentioned denoised event information group sequence according to the above-mentioned target time window size to update the denoised event information group sequence. In practice, the above-mentioned execution subject can merge the denoised event information groups belonging to the same target time window size in the above-mentioned denoised event information group sequence according to the above-mentioned target time window size to update the denoised event information group sequence. For example, if the above-mentioned target time window size is 1s, the above-mentioned execution subject can merge the denoised event information groups in the same time window in the denoised event information group sequence.

[0061] The fifth step is to build a model based on the updated denoised event information group sequence and the pre-trained video frames to generate an initial recorded video frame sequence.

[0062] In some optional implementations of some embodiments, the execution subject may construct a model according to the updated noise-reduced event information group sequence and the pre-trained video frames to generate an initial recorded video frame sequence through the following steps:

[0063] In the first step, for each denoised event information group in the updated denoised event information group sequence, the following video frame construction steps are performed:

[0064] The first sub-step is to sort the above-mentioned denoised event information group to obtain a denoised event information sequence. In practice, the above-mentioned execution subject can sort the denoised time information included in the denoised event information group according to the time sequence represented by the included acquisition time to obtain the denoised event information sequence.

[0065] The second sub-step is to input the above-mentioned denoised event information sequence into the above-mentioned video frame construction model to generate an initial recorded video frame. Among them, the above-mentioned video frame construction model can be a temporal neural network model with the denoised event information sequence as input and the initial video frame as output. The above-mentioned video frame construction model can be a neural network model obtained by improving the U-Net model. The above-mentioned video frame construction model can include a first convolutional network, a first two-dimensional convolutional layer, a first convolutional long short-term memory network (ConvLSTM, convolutional long short-term memory network), a second two-dimensional convolutional layer, a second convolutional long short-term memory network, a residual block, a first upsampling convolutional layer, a second upsampling sample convolutional layer and an image prediction layer. There is a jump connection between the output of the above-mentioned first convolutional layer and the input of the above-mentioned image prediction layer. There is a jump connection between the output of the above-mentioned first convolutional long short-term memory network and the second two-dimensional sample convolutional layer. There is a jump connection between the input and output of the above-mentioned residual block. The above-mentioned first convolutional network can be a convolutional neural network. The convolution kernel size contained in the above-mentioned first two-dimensional convolutional layer and the above-mentioned second two-dimensional convolutional layer can be 5. The convolution kernel of the first convolutional long short-term memory network and the second convolutional long short-term memory network can be 3. The number of channels of the first convolutional long short-term memory network is The convolution kernel of the residual block can be 3. The first upsampling convolution layer can include a bilinear interpolation upsampling layer and a convolution layer with a convolution kernel size of 5. The second upsampling convolution layer has the same network structure as the first upsampling convolution layer. Each of the above convolution layers uses a ReLU activation function and batch normalization processing. The above image prediction layer can be a deep convolution layer. The convolution kernel size of the deep convolution layer is 1, and a sigmoid activation function is used.

[0066] The second step is to determine the generated initial video frames as an initial video frame sequence. In practice, the execution subject may determine the generated initial video frames as an initial video frame sequence according to the generation order.

[0067] Step 104 , performing frame reconstruction processing on the initial recorded video frame sequence to obtain a recorded video frame sequence.

[0068] In some embodiments, the execution subject may perform frame reconstruction processing on the initial recorded video frame sequence to obtain a recorded video frame sequence. The number of recorded video frames contained in the recorded video frame sequence is greater than the number of initial recorded video frames contained in the initial recorded video frame sequence. In practice, the execution subject may perform frame reconstruction processing on the initial recorded video frame sequence through a video frame insertion algorithm to obtain a recorded video frame sequence. As an example, the video frame insertion algorithm may be, but is not limited to, the DAIN (Depth-Aware Video Frame Interpolation) algorithm, the DeepVO algorithm, and the FlowNet algorithm based on optical flow.

[0069] In some optional implementations of some embodiments, the execution subject may perform frame reconstruction processing on the initial recorded video frame sequence through the following steps to obtain a recorded video frame sequence:

[0070] The first step is to determine the target number of video frames according to the preset video frame rate and the acquisition time. In practice, the execution subject may determine the target number of video frames as the product of the preset video frame rate and the acquisition time.

[0071] In the second step, the difference between the target video frame quantity and the initial video frame quantity in the initial video frame sequence is determined as the total number of reconstructed video frames.

[0072] In the third step, the ratio of the total amount of the reconstructed video frames to the acquisition time is used as the unit reconstructed video frame number.

[0073] Step 4: Update the target time window size according to the number of unit reconstructed video frames. In practice, the execution subject may round up the ratio of the target time window size to the number of unit reconstructed video frames as the updated target time window size.

[0074] The fifth step is to perform secondary window division on the updated denoised event information group sequence according to the updated target time window size to obtain a divided event information group sequence. In practice, first, the above-mentioned execution subject can sort the denoised event information included in the denoised event information group sequence according to the time sequence represented by the included acquisition times to obtain a denoised event information sequence. Then, the above-mentioned execution subject can perform secondary window division on the denoised event information sequence according to the above-mentioned updated target window size to obtain each divided event information group. For example, the above-mentioned updated target time window size can be 20ms, and the first denoised event information in the denoised event information sequence includes the acquisition time represented by 12 o'clock. The above-mentioned execution subject can divide each denoised event information in the denoised event information sequence whose acquisition time is between 12 o'clock and 12:20 milliseconds as each divided event information into the same divided event information group to obtain a divided event information group.

[0075] In the sixth step, a pre-built video frame quality enhancement network is added to the video frame construction model, so as to use the added video frame construction model as a video frame reconstruction model. The video frame quality enhancement network may be a neural network model for enhancing image quality.

[0076] Step 7: For each divided event information group in the divided event information group sequence, perform the following video frame reconstruction steps:

[0077] The first sub-step is to sort the above-mentioned event information groups after division to obtain a sequence of event information after division. In practice, the execution subject may sort the various event information after division included in the above-mentioned event information groups after division in chronological order to obtain a sequence of event information after division.

[0078] In the second sub-step, the divided event information sequence is input into the video frame construction model included in the above-mentioned video frame reconstruction model to obtain the reconstructed recorded video frame.

[0079] In the third sub-step, the initial video frame corresponding to the reconstructed video frame and the reconstructed video frame are input into the video frame quality enhancement network included in the video frame reconstruction model to obtain the video frame. The video frame may be a quality-enhanced reconstructed video frame. The initial video frame corresponding to the reconstructed video frame may be an initial video frame in which each event information used is repeated with each event information used by the reconstructed video frame. The video frame quality enhancement network may be a neural network model that takes the reconstructed video frame and the initial video frame corresponding to the reconstructed video frame as input and the video frame as output. In practice, since the initial video frame corresponding to the reconstructed video frame uses more event information and has repeated event information used to construct the reconstructed video frame, the quality of the reconstructed video frame can be enhanced through the initial video frame. The video frame quality enhancement model may enhance the quality of the reconstructed video frame through the corresponding initial video frame. As examples, the video frame quality enhancement model can be but is not limited to a dual convolutional neural network model, a DCGAN network (Deep Convolutional Generative Adversarial Networks) and a U-Net neural network model.

[0080] Step 8: Update the video frame sequence according to the obtained video frames and the initial video frame sequence. In practice, first, for each initial video frame in the initial video frame sequence, the execution subject may insert the video frames corresponding to the initial video frame into the initial video frame in the order of generation. Then, the execution subject may determine the inserted initial video frame sequence as the video frame sequence.

[0081] The relevant contents of the above-mentioned embodiments, as an inventive point of the embodiments of the present disclosure, can solve the technical problem three "When the collected event information is relatively sparse, the number of video frames constructed is small, resulting in a low frame rate of the video, and the quality of the reconstructed video is poor". The factors that lead to the poor quality of the video are often as follows: When the collected event information is relatively sparse, the number of video frames constructed is small, resulting in a low frame rate of the video. If the above factors are solved, the effect of improving the quality of the video can be achieved. In order to achieve this effect, the present disclosure adopts a secondary window division of the noise-reduced event information group sequence to obtain a divided event information group sequence. Then, the video frame is reconstructed with each divided event information group as a unit. Then, the quality of the reconstructed video frame is enhanced by the corresponding initial video frame, thereby improving the picture quality of the reconstructed video frame, and then improving the quality of the reconstructed video. Because the number of video frames is increased, the frame rate of the reconstructed video is higher, and the quality of the video is improved.

[0082] Step 105 , performing video encoding processing on the recorded video frame sequence to generate a recorded video for the target monitoring area.

[0083] In some embodiments, the execution subject may perform video encoding on the processed video frame sequence to generate a video for the target surveillance area. In practice, the execution subject may perform video encoding on the processed video frame sequence through a video encoder to generate a video for the target surveillance area. As an example, the video encoder may be, but is not limited to, an AVC encoder, an AV1 encoder, or an FFmpeg encoder.

[0084] Step 106, storing the recorded video.

[0085] In some embodiments, the execution entity may store the recorded video. In practice, the execution entity may store the recorded video in a cache device.

[0086] In some optional implementations of some embodiments, the execution subject may store the recorded video through the following steps, including:

[0087] The first step is to compress the recorded video to obtain a compressed recorded video. In practice, the execution subject may compress the recorded video by using the video encoder to obtain a compressed recorded video.

[0088] The second step is to store the compressed recorded video in a cache device.

[0089] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: the event-based video recording storage method of some embodiments of the present disclosure can reduce the storage of redundant video frames, thereby reducing the waste of storage space. Specifically, the reason for the waste of storage space is that the infrared monitoring false touch rate is high (for example, the hot air flow generated by the air conditioner outdoor unit, etc.), and the video recording collected in the case of false touch is often redundant video, that is, it does not contain dynamic events, thereby causing a waste of storage space. Based on this, the event-based video recording storage method of some embodiments of the present disclosure first obtains an event information set. Among them, the event information in the above-mentioned event information set is collected by a monitoring device including an event sensor for a target monitoring area. The event information in the above-mentioned event information set includes a pixel position, a collection time, and a brightness identifier that characterizes the brightness change. The above-mentioned event information set corresponds to a collection duration. Therefore, through the event sensor, event information can be collected when a brightness change event greater than a preset brightness threshold is detected in the monitoring area, thereby reducing redundant recording caused by false triggering. Then, each event information included in the above-mentioned event information set is subjected to spatiotemporal noise reduction processing to obtain a noise-reduced event information group sequence. Thus, the noise information in the event information set can be reduced, thereby reducing the additional computing power overhead caused by processing the noise event information. Afterwards, each initial video frame is constructed according to the above-mentioned noise-reduced event information group sequence to obtain an initial video frame sequence. Thus, each discrete event information can be converted into a continuous video frame, so that the generated video frame contains less redundant information. Next, the above-mentioned initial video frame sequence is subjected to frame reconstruction processing to obtain a video frame sequence. Among them, the number of video frames contained in the above-mentioned video frame sequence is greater than the number of initial video frames contained in the above-mentioned initial video frame sequence. Thus, the number of video frames is increased by frame reconstruction processing to fill the time gaps between dynamic events and enhance the continuity and smoothness of the video stream. Secondly, the above-mentioned initial video frame sequence is subjected to video encoding processing to generate a video for the above-mentioned target monitoring area. Among them, the video duration of the above-mentioned video is equal to the above-mentioned acquisition duration. Thus, the video encoding of the video frame sequence can obtain a complete video. Finally, the above-mentioned video is stored. Because the video frame construction and reconstruction method based on event sensors is adopted, the collection of redundant videos can be reduced and video containing less redundant background information can be generated, thereby reducing the waste of storage space.

[0090] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of an event-based video storage device. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the event-based recorded video storage device can be specifically applied to various electronic devices.

[0091] like Figure 2 As shown, some embodiments of the event-based video recording storage device 200 include: an acquisition unit 201, a spatiotemporal noise reduction unit 202, a construction unit 203, a frame reconstruction unit 204, a video encoding unit 205 and a storage unit 206. The acquisition unit 201 is configured to acquire an event information set, wherein the event information in the event information set is collected by a monitoring device including an event sensor for a target monitoring area, and the event information in the event information set includes a pixel position, a collection time and a brightness identifier representing a brightness change, and the event information set corresponds to a collection duration; the spatiotemporal noise reduction unit 202 is configured to perform spatiotemporal noise reduction processing on each event information included in the event information set to obtain a noise-reduced event information group sequence; the construction unit 203 is configured to construct each initial recording according to the noise-reduced event information group sequence. The image video frame is obtained to obtain an initial video frame sequence; the frame reconstruction unit 204 is configured to perform frame reconstruction processing on the above-mentioned initial video frame sequence to obtain a video frame sequence, wherein the number of video frames included in the above-mentioned video frame sequence is greater than the number of initial video frames included in the above-mentioned initial video frame sequence; the video encoding unit 205 is configured to perform video encoding processing on the above-mentioned initial video frame sequence to generate a video for the above-mentioned target monitoring area, wherein the video duration of the above-mentioned video is equal to the above-mentioned acquisition duration; the storage unit 206 is configured to store the above-mentioned video.

[0092] It is understandable that the units recorded in the event-based video storage device 200 and the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the event-based video storage device 200 and the units contained therein, and will not be described in detail here.

[0093] Reference below Figure 3 , which shows a structural schematic diagram of an electronic device 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0094] like Figure 3As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory 302 or a program loaded from a storage device 308 into a random access memory 303. Various programs and data required for the operation of the electronic device 300 are also stored in the random access memory 303. The processing device 301, the read-only memory 302, and the random access memory 303 are connected to each other via a bus 304. An input / output interface 305 is also connected to the bus 304.

[0095] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0096] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the read-only memory 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0097] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0098] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0099] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains an event information set, wherein the event information in the event information set is collected by a monitoring device including an event sensor for a target monitoring area, and the event information in the event information set includes a pixel position, a collection time, and a brightness identifier representing a brightness change. The event information set corresponds to a collection duration; performs spatiotemporal noise reduction processing on each event information included in the event information set to obtain a noise-reduced event information group sequence; constructs each initial video recording frame according to the noise-reduced event information group sequence to obtain an initial video recording frame sequence; performs frame reconstruction processing on the initial video recording frame sequence to obtain a video recording frame sequence, wherein the number of video recording frames included in the video recording frame sequence is greater than the number of initial video recording frames included in the initial video recording frame sequence; performs video encoding processing on the initial video recording frame sequence to generate a video recording for the target monitoring area, wherein the video duration of the video recording is equal to the collection duration; and performs storage processing on the video recording.

[0100] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0101] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0102] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor comprising: an acquisition unit, a spatiotemporal noise reduction unit, a construction unit, a frame reconstruction unit, a video encoding unit, and a storage unit. The names of these units do not, in some cases, constitute limitations on the units themselves, for example, the acquisition unit may also be described as a "unit for acquiring a set of event information".

[0103] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0104] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A method for storing recorded video based on an event, comprising: Acquire an event information set, wherein the event information in the event information set is collected by a monitoring device including an event sensor for a target monitoring area, the event information in the event information set includes a pixel position, a collection time, and a brightness identifier representing a brightness change, and the event information set corresponds to a collection duration; Performing spatiotemporal noise reduction processing on each event information included in the event information set to obtain a noise-reduced event information group sequence; According to the noise-reduced event information group sequence, construct each initial video frame to obtain an initial video frame sequence; Performing frame reconstruction processing on the initial recorded video frame sequence to obtain a recorded video frame sequence, wherein the number of recorded video frames included in the recorded video frame sequence is greater than the number of initial recorded video frames included in the initial recorded video frame sequence; Performing video encoding processing on the initial video frame sequence to generate a video for the target monitoring area, wherein the video duration of the video is equal to the acquisition duration; The recorded video is stored.

2. The method according to claim 1, wherein: The performing spatiotemporal noise reduction processing on each event information included in the event information set to obtain a noise-reduced event information group sequence includes: According to the collection time included in the event information in the event information set, the event information set is divided and processed to obtain various event information groups; Performing time sequence sorting on each of the event information groups to obtain an event information group sequence; Selecting an event information from the first event information group in the event information group sequence as signal event information and inserting it into a pre-constructed signal event information queue to update the signal event information queue; Inserting the selected event information as noise event information into a pre-constructed noise event information queue to update the noise event information queue, wherein the length of the signal event information queue is equal to the length of the noise event information queue; According to the updated signal event information queue and the updated noise event information queue, spatial noise reduction processing is performed on the event information group sequence to obtain a noise-reduced event information group sequence.

3. The method according to claim 1, wherein: The step of constructing each initial video frame according to the noise-reduced event information group sequence to obtain an initial video frame sequence includes: Performing clustering processing on each piece of denoised event information in the denoised event information group sequence to obtain a denoised event information cluster set; Determine the time window size corresponding to each denoised event information cluster in the denoised event information cluster set; Determine the average value of each determined time window size as the target time window size; According to the target time window size, performing window division processing on the noise-reduced event information group sequence to update the noise-reduced event information group sequence; A model is constructed based on the updated denoised event information group sequence and the pre-trained video frames to generate an initial recorded video frame sequence.

4. The method according to claim 3, wherein: The method of constructing a model based on the updated noise-reduced event information group sequence and the pre-trained video frames to generate an initial video frame sequence includes: For each post-noise reduction event information group in the updated post-noise reduction event information group sequence, the following video frame construction steps are performed: Sorting the noise-reduced event information groups to obtain a noise-reduced event information sequence; Inputting the noise-reduced event information sequence into the video frame construction model to generate an initial recorded video frame; The generated initial recorded video frames are determined as an initial recorded video frame sequence.

5. The method according to claim 1, wherein: The storing and processing of the recorded video includes: Compressing the recorded video to obtain a compressed recorded video; The compressed recorded video is stored in a cache device.

6. A method and device for storing recorded video based on events, comprising: An acquisition unit is configured to acquire an event information set, wherein the event information in the event information set is collected by a monitoring device including an event sensor for a target monitoring area, and the event information in the event information set includes a pixel position, a collection time, and a brightness identifier representing a brightness change, and the event information set corresponds to a collection duration; a spatiotemporal noise reduction unit configured to perform spatiotemporal noise reduction processing on each event information included in the event information set to obtain a noise-reduced event information group sequence; A construction unit is configured to construct each initial video frame according to the noise-reduced event information group sequence to obtain an initial video frame sequence; a frame reconstruction unit configured to perform frame reconstruction processing on the initial recorded video frame sequence to obtain a recorded video frame sequence, wherein the number of recorded video frames included in the recorded video frame sequence is greater than the number of initial recorded video frames included in the initial recorded video frame sequence; A video encoding unit is configured to perform video encoding processing on the initial video frame sequence to generate a video for the target monitoring area, wherein the video duration of the video is equal to the acquisition duration; The storage unit is configured to store the recorded video.

7. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

8. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images