System and method for encoding regions containing elements of interest in a sequence of images with high resolution
By detecting and identifying elements of interest in the processor and using encoding strategies with different resolutions, the problem of efficient encoding of important areas in video surveillance systems is solved, reducing unnecessary storage costs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- DOMAINS USA LLC
- Filing Date
- 2021-04-14
- Publication Date
- 2026-04-10
AI Technical Summary
It is known that video surveillance systems fail to effectively distinguish between important and unimportant areas when encoding regions containing elements of interest in image sequences, leading to unnecessary increases in high-resolution encoding and storage costs.
By detecting elements of interest in the image sequence identified by the processor, different regions are encoded using different resolutions, including encoding important regions at high resolution and minor regions at low resolution, and the encoding strategy is adjusted according to predefined parameters.
It achieves efficient encoding of elements of interest, reduces unnecessary storage requirements, and optimizes resource utilization.
Smart Images
Figure CN115428036B_ABST
Abstract
Description
[0001] Related Applications
[0002] This application claims priority to U.S. Application No. 16 / 865,612, filed May 4, 2020, the contents of which are incorporated by reference herein. TECHNICAL FIELD
[0003] The present invention relates generally to video surveillance systems. More particularly, the present invention relates to systems and methods for encoding regions of image sequences containing elements of interest at high resolution. BACKGROUND
[0004] Known video surveillance systems can encode regions of image sequences containing elements of interest at higher resolution than regions not containing elements of interest. However, known systems encode elements of interest at higher resolution in every one of the image sequences in which the element of interest appears, even if not every one of the image sequences is likely to contain additional relevant information about the element of interest. Thus, known systems can encode more of the image sequences at higher resolution than is necessary, which can result in large and unnecessary storage costs.
[0005] In light of the above, there is a need and opportunity for improved systems and methods. BRIEF DESCRIPTION OF DRAWINGS
[0006] Figure 1 is a block diagram of a system according to the disclosed embodiments;
[0007] Figure 2 is a block diagram of one of the image sequences processed by the detection processor according to the disclosed embodiments;
[0008] Figure 3 is a block diagram of one of the image sequences processed by the detection processor according to the disclosed embodiments;
[0009] Figure 4 is a block diagram of one of the image sequences processed by the detection processor according to the disclosed embodiments;
[0010] Figure 5 is a flowchart of a method according to the disclosed embodiments;
[0011] Figure 6 is a block diagram of an image sequence according to the disclosed embodiments; and
[0012] Figure 7 is a block diagram of an image sequence according to the disclosed embodiments. DETAILED DESCRIPTION
[0013] While the application is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that the application is not to be limited to the particular embodiments described, as this can deter individuals who have no prior knowledge with the field of the application from developing appropriate modifications within the true spirit and scope of the application.
[0014] Embodiments of the claimed application can include a video surveillance system that encodes elements of interest in a sequence of images of a monitored area at a high resolution. In some embodiments, the video surveillance system can include a camera, a detection processor, and an encoder, and in some embodiments, the camera, detection processor, and encoder can be integrated into a single device, such as a doorbell device. Alternatively, in some embodiments, at least one of the detection processor and encoder can be housed in or on a cloud server that is remote from the monitored area, and / or at least one of the camera, detection processor, and encoder can be housed in or on a control panel that is located inside or near the monitored area.
[0015] In some embodiments, the camera can capture a sequence of images of the monitored area and transmit the sequence of images to the detection processor. The detection processor can receive the sequence of images from the camera and identify a first region within the sequence of images that contains a first element of interest, and the encoder can receive the sequence of images from the camera or detection processor and receive an indication of the first region from the detection processor. The encoder can then encode the first region within a first subset of the sequence of images at a first resolution and can encode a second region within the first subset of the sequence of images that is outside of the first region at a second resolution that is less than the first resolution. In some embodiments, the first resolution can be a maximum resolution of the encoder and the second resolution can be a minimum resolution of the encoder. Additionally or alternatively, in some embodiments, one or both of the first resolution and the second resolution can be determined based on user input received by the video surveillance system.
[0016] In some embodiments, the encoder can encode any or all of the sequence of images at multiple resolutions using the H.265 standard in a single file, and in some embodiments, after encoding any or all of the sequence of images, the encoder can output the sequence of images to one or more of a plurality of destination devices, including but not limited to a remote user device associated with the monitored area, a local storage device associated with the monitored area, a remote storage device associated with a cloud server, a central monitoring station, etc.
[0017] In some embodiments, the number of image sequences in the first subset of the image sequence can be less than the entirety of the image sequence, and in some embodiments, the number of image sequences in the first subset of the image sequence can be based on a predefined parameter. Various embodiments for the predefined parameter can be contemplated, including a time value and an image number value. For example, in embodiments in which the predefined parameter comprises a time value, the first subset of the image sequence can include any of the image sequence captured by the camera during a time window that begins at a first time when the detection processor identifies the first region or the first element of interest, and ends at a second time equal to the first time plus the time value. Additionally or alternatively, in embodiments in which the predefined parameter comprises a time value, the first subset of the image sequence can include periodic images of the image sequence captured by the camera at periodic times equal to the time value. However, in embodiments in which the predefined parameter comprises an image number value, the first subset of the image sequence can include a sequential number of the image sequence captured by the camera, the sequential number beginning with the first of the captured image sequence in which the detector processor identifies the first region or the first element of interest, wherein the sequential number is equal to the image number value. Additionally or alternatively, in embodiments in which the predefined parameter comprises an image number value, the first subset of the image sequence can include periodic images of the image sequence captured by the camera at a periodicity equal to the image number value.
[0018] In some embodiments, the encoder can encode the second subset of the image sequence differently than the first subset of the image sequence. For example, in some embodiments, the encoder can encode all regions within the second subset of the image sequence at a second resolution. Alternatively, in some embodiments, the encoder can encode the first region within the second subset of the image sequence at a third resolution, and can encode the second region within the second subset of the image sequence at a second resolution. In these embodiments, the third resolution can be less than the first resolution and greater than the second resolution.
[0019] In some embodiments, the second subset of the image sequence can include all image sequences of the image sequence except for the first subset of the image sequence. However, in some embodiments, the number of image sequences in the second subset of the image sequence can be less than the entirety of the image sequence, and in some embodiments, the number of image sequences in the second subset of the image sequence can be determined based on a predefined parameter.
[0020] In some embodiments, the detection processor and / or the encoder can identify a third region within the first subset of the sequence of images or the second subset of the sequence of images, and the encoder can encode the third region within the first subset of the sequence of images or the second subset of the sequence of images at a third resolution. In these embodiments, the third region within the first subset of the sequence of images or the second subset of the sequence of images can contain a second element of interest, and the second element of interest can be a different, less important type than the first element of interest. For example, in some embodiments, the first element of interest can comprise a face of a first person in a monitored area, and the second region can comprise a body of the first person and / or an immediate surrounding environment of the first person. Additionally or alternatively, in some embodiments, the first element of interest can comprise a first moving object in a monitored area, and the second element of interest can comprise an immediate surrounding environment of the first moving object.
[0021] However, in some embodiments, the encoder can encode the third region within the first subset of the sequence of images or the second subset of the sequence of images at the first resolution. In these embodiments, the third region within the first subset of the sequence of images or the second subset of the sequence of images can contain a second element of interest, but the second element of interest can be a new element of interest of the same type as the first element of interest. For example, in some embodiments, the second element of interest can comprise a face of a second person in a monitored area or a second moving object in a monitored area.
[0022] In some embodiments, the encoder and / or the detection processor can identify the third region from the indication of the first region and at least one relationship parameter. For example, in some embodiments, the at least one relationship parameter can comprise a scaling parameter that can define a size of the third region with respect to a size of the first region. Additionally or alternatively, in some embodiments, the at least one relationship parameter can comprise an offset parameter that can define a position of the third region with respect to a position of the first region. For example, in embodiments in which the first element of interest comprises a face of a first person, the at least one relationship parameter can comprise an average position of a general body with respect to a general face.
[0023] Figure 1 is a block diagram of a video monitoring system 20 in accordance with the disclosed embodiments. As seen in Figure 1 In some embodiments, the video monitoring system 20 can comprise a camera 22 capable of capturing a sequence of images 24, a detection processor 26 capable of receiving the sequence of images 24 from the camera 22, and an encoder 28 capable of receiving the sequence of images 24 from the camera 22 or the detection processor 26 and capable of receiving an indication of one or more regions in the context of the sequence of images from the detection processor 26, as seen in
[0024] Figure 2 and Figure 3 is a block diagram of a first image 24-1 in the sequence of images 24 processed by the detection processor 26 in accordance with the disclosed embodiments. As seen in Figure 2 The first image 24-1 can include a first element of interest 30, such as a face of a first person, as seen in Figure 3 As seen in
[0025] Figure 4 is a block diagram of a second image 24-2 in the sequence of images 24 processed by the detection processor 26 in accordance with the disclosed embodiments. The second image 24-2 can be captured after the first image 24-1 and, as seen, can include the first element of interest 30 and a second element of interest 35, such as a face of a second person. Accordingly, the detection processor 26 can identify a region 32 within the second image 24-2 containing the first element of interest 30, a region 36 within the second image 24-2 containing the second element of interest 35, and a region 37 within the second image 24-2 outside of both the region 32 and the region 36. In some embodiments, the detection processor can also identify a region 38 similar to the region 34 but associated with the region 36 instead of the region 32 (e.g., a body of the second person). In these embodiments, the detection processor 26 can identify the region 37 as being outside of all of the region 32, the region 36, and the region 38 (and the region 34, if identified) within the second image 24-2.
[0026] In operation, the detection processor 26 can transmit an indication of the region 32, the region 33A, the region 33B, the region 34, the region 36, and / or the region 37 to the encoder 28. The encoder 28 can then encode one or more of the region 32, the region 33A, the region 33B, the region 34, the region 36, and / or the region 37 in the first image 24-1 and / or the second image 24-2 at a resolution that can be determined according to the values of the predefined parameters and whether the first element of interest 30 and / or the second element of interest 35 is a newly identified element of interest or a previously identified element of interest, as described in more detail below. However, in some embodiments, the detection processor 26 need not identify and transmit an indication of the region 33A, the region 33B, and / or the region 37, and in these embodiments, the encoder 28 can identify the region 33A, the region 33B, and / or the region 37 directly from the indication of the region 32, the third region 34, and / or the region 36, if necessary.
[0027] Figure 5 is a flowchart of a method 100 according to the disclosed embodiments. The method 100 can include the camera 22 capturing a new image of the monitored region (e.g., as the first image 24-1 in the sequence of images 24 in Figure 2 , or as the second image 24-2 in the sequence of images in Figure 4 , as in 102. The method 100 can then include determining whether the new image contains a new element of interest (e.g., the first element of interest 30 or the second element of interest 35), as in 104.
[0028] When the new image contains a new element of interest, the method 100 can include determining whether an additional element of interest has previously been identified in a previous image of the monitored region, as in 106. When the additional element of interest has not previously been identified, the method 100 can include the encoder 28 encoding a region within the new image containing the new element of interest at a first resolution and encoding all regions within the new image outside of the region containing the new element of interest at a second resolution that is less than the first resolution, as in 108. For example, when the first element of interest 30 is first identified in the first image 24-1 of Figure 2 , the method 100 can include the encoder 28 encoding the region 32 in the first image 24-1 at a first resolution and encoding the region 33A in the first image 24-1 at a second resolution.
[0029] However, when the additional element of interest has been previously identified, the method 100 can include determining whether a predefined parameter, such as, for example, a time value and an image number value, indicates enhanced encoding for the additional element of interest in the new image, as in 110. If so, the method 100 can include the encoder 28 encoding regions within the new image containing both the new element of interest and the additional element of interest at a first resolution, and encoding all regions within the new image outside of the regions containing the new element of interest and the additional element of interest at a second resolution, as in 112. For example, when the second image 24-2 is captured after the first image 24-1, and the predefined parameter indicates enhanced encoding for the region 32 within the second image 24-2, the method 100 can include the encoder 28 encoding the region 32 and the region 36 within the second image 24-2 at the first resolution, and encoding the region 37 within the second image 24-2 at the second resolution.
[0030] However, when the predefined parameter does not indicate enhanced encoding for the additional element of interest in the new image, the method 100 can include the encoder 28 encoding only regions within the new image containing the new element of interest at a first resolution, and encoding all regions within the new image outside of the regions containing the new element of interest at a second resolution, as in 108. For example, when the second image 24-2 is captured after the first image 24-1, but the predefined parameter does not indicate enhanced encoding for the region 32 within the second image 24-2, the method 100 can include the encoder 28 encoding the region 36 within the second image 24-2 at the first resolution, and encoding the region 32 and the region 37 within the second image 24-2 at the second resolution. However, in some embodiments, when the predefined parameter does not indicate enhanced encoding for the region 32 within the second image 24-2, the method 100 can include the encoder 28 encoding the region 32 within the second image 24-2 at a third resolution that is less than the first resolution and greater than the second resolution.
[0031] When the new image fails to contain the new element of interest, the method 100 can include determining whether the additional element of interest has been previously identified in a previous image, as in 114. When the additional element of interest has not been previously identified, the method 100 can include the encoder 28 encoding all regions within the new image at the second resolution, as in 116.
[0032] However, when the additional element of interest has been previously identified, the method 100 can include determining whether the predefined parameter indicates enhanced encoding for the additional element of interest in the new image, as in 118. If not, the method 100 can include the encoder 28 encoding all regions within the new image at the second resolution, as in 116.
[0033] However, when the predefined parameter indicates enhanced encoding for the additional element of interest in the new image, the method 100 can include the encoder 28 encoding regions within the new image containing the additional element of interest at the first resolution and encoding all regions within the new image outside of the regions containing the additional element of interest at the second resolution, as in 120. For example, when the first element of interest 30 was previously identified in a previous image captured prior to the first image 24-1 and the predefined parameter indicates enhanced encoding for the first element of interest 30 in the first image 24-1, the method 100 can include the encoder 28 encoding the region 32 within the first image 24-1 at the first resolution and encoding the region 33A within the first image 24-1 at the second resolution.
[0034] Figure 6 and Figure 7 is a block diagram of an image sequence 24 in accordance with the disclosed embodiments. As seen in Figure 6 and Figure 7 In some embodiments, the image sequence 24 can include a finite or infinite grouping of images from a first image 24-1 to a last image 24-(N+1) in the image sequence 24, as seen in some embodiments. When each of the image sequence 24 is processed as in Figure 5 The encoder can encode regions in a first subset 25A, 25B of the image sequence 24 and / or a second subset 27A, 27B of the image sequence 24 as indicated by a predefined parameter, such as, for example, a time value and an image number value.
[0035] For example, as seen in Figure 6 the time value or image number value can define a consecutive time window or sequential number of the image sequence. In these embodiments, the consecutive time window can begin at a first time when the object or region for encoding is captured or identified and end at a second time equal to the first time plus the time value. Similarly, the sequential number of the image sequence 24 can begin when the object or region for encoding is captured or identified, where the sequential number is equal to the image number value. Alternatively, as seen in Figure 7As seen in the middle, the time value or image number value can define a periodic image in the image sequence 24. In these embodiments, the periodic time can equal the time value or image number value, and the periodic image in the image sequence 24 captured at the periodic time can be encoded (e.g., every third image in the image sequence 24).
[0036] Although several embodiments have been described in detail above, other modifications are possible. For example, the logic flows described above do not require the particular order or ordering of steps described. Other steps can be provided, steps can be eliminated, and other components can be added to or removed from the described systems. Other embodiments are within the scope of the invention.
[0037] From the foregoing, it will be observed that numerous variations and modifications can be effected. It is to be understood that no limitation is intended or should be inferred in this regard unless otherwise specifically stated. It is intended to cover by the appended claims all such modifications as fall within the scope of the invention.
Claims
1. A system comprising: a camera that captures a sequence of images of a monitored area; a detection processor that identifies a first region within the sequence of images that contains a first element of interest and a second region within the sequence of images that contains a second element of interest, wherein the first element of interest comprises a moving object and the second element of interest comprises an area adjacent to the moving object, wherein the detection processor is configured to determine whether each of the first element of interest and the second element of interest is a newly identified element of interest or a previously identified element of interest; and an encoder that encodes the first region within a first subset of the sequence of images at a first resolution, encodes a second region within the first subset of the sequence of images that is outside the first region at a second resolution that is less than the first resolution when the detection processor determines that the second element of interest is a previously identified element of interest, and encodes the second region within the first subset of the sequence of images that is outside the first region at a third resolution that is lower than the first resolution but higher than the second resolution when the detection processor determines that the second element of interest is a newly identified element of interest, wherein a number of the sequence of images in the first subset of the sequence of images is less than all of the sequence of images and is based on a predefined parameter.
2. The system of claim 1, wherein the predefined parameter comprises a time value, and wherein the first subset of the sequence of images comprises any of the sequence of images that are captured during a time window that starts at a first time when the first region is identified and ends at a second time that is equal to the first time plus the time value.
3. The system of claim 1, wherein the predefined parameter comprises a time value, and wherein the first subset of the sequence of images comprises periodic images in the sequence of images that are captured at periodic times that are equal to the time value.
4. The system of claim 1, wherein the predefined parameter comprises an image number value, and wherein the first subset of the sequence of images comprises sequential numbers of the sequence of images that are equal to the image number value and start when the first region is identified.
5. The system of claim 1, wherein the predefined parameter comprises an image number value, and wherein the first subset of the sequence of images comprises periodic images in the sequence of images that are captured at a periodicity that is equal to the image number value.
6. The system of claim 1, wherein the detection processor identifies a third region within the sequence of images that contains a third element of interest, and wherein the encoder encodes the third region within the first subset of the sequence of images at a fourth resolution that is less than the first resolution and greater than the second resolution.
7. The system of claim 1, wherein the encoder encodes a third region within the first subset of the sequence of images at a fourth resolution that is less than the first resolution and greater than the second resolution, and wherein the third region is predefined relative to the first region by at least one relationship parameter.
8. The system of claim 1, wherein when the detection processor identifies a third region within the sequence of images containing a new element of interest, the encoder encodes the third region within a second subset of the sequence of images at a first resolution, and wherein a number of the sequence of images in the second subset of the sequence of images is less than all of the sequence of images and is based on the predefined parameter.
9. The system of claim 1, wherein the encoder encodes all regions within the second subset of the sequence of images at a second resolution, and wherein the second subset of the sequence of images includes all of the sequence of images outside of the first subset of the sequence of images.
10. The system of claim 1, wherein the encoder encodes the first region within the second subset of the sequence of images at a fourth resolution and encodes the second region within the second subset of the sequence of images at the second resolution, wherein the second subset of the sequence of images includes all of the sequence of images outside of the first subset of the sequence of images, and wherein the fourth resolution is less than the first resolution and greater than the second resolution.
11. A method comprising: capturing a sequence of images of a monitored area; identifying a first region within the sequence of images containing a first element of interest and a second region containing a second element of interest, wherein the first element of interest includes a moving object and the second element of interest includes an area adjacent to the moving object; determining whether each of the first element of interest and the second element of interest is a newly identified element of interest or a previously identified element of interest; and encoding the first region within a first subset of the sequence of images at a first resolution, encoding the second region outside of the first region within the first subset of the sequence of images at a second resolution less than the first resolution when the second element of interest is determined to be a previously identified element of interest, and encoding the second region outside of the first region within the first subset of the sequence of images at a third resolution lower than the first resolution but higher than the second resolution when the second element of interest is determined to be a newly identified element of interest, wherein a number of the sequence of images in the first subset of the sequence of images is less than all of the sequence of images and is based on a predefined parameter.
12. The method of claim 11, wherein the predefined parameter includes a time value, and wherein the first subset of the sequence of images includes any of the sequence of images captured during a time window that begins at a first time when the first region is identified and ends at a second time equal to the first time plus the time value.
13. The method of claim 11, wherein the predefined parameter includes a time value, and wherein the first subset of the sequence of images includes periodic images of the sequence of images captured at periodic times equal to the time value. 14. The method of claim 11, wherein the predefined parameter comprises an image number value, and wherein the first subset of the image sequence comprises sequentially numbered images of the image sequence that equal the image number value and begin when the first region is identified.
15. The method of claim 11, wherein the predefined parameter comprises an image number value, and wherein the first subset of the image sequence comprises periodic images in the image sequence that are captured with a periodicity equal to the image number value.
16. The method of claim 11, further comprising: identifying a third region within the image sequence that contains a third element of interest; and encoding the third region within the first subset of the image sequence at a fourth resolution that is less than the first resolution and greater than the second resolution.
17. The method of claim 11, further comprising: encoding the third region within the first subset of the image sequence at a fourth resolution that is less than the first resolution and greater than the second resolution, wherein the third region is predefined relative to the first region by at least one relationship parameter.
18. The method of claim 11, further comprising: in response to identifying a third region within the image sequence that contains a new element of interest, encoding the third region within a second subset of the image sequence at the first resolution, wherein the second subset of the image sequence comprises fewer images of the image sequence than all of the image sequence and is based on the predefined parameter.
19. The method of claim 11, further comprising: encoding all regions within a second subset of the image sequence at the second resolution, wherein the second subset of the image sequence comprises all images of the image sequence outside of the first subset of the image sequence.
20. The method of claim 11, further comprising: encoding the first region within a second subset of the image sequence at the fourth resolution; and encoding a second region within the second subset of the image sequence at the second resolution, wherein the second subset of the image sequence comprises all images of the image sequence outside of the first subset of the image sequence, and wherein the fourth resolution is less than the first resolution and greater than the second resolution.
Citation Information
Patent Citations
Image processing apparatus, image processing method, image processing system and computer readable medium
US20100119156A1
Techniques for managing video streaming
US20140198838A1