Image downsampling method and image processing system

By using interlaced odd pixel groups in the image downsampling method, the sawtooth image problem caused by traditional downsampling is solved, and the event detection accuracy and power consumption management of the surveillance camera are improved.

CN119012032APending Publication Date: 2024-11-22PIXART IMAGING INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410587391.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2024-03-24
Filing Date
2024-05-13
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

Traditional image downsampling methods may lead to serrated images, affecting the accurate judgment of events by surveillance cameras.

Method used

Using an image downsampling method of the first and second pixels arranged in an interlaced array, an output pixel value is generated by using odd pixels in the first downsampling group to ensure that one output pixel value is generated per N pixels, and N is an odd number.

Benefits of technology

Effectively improves the serrated image problem, reduces power consumption in standby mode, and improves the accuracy of event detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119012032A_ABST
    Figure CN119012032A_ABST
Patent Text Reader

Abstract

An image down-sampling method and an image processing system, the image down-sampling method for down-sampling a sensing image including first pixels and second pixels, the first pixels and the second pixels being arranged in a staggered manner, includes: generating a first output pixel value according to a pixel value of the first pixel in a first down-sampling group; wherein the first downsampling group comprises an odd number of pixels. Therefore, the problem of sawtooth images in the prior art can be improved. If the image downsampling method is applied to the surveillance camera, the event detection accuracy of the surveillance camera can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to an image downsampling method and an image processing system, and in particular to an image downsampling method and an image processing system capable of preventing aliased images. Background Art

[0002] The image sensor of existing surveillance cameras uses pre-roll mode (also known as buffer mode) to store images captured in the last N seconds, so that when an event (such as a person) is recognized, the image sensor can save images that occurred before and after the event. In pre-roll mode, the image sensor can generate images by down sampling to reduce power consumption. However, traditional downsampling may result in jagged images, which may be caused by uneven sampling or aliasing. Jagged images may cause surveillance cameras to make incorrect event judgments. Summary of the invention

[0003] An object of the present invention is to disclose an image downsampling method capable of improving the aliased image problem.

[0004] Another object of the present invention is to disclose an image processing system capable of improving the aliased image problem.

[0005] An embodiment of the present invention discloses an image downsampling method for downsampling a sensing image including a first pixel and a second pixel, wherein the first pixel and the second pixel are arranged alternately, and comprises: generating a first output pixel value according to a pixel value of a first pixel in a first downsampling group; wherein the first downsampling group includes an odd number of pixels.

[0006] Another embodiment of the present invention discloses an image processing system, comprising: an image sensor for generating a sensing image including a first pixel and a second pixel, wherein the first pixel and the second pixel are arranged alternately; and a processing circuit for generating a first output pixel value according to a pixel value of a first pixel in a first down-sampling group; wherein the first down-sampling group includes an odd number of pixels.

[0007] According to the above embodiments, pixel groups with odd-numbered pixels can be used to perform downsampling of pixel values ​​to generate output images, which can improve the problem of jagged images in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 A block diagram of an image processing system according to an embodiment of the present invention is shown.

[0009] Figure 2 According to an embodiment of the present invention, Figure 1 A block diagram of the detailed structure of the image sensor is shown.

[0010] Figure 3 , Figure 4 , Figure 5 , Figure 6 as well as Figure 7 A schematic diagram of an image downsampling method according to an embodiment of the present invention is shown.

[0011] The reference numerals are described as follows:

[0012] 100 Image Processing System

[0013] 101 processing circuit

[0014] 103 Storage device

[0015] 105 Image Sensor

[0016] 107 Buffer

[0017] 201 Lenses

[0018] 205ISP

[0019] 206 pixel array

[0020] 207 Reading Circuit

[0021] 209 image signal amplifier circuit

[0022] 211ADC

[0023] DSG_11,DSG_12 first downsampling group

[0024] DSG_21 Second downsampling group

[0025] AIS amplifies image signal

[0026] IS image signal

[0027] DIS digital image signal

[0028] Img_O output image

[0029] Img_S sensing image

[0030] OPV1 first output pixel value

[0031] OPV2 second output pixel value

[0032] P1,P11,P12,P13,P14,P15,P16,P17First Pixel

[0033] P2, P21, P22, P23, P24, P25, P26, P27 second pixel DETAILED DESCRIPTION

[0034] The present invention will be described below with multiple embodiments. Please note that the "first", "second" and similar descriptions in the following description are only used to define different elements, parameters, data, signals or steps. They are not used to limit their order. For example, the first device and the second device can be different devices with the same structure.

[0035] Figure 1 FIG. 1 is a block diagram of an image processing system 100 according to an embodiment of the present invention. Figure 1 As shown, the image processing system 100 includes a processing circuit 101, a storage device 103, an image sensor 105, and a buffer 107. The processing circuit 101 is used to execute at least one program in the storage device 103 to control the image sensor 105 to execute the image downsampling method disclosed in the present invention. The processing circuit 101 can also control the actions of other devices of the image processing system 100. The image sensor 105 is used to execute the image downsampling method to generate an output image Img_O. In one embodiment, the buffer 107 is used to buffer the output image Img_O for further processing. In one embodiment, the image processing system 100 is a surveillance camera, and the buffer 107 is used to store the output image Img_O generated within a predetermined time (e.g., the last N seconds), so that when an event (e.g., a person) is recognized, the processing circuit 101 can save the images that occurred before and after the event.

[0036] In another embodiment, the surveillance camera does not perform the image downsampling method in the normal mode, but generates a sensing image Img_S for event detection. The surveillance camera performs the image downsampling method in the standby mode, and generates a downsampled output image Img_O for event detection. This approach can reduce the power consumption in the standby mode. The specific steps of the image downsampling method will be described in subsequent embodiments.

[0037] Figure 2 According to an embodiment of the present invention, Figure 1 Please also understand that the method described in the following embodiments is not limited to the method described in the following embodiments. Figure 1 and Figure 2 The image processing system 100 shown in the figure is implemented. In one embodiment, the image processing system 100 is a camera, which can be an independent electronic device or integrated into other electronic devices such as mobile phones and tablet computers.

[0038] like Figure 2As shown, the image processing system 100 includes a lens 201, an image sensor 105 and an ISP 205. The image sensor 105 includes a pixel array 206, a readout circuit 207, an image signal amplification circuit 209 and an ADC (Analog to digital converter) 211. The pixel array 206 includes a plurality of pixels, which generate sensing charges corresponding to the light received through the lens 201. The readout circuit 207 is used to read the sensing charges to generate an image signal IS.

[0039] The image signal amplification circuit 209 is used to amplify the image signal IS to generate an amplified image signal AIS. The amplified image signal AIS is transmitted to the ADC 211 to generate a digital image signal DIS (pixel value). The digital image signal DIS is transmitted to the ISP 205, and the ISP 205 can perform the above-mentioned image downsampling method to downsample the digital image signal DIS to generate the output image Img_O. If the digital image signal DIS is not downsampled, the ISP 205 generates the above-mentioned sensed image Img_S. The ISP 205 can be independent of the image sensor 105 or integrated into the image sensor 105, or integrated into the processing circuit 101.

[0040] In other words, the digital image signal DIS generated by ADC 211 can be regarded as a raw image signal. These raw image signals can be processed by ISP 205 but not downsampled to generate a sensed image Img_S, or processed by ISP 205 and downsampled to generate an output image Img_O. For the convenience of explanation, the following embodiments illustrate the image downsampling method using the sensed image Img_S. However, the image downsampling method disclosed in the present invention can be applied to raw image signals or to the sensed image Img_S. Therefore, in the following embodiments, the description of "downsampling the sensed image" can refer to downsampling the raw image signal of the sensed image Img_S, or to downsampling the sensed image Img_S.

[0041] Figure 3 , Figure 4 , Figure 5 , Figure 6 as well as Figure 7 FIG. 1 is a schematic diagram of an image downsampling method according to an embodiment of the present invention. Figure 5 , Figure 6 and Figure 7 As shown, the sensing image Img_S includes a plurality of first pixels P1 and a plurality of second pixels P2 arranged alternately. The first pixel P1 and the second pixel P2 can be any type of pixel. In one example, the first pixel P1 is a G pixel and the second pixel P2 is a B pixel. In another example, the first pixel P1 is an R pixel and the second pixel P2 is an IR pixel. Figure 3 , Figure 4 The embodiment shown also includes a plurality of first pixels and second pixels, but for the convenience of description, the first pixels and the second pixels are marked with different symbols. Figure 3 , Figure 4 , Figure 5 , Figure 6 and Figure 7 In the embodiment of the present invention, the first output pixel value is generated according to at least one pixel value of at least one first pixel in the first down-sampling group. The first down-sampling group includes an odd number of pixels. Please note that in the following embodiments, each down-sampling group only generates one output pixel value. However, each down-sampling group can also generate multiple output pixel values.

[0042] The pixels in the downsampled group can be configured in a variety of ways. In addition, the output pixel value can be generated in a variety of ways. In one embodiment, the first downsampled group includes an even number of first pixels (e.g., first pixels P11, P12) and an odd number of second pixels (e.g., second pixel P21), for example Figure 3 In another embodiment, the first down-sampling group includes an odd number of first pixels (eg, first pixel P16) and an even number of second pixels (eg, second pixels P25, P26), for example Figure 3 In such an embodiment, the pixel value of the first pixel can be downsampled to generate a first output pixel value OPV1, or the pixel value of the second pixel can be downsampled to generate a second output pixel value OPV2. For example, if the first downsampled group DSG_11 includes five pixels, the pixel value of the first pixel can be downsampled to generate a first output pixel value OPV1, or the pixel value of the second pixel can be downsampled to generate a second output pixel value OPV2. In the following embodiments, the symbol Pxv is used to illustrate, which represents the value of the pixel Px. For example, P11v represents Figure 3 and Figure 4 The pixel value of the first pixel P11 in .

[0043] exist Figure 3In the embodiment of, only the pixel value of the rightmost or leftmost first pixel in the first down-sampling group may be output as the first output pixel value OPV1. Therefore, the first pixel is down-sampled. In this case, the pixel value of the second pixel P21 that has not been down-sampled may also be output as the first output pixel value OPV1. Accordingly, the pixel values ​​(P11v, P21v, P12v) in the first down-sampling group DSG_11 in the sensed image Img_S are down-sampled to generate pixel values ​​P11v, P21v or P12v in the first down-sampling group DSG_11 in the output image Img_O. According to the same rule, the pixel values ​​(P25v, P16v, P26v) in the first down-sampling group DSG_12 in the sensed image Img_S may be down-sampled to generate pixel values ​​P25v, P16v or P26v of the first down-sampling group DSG_12 in the output image Img_O.

[0044] In one embodiment, the first output pixel value OPV1 or the second output pixel value OPV2 is determined according to the number of pixels of the same type. Figure 3 In the first down-sampling group DSG_11, the first pixels P11 and P12 account for the majority, so the pixel value P11v or P12v is selected as the first output pixel value OPV1. Figure 3 In the first down-sampling group DSG_12, the second pixels P25 and P26 account for the majority, so the pixel value P25v or P26v is selected as the second output pixel value OPV2. Such a rule can also be applied in other embodiments.

[0045] In one embodiment, the first output pixel value is generated by averaging the pixel values ​​of at least two first pixels in the first down-sampled group. Figure 4 In the embodiment of the present invention, the pixel configurations of the first down-sampling group DSG_11 and the first down-sampling group DSG_12 are the same as Figure 3 The pixel configurations of the first down-sampling group DSG_11 and the first down-sampling group DSG_12 are the same. Figure 4 In the embodiment of the present invention, the pixel values ​​of the first pixels in the first down-sampling group are averaged to generate the first output pixel value OPV1.

[0046] Accordingly, the pixel values ​​(P11v, P21v, P12v) in the first down-sampling group DSG_11 in the sensed image Img_S are down-sampled to generate the pixel values ​​P21v or P1av in the first down-sampling group DSG_11 in the output image Img_O. The pixel values ​​P21v and P1av can be selectively used in accordance with different design requirements. According to the same rule, the pixel values ​​(P25v, P16v, P26v) in the first down-sampling group DSG_12 in the sensed image Img_S can be down-sampled to generate the pixel values ​​P16v or P2av in the first down-sampling group DSG_12 in the output image Img_O. The pixel values ​​P16v and P2av can be selectively used in accordance with different design requirements.

[0047] In one embodiment, the first output pixel value OPV1 or the second output pixel value OPV2 is determined according to the number of pixels of the same type. Figure 4 In the first down-sampling group DSG_11, the first pixels P11 and P12 are in the majority, so the pixel value P1av is selected as the first output pixel value OPV1. Figure 4 In the first down-sampling group DSG_12, the second pixels P25 and P26 are in the majority, so the pixel value P2av is selected as the second output pixel value OPV2. Such a rule can also be applied to other embodiments.

[0048] According to the above description, in one embodiment, the first down-sampling group including an even number of first pixels and an odd number of second pixels is down-sampled, such as the first down-sampling group DSG_11. The first down-sampling group including an odd number of first pixels and an even number of second pixels is not down-sampled, such as the first down-sampling group DSG_12. On the contrary, in another embodiment, the first down-sampling group including an even number of first pixels and an odd number of second pixels is not down-sampled, and the first down-sampling group including an odd number of first pixels and an even number of second pixels is down-sampled.

[0049] However, in another embodiment, the first down-sampling group including an even number of first pixels and an odd number of second pixels and the first down-sampling group including an odd number of first pixels and an even number of second pixels are all down-sampled. In addition, in one embodiment, the pixel values ​​of the second pixels in the first down-sampling group are down-sampled, or the pixel values ​​of the first pixels in the first down-sampling group are down-sampled. Such changes may also be included in the scope of the present invention.

[0050] However, these two embodiments can be combined. Figure 5 In the embodiment of FIG. 5 , the sensed image Img_S includes a first down-sampling group DSG_11 and also includes a second down-sampling group DSG_21 . Figure 5 The pixel configuration of the second downsampling group DSG_21 in Figure 4 The pixel configuration of the first down-sampling group DSG_12 in the sensed image Img_S is the same. Accordingly, through the above steps, the second down-sampling group DSG_21 in the sensed image Img_S is down-sampled to generate the pixel values ​​P2av or P1v in the second down-sampling group DSG_21 in the output image Img_O. Figure 5 In the embodiment, both the first down-sampling group DSG_11 and the second down-sampling group DSG_21 are down-sampled. Therefore, since the first down-sampling group DSG_11 and the second down-sampling group DSG_21 are arranged alternately, the entire sensing image Img_S is down-sampled.

[0051] exist Figure 5 In some embodiments, the first down-sampling group and the second down-sampling group have completely different pixels. However, in some embodiments, the first down-sampling group and the second down-sampling group have at least one pixel in common. Figure 5 In the embodiment of the present invention, the first down-sampling group DSG_11 and the second down-sampling group DSG_21 share the first pixel P1 and the second pixel P2. In addition, the pixel values ​​of the first pixel P1 in the first down-sampling group DSG_11 of the sensed image Img_S may be averaged to generate the first output pixel value OPV1. Similarly, the pixel values ​​of the second pixel P2 in the second down-sampling group DSG_21 of the sensed image Img_S may be averaged to generate the second output pixel value OPV2.

[0052] In this way, the pixel values ​​(P1v, P2v, P1v, P2v, P1v) of the first down-sampling group DSG_11 of the sensed image Img_S are down-sampled to the pixel values ​​P1av or P2v of the output image Img_O. Similarly, the pixel values ​​(P2v, P1v, P2v, P1v, P2v) of the second down-sampling group DSG_21 of the sensed image Img_S are down-sampled to the pixel values ​​P1v or P2av of the output image Img_O. In addition, Figure 6 In the embodiment of FIG. 1 , since the first down-sampling group DSG_11 and the second down-sampling group DSG_21 can be arranged alternately, all pixels of the sensed image Img_S can be down-sampled. Figure 6 In the embodiment of the present invention, the sensing image Img_S further includes a first down-sampling group DSG_12 after the first down-sampling group DSG_11. The first down-sampling group DSG_12 also shares the first pixel P1 and the second pixel P2 with the second down-sampling group DSG_21.

[0053] exist Figure 6In the embodiment, the pixel values ​​of all first pixels P1 in the first down-sampling group DSG_11 and the pixel values ​​of all second pixels P2 in the second down-sampling group DSG_21 are averaged to generate the pixel value. However, in an embodiment, the first output pixel value OPV1 may be generated by the pixel values ​​of only a portion of the first pixels P1 in the first down-sampling group DSG_11, and the second output pixel value OPV2 may be generated by the pixel values ​​of only a portion of the second pixels P2 in the second down-sampling group DSG_21. Figure 7 As shown in the embodiment, in the first down-sampling group DSG_11, only the pixel values ​​of the rightmost first pixel P1 and the leftmost first pixel P1 are averaged to generate the first output pixel value OPV1. In addition, in the second down-sampling group DSG_21, only the pixel values ​​of the rightmost second pixel P2 and the leftmost second pixel P2 are averaged to generate the second output pixel value OPV2.

[0054] The image downsampling method disclosed in the present invention is not limited to the above embodiments. All or part of the steps disclosed in each embodiment can be combined into another embodiment. Figure 6 In the embodiment, the pixel value of the rightmost first pixel P1 in the first down-sampling group DSG_11 can be directly used as the first output pixel value. In addition, the image down-sampling method disclosed in the present invention can be applied to images with different pixels arranged alternately, such as a Bayer pattern.

[0055] according to Figure 3 , Figure 4 , Figure 5 , Figure 6 and Figure 7 In the illustrated embodiment, the image downsampling method disclosed in the present invention may have a rule: the first downsampling group includes pixels that are not downsampled. For example, Figure 3 The second pixel P21 in the first down-sampling group DSG_11 is not down-sampled. For another example, Figure 5 The second pixel P2 in the first down-sampling group DSG_11 is not down-sampled.

[0056] In addition, the image downsampling method disclosed in the present invention may also have another rule: the number of pixels in the first downsampling group may be an odd multiple of the number of the first output pixel values. Figure 3 In the embodiment of , the number of pixels of the first down-sampling group DSG_11 is 3, and the number of the first output pixel value OPV1 is 1. Figure 5In the embodiment of FIG. 1 , the number of pixels of the first down-sampling group DSG_11 is 5, and the number of the first output pixel value OPV1 is 1. In other words, every N pixels of the sensed image Img_S will generate an output pixel value of the output image Img_O, and N is an odd number. The above rule can also be applied to Figure 5 , Figure 6 and Figure 7 The second down-sampling group DSG_21 shown in the embodiment of FIG. As described above, the number of output pixel values ​​can be greater than 1.

[0057] According to the above embodiment, an image downsampling method can be obtained. The image downsampling method is used to downsample a sensed image including a plurality of first pixels and a plurality of second pixels arranged alternately. The image downsampling method comprises: according to the first pixel in the first downsampling group (for example Figure 3 , Figure 4 , Figure 5 , Figure 6 and Figure 7 The pixel value of the first pixel P1 in the first down-sampling group is used to generate a first output pixel value. The first down-sampling group includes an odd number of pixels. Other detailed steps can be known by referring to the above embodiment, so they are not repeated here.

[0058] According to the above embodiments, pixel groups with odd-numbered pixels can be used to perform downsampling of pixel values ​​to generate output images, which can improve the problem of jagged images in the prior art.

[0059] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An image downsampling method for downsampling a sensing image comprising a first pixel and a second pixel, wherein the first pixel and the second pixel are arranged alternately, characterized in that: include: generating a first output pixel value based on the pixel value of the first pixel in the first downsampled group; The first down-sampling group includes an odd number of pixels.

2. The image downsampling method according to claim 1, characterized in that: The first down-sampling group includes an even number of the first pixels and an odd number of the second pixels.

3. The image downsampling method according to claim 1, characterized in that: The first down-sampling group includes an odd number of the first pixels and an even number of the second pixels.

4. The image downsampling method according to claim 1, characterized in that: Further including: A second output pixel value is generated according to the pixel value of the second pixel in the second down-sampled group.

5. The image downsampling method according to claim 4, characterized in that: The first down-sampling group includes an even number of the first pixels and an odd number of the second pixels; The second down-sampling group includes an odd number of the first pixels and an even number of the second pixels.

6. The image downsampling method according to claim 4, characterized in that: The first down-sampling group and the second down-sampling group share the same pixels.

7. The image downsampling method according to claim 1, characterized in that: The first output pixel value is generated by averaging the pixel values ​​of the first pixels in the first down-sampled group.

8. The image downsampling method according to claim 1, characterized in that: The first output pixel value is generated based on pixel values ​​of only a portion of the first pixels in the first down-sampled group.

9. The image downsampling method according to claim 1, characterized in that: The first down-sampling group includes pixels that are not down-sampled, and the number of the pixels included in the first down-sampling group is an odd multiple of the first output pixel value.

10. The image downsampling method according to claim 1, characterized in that: The pixels of the sensed image form a Bayer array.

11. The image downsampling method according to claim 1, characterized in that: The image downsampling method is used in a surveillance camera, which does not execute the image downsampling method in a normal mode but executes the image downsampling method in a standby mode.

12. An image processing system, characterized in that: include: An image sensor for generating a sensing image including a first pixel and a second pixel, wherein the first pixel and the second pixel are arranged alternately; as well as a processing circuit for generating a first output pixel value according to a pixel value of a first pixel in a first down-sampled group; The first down-sampling group includes an odd number of pixels.

13. The image processing system according to claim 12, wherein: The first down-sampling group includes an even number of the first pixels and an odd number of the second pixels.

14. The image processing system according to claim 12, wherein: The first down-sampling group includes an odd number of the first pixels and an even number of the second pixels.

15. The image processing system according to claim 12, wherein: The processing circuit further generates a second output pixel value according to the pixel value of the second pixel in the second down-sampled group.

16. The image processing system according to claim 15, characterized in that: The first down-sampling group includes an even number of the first pixels and an odd number of the second pixels; The second down-sampling group includes an odd number of the first pixels and an even number of the second pixels.

17. The image processing system according to claim 15, characterized in that: The first down-sampling group and the second down-sampling group share the same pixels.

18. The image processing system according to claim 12, wherein: The first output pixel value is generated by averaging the pixel values ​​of the first pixels in the first down-sampled group.

19. The image processing system according to claim 12, wherein: The first output pixel value is generated based on pixel values ​​of only a portion of the first pixels in the first down-sampled group.

20. The image processing system according to claim 12, wherein: The first down-sampling group includes pixels that are not down-sampled, and the number of the pixels included in the first down-sampling group is an odd multiple of the first output pixel value.

21. The image processing system according to claim 12, wherein: The pixels of the sensed image form a Bayer array.

22. The image processing system according to claim 12, wherein: The image processing system is used in a monitoring camera, which does not execute the image downsampling method in a normal mode but executes the image downsampling method in a standby mode.