Pre-storage circuit and image sensing system and method thereof

Abstract

The invention relates to a pre-storage circuit and an image sensing system and method thereof. A pre-storage circuit is used for an image sensing system, and the pre-storage circuit is used for receiving pre-stored image data through an image sensor and providing the pre-stored image data to a camera device in the image sensing system. The pre-storage circuit comprises a first memory and a compressor circuit. The first memory is used for storing the pre-stored image data. The compressor circuit is coupled to the first memory and is used for compressing the pre-stored image data before the pre-stored image data is stored in the first memory.

Description

Technical Field

[0001] The present invention relates to a pre-storage circuit, and in particular to a pre-storage circuit for an image sensing system and an image sensing system and method thereof. Background Art

[0002] In a common long-power camera system, the front end is an image sensor that captures the image and transmits it to the camera device through a transmission interface. The image signal processor (ISP), video encoder, image encoder and other devices in the camera device perform a series of image signal processing and then perform subsequent output. This type of camera system can transmit the processed image data to the back end, such as through a wired or wireless transmission interface to the user end for real-time online viewing.

[0003] Because the image sensor needs to obtain images around the clock, and the camera system continuously processes images in real time, this system consumes a lot of power and needs to be connected to an external AC power supply. Another common camera system is a battery-based system that provides power through batteries. In a battery-based system, the camera device cannot be turned on for a long time to process image signals, but must be activated when a specific event is triggered to save power. Summary of the invention

[0004] The main purpose of the present invention is to provide a pre-storage circuit that can be used in a battery-based image sensing system to perform a pre-storage (or pre-roll) function (or pre-roll function) of a camera device to provide a pre-stored image before the camera device is activated, while achieving good resolution to enhance the user experience.

[0005] One embodiment of the present invention discloses a pre-storage circuit for an image sensing system, the pre-storage circuit is used to receive a pre-stored image data through an image sensor and provide the pre-stored image data to a camera device. The pre-storage circuit includes a first memory and a compressor circuit. The first memory is used to store the pre-stored image data. The compressor circuit is coupled to the first memory and is used to compress the pre-stored image data before storing the pre-stored image data in the first memory.

[0006] Another embodiment of the present invention discloses an image sensing system, which includes an image sensor, a camera device, a motion sensor and a pre-storage circuit. The image sensor is used to obtain a plurality of image data. The camera device is coupled to the image sensor and is used to receive a real-time image data from the image sensor among the plurality of image data. The motion sensor is coupled to the camera device and is used to wake up the camera device when an object movement is detected. The pre-storage circuit is coupled to the image sensor and the camera device and is used to receive a pre-stored image data from the image sensor among the plurality of image data and provide the pre-stored image data to the camera device. The pre-storage circuit includes a first memory and a compressor circuit. The first memory is used to store the pre-stored image data. The compressor circuit is coupled to the first memory and is used to compress the pre-stored image data before the pre-stored image data is stored in the first memory.

[0007] Another embodiment of the present invention discloses an image sensing method, comprising obtaining a plurality of image data; receiving a pre-stored image data among the plurality of image data; compressing the pre-stored image data and storing the compressed pre-stored image data in a first memory; waking up a camera device when detecting the movement of an object to receive a real-time image data among the plurality of image data; and providing the pre-stored image data stored in the first memory to the camera device. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 FIG. 4 is a schematic diagram of a battery-based image sensing system according to an embodiment of the present invention.

[0009] Figure 2 FIG. 4 is a schematic diagram of an image sensing system according to another embodiment of the present invention.

[0010] Figure 3 The diagram shows a coding structure of a series of input frame image data using single reference frame prediction.

[0011] Figure 4 FIG. 4 is a schematic diagram of an image sensing process according to an embodiment of the present invention. DETAILED DESCRIPTION

[0012] Figure 1FIG. 1 is a schematic diagram of a battery-based image sensing system 10 according to an embodiment of the present invention. The image sensing system 10 includes an image sensor 102, a camera device 104, a motion sensor 106, and a pre-storage circuit 108. The image sensor 102 can be used to obtain image data. It can be a monitoring device with a camera lens, which is set at a location that the user wants to monitor, such as a store, a courtyard, a door, etc. The camera device 104 is coupled to the image sensor 102 and can receive real-time image data from the image sensor 102. The camera device 104 can include various signal processing modules, such as an image signal processor (ISP), a video encoder, an image encoder, etc., to select a suitable method for signal processing according to the type of received image data. In one embodiment, the camera device 104 can be a system on chip (SoC), which can integrate various signal processing modules into an integrated circuit in a chip. In addition, according to system requirements, the camera device 104 may also include other necessary control devices or peripheral modules, such as a central processing unit (CPU), a memory, a wireless communication device (such as a Wi-Fi communication circuit), etc. Without affecting the description of this embodiment, these devices or modules are omitted. Figure 1 middle.

[0013] The motion sensor 106 is coupled to the camera device 104 and can be used to control the operation of the camera device 104. As described above, in the battery-based image sensing system 10, the battery can provide limited power and the system cannot work for a long time. Therefore, the camera device 104 can start to operate when a specific event is triggered. In this example, the camera device 104 is usually in a dormant state. When the motion sensor 106 detects the movement of an object, the motion sensor 106 can output an activation signal to wake up the camera device 104, so that the camera device 104 starts to receive real-time image data from the image sensor 102 and performs appropriate image signal processing on the received image data. The camera device 104 can also return to the dormant state when there is no image change for a period of time. In one embodiment, the motion sensor 106 can be a passive infrared (PIR) motion sensor, which is used to detect moving objects in the scene.

[0014] Generally speaking, there is no object moving in the scene most of the time, so the camera device 104 will not be activated, so that a large amount of power consumption required for the camera device 104 to perform signal processing is saved. The camera device 104 starts recording images and processing signals only when it receives a notification from the motion sensor 106, which can avoid long-term operation, achieve the purpose of saving power, and extend the cycle time of replacing batteries. However, the above approach allows the user to only observe the video recorded after the moving object is detected, and cannot obtain the image before the moving object enters the scene. Therefore, a pre-storage circuit 108 is further provided in the image sensing system 10. The pre-storage circuit 108 is coupled between the image sensor 102 and the camera device 104, and can be used to pre-store image data from the image sensor 102. Regardless of whether there is a moving object in the scene, the pre-storage circuit 108 will continue to capture images and continuously write the image data into the static random access memory (SRAM) 110 in the pre-storage circuit 108 through a ring buffer. When the motion sensor 106 detects a moving object, the camera device 104 starts to receive real-time image data from the image sensor 102, and the pre-storage circuit 108 provides the pre-stored image data to the camera device 104. In this way, the user can observe the front and back images when the moving object enters the scene.

[0015] However, due to cost considerations, the static random access memory 110 inside the pre-storage circuit 108 has a relatively small space, and the number of image frames that can be pre-stored is very limited. In order to meet the requirement of a certain number of frames, the pre-storage circuit 108 usually adopts a low-resolution configuration to increase the length of the pre-stored image. However, this will make it easy for users to see the difference in image quality between the pre-stored image and the real-time image, resulting in a decrease in user experience.

[0016] To solve the problems of the above embodiments, the present invention proposes a system and method for storing pre-stored image data using a pseudo-static random access memory (Pseudo SRAM, PSRAM), and writing the pre-stored image data into the pseudo-static random access memory through a ring buffer. Figure 2 FIG. 2 is a schematic diagram of an image sensing system 20 according to another embodiment of the present invention. The difference between the image sensing system 20 and the image sensing system 10 is that the image sensing system 20 is provided with a pre-storage circuit 208 to replace the pre-storage circuit 108, and the camera device 104 of the image sensing system 20 further includes a decompressor circuit 204. The other devices or modules in the image sensing system 20 are similar to Figure 1 The image sensing system 10 is therefore represented by the same symbols, and its related operations can be referred to the description in the above paragraphs and will not be repeated here.

[0017] like Figure 2 As shown, the pre-storage circuit 208 includes an internal circuit 210 and a pseudo-static random access memory 220. The internal circuit 210 may be an integrated circuit disposed in a companion chip, and is used to operate in conjunction with the system chip of the camera device 104. The pseudo-static random access memory 220 may be an external memory device, which may be disposed on the same circuit board as the chip of the internal circuit 210, and is externally connected to the internal circuit 210 through a wire. It should be noted that in another embodiment, the pseudo-static random access memory 220 may also be built into the internal circuit 210, that is, the pseudo-static random access memory 220 and the internal circuit 210 may be integrated into the same chip.

[0018] Compared with the smaller space of a general static random access memory, the pseudo-static random access memory 220 used in the present invention has a larger capacity (generally tens of megabytes (MB)), so it can store more frames or higher resolution image data, and can be properly configured for resolution, frame rate, and time length to meet the requirements of product specifications. For example, assuming that the capacity of the pseudo-static random access memory 220 is 32MB, in terms of the 12-bit Bayer Raw format, it can store 24 frames of 720p image screens, such as pre-stored image data with a length of 6 seconds and a frame rate of 4 (frames / second).

[0019] However, for data reading and writing, the power consumption of the pseudo-static random access memory 220 is about 5 to 10 times that of a general static random access memory. It should be noted that the image sensing system 20 of the present invention is applied to battery-type products, and its power consumption is often one of the important considerations of the product. In order to reduce the power consumption generated by the reading and writing of data in the pseudo-static random access memory 220, the amount of data written into the pseudo-static random access memory 220 for each frame of pre-stored image should be reduced. Therefore, the internal circuit 210 includes a compressor circuit 212, which can compress the pre-stored image data before storing it in the pseudo-static random access memory 220. In this way, the pre-stored image data stored in the pseudo-static random access memory 220 is compressed image data, which greatly reduces the power consumption required for reading and writing it. The compressor circuit 212 can use any suitable algorithm to achieve compression of image data, such as lossless compression or lossy compression. Lossless compression can achieve the compression effect without affecting the image quality, while lossy compression can achieve a higher compression rate. A suitable compression method can be selected according to the product's requirements for image quality, and the selection of the compression method should not limit the scope of the present invention.

[0020] In one embodiment, to reduce the complexity of the compression algorithm and the power consumption of the compressed data operation, the compressor circuit 212 may use Differential Pulse Code Modulation (DPCM) technology for compression, which includes spatial DPCM and temporal DPCM. Temporal DPCM may refer to image data of different frames for compression, and may achieve a better compression rate when the image data of the pre-stored image data does not change much.

[0021] In one embodiment, a single reference prediction image compression method may be used, and the internal circuit 210 is provided with a static random access memory 214 to store the reference frame image data required for compression coding. Figure 3 A series of input frame image data using a single reference frame prediction coding structure is shown, which includes S frames and T frames, wherein the S frame uses a spatial domain error signal coding method, and generates compressed data through intra-frame prediction and writes it into the pseudo-static random access memory 220. While the S frame is being compressed, the image data of the S frame can also be stored in the static random access memory 214 in the internal circuit 210. Since the space of the static random access memory 214 is limited, the image data of the S frame can be downsampled and then stored in the static random access memory 214. Therefore, the next received T frame can use a temporal domain error signal coding method, use the downsampled image data stored in the static random access memory 214 as a reference frame, generate compressed data through inter-frame prediction and write it into the pseudo-static random access memory 220. Similarly, the image data of the T frame will also be downsampled and stored in the static random access memory 214 as a reference frame for subsequent T frames to be compressed.

[0022] As described above, when the camera device 104 is awakened by the motion sensor 106, it starts to receive real-time image data from the image sensor 102. At the same time, the camera device 104 can also receive pre-stored image data from the pseudo-static random access memory 220. In this example, the image data stored in the pseudo-static random access memory 220 is compressed data. Correspondingly, a decompressor circuit 204 is further provided in the camera device 104 to decompress the pre-stored image data received from the pseudo-static random access memory 220, and then perform image processing through various signal processing modules.

[0023] It is worth noting that the purpose of the present invention is to provide a pre-storage circuit that can be used in an image sensing system to improve the quality of pre-stored image data of a battery-type system. A person skilled in the art can make modifications or changes accordingly, but is not limited to this. For example, the present invention uses a compressor circuit to compress the pre-stored image data and then store it in a pseudo-static random access memory. The compressor circuit can use various suitable compression algorithms, but is not limited to the methods mentioned in this specification. In one embodiment, if the static random access memory 214 has enough space, the S frame or T frame used as a reference frame can be directly stored in the static random access memory 214 and the sampling operation can be omitted. In another embodiment, spatial domain error signal encoding can be performed for each frame of image data without using temporal domain error signal encoding (that is, all input frames are S frames). In this way, there is no need to use a reference frame for inter-frame prediction, so the capacity of the static random access memory 214 can be further reduced to reduce the cost of the internal circuit 210. In addition, the compressor circuit of the present invention may also adopt other encoding architectures, such as JPEG (Joint Photographic Experts Group, JPEG) encoding. In fact, various transform-based image compression methods can be applied to the compressor circuit of the present invention.

[0024] In addition, the type of memory should not be used to limit the scope of the present invention. For example, the above-mentioned embodiment uses an external pseudo-static random access memory 220 to store pre-stored image data; in other embodiments, the pseudo-static random access memory 220 can also be integrated into the internal circuit 210. Alternatively, other types of memory can be used to store pre-stored image data, such as dynamic random access memory (DRAM) or flash memory (Flash Memory), but not limited to this. In addition, the static random access memory 214 used to store reference frames can also be replaced by other types of memory, but not limited to this. In another embodiment, in order to further reduce the cost of the image sensing system, the pre-storage circuit and / or the pseudo-static random access memory can be integrated with the camera device to realize a single-chip system.

[0025] The operation of the image sensing system of the present invention can be summarized as an image sensing process 40. Figure 4 The image sensing process 40 can be applied to Figure 2 The image sensing system 20 comprises the following steps:

[0026] Step 402: The image sensor 102 obtains a plurality of image data.

[0027] Step 404: The pre-storage circuit 208 receives pre-stored image data among the plurality of image data.

[0028] Step 406: The compressor circuit 212 compresses the pre-stored image data and stores the compressed pre-stored image data in a first memory.

[0029] Step 408: When the motion sensor 106 detects movement of an object, the camera device 104 is awakened to enable the camera device 104 to receive the real-time image data from the plurality of image data.

[0030] Step 410 : The pre-storage circuit 208 provides the pre-stored image data stored in the first memory to the camera device 104 .

[0031] In the image sensing process 40, the first memory may be the pseudo-static random access memory 220 in the above embodiment or other suitable memory devices. The detailed operation and variation of the image sensing process 40 may be referred to the description in the above paragraphs, which will not be repeated here.

[0032] In summary, the present invention proposes a pre-storage circuit and an image sensing method thereof that can be used in a battery-type image sensing system. The pre-storage circuit can use a pseudo-static random access memory to store pre-stored image data. Compared with a general static random access memory, a pseudo-static random access memory has a larger capacity and can store more frames or higher resolution image data. In order to save power consumption in reading and writing the pseudo-static random access memory, a compressor circuit can be set in the pre-storage circuit to compress the pre-stored image data before writing it to the pseudo-static random access memory. When the camera device is awakened and needs to read the pre-stored image data, it can first be decompressed through a decompressor circuit, and then various image signal processing can be performed on the restored pre-stored image data. In this way, the pre-stored image data obtained by the camera device can have a resolution and image quality comparable to that of the real-time image data, and a good user experience can be achieved.

[0033] The above description is only a preferred embodiment of the present invention. All equivalent changes and modifications made according to the scope of the patent application of the present invention should fall within the scope of the present invention.

[0034]

Explanation of symbols

[0035] 10,20: Image sensing system

[0036] 102: Image sensor

[0037] 104: Camera device

[0038] 106: Motion sensor

[0039] 108,208: Pre-storage circuit

[0040] 110,214: Static Random Access Memory

[0041] 210: Internal Circuit

[0042] 212: Compressor circuit

[0043] 220: Pseudo-static random access memory

[0044] 204: Decompressor circuit

[0045] 40: Image Sensing Process

[0046] 402~410: Steps

Claims

1. A pre-storage circuit for an image sensing system, the pre-storage circuit is used to receive pre-stored image data through an image sensor and provide the pre-stored image data to a camera device, the pre-storage circuit comprising: a first memory, used to store the pre-stored image data; and A compressor circuit is coupled to the first memory and is used to compress the pre-stored image data before storing the pre-stored image data in the first memory.

2. The pre-storage circuit according to claim 1, wherein the first memory is a pseudo-static random access memory.

3. The pre-storage circuit according to claim 1, wherein the camera device comprises: A decompressor circuit is used to decompress the pre-stored image data received from the first memory. 4 . The pre-storage circuit according to claim 1 , wherein the pre-storage circuit is coupled to a motion sensor, and the motion sensor is used to wake up the camera device when a movement of an object is detected. 5 . The pre-storage circuit according to claim 4 , wherein when the camera device is awakened by the motion sensor, it starts to receive real-time image data from the image sensor and receives the pre-stored image data from the first memory.

6. The pre-storage circuit according to claim 1 further comprises: a second memory, used for storing a reference frame image data in the pre-stored image data; in, The compressor circuit compresses an input frame image data in the pre-stored image data according to the reference frame image data.

7. The pre-storage circuit according to claim 6, wherein the second memory is a static random access memory.

8. An image sensing system comprising: An image sensor for acquiring a plurality of image data; A camera device, coupled to the image sensor, for receiving a real-time image data among the plurality of image data from the image sensor; a motion sensor coupled to the camera device, for waking up the camera device when a movement of an object is detected; as well as A pre-storage circuit is coupled to the image sensor and the camera device, and is used to receive a pre-stored image data from the image sensor and provide the pre-stored image data to the camera device. The pre-storage circuit includes: a first memory, used to store the pre-stored image data; as well as A compressor circuit is coupled to the first memory and is used to compress the pre-stored image data before storing the pre-stored image data in the first memory.

9. The image sensing system according to claim 8, wherein the pre-storage circuit further comprises: a second memory, used for storing a reference frame image data in the pre-stored image data; in, The compressor circuit compresses an input frame image data in the pre-stored image data according to the reference frame image data.

10. An image sensing method, comprising: Acquire a plurality of image data; receiving a pre-stored image data from the plurality of image data; compressing the pre-stored image data, and storing the compressed pre-stored image data in a first memory; When a movement of an object is detected, waking up a camera device to receive a real-time image data from the plurality of image data; and The pre-stored image data stored in the first memory is provided to the camera device.

Images