Image processing system and image processing method
By combining visible and invisible light sensors in the monitoring system, the start and stop of the invisible ISP is controlled by using an event detector, which solves the problems of system energy waste and insufficient event detection, and achieves efficient dangerous event detection and low-power operation.
Patent Information
- Application Number
- CN202510116858.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-01-28
- Filing Date
- 2025-01-24
- Publication Date
- 2025-07-29
AI Technical Summary
The existing monitoring system continuously operates the invisible ISP when invisible images are not required, resulting in waste of energy and is unable to effectively detect invisible events such as arcs or high temperatures.
Use a visible light sensor and an invisible light sensor combined with an event detector to start or disable the invisible light ISP according to a specific event, and generate invisible light images only if needed.
It realizes reducing system power consumption when invisible images are not required, and can effectively detect invisible light events, save energy, and improve the detection ability of dangerous events.
Smart Images
Figure CN120390128A_ABST
Abstract
Description
Background Art
[0002] In modern surveillance systems, it is often necessary to process visible light and invisible light (such as infrared) images simultaneously. These systems typically include sensors for sensing visible light and invisible light, as well as corresponding image signal processors (ISPs) to process the sensed signals. However, the operation of the invisible light ISP may consume more power. Therefore, when invisible light images are not required, continuously operating the invisible light ISP will cause unnecessary energy waste. However, if the image sensor in the surveillance system is only a visible light sensor, it can only sense visible light. Therefore, it is difficult to detect some dangerous events that emit invisible light, such as electric arcs or high temperatures.
Summary of the Invention
[0003] An object of the present application is to provide an image processing system that can detect dangerous events that cannot be discovered based on visible light images.
[0004] Another object of the present application is to provide an image processing method that can detect dangerous events that cannot be discovered based on visible light images.
[0005] An embodiment of the present application is to provide an image processing system, including: a visible light sensor configured to generate a visible light sensing signal; a visible light image signal processor (ISP) configured to generate a visible light image based on the visible light sensing signal; an invisible light sensor configured to generate an invisible light sensing signal; an invisible light ISP configured to generate an invisible light image based on the invisible light sensing signal; and an event detector configured to start or stop the invisible light ISP according to whether a specific event occurs.
[0006] Another embodiment of the present application is to provide an image processing method applied to an image processing system including a visible light sensor, a visible light ISP, an invisible light sensor, and an invisible light ISP, including: (a) generating a visible light sensing signal by the visible light sensor; (b) generating a visible light image by the visible light ISP based on the visible light sensing signal; (c) generating an invisible light sensing signal by the invisible light sensor; (d) starting or stopping the invisible light ISP according to whether a specific event occurs, where the invisible light ISP is configured to generate an invisible light image based on the invisible light sensing signal.
[0007] The above-mentioned image processing system and image processing method can be applied to a surveillance system.
[0008] According to the above embodiments, dangerous events that cannot be detected based on visible light images can be detected. In addition, if no dangerous event occurs, components that generate invisible light images or combined images including invisible light images can be deactivated. In this way, the power consumption of the entire system can be reduced.
[0009] These and other objects of the present application will no doubt become apparent to those of ordinary skill in the art upon reading the preferred embodiments described in detail below and viewing the various figures and diagrams.
Description of the Drawings
[0010] Figure 1 is a block diagram showing an image processing system according to an embodiment of the present application.
[0011] Figure 2 is a schematic diagram showing an invisible light image with a curved image according to an embodiment of the present application.
[0012] Figure 3 、 Figure 4 and Figure 5 are block diagrams showing an image processing system according to different embodiments of the present application.
[0013] Figure 6 is according to an embodiment of the present application, showing Figures 1 to 5 the power consumption rate of the image processing system shown in
[0014] Figure 7 is a block diagram showing a monitoring system according to an embodiment of the present application.
[0015] Figure 8 is a flowchart showing an image processing method according to an embodiment of the present application.
Detailed Description of the Embodiments
[0016] Several embodiments are provided in the following description to explain the concept of the present invention. The methods in the following description can be executed by a program stored in a non-transitory calculator-readable recording medium (such as a hard disk, optical disk, or memory). In addition, the terms "first", "second", "third" in the following description are only used to distinguish different elements and do not mean the order of the elements. For example, the first device and the second device only mean that these devices can have the same structure, but different devices.
[0017] Figure 1 is a block diagram of an image processing system 100 shown according to an embodiment of the present application. As Figure 1As shown, the image processing system 100 includes a visible light sensor 101, a visible light image signal processor (ISP) 103, an invisible light sensor 105, an invisible light ISP 107, and an event detector 109. The visible light sensor 101 is configured to generate a visible light sensing signal SS_VL. For example, the visible light sensor 101 includes a plurality of pixel circuits that generate electric charges as the visible light sensing signal SS_VL in response to the received visible light. The visible light ISP 103 is configured to generate a visible light image Img_VL based on the visible light sensing signal SS_VL.
[0018] The invisible light sensor 105 is configured to generate an invisible light sensing signal SS_IVL. For example, the invisible light sensor 105 includes a plurality of pixel circuits that generate electric charges as the invisible light sensing signal SS_IVL in response to the received invisible light. In one embodiment, the invisible light sensor 105 is configured to sense ultraviolet light, near-infrared light, or thermal light (LWIR light), but is not limited thereto. The invisible light ISP 107 is configured to generate an invisible light image Img_IVL based on the invisible light sensing signal SS_IVL. The event detector 109 is configured to activate or deactivate the invisible light ISP 107 according to whether a specific event is detected. The event detector 109 may include various circuits, such as processing circuits or logic gates, to perform its functions.
[0019] In Figure 1 an embodiment, the image processing system 100 further includes a gate 111, which can be implemented by at least one logic gate, but is not limited thereto. In this case, the event detector 109 is configured to open or close the gate 111. If the gate 111 is open, the invisible light sensing signal SS_IVL is transmitted to the invisible light ISP 107, thereby activating the invisible light ISP 107. On the contrary, if the gate 111 is closed, the invisible light sensing signal SS_IVL is not transmitted to the invisible light ISP 107, thereby deactivating the invisible light ISP 107. However, in other embodiments, the gate 111 can be removed, and the invisible light ISP 107 can be activated or deactivated by other mechanisms.
[0020] In one embodiment, the specific event is that the invisible light sensing signal SS_IVL conforms to a specific rule. For example, the specific rule is that the change in the invisible light sensing signal SS_VL exceeds a change threshold. More specifically, if a dangerous event that generates invisible light occurs, two or more consecutive invisible light images Img_IVL may have different image contents. Figure 2It is a diagram schematically showing an invisible light image with an electric arc. In the invisible light image Img_IVL1, the wire 201 is operating normally. However, in the invisible light image Img_IVL2 following the invisible light image Img_IVL1, an electric arc 203 occurs, so the invisible light sensor 105 can sense the ultraviolet light generated by the electric arc 203. Therefore, the consecutive invisible light images Img_IVL1 and Img_IVL2 have different image contents, which means there is a large change in the invisible light sensing signal SS_IVL. Therefore, if the change in the invisible light sensing signal SS_IVL exceeds the change threshold, it may mean that a dangerous event has occurred.
[0021] In another embodiment, the specific rule is that the pixel value distribution of the invisible light sensing signal SS_IVL conforms to a predetermined distribution. As Figure 2 in the example described, the invisible light sensor 105 may sense the ultraviolet light generated by the electric arc 203 in the invisible light image Img_IVL2, and the pixel value of the ultraviolet light may conform to the predetermined distribution that may be recorded in the event detector 109. Therefore, if the pixel value distribution of the invisible light sensing signal SS_IVL conforms to the predetermined distribution, it may mean that a dangerous event has occurred. Note that Figure 2 the electric arc 203 is used as an example for explanation. However, the concept provided in this application can be applied to any other dangerous event that generates invisible light.
[0022] Figure 3 It is a block diagram of an image processing system according to another embodiment of the present application. In Figure 3 this embodiment, the components of the image processing system 300 are the same as those of the image processing system 100 in Figure 4 but the specific event is the trigger signal TS for triggering the event detector 109 to activate the invisible light ISP 107. The trigger signal TS can be generated by various methods. In one embodiment, the trigger signal TS is generated by a thermal sensor. For example, if the thermal sensor senses that the temperature of a part of the device or building exceeds the temperature threshold, the trigger signal TS can be sent to activate the invisible light ISP 107, so that a fire that cannot be found in the visible light image Img_VL can be rechecked or confirmed according to the invisible light image Img_IVL.
[0023] In another embodiment, the trigger signal TS is generated by a smoke detector. Similarly, if the smoke detector detects smoke, the trigger signal TS can be sent to activate the invisible light ISP 107, so that a fire that cannot be found in the visible light image Img_VL can be checked or confirmed again based on the invisible light image Img_IVL. In addition to the thermal sensor and the smoke detector, the trigger signal TS can also be generated by other methods. For example, the trigger signal TS can be generated by a voltage detector or manually.
[0024] In Figure 1 and Figure 3 embodiments, the visible light image signal processor (ISP) 103 may also operate according to a specific event. In one embodiment, if a specific event is not detected, the visible light ISP 103 is activated and the invisible light ISP 107 is deactivated; if a specific event occurs, the visible light ISP 103 is deactivated and the invisible light ISP 107 is activated. In this way, the power consumption of the image processing system 100 can be saved. However, in another embodiment, the visible light ISP 103 may always remain active to generate the visible light image Img_VL.
[0025] The image processing system provided by this application may have other structures. Figure 4 is a block diagram of an image processing system 400 shown according to an embodiment of this application. As Figure 4 shown, the image processing system 400 includes a visible light sensor 101, a visible light ISP 103, an invisible light sensor 105, an invisible light ISP 107, an event detector 109, a buffer 401, and an image combining circuit 403.
[0026] The visible light sensor 101 is configured to generate a visible light sensing signal SS_VL. The visible light ISP 103 is configured to generate a visible light image Img_VL according to the visible light sensing signal SS_VL. The invisible light sensor 105 is configured to generate an invisible light sensing signal SS_IVL. In one embodiment, the invisible light sensor 105 is configured to sense ultraviolet light, near-infrared light, or thermal light (long-wave infrared light), but is not limited thereto. The invisible light ISP 107 is configured to generate an invisible light image Img_IVL according to the invisible light sensing signal SS_IVL. The event detector 109 is configured to activate or deactivate the invisible light ISP 107 according to whether a specific event occurs. The event detector 109 may include various circuits, such as processing circuits or logic gates, to perform its functions.
[0027] In Figure 4In an embodiment, the image processing system 100 further includes a shutter 111, which may be implemented by at least one logic gate, but is not limited thereto. In this case, the event detector 109 is configured to open or close the shutter 111. If the shutter 111 is open, the invisible light sensing signal SS_IVL is transmitted to the invisible light ISP 107, thereby activating the invisible light ISP 107. Conversely, if the shutter 111 is closed, the invisible light sensing signal SS_IVL is not transmitted to the invisible light ISP 107, thereby deactivating the invisible light ISP 107. However, in other embodiments, the shutter 111 may be removed, and the invisible light ISP 107 may be activated / deactivated by other mechanisms.
[0028] The image combining circuit 403, such as a graphics processing unit (GPU), is configured to combine the visible light image Img_VL and the invisible light image Img_IVL to generate a combined image Img_C. For example, the image corresponding to the invisible light in the invisible light image Img_IVL may be combined into the visible light image Img_VL. In one embodiment, the event detector 109 is further configured to activate or deactivate the image combining circuit 403 according to whether a specific event occurs. The buffer 401 is configured to buffer the visible light image Img_VL and the invisible light image Img_IVL for combination. However, if the visible light image Img_VL and the invisible light image Img_IVL are well synchronized, the buffer 401 may be removed.
[0029] In Figure 4 an embodiment, the specific event is that the invisible light sensing signal SS_IVL conforms to a specific rule. As described above, in one embodiment, the specific rule is that the change in the invisible light sensing signal SS_VL exceeds a change threshold. In addition, in another embodiment, the specific rule is that the pixel value distribution of the invisible light sensing signal SS_IVL conforms to a predetermined distribution.
[0030] Figure 5 is a block diagram of an image processing system according to another embodiment of the present application. In Figure 5 an embodiment, the components of the image processing system 500 are the same as those of the Figure 4 image processing system 400, but the specific event is a trigger signal TS for triggering the event detector TS to activate the invisible light ISP 107. The trigger signal TS can be generated by various methods. As described above, in one embodiment, the trigger signal TS is generated by a thermal sensor. For example, if the thermal sensor senses that the temperature of a part of a device or building exceeds a temperature threshold, the trigger signal TS can be sent to activate the invisible light ISP 107, so that a fire that cannot be detected in the visible light image Img_VL can be rechecked or confirmed according to the invisible light image Img_IVL.
[0031] As described above, in another embodiment, the trigger signal TS is generated by the smoke detector. Similarly, if the smoke detector detects smoke, the trigger signal TS can be sent to activate the invisible light ISP 107, so that the fire that cannot be found in the visible light image Img_VL can be checked or confirmed again based on the invisible light image Img_IVL. The trigger signal TS can be generated by other methods in addition to the thermal sensor and the smoke detector. For example, the trigger signal TS can be generated by a voltage detector or manually.
[0032] In Figure 4 and Figure 5 's embodiment, the visible light ISP 103 is activated both when a specific event occurs and does not occur. If the specific event does not occur, the invisible light ISP 107 is deactivated, so the invisible light image Img_IVL is not generated. Therefore, in this case, the combined image Img_C is not generated or only contains the content of the visible light image Img_VL. On the contrary, if the specific event occurs, the invisible light ISP 107 is activated to generate the invisible light image Img_IVL, so the combined image Img_C contains the content of the visible light image Img_VL and the content of the invisible light image Img_IVL.
[0033] As described above, when the specific event does not occur, the invisible light image signal processor (invisible light ISP) 107 and / or the image combining circuit 403 may be in a deactivated state. Therefore, the power consumption rate of the image processing system may be different in different states. Figure 6 is a schematic diagram showing Figures 1 to 4 the power consumption rate of the image processing system shown in Figure 6 's embodiment. Note that in Figure 6 's embodiment, the visible light image signal processor (visible light ISP) may be in an activated state in all states. As Figure 6 shown, at the beginning of state 1, the specific event does not occur (low logic level), the invisible light ISP 103 and / or the image combining circuit 403 are in a deactivated state, so the power consumption rate of the image processing system is the lower first power consumption rate. At the end of state 1, the specific event occurs (high logic level), so the image processing system enters state 2. In state 2, the invisible light ISP 103 and / or the image combining circuit 403 switch from deactivated to activated. In state 3, the invisible light ISP 103 and / or the image combining circuit 403 are in an activated state, so the power consumption rate increases to the higher second power consumption rate.
[0034] The image processing system provided by this application can be applied to various applications. In one embodiment, the image processing system is applied to a monitoring system, such as a surveillance camera. Figure 7 is a schematic diagram. According to an embodiment of the present application, the monitoring system 700 is illustrated. As Figure 7 shown, the monitoring system 700 includes an image processing system 701 and a screen 703. The image processing system 701 may include Figure 1 , Figure 3 , Figure 4 and Figure 5 the structures shown therein, and can output the aforementioned visible light image Img_VL, invisible light image Img_IVL, or combined image Img_C to the screen 703 for display. In this way, the user can monitor the target to be monitored according to the visible light image Img_VL, invisible light image Img_IVL, or combined image Img_C.
[0035] In one embodiment, the monitoring system 700 may further include a control circuit 705, which is configured to control an alarm generation circuit 709 to generate an alarm according to the invisible light image Img_IVL or the combined image Img_C. The alarm can be, for example, a sound alarm, a light alarm, or a message transmitted to the user's mobile device.
[0036] According to the above embodiment, an image processing method can be obtained. Figure 8 is a flowchart. According to an embodiment of the present application, the image processing method is illustrated. The image processing method includes the following steps:
[0037] Step 801
[0038] Generate a visible light sensing signal by a visible light sensor (for example, visible light sensor 101).
[0039] Step 803
[0040] Generate a visible light image by a visible light image signal processor (for example, visible light ISP 103) according to the visible light sensing signal.
[0041] Step 805
[0042] Generate an invisible light sensing signal by an invisible light sensor (for example, invisible light sensor 105).
[0043] Step 807
[0044] Start or stop an invisible light ISP (for example, invisible light ISP 107) according to whether a specific event occurs.
[0045] The invisible light ISP is configured to generate an invisible light image according to the invisible light sensing signal.
[0046] According to the above-described embodiments, dangerous events that cannot be detected based on visible light images can be detected. Additionally, if no dangerous event occurs, components for generating a combined image including an invisible light image can be deactivated. In this way, power consumption of the entire system can be reduced.
[0047] Those skilled in the art of technology will readily observe that many modifications and changes can be made to the apparatus and method while retaining the teaching of the invention. Accordingly, the above disclosure should be limited only to the scope and bounds of the appended claims.
Claims
1. An image processing system, comprising: A visible light sensor configured to generate a visible light sensing signal; A visible light image signal processor (ISP) configured to generate a visible light image according to the visible light sensing signal; An invisible light sensor configured to generate an invisible light sensing signal; An invisible light ISP configured to generate an invisible light image according to the invisible light sensing signal; and An event detector configured to activate or deactivate the invisible light ISP according to whether the event detector detects a specific event.
2. The image processing system according to claim 1, wherein the specific event is that the invisible light sensing signal conforms to a specific rule.
3. The image processing system according to claim 2, wherein the specific rule is that the change of the invisible light sensing signal exceeds a change threshold.
4. The image processing system according to claim 2, wherein the specific rule is that the pixel value distribution of the invisible light sensing signal conforms to a predetermined distribution.
5. The image processing system according to claim 1, wherein the specific event is a trigger signal for triggering the event detector to activate the invisible light ISP.
6. The image processing system according to claim 5, wherein the trigger signal comes from a thermal sensor or a smoke detector.
7. The image processing system according to claim 6, wherein the event detector is triggered to allow the invisible light sensing signal to be transmitted to the invisible light ISP.
8. The image processing system according to claim 1, further comprising: An image combining circuit configured to combine the visible light image and the invisible light image to generate a combined image; wherein the event detector is further configured to activate or deactivate the image combining circuit according to whether the specific event is detected.
9. The image processing system according to claim 1, wherein the image processing system has a first power consumption rate when the specific event is not detected and a second power consumption rate after the specific event is detected, and the second consumption rate is higher than the first consumption rate.
10. An image processing method applied to an image processing system including a visible light sensor, a visible light image signal processor (ISP), an invisible light sensor, and an invisible light ISP, comprising: (a) Generating a visible light sensing signal by the visible light sensor; (b) Generating a visible light image by the visible light ISP according to the visible light sensing signal; (c) Generating an invisible light sensing signal by the invisible light sensor; (d) Activating or deactivating the invisible light ISP according to whether a specific event is detected, wherein the invisible light ISP is configured to generate an invisible light image according to the invisible light sensing signal.
11. The image processing method according to claim 10, wherein the specific event is that the invisible light sensing signal conforms to a specific rule.
12. The image processing method according to claim 11, wherein the specific rule is that the change of the invisible light sensing signal exceeds a change threshold.
13. The image processing method according to claim 11, wherein the specific rule is that the pixel value distribution of the invisible light sensing signal conforms to a predetermined distribution.
14. The image processing method according to claim 10, wherein the specific event is a trigger signal for triggering the event detector to activate the invisible light ISP.
15. The image processing method according to claim 14, wherein the trigger signal comes from a thermal sensor or a smoke detector.
16. The image processing method according to claim 14, wherein the event detector is triggered to allow the invisible light sensing signal to flow to the invisible light ISP.
17. The image processing method according to claim 10, further comprising: combining the visible light image and the invisible light image to generate a combined image; wherein step (d) further activates or deactivates the image combining circuit according to whether the specific event is detected.
18. The image processing method according to claim 10, wherein the image processing system has a first power consumption rate when the specific event does not occur, and a second power consumption rate after the specific event occurs, wherein the second consumption rate is higher than the first consumption rate.