Security event determination system and method
By combining depth images and thermal images, the security event judgment system utilizes image overlay and matching operations to solve the problem of high false judgment rate in traditional monitoring systems, achieving more accurate security event judgment.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- IND TECH RES INST
- Filing Date
- 2025-02-05
- Publication Date
- 2026-06-23
AI Technical Summary
Traditional home surveillance systems are prone to misjudgment when using depth or thermal imaging, making it difficult to accurately distinguish the posture of the target object, resulting in a high misjudgment rate.
A security event judgment system combining depth and thermal images is adopted. The first and second recognition devices identify the pose of the target object, and the first and second processors perform image overlay and matching operations. Combined with confidence and weight calculations, the security status of the target object is finally determined.
It improves the accuracy of security incident judgment, reduces misjudgments, and can more accurately identify the posture of the target object and judge its security status.
Smart Images
Figure CN122265931A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to a judgment mechanism, and more particularly to a judgment system and method for judging security incidents. Background Technology
[0002] With the significant increase in the elderly population, home safety for seniors living alone has become a critical issue. Traditional home monitoring systems typically use depth imaging or thermal imaging. However, depth imaging alone is prone to misjudgments; for example, it may be difficult to distinguish whether a target in the image is an elderly person or a blanket. Similarly, thermal imaging alone can also lead to misjudgments; for example, the target in the image may be the residual warmth of an elderly person.
[0003] The aforementioned situations often lead to a high false alarm rate in traditional home monitoring systems. Therefore, there is a need for improved monitoring systems that combine the advantages of depth imaging and thermal imaging, while also improving the accuracy of home security incident detection. Summary of the Invention
[0004] According to this disclosure, a security incident determination system is provided for determining whether a target object is safe. The security incident determination system includes the following components: a first identification device for identifying the pose of the target object based on a first image to generate a first identification result, wherein the first identification result is associated with a first confidence level. A second identification device for identifying the pose of the target object based on a second image to generate a second identification result, wherein the second identification result is associated with a second confidence level, and the second image is heterogeneous to the first image. A first processor for overlaying the first image and the second image to generate an overlaid image, and performing a matching operation based on the first identification result, the second identification result, and the overlaid image to obtain a set of matching results, wherein the set of matching results includes multiple matching results. A second processor for analyzing each matching result to obtain a pose category result and a security incident determination result.
[0005] According to another aspect of this disclosure, a security event determination method is provided for determining whether a target object is safe. The security event determination method includes the following steps: A first identification device identifies the pose of the target object based on a first image to generate a first identification result, wherein the first identification result is associated with a first confidence level. A second identification device identifies the pose of the target object based on a second image to generate a second identification result, wherein the second identification result is associated with a second confidence level, and the second image is heterogeneous to the first image. A first processor overlays the first image and the second image to generate an overlaid image, and performs a matching operation based on the first identification result, the second identification result, and the overlaid image to obtain a matching result set, wherein the matching result set includes multiple matching results. A second processor analyzes each matching result to obtain a pose category result and a security event determination result.
[0006] Other aspects and advantages of this disclosure will become apparent from the following figures, detailed description, and scope of the patent application. Attached Figure Description
[0007] Figure 1 This is a block diagram of a security event determination system according to an embodiment of this disclosure.
[0008] Figure 2A This is a schematic diagram of the first recognition device analyzing the first image.
[0009] Figure 2B This is a schematic diagram of the region in the first image.
[0010] Figure 3A This is a schematic diagram of the second recognition device analyzing the second image.
[0011] Figure 3B This is a schematic diagram of the region in the second image.
[0012] Figure 4 This is a schematic diagram of the overlapping images in an embodiment.
[0013] Figure 5 The flowchart shows the first processor performing a matching operation with the overlapping image based on the first recognition result, the second recognition result, and the image.
[0014] Figure 6 This is a flowchart illustrating how the second processor analyzes and processes the matching result set to obtain attitude category results and safety event judgment results.
[0015] Figure 7 This is a schematic diagram of an overlapping image for another embodiment.
[0016] Figure 8 This is a schematic diagram of an overlapping image according to yet another embodiment.
[0017] [Symbol Explanation]
[0018] 1000: Security Incident Judgment System
[0019] 10: First image capture device
[0020] 20: Second image capture device
[0021] 50: Target object
[0022] 51, 52, 53, 54, 55, 56, 57: Target object projection
[0023] 60: Bed
[0024] 100: First identification device
[0025] 200: Second identification device
[0026] 300: First Processor
[0027] 400: Second Processor
[0028] 500: Warning device
[0029] R0, R11, R12, R2, R21, R22, R31, R32: Regions
[0030] I1: First image
[0031] I2: Second image
[0032] I3: Overlapping Images
[0033] D1: First recognition result
[0034] D2: Second recognition result
[0035] A: First level of trust
[0036] B: Second level of trust
[0037] CLS: Pose Classification Results
[0038] S: Security incident assessment result
[0039] WR: Warning signal
[0040] {M(i)}: Set of matching results
[0041] (x1, y1), (x2, y2): Coordinates of the origin
[0042] h1, h2: Height
[0043] w1, w2: Width
[0044] offsetX: Displacement amount
[0045] S500~S506, S600~S616: Steps Detailed Implementation
[0046] The technical terms used in this specification are based on common terminology in the field. Where this specification provides explanations or definitions for certain terms, the interpretation of those terms shall be based on the explanations or definitions provided in this specification. Each embodiment disclosed herein has one or more technical features. Where feasible, those skilled in the art may selectively implement some or all of the technical features in any embodiment, or selectively combine some or all of the technical features in these embodiments.
[0047] Figure 1 This is a block diagram of a security event determination system 1000 according to an embodiment of this disclosure. The security event determination system 1000 is used to determine security events of a target object 50; the target object 50 is, for example, an elderly person residing at home. In this example, the security event determination system 1000 is a processor in the form of hardware circuitry, such as a digital signal processor (DSP), a central processing unit (CPU), or a micro control unit (MCU), but is not limited thereto. Figure 1 As shown, the security event determination system 1000 includes a first identification device 100, a second identification device 200, a first processor 300, a second processor 400, and an alarm device 500. The first identification device 100, the second identification device 200, the first processor 300, the second processor 400, and the alarm device 500 are all hardware circuit units within the security event determination system 1000.
[0048] In another example, the security event determination system 1000 is a software program module, whose functions are implemented by software program code within a central processing unit (CPU) or digital signal processor (DSP). Specifically, the first identification device 100, the second identification device 200, the first processor 300, the second processor 400, and the warning device 500 within the security event determination system 1000 are all software modules, each implementing its function through software program code (which includes multiple instructions). This software program code can be stored on a non-transitory computer-readable storage medium. Examples of non-transitory computer-readable storage media include various forms of non-transitory (non-volatile) memory, hard disks, flash drives, and other storage devices. The non-transitory computer-readable storage medium can be electrically connected to or located within the CPU or DSP. The central processing unit or digital signal processor reads the aforementioned software program code from a non-transitory computer-readable storage medium and executes the instructions of the software program code to realize the functions of the first identification device 100, the second identification device 200, the first processor 300, the second processor 400, and the warning device 500.
[0049] The security incident determination system 1000 operates in conjunction with an external first image capture device 10 and a second image capture device 20. The first image capture device 10 captures an image of the target object 50 to generate a first image I1. Similarly, the second image capture device 20 captures an image of the target object 50 to generate a second image I2. The first image capture device 10 and the second image capture device 20 capture images of the target object 50 using different mechanisms. In this embodiment, the first image capture device 10 is, for example, a time-of-flight (ToF) camera, which captures images based on the depth coordinates of various parts of the target object 50. The first image I1 generated by the first image capture device 10 is a depth image of the target object 50. On the other hand, the second image capture device 20 is, for example, an infrared camera, which captures images based on the temperature of various parts of the target object 50. The second image I2 generated by the second image capture device 20 is an infrared image of the target object 50 (which may be referred to as a "thermal image"). The first image I1 is different from the second image I2 in terms of properties; that is, the first image I1 is heterogeneous to the second image I2.
[0050] The first identification device 100 of the security incident judgment system 1000 receives a first image I1 generated by the first image capture device 10. The first identification device 100 uses an artificial intelligence algorithm with a neural network-like model (e.g., the "Yolo-v3" model) to perform posture recognition, thereby identifying the posture of the target object 50 and generating a first identification result D1. The first identification result D1 includes a first posture category cls_1; the first posture category cls_1 represents the identified posture of the target object 50. For example, the first posture category cls_1 indicates that the target object 50 is in a standing, sitting, or lying posture. Furthermore, the first identification device 100 evaluates a first confidence level A of the first image capture device 10. The first confidence level A represents the degree to which the first image I1 of the first image capture device 10 can be trusted, and it also reflects the accuracy of the first identification result D1. In this example, the first confidence level A may be included in the first identification result D1. The first identification device 100 transmits the first identification result D1 and the first confidence level A to the first processor 300.
[0051] On the other hand, the second identification device 200 of the security incident judgment system 1000 receives the second image I2 generated by the second image capture device 20. Similar to the operation of the first identification device 100, the second identification device 200 also uses a neural network-like model and artificial intelligence algorithm to identify the posture of the target object 50 to generate a second identification result D2. The second identification result D2 includes a second posture category cls_2, which represents the posture of the target object 50 that is identified (e.g., standing, sitting, or lying down). Furthermore, the second identification device 200 evaluates the second confidence level B of the second image capture device 20. The second confidence level B represents the degree to which the second image I2 of the second image capture device 20 can be trusted, and it also reflects the accuracy of the identification of the second identification result D2. In this example, the second confidence level B may be included in the second identification result D2. Then, the second identification device 200 transmits the second identification result D2 and the second confidence level B to the first processor 300.
[0052] The first processor 300 receives a first recognition result D1 and a first confidence level A from the first recognition device 100, and a second recognition result D2 and a second confidence level B from the second recognition device 200. Furthermore, the first processor 300 receives a first image I1 from the first image capture device 10, and a second image I2 from the second image capture device 20. The first processor 300 performs image overlay based on the first image I1 and the second image I2. Then, the first processor 300 performs matching operations based on the first recognition result D1, the second recognition result D2, and the image overlay results to obtain multiple matching results M(i), and aggregates them into a matching result set {M(i)}. The first processor 300 then transmits the matching result set {M(i)} to the second processor 400.
[0053] The second processor 400 analyzes and processes the matching result set {M(i)} to obtain the posture category result CLS and the safety event judgment result S. The second processor 400 sequentially analyzes each matching result M(i) in the matching result set {M(i)}. Based on the matching result M(i), the second processor 400 compares the first posture category cls_1 of the first identification result D1 with the second posture category cls_2 of the second identification result D2. When the first posture category cls_1 is the same as the second posture category cls_2, the second processor 400 uses the first posture category cls_1 (or the second posture category cls_2) as the posture category result CLS. Furthermore, the second processor 400 determines whether the target object 50 is safe based on the posture category result CLS, and generates the safety event judgment result S accordingly. For example, if both the first posture category cls_1 and the second posture category cls_2 are "standing posture", the second processor 400 uses "standing posture" as the posture category result CLS. Since the target object 50 is in a standing position, it means that the target object 50 has not fallen. Therefore, the second processor 400 determines that the target object 50 is safe and outputs the safety event judgment result S as "safe".
[0054] On the other hand, the second processor 400 can also evaluate the first weight WE1 of the first image I1 and the second weight WE2 of the second image I2. When the first pose category cls_1 is different from the second pose category cls_2, the second processor 400 cannot temporarily decide whether to use the first pose category cls_1 (or the second pose category cls_2) as the pose category result CLS. Therefore, the second processor 400 determines the pose category result CLS using the first weight WE1 and the second weight WE2. For example, the second processor 400 calculates based on the first confidence level A, the second confidence level B, the first weight WE1, and the second weight WE2 to determine whether the first pose category cls_1 or the second pose category cls_2 can be adopted as the pose category result CLS.
[0055] More specifically, the first weight WE1 is associated with the first image capture device 10. The first image capture device 10 measures and captures images of the target object 50 based on the time-of-flight principle. The measured value is proportional to the distance to the target object 50, and the first image capture device 10 can provide relatively accurate distance measurements. On the other hand, the second weight WE2 is associated with the second image capture device 20. The second image capture device 20 performs imaging based on temperature differences, and can detect the heat of the target object 50 and capture its thermal image. In an indoor environment, the first weight WE1 can be set higher than the second weight WE2. In one example, the first weight WE1 is set to 0.6, and the second weight WE2 is set to 0.4.
[0056] Furthermore, the second processor 400 transmits the attitude category result CLS and the safety event judgment result S to the warning device 500; the warning device 500 can selectively generate a warning signal WR based on the safety event judgment result S. For example, when the safety event judgment result S is "safe", the warning device 500 does not generate a warning signal WR; conversely, when the safety event judgment result S is "dangerous", the warning device 500 generates a warning signal WR.
[0057] Next, please see Figure 2A This is a schematic diagram of the first recognition device 100 analyzing the first image I1. The first recognition device 100 performs posture recognition of the target object 50 based on the first image I1 to generate a first recognition result D1. More specifically, the first recognition device 100 defines multiple regions in the first image I1, including region R0 and region R11. The first recognition device 100 uses the bed 60 as a reference object and defines region R0 based on the bed 60. Furthermore, the first recognition device 100 analyzes the target object projection 51 from the first image I1; the target object projection 51 is, for example, the sitting posture projection of the target object 50. The first recognition device 100 defines region R11 based on the target object projection 51.
[0058] Next, please see Figure 2B This is a schematic diagram of region R11 in the first image I1. The first recognition device 100 analyzes region R11 to obtain multiple parameters of region R11, including: the origin coordinates (x1, y1), height h1, and width w1 of region R11. The first recognition result D1 of the first recognition device 100 includes: the above parameters of region R11, the first confidence level A, and the first pose category cls_1, as shown in equation (1-1):
[0059] D1=[x1,y1,w1,h1,A,cls_1] (1-1)
[0060] On the other hand, the second identification device 200 also performs a similar operation. See also... Figure 3A This is a schematic diagram illustrating the analysis of the second image I2 by the second recognition device 200. The second recognition device 200 defines multiple regions within the second image I2, including region R0, region R12, and region R2. Region R0 is defined with the bed 60 as a reference object. Furthermore, the second recognition device 200 analyzes the target object projection 52 and target object projection 53 from the second image I2. Target object projection 52 is, for example, a sitting projection of the target object 50, and target object projection 53 is, for example, a lying projection of the target object 50. The second recognition device 200 defines region R12 based on target object projection 52 and region R2 based on target object projection 53.
[0061] Next, please see Figure 3BThis is a schematic diagram of region R12 in the second image I2. The second recognition device 200 analyzes region R12 to obtain multiple parameters, including: the origin coordinates (x2, y2), height h2, and width w2 of region R12. Furthermore, the second recognition result D2 includes the aforementioned parameters of region R12, the second confidence level B, and the second pose category cls_2, as shown in equation (1-2):
[0062] D2=[x2,y2,w2,h2,B,cls_2] (1-2)
[0063] Then, the first processor 300 performs image overlay based on the first image I1 and the second image I2 to generate an overlaid image I3. Figure 4 This is a schematic diagram of an overlapping image I3 according to an embodiment. The first processor 300 uses the region R0 corresponding to the bed 60 as a reference. First, it aligns the region R0 of the first image I1 with the region R0 of the second image I2. Then, based on the aligned region R0, it overlaps the first image I1 with the second image I2 to obtain the overlapping image I3. Subsequently, the first processor 300 performs a matching operation based on the overlapping image I3, the first recognition result D1, and the second recognition result D2 to obtain multiple matching results M(i).
[0064] First, the first processor 300 performs a matching operation on regions R11 and R12 of the overlapping image I3 to obtain the matching result M(1) of regions R11 and R12. The matching result M(1) consists of the following parameters: the matching degree ASSO of regions R11 and R12, and the first recognition result D1 and the second recognition result D2. The matching result M(1) is shown in equation (2-1):
[0065] M(1) ={ (1,ASSO),D1,D2} (2-1)
[0066] In equation (2-1), the independent variable "1" of the first field of parameter "(1,ASSO)" represents the "first" matching result M(1); and the matching degree ASSO of the second field represents the degree of association between region R11 and region R12. In one example, based on the parameters of region R11 and region R12 (including: the origin coordinates (x1,y1), height h1 and width w1 of region R11, and the origin coordinates (x2,y2), height h2 and width w2 of region R12), the first processor 300 calculates the matching degree ASSO of region R11 and region R12 according to the intersection over union (IOU), as shown in equation (2-2):
[0067]
[0068] In equation (2-2), parameter XR is defined as in equation (2-3), and parameter XL is defined as in equation (2-4). Here, the displacement offsetX represents the possible displacement between the origin coordinates (x1, y1) of region R11 and the origin coordinates (x2, y2) of region R12. More specifically, the reference coordinate system of region R11 may originally be aligned with the reference coordinate system of region R12; in this case, the origin coordinates (x1, y1) of region R11 coincide with the origin coordinates (x2, y2) of region R12, indicating no displacement between the reference coordinate systems of region R11 and R12. Therefore, the value of the displacement offsetX between the origin coordinates (x1, y1) of region R11 and the origin coordinates (x2, y2) of region R12 is "0". On the other hand, if the reference coordinate system of region R11 is not aligned with the reference coordinate system of region R12, then when calculating the parameters XR and XL of the matching degree ASSO, the displacement offsetX (which is not "0") between the origin coordinates (x1, y1) of region R11 and the origin coordinates (x2, y2) of region R12 must be taken into account.
[0069] XR=min ( (x2 +w2+offsetX), (x1 +w1) ) (2-3)
[0070] XL=max ( (x2+offsetX),x1) (2-4)
[0071] Furthermore, in equation (2-2), the parameter YB is defined as in equation (2-5), and the parameter YT is defined as in equation (2-6):
[0072] YB=min ( (y2+h2), (y1+h1) ) (2-5)
[0073] YT = max ( y2, y1) (2-6)
[0074] exist Figure 4 In the overlapping image I3, regions R11 and R12 have a high degree of correlation. The first processor 300 calculates the matching degree ASSO of regions R11 and R12 according to equation (2-2), which has a high value, for example, equal to 0.91. Based on this, the first processor 300 calculates the first matching result M(1) between the first image I1 and the second image I2, as shown in equation (2-7):
[0075] M(1)={(1,0.91),
[0076] [x1,y1,w1,h1,A,cls_1],
[0077] [x2,y2,w2,h2,B,cls_2]}
[0078] (2-7)
[0079] Next, the first processor 300 performs a matching operation based on region R2 of the overlapping image I3 to obtain a second matching result M(2), as shown in equation (3):
[0080] M(2) = {(2,ASSO),NULL,D2} (3)
[0082] In equation (3), the independent variable "2" of the first field of parameter "(2,ASSO)" represents the "second" matching result M(2). The region R2 of the overlapping image I3 is identified by the second recognition device 200 in the second image I2, and thus the second recognition result D2 corresponding to the region R2 can be generated.
[0083] However, the first recognition device 100 did not recognize the corresponding area in the first image I1, so no first recognition result D1 was generated; therefore, the field corresponding to the first recognition result D1 in the second matching result M(2) is "NULL", indicating that the first recognition device 100's recognition result based on the first image I1 is an "empty event".
[0084] Furthermore, the matching degree ASSO of the second field of parameter "(2,ASSO)" in equation (3) represents the degree of association between region R2 and the possible corresponding region. Since the first recognition device 100 does not recognize the corresponding region (which may correspond to region R2) in the first image I1, the matching degree ASSO of region R2 is equal to 0.
[0085] Figure 5 The flowchart illustrating the matching operation performed by the first processor 300 based on the first recognition result D1, the second recognition result D2, and the overlapping image I3 is part of the security event judgment method disclosed herein. Figure 2A , Figure 2B , Figure 3A , Figure 3B and Figure 4 An embodiment can be represented as follows Figure 5 The flowchart. For example... Figure 5 As shown, firstly, step S500 is executed: the first processor 300 performs image overlay based on the first image I1 and the second image I2 to obtain the overlaid image I3.
[0086] Next, step S502 is executed: In the overlapping image I3, the first processor 300 calculates the matching degree based on the intersection-union ratio. The matching degree is the projection of one of the target objects in the first image I1 (e.g., Figure 2AThe area corresponding to the target object projection 51 (e.g.) Figure 2A The region R11) and the region corresponding to the second image I2 (e.g.) Figure 2B The degree of correlation between regions R12).
[0087] Next, step S504 is executed: based on the first recognition result D1 obtained by the first recognition device 100, the second recognition result D2 obtained by the second recognition device 200 and the matching degree obtained in step S502, the first processor 300 generates the first matching result M(1) in the overlapping image I3.
[0088] Next, steps S502 and S504 are repeated to project other target objects onto the first image I1 and / or the second image I2 (e.g., Figure 3A The first processor 300 generates other matching results in the overlapping image I3 (e.g., the second matching result M(2), the third matching result M(3), the fourth matching result M(4), and so on).
[0089] Next, step S506 is executed: the multiple matching results M(1), M(2), M(3), M(4) are combined into a matching result set {M(i)}.
[0090] Figure 6 The flowchart illustrating how the second processor 400 analyzes and processes the matching result set {M(i)} to obtain the attitude category result CLS and the safety event judgment result S is part of the safety event judgment method disclosed herein. Figure 6 As shown, first, step S600 is executed: the second processor 400 parses the first matching result M(1) from the matching result set {M(i)}.
[0091] Next, step S602 is executed: the second processor 400 determines whether the first identification result D1 or the second identification result D2 in the matching result M(1) is an empty event. If the determination result of step S602 is "yes", it means that the currently analyzed areas of the first image I1 and the second image I2 do not match, and the second processor 400 cannot determine the attitude category result CLS, nor can it determine the safety event judgment result S. Therefore, step S604 is executed next: the second processor 400 does not output the safety event judgment result S to the warning device 500.
[0092] If the judgment result of step S602 is "No", indicating that the matching result M(1) includes the first identification result D1 and the second identification result D2, then step S606 is executed: determining whether the first posture category cls_1 in the first identification result D1 is the same as the second posture category cls_2 in the second identification result D2. If the judgment result of step S606 is "Yes", then the second processor 400 takes the first posture category cls_1 (or the second posture category cls_2) as the posture category result CLS, and the second processor 400 outputs the posture category result CLS to the warning device 500. For example Figure 2B and Figure 3B In the embodiment, in the matching result M(1) of region R11 and region R12, both the first posture category cls_1 and the second posture category cls_2 are "sitting posture", then the second processor 400 determines the posture category result CLS as "sitting posture".
[0093] If the judgment result of step S606 is "No", it means that the second processor 400 cannot decide to take the first pose category cls_1 (or the second pose category cls_2) as the pose category result CLS. Therefore, step S610 is then executed: the second processor 400 calculates based on the first confidence level A, the second confidence level B, the first weight WE1, and the second weight WE2 to obtain its final determined pose category result CLS. In this embodiment, the second processor 400 calculates the first product PD1 of the first confidence level A and the first weight WE1, and calculates the second product PD2 of the second confidence level B and the second weight WE2, as shown in equations (4-1) and (4-2):
[0094] PD1 = A * WE1
[0095] (4-1)
[0096] PD2 = B * WE2
[0097] (4-2)
[0098] Furthermore, the second processor 400 compares the magnitudes of the first product PD1 and the second product PD2 to obtain the maximum value of the first product PD1 and the second product PD2, thereby obtaining the final determined attitude category result CLS. Please refer to equation (4-3), where the index T is the maximum value of the first product PD1 and the second product PD2; the index T points to either the first attitude category cls_1 or the second attitude category cls_2.
[0099] T = max {PD1, PD2} (4-3)
[0100] If the first product PD1 is greater than the second product PD2, and the index T points to the first attitude category cls_1, then the second processor 400 uses the first attitude category cls_1 as the final attitude category result CLS. Conversely, if the first product PD1 is less than the second product PD2, and the index T points to the second attitude category cls_2, then the second processor 400 uses the second attitude category cls_2 as the final attitude category result CLS.
[0101] After the judgment result of step S606 is "yes" or after step S610, step S608 is executed: the second processor 400 outputs the attitude category result CLS it determines to the warning device 500.
[0102] Next, after step S608, step S612 is executed: the second processor 400 determines whether the target object 50 is safe based on the final determined posture category result CLS, and generates a safety event judgment result S accordingly. If the judgment result of step S612 is "yes", it means that the target object 50 is safe. For example, in the matching result M(1), the first posture category cls_1 is the same as the second posture category cls_2, both of which are "sitting posture", so the posture category result CLS is "sitting posture". The second processor 400 can analyze that the region R11 of the target object projection 51 and the region R12 of the target object projection 52 roughly fall within the region R0 corresponding to the bed 60. Therefore, the second processor 400 determines that the target object 50 is sitting on the bed and determines that the target object 50 is safe.
[0103] After the judgment result of step S612 is "yes", step S616 is then executed: the second processor 400 outputs the security event judgment result S as "safe" and transmits the security event judgment result S to the warning device 500. The warning device 500 knows that the security event judgment result S is "safe", therefore the warning device 500 does not generate a warning signal WR.
[0104] If the judgment result of step S612 is "No", it means that the target object 50 is judged to be dangerous. Then, step S614 is executed: the second processor 400 outputs the safety event judgment result S as "dangerous" and transmits it to the warning device 500. In response to the safety event judgment result S being "dangerous", the warning device 500 generates a warning signal WR.
[0105] Next, step S600 is repeated: the second processor 400 parses the second matching result M(2) from the matching result set {M(i)}. Subsequent steps S602 to S616 are executed based on the second matching result M(2). For example, for region R2 in the overlapping image I3, there is no corresponding recognition result for the first image I1, therefore the field of the first recognition result D1 in the matching result M(2) is an empty event (as shown in equation (3)). Therefore, in the subsequent step S602 of the matching result M(2), since the first recognition result D1 of the matching result M(2) is an empty event, the second processor 400 cannot determine the pose category result CLS, nor can it determine the safety event judgment result S. Therefore, step S604 is then executed: the second processor 400 does not output the safety event judgment result S to the warning device 500.
[0106] Figure 7 This is a schematic diagram of an overlapping image I3 according to another embodiment. Figure 7 As shown, in the overlapping image I3, the target object projection 54 is, for example, a supine projection of the target object 50. The first recognition device 100 defines a region R21 based on the target object projection 54. On the other hand, the target object projection 55 is also a supine projection of the target object 50. The second recognition device 200 defines a region R22 based on the target object projection 55.
[0107] The first processor 300 performs a matching operation on regions R21 and R22 of the overlapping image I3 to obtain the matching degree ASSO between regions R21 and R22. Regions R21 and R22 have a high degree of correlation, so their matching degree ASSO has a high value, for example, equal to 0.85. The matching result M(1) of regions R21 and R22 is shown in equation (5-1):
[0108] M(1) ={ (1,0.85),D1,D2} (5-1)
[0109] In the matching result M(1), the first posture category cls_1 of the first recognition result D1 is "reclining", and the second posture category cls_2 of the second recognition result D2 is also "reclining". That is, the first posture category cls_1 is the same as the second posture category cls_2, which determines that the posture category result CLS is "reclining".
[0110] Furthermore, the second processor 400 analyzes that regions R21 and R22 of the overlapping image I3 are outside the range of region R0 of the bed 60, indicating that the target object 50 is lying outside the bed 60. Therefore, the second processor 400 determines that the target object 50 is not lying on the bed, but may have fallen next to the bed 60. The second processor 400 outputs the posture category result CLS as "lying position" and the safety event judgment result S as "dangerous".
[0111] Figure 8 This is a schematic diagram of an overlapping image I3 according to another embodiment. (As shown...) Figure 8 As shown, in the overlapping image I3, the first recognition device 100 obtains the target object projection 56 based on time-of-flight ranging and defines a region R31 based on the target object projection 56. The target object projection 56 is, for example, the seated posture projection of the target object 50. The first posture category cls_1 of the first recognition result D1 is "seated posture".
[0112] On the other hand, the second identification device 200 obtains the target object projection 57 based on infrared imaging and defines a region R32 based on the target object projection 57. The target object projection 57 is, for example, a supine projection of the target object 50. The second posture category cls_2 of the second identification result D2 is "suppressive posture". Since the target object projection 57 is obtained based on infrared imaging, the target object projection 57 may be an afterimage of the target object 50 in a previous time period, rather than the posture of the target object 50 in the current time period.
[0113] The first processor 300 calculates the matching degree ASSO between region R31 and region R32. Region R31 and region R32 have a low degree of correlation, so their matching degree ASSO has a low value, for example, equal to 0.05.
[0114] Since the first pose category cls_1 is "sitting," which differs from the second pose category cls_2's "lying down," the second processor 400 cannot yet determine whether the pose category result CLS is "sitting" or "lying down." Therefore, the second processor 400 must further determine the final pose category result CLS based on the first confidence level A, the second confidence level B, the first weight WE1, and the second weight WE2.
[0115] If the posture category result CLS is determined to be "sitting posture", the second processor 400 analyzes whether region R31 of the overlapping image I3 is within the range of region R0 of bed 60. If region R31 is within the range of region R0 of bed 60, it means that the target object 50 is sitting on bed 60 and has not fallen. Therefore, the second processor 400 outputs the posture category result CLS as "sitting posture" and outputs the safety event judgment result S as "safe".
[0116] In summary, the security incident judgment system 1000 and its execution method disclosed herein perform dual-mode judgment based on two imaging mechanisms, which can take into account the advantages of depth image monitoring and thermal image monitoring. Furthermore, by calculating the matching degree and comparing the posture category, it can eliminate false judgments from depth image monitoring or thermal image monitoring, thereby improving the accuracy of security incident judgment.
[0117] While this disclosure has been detailed above with reference to preferred embodiments and examples, it is understood that these examples are intended to be illustrative rather than limiting. It is anticipated that various modifications and combinations will arise for those skilled in the art, all of which fall within the spirit of this disclosure and the scope of the appended claims.
Claims
1. A security incident determination system for determining whether a target object is safe, the security incident determination system comprising: A first identification device is configured to identify the pose of the target object based on a first image to generate a first identification result, wherein the first identification result is associated with a first confidence level; A second recognition device is configured to recognize the pose of the target object based on a second image to generate a second recognition result, wherein the second recognition result is associated with a second confidence level, and the second image is heterogeneous to the first image; A first processor is configured to overlap the first image and the second image to generate an overlapping image, and to perform a matching operation based on the first recognition result, the second recognition result, and the overlapping image to obtain a matching result set, wherein the matching result set includes multiple matching results; as well as The second processor is used to analyze the matching results to obtain attitude category results and safety event judgment results.
2. The security event determination system of claim 1, wherein the first image is generated by a first image capture device, the first image capture device is a time-of-flight camera, and the first image is a depth image of the target object.
3. The security event determination system of claim 1, wherein the second image is generated by a second image capturing device, the second image capturing device is an infrared camera, and the second image is a thermal image of the target object.
4. The security event determination system as claimed in claim 1, wherein each of the matching results includes a matching degree, the first identification result, and the second identification result.
5. The security event determination system of claim 4, wherein the matching degree is the degree of correlation between a first region of the projection of a first target object in the first image and a second region of the projection of a second target object in the second image.
6. The security event determination system of claim 5, wherein the first region has a plurality of first parameters, and the second region has a plurality of second parameters, and the first processor calculates the matching degree based on the first parameters and the second parameters with an intersection-union ratio.
7. The security event determination system of claim 6, wherein the first parameter includes the first origin coordinates, the first height and the first width of the first region, and the second parameter includes the second origin coordinates, the second height and the second width of the second region.
8. The security event determination system of claim 6, wherein the first identification result includes the first parameter, the first confidence level and the first attitude category, and the second identification result includes the second parameter, the second confidence level and the second attitude category.
9. The security event determination system of claim 8, wherein when the first posture category is the same as the second posture category, the second processor uses the first posture category or the second posture category as the posture category result.
10. The security event determination system of claim 8, wherein the second processor calculates a first product of the first confidence level and the first weight, and calculates a second product of the second confidence level and the second weight, and when the first attitude category is different from the second attitude category, the second processor determines the attitude category result according to the size relationship between the first product and the second product.
11. A method for determining a security incident, used to determine whether a target object is safe, the method comprising: The first recognition device identifies the pose of the target object based on the first image to generate a first recognition result, wherein the first recognition result is associated with a first confidence level; The second recognition device identifies the pose of the target object based on the second image to generate a second recognition result, wherein the second recognition result is associated with a second confidence level, and the second image is heterogeneous to the first image; An overlapping image is generated by superimposing the first image and the second image by a first processor, and a matching operation is performed on the first recognition result, the second recognition result and the overlapping image to obtain a matching result set, wherein the matching result set includes multiple matching results; as well as The second processor analyzes the matching results to obtain attitude category results and safety event judgment results.
12. The security event determination method as described in claim 11, further comprising: The first image is generated by the first image capture device. The first image capturing device is a time-of-flight camera, and the first image is a depth image of the target object.
13. The security event determination method as described in claim 11, further comprising: The second image is generated by a second image capture device. The second image capturing device is an infrared camera, and the second image is a thermal image of the target object.
14. The security event determination method of claim 11, wherein each of the matching results includes a matching degree, the first identification result, and the second identification result.
15. The security event determination method of claim 14, wherein the matching degree is the degree of correlation between a first region of a first target object projection in the first image and a second region of a second target object projection in the second image.
16. The security event determination method of claim 15, wherein the first region has a plurality of first parameters, and the second region has a plurality of second parameters, the security event determination method further includes: The first processor calculates the matching degree based on the first parameter and the second parameter using the intersection-union ratio.
17. The security event determination method of claim 16, wherein the first parameter includes the first origin coordinates, the first height and the first width of the first region, and the second parameter includes the second origin coordinates, the second height and the second width of the second region.
18. The security event determination method of claim 16, wherein the first identification result includes the first parameter, the first confidence level and the first attitude category, and the second identification result includes the second parameter, the second confidence level and the second attitude category.
19. The security event determination method as described in claim 18, further comprising: When the first pose category is the same as the second pose category, the second processor uses either the first pose category or the second pose category as the pose category result.
20. The security event determination method as described in claim 18, further comprising: The second processor calculates a first product of the first confidence level and the first weight, and calculates a second product of the second confidence level and the second weight. When the first pose category is different from the second pose category, the second processor determines the pose category result based on the relationship between the first product and the second product.