Electronic device for identifying image abstract and abnormal behavior pattern

Abstract

The invention discloses an electronic device for identifying an image abstract and an abnormal behavior pattern. The electronic device comprises a processor and a memory, and the memory stores instructions enabling the processor to execute the following steps when the memory runs: first image data and second image data are obtained from a first camera and a second camera in a specified time period when a shooting object is shot by the first camera and the second camera; identifying a first main observation part, a second main observation part, a third main observation part and a fourth main observation part, identifying first to fourth movement data in a first unit time, identifying first to fourth behavior pattern groups based on the first to fourth movement data, respectively, and scoring behavior patterns included in each of the first to fourth behavior pattern groups based on an association rule, identifying a first behavior pattern having a score lower than a reference score or a plurality of mutually associated behavior patterns, and identifying a first specific unit time and first camera information.

Description

Technical Field

[0001] The invention relates to an electronic device for identifying image summaries and abnormal behavior patterns, and to an electronic device for identifying abnormal behavior patterns and summarizing images in a multi-camera network environment. Background Art

[0002] The contents described in this section are only used to provide background information for this embodiment and do not constitute prior art.

[0003] Multi-camera networks can be used to identify data patterns in surveillance, etc. However, as the number of cameras increases, the recorded images also increase, making it difficult to confirm all images.

[0004] Therefore, there is a need to identify abnormal data patterns of monitored objects with minimal images. Summary of the invention

[0005] Problem that the invention aims to solve

[0006] An object of the present invention is to provide an electronic device for identifying image summaries and abnormal behavior patterns, that is, capable of summarizing images obtained from multiple cameras in a multi-camera network environment.

[0007] Furthermore, an object of the present invention is to provide an electronic device for identifying image summaries and abnormal behavior patterns, that is, capable of distinguishing normal behavior patterns and abnormal behavior patterns of a monitored object based on images obtained from multiple cameras in a network environment.

[0008] The purpose of the present invention is not limited to the purpose mentioned above, and other purposes and advantages of the present invention not mentioned can be understood by the following description, and can be more clearly understood by the embodiments of the present invention. In addition, the purposes and advantages of the present invention can obviously be achieved by the methods and combinations thereof shown in the scope of the invention claims.

[0009] Means used to solve problems

[0010] The electronic device for identifying image summaries and abnormal behavior patterns of the present invention includes: a processor; and a memory, which is operatively connected to the processor, wherein the memory stores instructions that cause the processor to execute the following steps when the memory is running: identifying a first main observation portion and a second main observation portion of the object photographed by the first camera and a second camera of a plurality of cameras that photograph the object within a specified time, identifying a third main observation portion and a fourth main observation portion of the object photographed by the second camera, identifying first movement data that measures the frequency of the first main observation portion and the second main observation portion moving along the first direction to the fourth direction in a first unit time, identifying second movement data that measures the frequency of the first main observation portion and the second main observation portion moving along the first direction to the fourth direction in a second unit time continuous with the first unit time, identifying third movement data that measures the frequency of the third main observation portion and the fourth main observation portion moving along the first direction to the fourth direction in the first unit time, identifying third movement data that measures the frequency of the third main observation portion and the fourth main observation portion moving along the first direction to the fourth direction in the second unit time. a fourth movement data for measuring the frequency at which each main observation part moves along the first direction to the fourth direction respectively; based on the first movement data, a first behavior pattern group related to the movement of the first main observation part and the second main observation part of the first camera is identified within the first unit time; based on the second movement data, a second behavior pattern group related to the movement of the first main observation part and the second main observation part of the first camera is identified within the second unit time; based on the third movement data, a third behavior pattern group related to the movement of the third main observation part and the fourth main observation part of the second camera is identified within the first unit time. , based on the above-mentioned fourth movement data, identify the fourth behavior pattern group related to the movement of the above-mentioned third main observation part and the fourth main observation part of the above-mentioned second camera within the above-mentioned second unit time, and within the above-mentioned first unit time and the above-mentioned second unit time, score the behavior patterns included in the above-mentioned first behavior pattern group to the fourth behavior pattern group based on the association rule, identify the first behavior pattern or the first multiple mutually related behavior patterns with a score lower than the benchmark score, and identify the first specific unit time for identifying the above-mentioned first behavior pattern or the above-mentioned first multiple mutually related behavior patterns and the first camera information for photographing the above-mentioned first behavior pattern or the above-mentioned first multiple mutually related behavior patterns.

[0011] Furthermore, the above-mentioned instructions cause the above-mentioned processor to perform the following steps: identifying the cumulative frequency of each of the multiple pixels included in the above-mentioned first image data moving more than a critical distance along at least one direction among the above-mentioned first direction to the fourth direction, identifying mutually similar pixels based on the above-mentioned cumulative frequency associated with each of the above-mentioned multiple pixels, and identifying a part of the above-mentioned photographic object corresponding to the above-mentioned mutually similar pixels as the above-mentioned first main observation part.

[0012] Furthermore, the above-mentioned multiple pixels include a first pixel and a second pixel, and the above-mentioned instruction enables the above-mentioned processor to perform the following steps: identifying whether the difference between the cumulative frequency of the above-mentioned first pixel and the cumulative frequency of the above-mentioned second pixel is included in a specified range, identifying whether the distance between the above-mentioned first pixel and the above-mentioned second pixel is below a critical distance, and based on the fact that the above-mentioned difference is included in the above-mentioned specified range and the above-mentioned distance is below the above-mentioned critical distance, identifying the above-mentioned first pixel and the second pixel as the above-mentioned mutually similar pixels.

[0013] Furthermore, the instructions enable the processor to execute the following steps: performing topic modeling on the first mobile data to identify the first behavior pattern group.

[0014] Furthermore, the above-mentioned instruction causes the above-mentioned processor to perform the following steps: for the above-mentioned first movement data, the above-mentioned first unit time is defined as a row, the above-mentioned first direction to the fourth direction related to the above-mentioned first main observation part and the above-mentioned first direction to the fourth direction related to the above-mentioned second main observation part are defined as columns respectively, and a first matrix is ​​identified, and the above-mentioned first matrix includes the frequencies of the above-mentioned first main observation part and the second main observation part moving along the above-mentioned first direction to the fourth direction respectively during the above-mentioned first unit time, and the above-mentioned topic modeling is performed by inputting the above-mentioned first matrix to identify the above-mentioned first behavior pattern group including the main behavior patterns in the above-mentioned first movement data.

[0015] Furthermore, the instructions enable the processor to execute the following steps: executing the association rule in two consecutive unit times.

[0016] Furthermore, the above-mentioned instructions cause the above-mentioned processor to execute the following steps: identifying the second behavior pattern or the second multiple behavior patterns that are mutually related, whose scores are higher than the above-mentioned baseline score, and identifying the second behavior pattern or the second multiple behavior patterns that are mutually related as the normal behavior pattern of the above-mentioned photographed object.

[0017] Furthermore, the above-mentioned instructions cause the above-mentioned processor to perform the following steps: identifying the second specific unit time for identifying the above-mentioned second behavior pattern or the above-mentioned second multiple behavior patterns that are mutually related, and identifying the second camera information for photographing the above-mentioned second behavior pattern or the above-mentioned second multiple behavior patterns that are mutually related.

[0018] Furthermore, the above-mentioned instructions cause the above-mentioned processor to simultaneously execute the following steps: for the above-mentioned first specific unit time in which the above-mentioned first behavior pattern or the above-mentioned first multiple behavior patterns are identified, image data captured by at least one camera among the above-mentioned multiple cameras identified based on the above-mentioned first camera information is provided; for the above-mentioned second specific unit time in which the above-mentioned second behavior pattern or the above-mentioned second multiple behavior patterns are identified, image data captured by at least one camera among the above-mentioned multiple cameras identified based on the above-mentioned second camera information is provided.

[0019] Furthermore, the above-mentioned instructions cause the above-mentioned processor to perform the following steps: for the above-mentioned first specific unit time in which the above-mentioned first behavior pattern or the above-mentioned first multiple interrelated behavior patterns are identified, simultaneously providing image data taken by at least one camera among the above-mentioned multiple cameras identified based on the above-mentioned first camera information.

[0020] Effects of the Invention

[0021] The electronic device for identifying image summaries and abnormal behavior patterns of the present invention can increase user convenience and efficiency by allowing summarization of images obtained from multiple cameras in a multi-camera network environment.

[0022] Furthermore, the electronic device for identifying image summaries and abnormal behavior patterns of the present invention can distinguish normal behavior patterns and abnormal behavior patterns of the monitored object based on images obtained from multiple cameras in a multi-camera network environment, thereby reducing the time required to determine whether a monitored object has a fault and increasing user convenience and efficiency.

[0023] The specific effects of the present invention together with the above contents will be described in the following description of the specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 FIG. 1 is a diagram for explaining an electronic device according to an embodiment of the present invention.

[0025] Figure 2 A diagram for explaining the operation of a processor included in the electronic device according to an embodiment of the present invention.

[0026] Figure 3 , Figure 4 and Figure 5 For illustration Figure 2 FIG. 100 of step S100.

[0027] Figure 6 and Figure 7 For illustration Figure 2 FIG. 10 is a diagram of step S200.

[0028] Figure 8 and Fig. 9 For illustration Figure 2 FIG. 10 is a diagram of step S300.

[0029] Fig.10 and Fig.11 For illustration Figure 2 FIG. 5 is a diagram of step S400 and step S500. DETAILED DESCRIPTION

[0030] The terms or words used in this specification and the scope of protection of the invention should not be interpreted as limited to the meaning in the general or dictionary. According to the principle that the inventor can define the concept of the term or word in order to explain his invention in the best way, it should be interpreted in accordance with the meaning and concept of the technical idea of ​​the present invention. In addition, the embodiments described in this specification and the structures shown in the figures are only one embodiment of the present invention and do not represent all the technical ideas of the present invention. Therefore, when submitting this application, there may be various equivalent technical solutions that can replace them and examples that can be deformed and applied.

[0031] The terms "first", "second", "A", "B", etc. used in this specification and the scope of protection of the invention may be used to describe various structural elements, but the above-mentioned structural elements should not be limited to the above-mentioned terms. The above-mentioned terms are only used to distinguish one structural element from other structural elements. For example, without departing from the scope of protection of the present invention, the first structural element may be named as the second structural element, and similarly, the second structural element diagram may also be named as the first structural element. The term "and / or" may include a combination of multiple related recorded items or any item in multiple related recorded items.

[0032] The terms used in this specification and the scope of protection of the invention are only used to illustrate specific embodiments and do not limit the present invention. Unless otherwise defined in the context, singular expressions include plural expressions. The terms "including" or "having" in this application should be understood as not excluding in advance the existence or additional possibility of the features, numbers, steps, actions, structural elements, components or their combinations recorded in the specification.

[0033] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meanings as commonly understood by one of ordinary skill in the art to which the present invention belongs.

[0034] Terms defined in commonly used dictionaries should be interpreted as meanings consistent with the meanings of the relevant technology in the context, and should not be interpreted in an idealized or overly formalized sense unless explicitly defined in this application. In addition, the structures, processes, techniques or methods included in the various embodiments of the present invention can be shared within the scope of technical non-contradiction.

[0035] Below, we will refer to Figures 1 to 11 The electronic device (hereinafter referred to as the electronic device) for identifying image summaries and abnormal behavior patterns according to an embodiment of the present invention is described.

[0036] Figure 1 FIG. 1 is a diagram for explaining an electronic device according to an embodiment of the present invention.

[0037] Reference Figure 1 , the electronic device 100 of the embodiment of the present invention can communicate with multiple cameras C1, C2, ..., Cn. The electronic device 100 can receive image data of the object 200 captured by the multiple cameras C1, C2, ..., Cn within a specified time from the multiple cameras C1, C2, ..., Cn. Figure 1 In addition to the structural elements shown, the electronic device 100 may further include at least one additional structural element.

[0038] The electronic device 100 may further include a processor 110 and a memory 120 .

[0039] The memory 120 may store various data used by at least one structural element (e.g., the processor 110) of the electronic device 100. For example, the data may include input data or output data of software (e.g., a program) and instructions related thereto. The memory 120 may include a volatile memory or a non-volatile memory.

[0040] The memory 120 may store instructions, information or data related to the operations of the components included in the electronic device 100. For example, the memory 120 may store instructions that enable the processor 110 to perform various operations described in this specification when executed.

[0041] The processor 110 may be operatively coupled to the memory 120 to perform the overall functions of the electronic device 100. For example, the processor 110 may include more than one processor. For example, the more than one processor may include an image signal processor (ISP), an application processor (AP), or a communication processor (CP).

[0042] For example, the processor 110 can control at least one other structural element (e.g., hardware structural element or software structural element) of the electronic device 100 connected to the processor 110 by running software (e.g., program), and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operation, the processor 110 can load instructions or data received from other structural elements (e.g., communication module) to the memory 120, process the instructions or data stored in the memory 120, and store the result data in the memory 120. According to one embodiment, the processor 110 may include a main processor (e.g., central processing unit or application processor) and an auxiliary processor (e.g., graphics processing unit, image signal processor, sensor hub processor or communication processor) that can operate independently or jointly. Additionally or alternatively, the auxiliary processor can be configured to use lower power than the main processor or be dedicated to a specified function. The auxiliary processor can be implemented separately from the main processor or as part of it. The program can be stored in the memory 120 as software, for example, it can include an operating system, middleware or application.

[0043] Reference Figure 2 The operation of processor 110 will be described.

[0044] Figure 2 A diagram for explaining the operation of a processor included in the electronic device according to an embodiment of the present invention.

[0045] Reference Figure 1 and Figure 2 , the processor 110 may identify the main observation portion (step S100). The main observation portion may be a portion determined as the main portion of the portions of the subject 200 photographed by the plurality of cameras C1, C2, ..., Cn.

[0046] The processor 110 may identify the first main observation portion and the second main observation portion of the photographed object 200 photographed by the first camera C1, and the third main observation portion and the fourth main observation portion of the photographed object 200 photographed by the second camera C2. In order to identify the main observation portion, the processor 110 may receive the first image data of the photographed object 200 photographed by the first camera C1 within a specified time, and the second image data of the photographed object 200 photographed by the second camera C2 within a specified time.

[0047] Figure 3 , Figure 4 and Figure 5 For illustration Figure 2 FIG. 100 of step S100. Figure 4 The reference numerals in FIG. 1 represent only a part of the plurality of pixels included in the first image data ID1 .

[0048] Reference Figures 1 to 5 For example, in order to identify the main observation parts of the first camera C1 and the second camera C2 among the multiple cameras C1, C2, ..., Cn, the processor 110 can first identify the cumulative frequency of multiple pixels included in the image data moving more than a critical distance along at least one direction of the first direction DR, the second direction DL, the third direction DU and the fourth direction DD (step S101).

[0049] For example, the processor 110 may identify a cumulative frequency of accumulating the frequency at which each of the plurality of pixels PX1, PX2, ..., PXn included in the first image data ID1 received from the first camera C1 moves over a critical distance along at least one of the first direction DR, the second direction DL, the third direction DU, and the fourth direction DD. The first direction DR, the second direction DL, the third direction DU, and the fourth direction DD may be different directions. The first direction DR and the second direction DL, the third direction DU, and the fourth direction DD may be directions that intersect with each other.

[0050] The processor 110 may identify similar pixels based on the cumulative frequencies of the pixels PX1, PX2, ..., PXn and the distances between the pixels (step S103). The processor 110 may identify a portion of the subject 200 corresponding to the similar pixels as a main observation portion (step S105).

[0051] For example, the plurality of pixels may include first, second, third, fourth, and fifth pixels PX1, PX2, PX3, PX4, and PX5. The processor 110 may identify the cumulative frequency of each of the first, second, third, fourth, and fifth pixels PX1, PX2, PX3, PX4, and PX5.

[0052] The processor 110 can identify whether the difference in cumulative frequency of at least two pixels among the multiple pixels PX1, PX2, ..., PXn is included in the specified range. In addition, the processor 110 can identify whether the distance between at least two pixels among the multiple pixels PX1, PX2, ..., PXn is less than the critical distance. The processor 110 can identify at least two pixels whose difference in cumulative frequency is included in the specified range and whose separation distance is less than the critical distance as mutually similar pixels. The processor 110 can make judgments related to the difference in cumulative frequency and distance for all the multiple pixels PX1, PX2, ..., PXn. The processor 110 can identify a portion of the photographed object 200 corresponding to the mutually similar pixels as the main observation portion.

[0053] For example, the processor 110 may identify whether the difference between the cumulative frequency of the first pixel PX1 and the cumulative frequency of the second pixel PX2 is included in the prescribed range. Furthermore, the processor 110 may identify whether the distance between the first pixel PX1 and the second pixel PX2 is below the critical distance. The processor 110 may identify the first pixel PX1 and the second pixel PX2 as mutually similar pixels based on the fact that the difference between the cumulative frequency of the first pixel PX1 and the cumulative frequency of the second pixel PX2 is included in the prescribed range and the distance between the first pixel PX1 and the second pixel PX2 is below the critical distance. On the other hand, the processor 110 may not identify the first pixel PX1 and the third pixel PX3 as mutually similar pixels based on the fact that the difference between the cumulative frequency of the first pixel PX1 and the cumulative frequency of the third pixel PX3 is included in the prescribed range, but the distance between the first pixel PX1 and the third pixel PX3 is greater than the critical distance. In addition, the processor 110 may not identify the first pixel PX1 and the fifth pixel PX5 as mutually similar pixels based on the fact that the distance between the first pixel PX1 and the fifth pixel PX5 is less than the critical distance, but the difference in the cumulative frequency between the first pixel PX1 and the fifth pixel PX5 is not included in the prescribed range. The processor 110 may identify a portion of the photographic object 200 corresponding to the first pixel PX1 and the second pixel PX2 identified as mutually similar pixels as the first main observation portion R1.

[0054] For example, the processor 110 may identify whether the difference between the cumulative frequency of the third pixel PX3 and the cumulative frequency of the fourth pixel PX4 is included in the prescribed range. And, the processor 110 may identify whether the distance between the third pixel PX3 and the fourth pixel PX4 is below the critical distance. The processor 110 may identify the third pixel PX3 and the fourth pixel PX4 as mutually similar pixels based on the difference between the cumulative frequency of the third pixel PX3 and the cumulative frequency of the fourth pixel PX4 being included in the prescribed range and the distance between the third pixel PX3 and the fourth pixel PX4 being below the critical distance. On the other hand, the processor 110 may not identify the third pixel PX3 and the fifth pixel PX5 as mutually similar pixels based on the difference between the cumulative frequency of the third pixel PX3 and the cumulative frequency of the fifth pixel PX5 being included in the prescribed range and the distance between the third pixel PX3 and the fifth pixel PX5 being greater than the critical distance. The processor 110 may identify a portion of the photographic object 200 corresponding to the third pixel PX3 and the fourth pixel PX4 identified as mutually similar pixels as the second main observation portion R2.

[0055] The processor 110 may identify the first main observation portion R1 and the second main observation portion R2 for the first camera C1 , and may identify the third main observation portion and the fourth main observation portion for the second camera C2 .

[0056] Re-reference Figure 1 and Figure 2 , the processor 110 may identify the movement data (step S200). The processor 110 may identify the movement data for measuring the frequency of movement of at least one main observation portion identified by one of the plurality of cameras C1, C2, ..., Cn along the first direction DR, the second direction DL, the third direction DU, and the fourth direction DD, respectively, per unit time.

[0057] Figure 6 and Figure 7 For illustration Figure 2 FIG. 10 is a diagram of step S200.

[0058] Reference Figure 1 , Figure 2 , Figure 5 , Figure 6 and Figure 7 The processor 110 may measure the frequency at which the main observation part moves along the first direction DR, the second direction DL, the third direction DU, and the fourth direction DD, respectively, within a unit time.

[0059] For example, for the first camera C1, the processor 110 can measure the frequency of the first main observation part R1 moving along the first direction DR, the second direction DL, the third direction DU, and the fourth direction DD within the first unit time T1, and can identify the first movement data measuring the frequency of the second main observation part R2 moving along the first direction DR, the second direction DL, the third direction DU, and the fourth direction DD. Figure 6 In FIG. 1 , the first movement data is shown to be expressed as a heat map, but is not limited thereto. For a unit time, if the frequency of the main observation part moving along the first direction DR, the second direction DL, the third direction DU, and the fourth direction DD can be presented, the movement data can also be expressed in other ways.

[0060] For example, for the first camera C1, the processor 110 can measure the frequency of the first main observation part R1 moving along the first direction DR, the second direction DL, the third direction DU, and the fourth direction DD within the second unit time T2 continuous with the first unit time T1, and identify the second movement data that measures the frequency of the second main observation part R2 moving along the first direction DR, the second direction DL, the third direction DU, and the fourth direction DD.

[0061] For example, for the second camera C2, the processor 110 can measure the frequency of the third main observation part moving along the first direction DR, the second direction DL, the third direction DU, and the fourth direction DD within the first unit time T1, and can identify the third movement data that measures the frequency of the fourth main observation part moving along the first direction DR, the second direction DL, the third direction DU, and the fourth direction DD.

[0062] For example, for the second camera C2, the processor 110 can measure the frequencies of the third main observation part moving along the first direction DR, the second direction DL, the third direction DU, and the fourth direction DD within the second unit time T2, and can identify fourth movement data that measures the frequencies of the fourth main observation part moving along the first direction DR, the second direction DL, the third direction DU, and the fourth direction DD.

[0063] For multiple cameras C1, C2, ..., Cn, the processor 110 can measure the frequency of at least one main observation part respectively identified by the multiple cameras C1, C2, ..., Cn moving along the first direction DR, the second direction DL, the third direction DU, and the fourth direction DD respectively in each unit time, thereby generating movement data.

[0064] The processor 110 may convert each movement data into a matrix. Figure 7 As shown, for the first movement data, the processor 110 can define the first unit time as a row, and define the first direction DR, the second direction DL, the third direction DU, and the fourth direction DD related to the first main observation part R1 and the first direction DR, the second direction DL, the third direction DU, and the fourth direction DD related to the second main observation part R2 as columns, respectively, and identify a first matrix including the frequencies in which the first main observation part R1 and the second main observation part R2 move along the first direction DR, the second direction DL, the third direction DU, and the fourth direction DD respectively during the first unit time T1.

[0065] For example, the rows of the first matrix for converting the first movement data may be the time that divides the first unit time T1 into sub-units, and the columns of the first matrix may be the movement of a main observation part along one of the first direction DR, the second direction DL, the third direction DU, and the fourth direction DD. For example, the first frequency a11 may be the frequency of the first main observation part R1 moving along the first direction DR during the first sub-unit time of the first unit time T1. For example, the second frequency a12 may be the frequency of the first main observation part R1 moving along the first direction DR during the second sub-unit time of the first unit time T1. For example, the third frequency a21 may be the frequency of the first main observation part R1 moving along the third direction DU during the first sub-unit time of the first unit time T1.

[0066] Re-reference Figure 1 and Figure 2 , the processor 110 may identify a behavior pattern group (step S300). The processor 110 may identify a behavior pattern group related to the movement of at least one main observation part of a camera in each unit time based on the movement data related to a camera among the multiple cameras C1, C2, ..., Cn. For example, the processor 110 may extract meaningful behavior patterns from the movement data using topic modeling to identify the behavior pattern group. For example, a meaningful behavior pattern may be a behavior pattern in which a main observation part of a camera mainly moves during a specific unit time period.

[0067] Figure 8 and Fig. 9 For illustration Figure 2 FIG. 10 is a diagram of step S300.

[0068] Reference Figure 1 , Figure 2 , Figure 8 and Fig. 9 , the processor 110 may convert a matrix based on the movement data (for example: Figure 7 The processor 110 may include or may not include the topic modeling module 301.

[0069] Taking one camera among the plurality of cameras C1, C2, ..., Cn as an object, the processor 110 may identify a behavior pattern group including at least one behavior pattern related to the movement of one camera during a unit time based on the movement data of the movement frequency of at least one main observation part measured in each unit time. The behavior pattern may be a pattern related to the behavior of the main observation part of one camera moving at a high frequency during a unit time. For example, the first behavior pattern Z1-1 may be a behavior pattern of the first main observation part R1 of the first camera C1 picking up an object within the first unit time T1.

[0070] The behavior pattern group may include at least one behavior pattern. For example, the first behavior pattern group G1 may include the behavior that the first main observation part R1 of the first camera C1 picks up an object (first behavior pattern Z1-1) and puts the object down again (second behavior pattern (Z1-2)) within the first unit time T1.

[0071] For example, based on the first movement data, the processor 110 may identify a first behavior pattern group G1 associated with the movement of the first main observation portion and the second main observation portion of the first camera C1 within the first unit time T1. For example, based on the second movement data, the processor 110 may identify a second behavior pattern group G2 associated with the movement of the first main observation portion and the second main observation portion of the first camera C1 within the second unit time T2. For example, based on the third movement data, the processor 110 may identify a third behavior pattern group G3 associated with the movement of the third main observation portion and the fourth main observation portion of the second camera C2 within the first unit time T1. For example, based on the fourth movement data, the processor 110 may identify a fourth behavior pattern group G4 associated with the movement of the third main observation portion and the fourth main observation portion of the second camera C2 within the second unit time T2.

[0072] Re-reference Figure 1 and Figure 2 The processor 110 may score each behavior pattern included in each behavior pattern group (step S400). Based on the scoring result, the processor 110 may extract the specific unit time including the abnormal behavior pattern and the camera information for shooting the abnormal behavior pattern (S500).

[0073] Fig.10 and Fig.11 For illustration Figure 2 FIG. 5 is a diagram of step S400 and step S500.

[0074] Reference Figure 1 , Figure 2 , Fig.10 and Fig.11 The processor 110 may score each behavior pattern of the behavior pattern group identified in each unit time based on an association rule in two consecutive unit times. For example, the processor 110 may input the behavior pattern group into the association rule module 303 and receive a scoring result from the association rule module 303.

[0075] The processor 110 may compare the score with the benchmark score and identify a behavior pattern with a score lower than the benchmark score or multiple behavior patterns that are related to each other. The processor 110 may identify a behavior pattern with a score lower than the benchmark score or identify a specific unit time and camera information for multiple behavior patterns that are related to each other. The processor 110 may identify a summary image that summarizes only a portion of the image data captured by at least one camera that corresponds to the specific unit time based on the specific unit time and camera information.

[0076] According to the result of executing the association rule process, only one behavior pattern may be associated and scored, or multiple behavior patterns may be associated with each other and the associated whole may be scored.

[0077] For example, the processor 110 may identify the first behavior pattern Z1-1 or the first plurality of mutually related behavior patterns P1 with a score higher than the benchmark score based on the scoring result. The first plurality of behavior patterns P1 may be behavior patterns mutually related according to the result of executing the association rule. The behavior patterns mutually related according to the result of executing the association rule may include at least one behavior pattern extracted from the plurality of cameras C1, C2, ..., Cn respectively within a unit time. The processor 110 may identify the first behavior pattern Z1-1 or the first plurality of mutually related behavior patterns P1 as a normal behavior pattern of the photographed object 200. For example, the processor 110 may identify the behavior of the photographed object 200 moving according to the first behavior pattern Z1-1 in the main observation part of the first camera C1 during the first unit time T1 as a normal behavior pattern. Furthermore, the processor 110 may identify the behavior of the photographed object 200 moving successively according to the second behavior pattern Z1-2, the third behavior pattern Z2-1, and the fourth behavior pattern Z2-2 in the main observation part of the first camera C1 and the main observation part of the second camera C2 during the first unit time T1 as a normal behavior pattern.

[0078] For example, the processor 110 may identify the fifth behavior pattern Z1-4 or the second plurality of mutually related behavior patterns P2 having a score lower than the benchmark score based on the scoring result. The processor 110 may identify the fifth behavior pattern Z1-4 or the second plurality of mutually related behavior patterns P2 as an abnormal behavior pattern of the photographed object 200. For example, the processor 110 may identify the behavior of the photographed object 200 moving according to the fifth behavior pattern Z1-4 in the main observation portion of the first camera C1 during the second unit time T2 as an abnormal behavior pattern. Furthermore, the processor 110 may identify the behavior of the photographed object 200 moving successively according to the first behavior pattern Z1-1 and the sixth behavior pattern Z1-3 in the main observation portion of the first camera C1 during the second unit time T2 as an abnormal behavior pattern.

[0079] The processor 110 may identify the specific unit time for identifying the abnormal behavior pattern and the camera information for photographing the abnormal behavior pattern. Also, the processor 110 may identify the specific unit time for identifying the normal behavior pattern and the camera information for photographing the normal behavior pattern.

[0080] For example, the processor 110 may identify the first unit time T1 of the first behavior pattern Z1-1 as a specific unit time, and identify the first camera C1 related to the first behavior pattern Z1-1, thereby identifying a portion of the image data related to the first unit time T1 in the image data captured by the first camera C1 as normal image data.

[0081] For example, the processor 110 may identify the first unit time T1 in which the first plurality of behavior patterns P1 associated with each other are identified as the specific unit time, and identify the first camera C1 and the second camera C2 associated with the first plurality of behavior patterns P1 associated with each other. The processor 110 may identify the image data captured by each of the first camera C1 and the second camera C2 during the first unit time T1 as the specific unit time as normal image data.

[0082] For example, the processor 110 may identify the second unit time T2 of the fifth behavior pattern Z1-4 as a specific unit time, and identify the first camera C1 related to the fifth behavior pattern Z1-4, thereby identifying a portion of the image data related to the second unit time T2 in the image data captured by the first camera C1 as abnormal image data.

[0083] For example, the processor 110 may identify the second unit time T2 in which the second plurality of behavior patterns P2 associated with each other are identified as the specific unit time, and identify the first camera C1 associated with the second plurality of behavior patterns P2 associated with each other. The processor 110 may identify the image data captured by the first camera C1 during the second unit time T2 as the specific unit time as abnormal image data.

[0084] The processor 110 may provide normal image data and abnormal image data simultaneously.

[0085] The electronic device 100 of the embodiment of the present invention can identify the specific time when the abnormal behavior pattern is identified and the camera that captured the abnormal behavior pattern in a multiple camera network environment, thereby reducing the time required to extract the abnormal behavior pattern without having to analyze all of the multiple cameras, and increasing work efficiency by only checking the images or pictures of at least a portion of the cameras within a specific time period.

[0086] The term "module" used in this specification may include a unit implemented in hardware, software or firmware, and may be used interchangeably with terms such as logic, logic block, component or circuit. A module may be an integral component or the smallest unit of the above components that performs one or more functions, or a portion thereof. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0087] Various embodiments of the present specification may be implemented as software (e.g., a program) including one or more instructions stored in a storage medium (e.g., memory 120) that can be read by a device (machine) (e.g., electronic device 100). For example, a processor (e.g., processor 110) of a device (e.g., electronic device 100) may call at least one of the one or more instructions stored in the storage medium and run it. This enables the device to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by an editor or code run by an interpreter. The storage medium readable by the device may be provided in the form of a non-transitory storage medium. Among them, "non-volatile" only means that the storage medium is a tangible device and does not include a signal (e.g., electromagnetic wave). The above term does not distinguish whether the data is semi-permanently stored in the storage medium or temporarily stored.

[0088] According to one embodiment, the methods of various embodiments disclosed in this specification may be included in a computer program product and provided. The computer program product may be traded between a seller and a buyer as a commodity. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or through an application store (e.g., Play Store). TM ) or directly online publishing (e.g., downloading or uploading) between two user devices (e.g., smartphones). In the case of online publishing, at least a portion of the computer program product may be temporarily stored on a device-readable storage medium such as a memory of a manufacturer's server, an application store's server, or a relay server, or may be temporarily generated.

[0089] According to various embodiments, each structural element (e.g., module or program) of the above-mentioned structural elements described may include a single or multiple individuals. According to various embodiments, one or more structural elements or actions in the corresponding above-mentioned structural elements may be omitted, or one or more other structural elements or actions may be added. Alternatively or additionally, multiple structural elements (e.g., modules or programs) may be integrated into one structural element. In this case, the integrated structural element may perform one or more functions of each structural element of the above-mentioned multiple structural elements in the same or similar manner as the functions performed by the corresponding structural elements in the above-mentioned multiple structural elements before the above-mentioned integration. According to various embodiments, the actions performed by modules, programs or other structural elements may be performed sequentially, in parallel, repeatedly or heuristically, or one or more actions in the above-mentioned actions may be performed in a different order, omitted, or one or more other actions may be added.

[0090] The above description is only an exemplary description of the technical idea of ​​the present embodiment. For ordinary technicians in the technical field to which the present embodiment belongs, various modifications and deformations can be made without departing from the essential characteristics of the present embodiment. Therefore, the present embodiment is not intended to limit the technical idea of ​​the present embodiment, but is used for illustration, and the scope of the technical idea of ​​the present embodiment is not limited to these embodiments. The protection scope of the present embodiment shall be interpreted by the attached invention claim protection scope, and all technical ideas within its equivalent range shall be interpreted as included in the invention claim protection scope of the present embodiment.

Claims

1. An electronic device for identifying image summaries and abnormal behavior patterns, characterized in that: include: Processor; and A memory is operably connected to the processor. The memory stores instructions that enable the processor to execute the following steps when the memory is executed: From first image data and second image data of the subject captured by a first camera and a second camera respectively within a specified time, among a plurality of cameras capturing the subject, a first main observation portion and a second main observation portion of the subject captured by the first camera are identified, and a third main observation portion and a fourth main observation portion of the subject captured by the second camera are identified, identifying first movement data for measuring the frequency at which the first main observation portion and the second main observation portion move along the first direction to the fourth direction, respectively, within a first unit time; identifying second movement data measuring the frequency at which the first main observation portion and the second main observation portion move along the first direction to the fourth direction, respectively, in a second unit time continuous with the first unit time, identifying third movement data for measuring the frequency at which the third main observation portion and the fourth main observation portion move along the first direction to the fourth direction, respectively, within the first unit time; identifying fourth movement data for measuring the frequency at which the third main observation portion and the fourth main observation portion move along the first direction to the fourth direction, respectively, within the second unit time; Based on the first movement data, a first behavior pattern group related to the movement of the first main observation part and the second main observation part of the first camera is identified within the first unit time, Based on the second movement data, a second behavior pattern group related to the movement of the first main observation part and the second main observation part of the first camera is identified within the second unit time, Based on the third movement data, a third behavior pattern group related to the movement of the third main observation part and the fourth main observation part of the second camera is identified within the first unit time, Based on the fourth movement data, a fourth behavior pattern group related to the movement of the third main observation part and the fourth main observation part of the second camera is identified within the second unit time, In the first unit time and the second unit time, the behavior patterns included in each of the first to fourth behavior pattern groups are scored based on association rules, identifying a first behavior pattern or a first plurality of behavior patterns that are correlated with each other with a score below a benchmark score, The first specific unit time for identifying the first behavior pattern or the first plurality of mutually related behavior patterns and the first camera information for photographing the first behavior pattern or the first plurality of mutually related behavior patterns are identified.

2. The electronic device according to claim 1, characterized in that: The above instructions cause the above processor to execute the following steps: identifying a cumulative frequency at which each of the plurality of pixels included in the first image data moves by more than a critical distance along at least one direction from the first direction to the fourth direction, identifying pixels that are similar to each other based on the cumulative frequencies associated with each of the plurality of pixels, A portion of the photographic subject corresponding to the mutually similar pixels is recognized as the first main observation portion.

3. The electronic device according to claim 2, characterized in that: The plurality of pixels include a first pixel and a second pixel. The above instructions cause the above processor to execute the following steps: identifying whether a difference between the cumulative frequency of the first pixel and the cumulative frequency of the second pixel is within a prescribed range, identifying whether the distance between the first pixel and the second pixel is less than a critical distance, Based on the fact that the difference is within the predetermined range and the distance is equal to or smaller than the critical distance, the first pixel and the second pixel are recognized as the pixels similar to each other.

4. The electronic device according to claim 1, characterized in that: The instructions enable the processor to execute the following steps: performing topic modeling on the first mobile data to identify the first behavior pattern group.

5. The electronic device according to claim 4, characterized in that: The above instructions cause the above processor to execute the following steps: For the first movement data, the first unit time is defined as a row. The first direction to the fourth direction related to the first main observation portion and the first direction to the fourth direction related to the second main observation portion are respectively defined as columns, identifying a first matrix including frequencies at which the first main observation portion and the second main observation portion move along the first direction to the fourth direction during the first unit time, The topic modeling is performed by inputting the first matrix to identify the first behavior pattern group including the main behavior patterns in the first mobile data.

6. The electronic device according to claim 1, characterized in that: The above instruction enables the above processor to execute the following steps: execute the above association rule in two consecutive unit times.

7. The electronic device according to claim 1, characterized in that: The above instructions cause the above processor to execute the following steps: identifying a second behavior pattern or a plurality of second behavior patterns related to each other with the score being higher than the benchmark score, The second behavior pattern or the second plurality of mutually related behavior patterns are identified as a normal behavior pattern of the photographed object.

8. The electronic device according to claim 7, characterized in that: The above instructions cause the above processor to execute the following steps: identifying the second specific unit time for identifying the above second behavior pattern or the above second multiple interrelated behavior patterns and the second camera information for photographing the above second behavior pattern or the above second multiple interrelated behavior patterns.

9. The electronic device according to claim 8, characterized in that: The above instructions cause the above processor to execute the following steps simultaneously: For the first specific unit time in which the first behavior pattern or the first plurality of mutually related behavior patterns is identified, image data captured by at least one camera of the plurality of cameras identified based on the first camera information is provided, For the second specific unit time in which the second behavior pattern or the second plurality of mutually related behavior patterns is identified, image data captured by at least one of the plurality of cameras identified based on the second camera information is provided.

10. The electronic device according to claim 1, characterized in that: The above instructions cause the above processor to execute the following steps: for the above first specific unit time for identifying the above first behavior pattern or the above first multiple interrelated behavior patterns, simultaneously provide image data taken by at least one camera among the above multiple cameras identified based on the above first camera information.

Images