Image analysis device

Abstract

The present invention relates to an image analysis device. The image analysis device according to the present invention calculates, on the basis of an input image, similarity scores that indicate degrees of similarity with a plurality of preset events, and provides the similarity scores for prompts corresponding to the plurality of events, so as to more effectively detect events occurring in images that appear in a video.

Description

Video analysis device

[0001] The present invention relates to an image analysis device.

[0002] Closed-Circuit Television (CCTV) is widely used for safety and security purposes in various environments, including public spaces, businesses, and residential areas. CCTV provides real-time surveillance and recording capabilities, enabling crime prevention, accident investigation, and abnormal situation detection, contributing to overall social safety. However, the recent surge in the number of CCTV installations and the explosive growth in the amount of data collected by individual CCTVs have revealed several limitations in effectively analyzing and utilizing this data.

[0003] In particular, the method of directly monitoring video data provided in real time by multiple CCTV cameras can lead to problems such as the inability to detect incidents and accidents in a timely manner due to limitations of surveillance personnel, accumulated fatigue, and human error. To address this, the need for technology capable of automatically analyzing video data to detect abnormal situations or specific events is emerging.

[0004] Existing image analysis technologies are primarily based on methodologies such as object detection, action recognition, and anomaly detection. While these technologies have demonstrated some success in detecting single events or individual anomalies, they still struggle to process multiple events simultaneously or analyze high-dimensional situations based on complex correlations.

[0005] Furthermore, defining specific situations or events and establishing criteria for their detection often relies on manual methods. For example, detecting specific anomalies requires configuring algorithms based on predefined criteria. This process is not only time-consuming and resource-intensive, but also makes it difficult to flexibly adapt to changing environments.

[0006] The technical problem to be achieved by the present invention is to provide a video analysis device capable of more effectively detecting events occurring in a video by calculating similarity scores indicating the degree of similarity between an input video and a plurality of prompts for detecting a plurality of preset events, providing detected events and the similarity scores based on the similarity scores, and deriving and providing a superior event by combining unit events as needed.

[0007] Here, the video may be a captured video received from a CCTV, and the prompt may be text, a video, or a combination of the two, which serve as criteria for detecting a specific event. Furthermore, the similarity score may be a value indicating the similarity between the video and the prompt, and an abnormal situation event may be defined as occurring when the similarity score between the video and the prompt exceeds a threshold.

[0008] To address these challenges, an image analysis device according to an embodiment of the present invention may include an input providing unit, a scoring unit, and a results unit. The input providing unit may provide an input image. The scoring unit may provide similarity scores indicating the degree of similarity between the input image and a plurality of preset prompts, as well as an indication of whether an event has occurred. The results unit may provide similarity scores for upper-level events corresponding to the plurality of events, as well as an indication of whether an event has occurred.

[0009] In one embodiment, the image analysis device may include a first judgment unit and a second judgment unit. The first judgment unit may determine whether the similarity scores for the prompts are greater than or equal to a predetermined reference score, thereby providing a first judgment result. The second judgment unit may determine whether the number of the prompts having a score greater than or equal to the reference score is greater than or equal to a predetermined reference number, thereby providing a second judgment result.

[0010] In one embodiment, the image analysis device may include a third judgment unit. The third judgment unit may determine that a higher-order event corresponding to a higher-order concept including the prompts has occurred when the number of prompts exceeding the reference score is greater than or equal to the reference number.

[0011] In one embodiment, the image analysis device may include a ranking unit and a weighting unit. The ranking unit may determine the priorities of the prompts. The weighting unit may provide weight values ​​applied to the similarity scores for the prompts based on the priorities.

[0012] In one embodiment, the image analysis device may further include a weighted score unit. The weighted score unit may apply the weight value to the similarity scores to calculate weighted scores.

[0013] In one embodiment, the image analysis device may further include a selection unit and a summation unit. The selection unit may select only some of the weighted scores based on the priority order and provide selection scores. The summation unit may sum the selection scores and provide a summation score.

[0014] In one embodiment, the image analysis device may further include a comparison unit and an event determination unit. The comparison unit may compare the combined score with a predetermined combined reference score and provide a comparison result. The event determination unit may determine that the upper event has occurred based on the comparison result.

[0015] In one embodiment, the image analysis device may further include a first measurement unit and a decision unit. The first measurement unit may measure an event interval corresponding to a time interval during which the similarity scores for the prompts remain above the reference score. The decision unit may determine whether the event has occurred based on the event interval.

[0016] In one embodiment, the image analysis device may further include a second measuring unit. When there are multiple event sections, the second measuring unit may measure an interval corresponding to a time interval between adjacent event sections.

[0017] In one embodiment, the image analysis device may further include an interval summing unit. The interval summing unit may sum adjacent event intervals and provide the sum as the event interval when the interval is smaller than a predetermined reference interval.

[0018] In addition to the technical problems of the present invention mentioned above, other features and advantages of the present invention are described below or may be clearly understood by a person skilled in the art to which the present invention pertains from such description and explanation.

[0019] According to the present invention as described above, the following effects are achieved.

[0020] The video analysis device according to the present invention can more effectively detect events occurring in a video by calculating similarity scores indicating the degree of similarity with a plurality of preset events based on an input video and providing similarity scores for prompts corresponding to the plurality of events.

[0021] FIG. 1 is a drawing showing an image analysis device according to embodiments of the present invention.

[0022] Figures 2 and 3 are drawings for explaining events and prompts applied to the image analysis device of Figure 1.

[0023] Figures 4 and 5 are drawings for explaining the operation of the judgment unit included in the image analysis device of Figure 1.

[0024] Figures 6 to 9 are drawings for explaining embodiments of the image analysis device of Figure 1.

[0025] FIGS. 10 and 11 are drawings for explaining another embodiment of the image analysis device of FIG. 1.

[0026] When adding reference numbers to components of each drawing in this specification, it should be noted that identical components are given the same numbers as much as possible even if they are shown in different drawings.

[0027] Meanwhile, the meanings of the terms described in this specification should be understood as follows.

[0028] Singular expressions should be understood to include plural expressions unless the context clearly defines otherwise, and the scope of rights should not be limited by these terms.

[0029] The terms "include" or "have" should be understood as not excluding in advance the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0030] Hereinafter, preferred embodiments of the present invention designed to solve the above problems will be described in detail with reference to the attached drawings.

[0031] FIG. 1 is a drawing showing an image analysis device according to embodiments of the present invention, FIGS. 2 and 3 are drawings for explaining events and prompts applied to the image analysis device of FIG. 1, and FIGS. 4 and 5 are drawings for explaining the operation of a judgment unit included in the image analysis device of FIG. 1.

[0032] Referring to FIGS. 1 to 5, an image analysis device (10) according to an embodiment of the present invention may include an input providing unit (100), a scoring unit (200), and a result unit (300). The input providing unit (100) may provide an input image (IM) to the scoring unit (200). For example, the input image (IM) may be a video provided from a CCTV. However, the spirit of the present invention is not limited to the input image (IM) being a video provided from a CCTV, and image(s) captured in other ways may also be used as the input image (IM).

[0033] The scoring unit (200) may provide similarity scores (YP) indicating the degree of similarity between the input image (IM) and a plurality of preset events (EV). The scoring unit (200) may include an artificial intelligence unit that has undergone artificial intelligence learning using a learning image and a plurality of events (EV) matching the learning image, and the method by which the artificial intelligence unit learns may be various methods that have already been disclosed. For example, the input image (IM) provided to the scoring unit (200) may be an image indicating the condition of a road provided by CCTV, and the plurality of preset events (EV) may include a first event (EV1), a second event (EV2), and a third event (EV3). The first event (EV1) may be “fire,” the second event (EV2) may be “smoke,” and the third event (EV3) may be “speed.” These types of events are provided as examples and do not limit the scope of the present invention.

[0034] The scoring unit (200) can provide a first similarity score (YP1), a second similarity score (YP2), and a third similarity score (YP3) corresponding to the similarity score (YP) for the degree of similarity of the first event (EV1), the second event (EV2), and the third event (EV3) from the input image (IM). For example, the first similarity score (YP1) provided by the scoring unit (200) can be 90 points, the second similarity score (YP2) can be 85 points, and the third similarity score (YP3) can be 40 points.

[0035] The result section (300) may provide similarity scores (YP) for prompts (PT) corresponding to a plurality of events (EV). For example, a first prompt (PT1) corresponding to a first event (EV1) may be "a fire broke out in the first vehicle", a second prompt (PT2) corresponding to a second event (EV2) may be "a high possibility of a fire in the first vehicle", and a third prompt (PT3) corresponding to a third event (EV3) may be "a speeding of the first vehicle". Here, the first similarity score (YP1), which is the similarity score (YP) corresponding to the first prompt (PT1), may be 90 points, and the second similarity score (YP2), which is the similarity score (YP) corresponding to the second prompt (PT2), may be 85 points. In addition, the third similarity score (YP3), which is the similarity score (YP) corresponding to the third prompt (PT3), may be 40 points. Here, the prompt can be text or an image or a combination of the two that serves as a criterion for detecting a specific event.

[0036] Figures 6 to 9 are drawings for explaining embodiments of the image analysis device of Figure 1.

[0037] Referring to FIGS. 1 to 9, in one embodiment, the image analysis device (10) may include a judgment unit (400), and the judgment unit (400) may include a first judgment unit (410), a second judgment unit (420), and a third judgment unit (430). The first judgment unit (410) may determine whether the similarity scores (YP) for the prompts (PT) are equal to or greater than a predetermined reference score (REP) and provide a first judgment result (PD1). For example, the first similarity score (YP1) for the first prompt (PT1) may be 90 points, the second similarity score (YP2) for the second prompt (PT2) may be 85 points, and the third similarity score (YP3) for the third prompt (PT3) may be 40 points. In addition, the predetermined reference score (REP) may be 70 points.

[0038] For example, the first judgment unit (410) can compare the reference score (REP) of 70 points with the first similarity score (YP1) to the third similarity score (YP3) and provide a first judgment result (PD1) that the first similarity score (YP1) and the second similarity score (YP2) are higher than the reference score (REP).

[0039] The second judgment unit (420) can provide a second judgment result (PD2) by judging whether the number of prompts (PT) having a reference score (REP) or higher is greater than or equal to a predetermined reference number (REN). For example, if the first similarity score (YP1) corresponding to the first prompt (PT1) and the second similarity score (YP2) corresponding to the second prompt (PT2) are higher than the reference score (REP), the number of prompts higher than the reference score (REP) may be 2. In addition, the predetermined reference number (REN) may be 2. In this case, the second judgment unit (420) can provide a second judgment result (PD2) that "the number of prompts (PT) having a reference score (REP) or higher is greater than or equal to the reference number (REN)."

[0040] In one embodiment, the image analysis device (10) may include a third judgment unit (430). The third judgment unit (430) may determine that a higher-order event (UV) corresponding to a higher-order concept including the prompts (PT) has occurred when the number of prompts (PT) having a reference score (REP) or greater is greater than or equal to a reference number (REN). For example, when events (EV) such as a vehicle fire, smoke, and speeding occur simultaneously, it may be determined that a traffic accident has occurred. In this case, the higher-order event (UV) corresponding to the higher-order concept of the prompts (PT) may be a traffic accident.

[0041] In one embodiment, the image analysis device (10) may include a priority determining unit (510) and a weighting unit (520). The priority determining unit (510) may determine priorities (PR) of prompts (PT). For example, the priorities (PR) may be input by a user through a user input (IN). Among the priorities (PR) determined according to the user input (IN), the first priority (PR1) may be the first prompt (PT1), and the second priority (PR2) may be the second prompt (PT2). In addition, among the priorities (PR) determined according to the user input (IN), the third priority (PR3) may be the third prompt (PT3).

[0042] The weighting unit (520) may provide weight values ​​applied to similarity scores (YP) for prompts (PT) according to priorities (PR). For example, a weight value applied to a first similarity score (YP1) of 90 points for a first prompt (PT1) corresponding to a first priority (PR1) may be a first weight value (WT1), and a weight value (WT) applied to a second similarity score (YP2) of 85 points for a second prompt (PT2) corresponding to a second priority (PR2) may be a second weight value (WT2). In addition, a weight value (WT) applied to a third similarity score (YP3) of 40 points for a third prompt (PT3) corresponding to a third priority (PR3) may be a third weight value (WT3). Here, the first weight value (WT1) may be 2, and the second weight value (WT2) may be 1. In addition, the third weight value (WT3) may be 0.5.

[0043] In one embodiment, the image analysis device (10) may further include a weighted score unit (530). The weighted score unit (530) may calculate weighted scores (WP) by applying a weight value (WT) to the similarity scores (YP). For example, the first weighted score (WP1) obtained by multiplying the first similarity score (YP1) by 90 points, which is the first weight value (WT1) of 2, may be 180 points, and the second weighted score (WP2) obtained by multiplying the second similarity score (YP2) by 85 points, which is the second weight value (WT2) of 1, may be 85 points. In addition, the third weighted score obtained by multiplying the third similarity score (YP3) by 40 points, which is the third weight value (WT3) of 0.5, may be 20 points.

[0044] In one embodiment, the image analysis device (10) may further include a selection unit (540) and a summation unit (550). The selection unit (540) may select only some of the weighted scores (WP) according to the priority (PR) and provide the selection scores (SP). For example, the first weighted score (WP1) may be 180 points, the second weighted score (WP2) may be 85 points, and the third weighted score (WP3) may be 20 points. The selection unit (540) may select the first weighted score (WP1) and the second weighted score (WP2) among the three weighted scores (WP) according to a user input (IN).

[0045] The summing unit (550) can provide a summed score (HP) by summing the selection scores (SP). For example, the summing unit (550) can provide a summed score (HP) of 265 points, which is the sum of 180 points, which is the first weighted score (WP1), and 85 points, which is the second weighted score (WP2).

[0046] In one embodiment, the image analysis device (10) may further include a comparison unit (560) and an event determination unit (570). The comparison unit (560) may provide a comparison result (BG) by comparing the combined score (HP) with a predetermined combined reference score (HRP). For example, the combined score (HP) may be 265 points, and the predetermined combined reference score (HRP) may be 200 points. In this case, the comparison unit (560) may provide a comparison result (BG) indicating that the combined score (HP) is higher than the reference score.

[0047] The event judgment unit (570) may determine that a higher-order event (UV) has occurred based on the comparison result (BG). For example, if the comparison result (BG) indicates that the total score (HP) is higher than the reference score, the event judgment unit (570) may determine that a traffic accident corresponding to the higher-order event (UV) has occurred.

[0048] FIGS. 10 and 11 are drawings for explaining another embodiment of the image analysis device of FIG. 1.

[0049] Referring to FIGS. 10 and 11, in one embodiment, the image analysis device (10) may further include a first measurement unit (610) and a determination unit (620). The first measurement unit (610) may measure an event interval (EI) corresponding to a time interval during which the similarity scores (YP) for the prompts (PT) are maintained above a reference score (REP). For example, the plurality of times may include a first time (T1) to a fourth time (T4). The event interval (EI) corresponding to the time interval during which the similarity score (YP) for the first event (EV1) corresponding to the first prompt (PT1) is maintained above the reference score may include a first_1 event interval (EI1_1) and a first_2 event interval (EI1_2). The first event interval (EI1_1) may be a time interval from a first time (T1) to a second time (T2), and the first event interval (EI1_2) may be a time interval from a third time (T3) to a fourth time (T4). The first measuring unit (610) may measure the first event interval (EI1_1) and the first event interval (EI1_2).

[0050] The decision unit (620) can determine whether an event occurs based on an event interval (EI). For example, the decision unit (620) can determine that the first event (EV1) has occurred (EB) if the first event interval (EI1_1) or the first event interval (EI1_2) is maintained for a certain period of time or longer.

[0051] In one embodiment, the image analysis device (10) may further include a second measuring unit (630). When there are multiple event intervals (EI), the second measuring unit (630) may measure an interval (BI) corresponding to a time interval between adjacent event intervals (EI). For example, an event interval adjacent to the 1_1st event interval (EI1_1) may be a 1_2nd event interval (EI1_2). In this case, the time interval arranged between the 1_1st event interval (EI1_1) and the 1_2nd event interval (EI1_2) may be a first interval (BI1). In this case, the second measuring unit (630) may measure the first interval (BI1).

[0052] In one embodiment, the image analysis device (10) may further include an interval summing unit (640). The interval summing unit (640) may provide an event interval (EI) by adding adjacent event intervals (EI) when the interval (BI) is smaller than a predetermined reference interval (REI). For example, when the first interval (BI1) measured by the second measuring unit (630) is smaller than the reference interval (REI), the interval summing unit (640) may provide an event interval by adding the first_1 event interval (EI1_1) and the first_2 event interval (EI1_2). Here, when the event interval (EI) obtained by adding the first_1 event interval (EI1_1) and the first_2 event interval (EI1_2) is maintained for a predetermined period of time or longer, it may be determined that the first event (EV1) has occurred.

[0053] The image analysis device (10) according to the present invention calculates similarity scores (YP) indicating the degree of similarity with a plurality of events (EV) set in advance based on an input image (IM), and provides similarity scores (YP) for prompts (PT) corresponding to the plurality of events (EV), thereby enabling more effective detection of events (EV) occurring in the video.

[0054] The following is a list of embodiments of the present invention.

[0055] Item 1 is an image analysis device including an input providing unit that provides an input image; a scoring unit that provides similarity scores indicating the degree of similarity with a plurality of events set in advance based on the input image; and a result unit that provides the similarity scores for prompts corresponding to the plurality of events.

[0056] Item 2 is an image analysis device according to Item 1, which includes a first judgment unit that determines whether the similarity scores for the above prompts are greater than or equal to a predetermined reference score and provides a first judgment result; and a second judgment unit that determines whether the number of the above prompts having the above reference score or greater is greater than or equal to a predetermined reference number and provides a second judgment result.

[0057] Item 3 is an image analysis device according to items 1 and 2, including a third judgment unit that determines that a higher event corresponding to a higher concept including the prompts has occurred when the number of the prompts having a score greater than or equal to the above-mentioned reference score is greater than or equal to the above-mentioned reference number.

[0058] Item 4 is an image analysis device according to items 1 to 3, including a ranking unit that determines the priorities of the prompts; and a weighting unit that provides weight values ​​to be applied to the similarity scores for the prompts according to the priorities.

[0059] Item 5 is an image analysis device according to items 1 to 4, further including a weighted score unit that calculates weighted scores by applying the weight values ​​to the similarity scores.

[0060] Item 6 is an image analysis device according to items 1 to 5, further comprising a selection unit that provides selection scores by selecting only some of the weighted scores according to the above priorities; and a summation unit that provides a summation score by summing the selection scores.

[0061] Item 7 is an image analysis device according to items 1 to 6, further comprising a comparison unit that compares the above-mentioned total score with a predetermined total reference score and provides a comparison result; and an event determination unit that determines that the above-mentioned upper event has occurred based on the comparison result.

[0062] Item 8 is an image analysis device according to items 1 to 7, further comprising a first measuring unit that measures an event interval corresponding to a time interval during which the similarity scores for the prompts remain above the reference score; and a decision unit that determines whether the event occurs based on the event interval.

[0063] Item 9 is an image analysis device according to items 1 to 8, further including a second measuring unit that measures an interval corresponding to a time interval between adjacent event sections when there are multiple event sections.

[0064] Item 10 is an image analysis device according to items 1 to 9, further including an interval summing unit that provides the sum of adjacent event intervals as the event interval when the interval is smaller than a predetermined reference interval.

[0065] While the embodiments of the present invention have been described above, they are merely illustrative and the present invention is not limited thereto, but should be construed to have the broadest scope consistent with the basic concepts disclosed in this specification. Those skilled in the art may combine or substitute the disclosed embodiments to implement embodiments not specified herein, but this also does not exceed the scope of the present invention. In addition, those skilled in the art may easily modify or alter the disclosed embodiments based on this specification, and it is clear that such modifications or alterations also fall within the scope of the present invention.

[0066] The present invention can be used in the field of image analysis equipment industry.

Claims

1. Input providing unit that provides input images; A scoring unit that provides similarity scores indicating the degree of similarity with a plurality of preset events based on the input image; and An image analysis device characterized by including a result section providing similarity scores for prompts corresponding to the plurality of events.

2. In paragraph 1, The above image analysis device, A first judgment unit that determines whether the similarity scores for the above prompts are greater than or equal to a predetermined reference score and provides a first judgment result; and An image analysis device characterized by including a second judgment unit that determines whether the number of the above prompts exceeding the above-mentioned reference score is greater than or equal to a predetermined reference number and provides a second judgment result.

3. In paragraph 2, The above image analysis device, An image analysis device characterized by including a third judgment unit that determines that a superior event corresponding to a superior concept including the prompts has occurred when the number of the prompts exceeding the above-mentioned reference score is greater than or equal to the above-mentioned reference number.

4. In paragraph 3, The above image analysis device, a priority determining unit for determining the priorities of the above prompts; and An image analysis device characterized by including a weighting unit that provides weight values ​​applied to the similarity scores for the prompts according to the above priorities.

5. In paragraph 4, The above image analysis device, An image analysis device further comprising a weighted score unit that calculates weighted scores by applying the weight values ​​to the similarity scores.

6. In paragraph 5, The above image analysis device, A selection unit that provides selection scores by selecting only some of the weighted scores according to the above priorities; and An image analysis device characterized by further including a summing unit that provides a summed score by summing the above selection scores.

7. In paragraph 6, The above image analysis device, A comparison unit that provides a comparison result by comparing the above-mentioned total score with a predetermined total reference score; and An image analysis device further comprising an event determination unit that determines that the upper event has occurred based on the comparison result.

8. In paragraph 7, The above image analysis device, A first measuring unit for measuring an event interval corresponding to a time interval during which the similarity scores for the above prompts are maintained above the reference score; and An image analysis device further comprising a decision unit that determines whether the event occurs or not according to the event section.

9. In paragraph 8, The above image analysis device, An image analysis device characterized in that, when there are multiple event sections, it further includes a second measuring unit that measures an interval corresponding to a time interval between adjacent event sections.

10. In paragraph 9, The above image analysis device, An image analysis device characterized in that it further includes an interval summing unit that provides the sum of adjacent event intervals as the event interval when the interval is smaller than a predetermined standard interval.

Images