Method and apparatus for behavior recognition based on virtual region of interest

Abstract

Disclosed are a method and apparatus for behavior recognition based on a virtual region of interest. One aspect of the present invention provides a computer-implemented method for recognizing the behavior of an object, the method comprising the processes of: detecting an object from an image; generating a virtual region of interest on the basis of a region in which the object is detected; and determining whether the object is adopting a predetermined type of behavior on the basis of the area of an overlapping region in which the region in which the object was detected and the virtual region of interest overlap each other.

Description

Method and device for virtual region-of-interest-based action recognition

[0001] The present disclosure relates to a method and device for virtual region-of-interest based action recognition.

[0002] The content described below merely provides background information related to the present embodiment and does not constitute prior art.

[0003] With recent advancements in IT, research into recognizing object behavior or situations from camera images is actively underway. These techniques primarily utilize AI model-based algorithms. Training AI models to recognize specific behaviors requires a large amount of training data, and building and training these data is both costly and time-consuming. Furthermore, trained models often suffer from poor generalization performance when applied to environments different from the training data.

[0004] The main purpose of the present disclosure is to provide a method and device capable of recognizing the behavior of an object in a rule-based manner by utilizing a virtual area of ​​interest adaptively set for each object and a mutual positional relationship between the objects.

[0005] The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

[0006] According to one aspect of the present disclosure, a computer-implemented method for recognizing an action of an object is provided, comprising: a step of detecting an object from an image; a step of generating a virtual region of interest based on an area where the object is detected; and a step of determining whether the object is taking a predetermined type of action based on an area of ​​an overlapping area where the area where the object is detected overlaps the virtual region of interest.

[0007] According to another aspect of the present disclosure, there is provided a device comprising: a memory storing commands; and at least one processor, wherein the at least one processor executes the commands to detect an object from an image, generate a virtual region of interest based on an area where the object is detected, and determine whether the object is performing a predetermined type of action based on an area of ​​an overlapping region where the area where the object is detected overlaps the virtual region of interest.

[0008] According to another aspect of the present disclosure, there is provided a computer-readable recording medium having stored thereon a command, which, when executed by the computer, causes the computer to perform the steps of: detecting an object from an image; generating a virtual region of interest based on an area where the object is detected; and determining whether the object is performing a predetermined type of action based on an area of ​​an overlapping area where the area where the object is detected overlaps the virtual region of interest.

[0009] According to an embodiment of the present disclosure, the behavior of an object can be recognized by utilizing the mutual positional relationship between the virtual area of ​​interest adaptively set for each object and the object.

[0010] According to an embodiment of the present disclosure, by automatically setting a virtual region of interest based on the object detection result, object behavior recognition is possible without limitations on the environment in which the camera is installed.

[0011] The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0012] FIG. 1 is a block diagram schematically illustrating an exemplary system to which the present disclosure can be applied.

[0013] FIGS. 2 to 4 are diagrams showing various examples of virtual regions of interest generated according to various embodiments of the present disclosure.

[0014] FIG. 5 is a diagram showing an example of setting a virtual region of interest for each frame according to one embodiment of the present disclosure.

[0015] FIG. 6 is a diagram showing an example of setting a virtual region of interest for each frame according to another embodiment of the present disclosure.

[0016] FIG. 7 is a flowchart illustrating an action recognition method according to one embodiment of the present disclosure.

[0017] FIG. 8 is a block diagram schematically illustrating an exemplary computing device that can be used to implement the devices and methods described in the present disclosure.

[0018] Hereinafter, some embodiments of the present disclosure will be described in detail using exemplary drawings. When designating components in each drawing, it should be noted that, where possible, identical components are given the same reference numerals, even if they appear in different drawings. Furthermore, when describing the present disclosure, detailed descriptions of related known structures or functions will be omitted if they are deemed to obscure the gist of the present disclosure.

[0019] In describing components of embodiments according to the present disclosure, symbols such as first, second, i), ii), a), b) may be used. These symbols are only for distinguishing the components from other components, and the nature, order, or sequence of the components are not limited by the symbols. When a part in the specification is said to "include" or "have" a component, this does not mean that other components are excluded, but rather that other components may be included, unless explicitly stated otherwise.

[0020] The detailed description set forth below, together with the accompanying drawings, is intended to explain exemplary embodiments of the present disclosure and is not intended to represent the only embodiments in which the present disclosure may be practiced.

[0021] In this disclosure, the term “image” may be used to encompass both still images and video.

[0022] FIG. 1 is a block diagram schematically illustrating an exemplary system to which the present disclosure can be applied.

[0023] The image processing device (10) can perform inference on images acquired from a camera (20). The image processing device (10) can recognize the actions of one or more objects in the image and / or recognize the situation shown in the image. The image processing device (10) can recognize the actions of the objects by analyzing the mutual positional relationship between the objects and the floor surface on which the objects are located in the image.

[0024] In an image, the floor can be identified based on a region-of-interest (ROI). For example, when the camera (20) is installed in a narrow space (e.g., an indoor space such as an elevator) where the floor and wall can be distinguished in the image, the floor can be identified based on a region-of-interest manually designated in advance by the user. However, when the camera (20) is installed in a space where it is difficult to specify the floor (e.g., an indoor / outdoor space that is wide compared to the field of view of the camera (20)), the region-of-interest cannot be designated in advance. For action recognition in a general space that is not limited to a narrow space, the image processing device (10) adaptively generates a virtual ROI according to an object and recognizes the object's falling action by utilizing the mutual positional relationship between the object and the virtual ROI.

[0025] To this end, the image processing device (10) may include all or part of an object detection module (100), a keypoint estimation module (120), an object tracking module (140), an ROI generation module (160), and an action recognition module (180). Not all blocks illustrated in FIG. 1 are essential components, and some blocks may be added, changed, or deleted in other embodiments. Meanwhile, the components illustrated in FIG. 1 represent functionally distinct elements, and at least one of the components may be implemented in a form in which they are integrated with each other in an actual physical environment.

[0026] The object detection module (100) can detect an object of a predefined type from an image. Here, the object of the predefined type may be, for example, a person. The object detection module (100) can detect a bounding box surrounding the object. Here, one bounding box may include at least one object. The object detection module (100) can detect a plurality of bounding boxes that each surround a plurality of objects of the same type. Detecting the bounding box may include, for example, estimating a combination of two or more of the upper left coordinate of the bounding box, the lower right coordinate of the bounding box, the center coordinate of the bounding box, and / or the size (width and height) of the bounding box.

[0027] The keypoint estimation module (120) can estimate the positions of keypoints corresponding to an object from an image. Keypoints may also be referred to as landmarks or joints. For example, the keypoint estimation module (120) can receive an image and output the coordinates of keypoints of arbitrary objects. As another example, the keypoint estimation module (120) can receive an image in which a portion where a specific object is detected is cropped and output the coordinates of one or more keypoints extracted for the object. As yet another example, the keypoint estimation module (120) can receive the coordinates of an image and a bounding box and output the coordinates of one or more keypoints extracted for the object. The keypoint estimation module (120) can estimate a confidence score indicating the probability that each keypoint is located at the corresponding coordinates, together with the coordinates of one or more keypoints.

[0028] The object tracking module (140) can track objects appearing in an image between multiple frames and identify objects having the same identity. For example, the object tracking module (140) can assign the same identifier to bounding boxes surrounding objects having the same identity among the bounding boxes detected in the multiple frames.

[0029] For object detection, pose estimation, and object tracking, various learning-based and / or rule-based algorithms may be employed, and the present disclosure is not limited thereto in any particular manner.

[0030] The ROI generation module (160) can generate a virtual region of interest based on an object region (i.e., a bounding box) in which an object is detected in an image. The ROI generation module (160) can adaptively change the size and location of the virtual region of interest depending on the size and location of the object appearing in the image.

[0031] FIGS. 2 to 4 are diagrams showing various examples of virtual regions of interest that can be generated according to various embodiments of the present disclosure. FIG. 2 shows an example of a virtual region of interest that can be generated when an object is standing (standing case), FIG. 3 shows an example of a virtual region of interest that can be generated when an object is squatting (squatting case), and FIG. 5 shows an example of a virtual region of interest that can be generated when an object is lying down (lying case).

[0032] The ROI generation module (160) can generate a virtual region of interest centered on the point where the object and the floor touch. The ROI generation module (160) can regard a point within the object area (BBOX) as the point where the object and the floor touch.

[0033] For example, the ROI generation module (160) can set the midpoint of the bottom of the object area (BBOX) as the center position (CTR) of the virtual region of interest (ROI). When the upper left and lower right coordinates of the object area (BBOX) are (x1, y1) and (x2, y2), respectively, the center position (CTR) of the virtual region of interest (ROI) can be calculated as in mathematical expression 1.

[0034]

[0035] As another example, the ROI generation module (160) may set the center position (CTR) of the virtual region of interest (ROI) to the location where a keypoint corresponding to a specific body part (e.g., a heel or an ankle, etc.) is extracted, or the perpendicular foot of a line drawn from the bottom of the object area (BBOX) at that location. For example, when the coordinates of the location where a specific keypoint is extracted are (x3, y3), the center position (CTR) of the virtual region of interest (ROI) may be set to (x3, y3) or (x3, y2).

[0036] The ROI generation module (160) can determine the size of a virtual region of interest (ROI) based on the length of the long side of the object area (BBOX). For example, the distance between the center position (CTR) and at least one vertex of the virtual region of interest (ROI) can be calculated as in mathematical expression 2.

[0037]

[0038] Here, W and H represent the width and height of the object area (BBOX), respectively, and α represents a coefficient that adjusts the size of the virtual area of ​​interest according to the size of the object. α can be set to, for example, 0.75, but is not limited thereto.

[0039] In this way, by changing the size of the virtual region of interest (ROI) based on the length of the long side of the object area (BBOX), the size of the virtual region of interest (ROI) can be flexibly changed during the object's collapse process. Furthermore, a high overlap rate between the object area (BBOX) and the virtual region of interest (ROI) of a horizontally lying object can be ensured.

[0040] For computational efficiency, the ROI generation module (160) can generate a virtual region of interest (ROI) in the shape of a convex polygon. For example, the virtual region of interest (ROI) can be a convex regular polygonal region that is tangent to a predetermined circle. The ROI generation module (160) can generate a circle having a center position (CTR) determined according to mathematical equations 1 to 3 and a length (RAD) determined according to mathematical equation 4 as a center point and a radius, respectively, and then sample a plurality of points (P0 to P7) on the circumference at equal intervals. The ROI generation module (160) can generate a virtual region of interest (ROI) having the sampled points as vertices.

[0041] The position and / or size of the virtual region of interest may be updated on a frame-by-frame basis, or may be fixed to a value set at a particular frame according to preset rules.

[0042] FIG. 5 is a diagram showing an example of setting a virtual region of interest for each frame according to one embodiment of the present disclosure. FIG. 6 is a diagram showing an example of setting a virtual region of interest for each frame according to another embodiment of the present disclosure. FIGS. 5 and 6 show object regions (BBOX1 to BBOX5) in which objects having the same identity are detected in a plurality of frames and corresponding virtual regions of interest (ROI1 to ROI5) in an overlapping manner. Although FIGS. 5 and 6 illustrate that the plurality of frames have a relatively wide time interval (i.e., a small FPS (Frame Per Second)), this is for easy identification of the drawings and the present disclosure is not limited thereto.

[0043] For example, the ROI generation module (160) can individually generate a virtual region of interest (ROI) for each frame, as illustrated in FIG. 5. That is, the virtual region of interest (ROI) in the Kth frame K , K is a natural number) is the size and position of the object area (BBOX) of the frame K ) can be determined by a virtual region of interest (ROI) within the video. K ) moves along with the movement of the object, and as the object gets closer to the camera (20), the virtual region of interest (ROI) K ) increases in size. In addition, as the pose of the object changes, the virtual region of interest (ROI) K ) can adaptively change in position and / or size.

[0044] As another example, the ROI generation module (160) creates a virtual region of interest (ROI) set in a specific frame, as shown in FIG. 6. fixed ) may also have the same position and / or size applied to subsequent frames.

[0045] The ROI generation module (160) generates a virtual region of interest (ROI) based on the change pattern of the detected object regions (BBOX1 to BBOX5) between multiple frames. fixed ) can identify the point at which to fix it.

[0046] For example, the ROI generation module (160) generates a virtual region of interest (ROI) based on changes in the aspect ratio of the object regions (BBOX1 to BBOX5). fixed ) can be identified at the point of fixation. Table 1 illustrates the aspect ratios of the object areas shown in FIGS. 2 and 3 of the object.

[0047] standing casesquatting casely caseAspect ratio2.01.20.7 ~ 0.5

[0048] As shown in Table 1, the aspect ratio of the object area surrounding the object changes depending on the pose of the object. Therefore, when the pose of the object changes significantly between multiple frames, the aspect ratio of the object areas (BBOX1 to BBOX5) also changes rapidly. Considering these characteristics, the ROI generation module (160) generates the object area (BBOX) in a specific frame. K ) is less than or equal to a predetermined threshold aspect ratio, the virtual region of interest (ROI) generated immediately before the frame K - 1) can be maintained. The critical aspect ratio is either pre-specified by the user or the object area in the preceding frames (BBOX1 Inland BBOX K - 1) It may be adaptively determined based on the statistics of the aspect ratio. For example, if the aspect ratio of the object area (BBOX5) in the 5th frame is rapidly reduced (e.g., reduced by more than 30% from the average value), the ROI generation module (160) may maintain the position and / or size of the virtual region of interest (ROI4) generated in the 4th frame in the 5th frame and subsequent frames.

[0049] Additionally or alternatively, the ROI generation module (160) generates a virtual region of interest (ROI) based on the movement distance or movement speed of the object regions (BBOX1 to BBOX5) between multiple frames. fixed ) can also be used to identify when to fix the issue.

[0050] The action recognition module (180) can recognize the action of an object based on the geometric relationship between the object area and the virtual area of ​​interest. The action recognition module (180) can recognize the state of an object falling or the action of an object falling based on the geometric relationship within a single frame and / or changes in the geometric relationship between multiple frames.

[0051] The geometric relationship between the object region and the virtual region of interest (ROI) can be defined based on the overlap rate between the object region (BBOX) and the virtual region of interest (ROI). Referring to FIGS. 2 to 4, when the object is fallen, the virtual region of interest (ROI) and the object region (BBOX) overlap relatively more than when the object is standing or squatting. Based on this, the action recognition module (180) can recognize the fall of the object based on the relative size of the overlapping region (OV), which is the portion where the virtual region of interest (ROI) and the object region (BBOX) overlap.

[0052] To this end, the action recognition module (180) can calculate the area of ​​the overlapping region (OV) using the position coordinates of the object area (BBOX) and the virtual region of interest (ROI). At this time, the virtual region of interest (ROI) is a virtual region of interest (ROI) generated from individual frames. K ) or a fixed virtual region of interest (ROI) fixed ) can be used. For example, in the example of Fig. 5, the overlapping area (OV) corresponding to the 5th frame 5-5 ) is calculated based on the object area (BBOX5) detected in the 5th frame and the virtual region of interest (ROI5) generated from it. On the other hand, in the example of Fig. 6, the overlapping area (OV) corresponding to the 5th frame 5-4 ) is calculated based on the object area (BBOX5) detected in the 5th frame and the virtual region of interest (ROI4) generated in the 4th frame.

[0053] The action recognition module (180) can use the Intersection over Human area (IoH), which is the ratio of the overlap area (OV) to the object area (BBOX), as an indicator for evaluating the relative size of the overlap area (OV). Here, the IoH can be expressed as in mathematical expression 3.

[0054]

[0055] In another example, the action recognition module (180) may use the Intersection over Floor area (IoF), which is the ratio of the overlap region (OV) to the virtual region of interest (ROI), as an indicator for evaluating the relative size of the overlap region (OV). Here, the IoF can be expressed as in mathematical equation 4.

[0056]

[0057] The action recognition module (180) can recognize that an object is in a fallen state based on whether the IoH or IoF calculated from a specific frame falls within a predefined numerical range. Here, the numerical range may be a value statistically derived from a plurality of images secured in advance. For example, the action recognition module (180) can determine that an object is fallen when the IoH has a value between 0.7 and 1.0. As another example, in order to reduce the probability of misrecognition, the action recognition module (180) may use a combination of conditions for IoH and conditions for IoE. Depending on the implementation example, the numerical ranges for IoH and / or IoE may be dynamically adjusted according to the installation environment of the camera. For example, if a large amount of data on the current camera installation environment is secured, the recognition performance can be further improved by applying a numerical range appropriate to the current environment using the statistics of the corresponding data.

[0058] The action recognition module (180) can recognize the motion of an object falling based on the temporal change pattern of the IoH or IoF calculated from each of a plurality of frames. For example, if the IoH has a pattern of increasing and then remaining at a high value between a predetermined number (N) of frames, as in the example of Table 2, the object can be determined to have fallen.

[0059] Frame 123...N-6N-5N-4N-3N-2N-1NIoH0.40.50.60.60.70.70.80.90.90.9

[0060] If an object is determined to have fallen, the action recognition module (180) may generate an alarm. The alarm may be provided visually, audibly, and / or tactilely to a control officer monitoring the surveillance space, and / or may be transmitted to a server operated by an emergency rescue agency (e.g., a fire department).

[0061] As described above, the image processing device (10) according to various embodiments of the present disclosure can recognize the action of an object based on the overlap rate between the virtual region of interest adaptively generated for each object instance and the object region. It has been experimentally confirmed that in non-fall cases where the object is in a posture such as standing, sitting, squatting, or kneeling, the IoH always has a value less than 0.7, and in the fall case, the IoH increases to a value exceeding 0.7 and close to 1. Therefore, even if this overlap rate is used alone, the fall action can be effectively recognized, and depending on the implementation example, the recognition performance can be further improved by combining the action recognition result based on the overlap rate with the action recognition result based on a learning-based or other rule-based algorithm.

[0062] FIG. 7 is a flowchart illustrating an action recognition method according to one embodiment of the present disclosure.

[0063] An image processing device (10) detects an object from an image (S700). The image processing device (10) can detect a bounding box surrounding an object of a pre-specified type from the image. Detecting the bounding box may include obtaining the coordinates of at least one point defining the position and / or size of the bounding box.

[0064] The image processing device (10) generates a virtual region of interest based on the region where the object is detected (S720). Here, the size and position of the virtual region of interest can be adaptively changed based on the size and position of the region where the object is detected. For example, the image processing device (10) can set a point within the region where the object is detected as the center position of the virtual region of interest. The image processing device (10) can set the length from the center point to the boundary of the virtual region of interest based on the length of the long side of the region where the object is detected. The image processing device (10) can generate a regular polygonal region that is tangent to a circle having the set center point and length as its center and radius, respectively, as the virtual region of interest.

[0065] The image processing device (10) determines whether the object is taking a predetermined type of action based on the area of ​​the overlapping area, which is the area where the object is detected and the virtual area of ​​interest overlap (S740). The image processing device (10) calculates the ratio of the overlapping area to the area where the object is detected or the area of ​​interest, and can recognize the object's action based on the calculated ratio. The image processing device (10) can independently recognize the object's action for each frame, or recognize the object's action by combining the ratios calculated from multiple frames.

[0066] For example, the image processing device (10) can determine whether an object has fallen based on whether a ratio calculated from a specific frame falls within a predefined numerical range. Here, the predefined numerical range may be a value statistically derived based on a camera installed in the same type of space as the camera that captured the image, or a plurality of images acquired from physically identical cameras.

[0067] As another example, the image processing device (10) can determine whether an object has fallen based on a change pattern of a ratio calculated for each of a plurality of frames. To this end, the image processing device (10) can generate a virtual region of interest for each frame based on an area where an object is detected in each of the plurality of frames. If any one target frame among the plurality of frames satisfies a predefined rule, the image processing device (10) can maintain at least one of the size and position of the virtual region of interest generated in a frame preceding the target frame in subsequent frames. Here, the predefined rule can be satisfied when the aspect ratio of the area where the object is detected is smaller than a preset threshold aspect ratio. The threshold aspect ratio may be a value specified in advance or a value adaptively set based on the aspect ratio of an area where an object having the same shape is detected in preceding frames.

[0068] FIG. 8 is a block diagram schematically illustrating an exemplary computing device that can be used to implement the devices and methods described in the present disclosure.

[0069] The computing device (80) may include some or all of a memory (800), a processor (820), storage (840), an input / output interface (860), and a communication interface (880). The computing device (80) may structurally and / or functionally include at least a portion of the image processing device (10). The computing device (80) may be a stationary computing device such as a desktop computer, a server, etc., as well as a mobile computing device such as a laptop computer, a smart phone, etc. The computing device (80) may be implemented with any specialized hardware accelerator capable of efficiently processing operations for an artificial intelligence model. For example, the computing device (80) may be implemented with a graphic processing unit (GPU), a tensor processing unit (TPU), a neural processing unit (NPU), etc.

[0070] The memory (800) may store a program that causes the processor (820) to perform a method or operation according to various embodiments of the present disclosure. For example, the program may include a plurality of instructions executable by the processor (820), and the method illustrated in FIG. 7 may be performed by executing the plurality of instructions by the processor (820).

[0071] The memory (800) may be a single memory or multiple memories. In this case, information required to perform the methods or operations according to various embodiments of the present disclosure may be stored in a single memory or divided and stored across multiple memories. If the memory (800) is comprised of multiple memories, the multiple memories may be physically separated.

[0072] The memory (800) may include at least one of volatile memory and non-volatile memory. The volatile memory includes SRAM (Static Random Access Memory) or DRAM (Dynamic Random Access Memory), and the non-volatile memory includes flash memory.

[0073] The processor (820) may include at least one core capable of executing at least one instruction. The processor (820) may execute instructions stored in the memory (800). The processor (820) may be a single processor or multiple processors.

[0074] Storage (840) maintains stored data even when power supplied to the computing device (80) is cut off. For example, storage (840) may include non-volatile memory, or may include storage media such as magnetic tape, optical disk, or magnetic disk.

[0075] A program stored in storage (840) may be loaded into memory (800) before being executed by processor (820). Storage (840) may store a file written in a programming language, and a program generated from the file by a compiler or the like may be loaded into memory (800). Storage (840) may store data to be processed by processor (820) and / or data processed by processor (820).

[0076] The input / output interface (860) may include input devices such as a keyboard, mouse, etc., and output devices such as a display device, printer, etc. A user may trigger execution of a program by the processor (820) and / or check the processing results of the processor (820) through the input / output interface (860).

[0077] The communication interface (880) may provide access to an external network. For example, the computing device (80) may communicate with other devices (e.g., a camera (20)) via the communication interface (880).

[0078] Each component of the device or method according to the present invention may be implemented in hardware, software, or a combination of hardware and software. Furthermore, the functions of each component may be implemented in software, with a microprocessor executing the software functions corresponding to each component.

[0079] Various implementations of the systems and techniques described herein may be implemented as digital electronic circuits, integrated circuits, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations of one or more computer programs executable on a programmable system. The programmable system includes at least one programmable processor (which may be a special purpose processor or a general purpose processor) coupled to receive data and instructions from and transmit data and instructions to a storage system, at least one input device, and at least one output device. Computer programs (also known as programs, software, software applications, or code) include instructions for the programmable processor and are stored on a "computer-readable recording medium."

[0080] A computer-readable recording medium includes any type of recording device that stores data that can be read by a computer system. Such computer-readable recording media may be non-volatile or non-transitory media such as ROMs, CD-ROMs, magnetic tapes, floppy disks, memory cards, hard disks, magneto-optical disks, storage devices, and may further include transitory media. Furthermore, the computer-readable recording medium may be distributed across network-connected computer systems, allowing computer-readable code to be stored and executed in a distributed manner.

[0081] Although the flowchart / timing diagram of this specification describes each process as being executed sequentially, this is merely an illustrative description of the technical idea of ​​one embodiment of the present disclosure. In other words, a person of ordinary skill in the art to which one embodiment of the present disclosure belongs may modify and apply various modifications and variations by changing the order described in the flowchart / timing diagram without departing from the essential characteristics of one embodiment of the present disclosure, or by executing one or more of the processes in parallel. Therefore, the flowchart / timing diagram is not limited to a chronological order.

[0082] The above description is merely an example of the technical idea of ​​the present embodiment, and those skilled in the art will appreciate that various modifications and variations can be made without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical idea of ​​the present embodiment, but rather to explain it, and the scope of the technical idea of ​​the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment should be interpreted by the claims below, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of rights of the present embodiment.

[0083]

[0084] CROSS-REFERENCE TO RELATED APPLICATION

[0085] This patent application claims priority to Korean Patent Application No. 10-2023-0141737, filed on October 23, 2023, which is incorporated herein by reference in its entirety.

Claims

1. A computer implementation method for recognizing the behavior of an object, The process of detecting objects from an image; A process of creating a virtual region of interest based on the region where the object is detected; and A process of determining whether the object is taking a predetermined type of action based on the area of ​​the overlapping area, which is the area where the object is detected and the virtual area of ​​interest overlap. A computer-implemented method comprising:

2. In paragraph 1, The size and location of the above virtual area of ​​interest are: A computer-implemented method in which the size and location of the area in which the object is detected are adaptively changed.

3. In paragraph 1, The above generating process is, A process of setting a point within the area where the object is detected as the center position of the virtual area of ​​interest; and A computer-implemented method comprising a process of setting a length from the center position to the boundary of the virtual region of interest based on the length of the long side of the region where the object is detected.

4. In paragraph 3, The above generating process is, A computer-implemented method further comprising a process of generating a convex polygonal area tangent to a circle having the center and radius of the set center position and length, respectively, as the virtual area of ​​interest.

5. In paragraph 1, The above judgment process is, A computer-implemented method comprising a process of calculating a ratio of the overlapping area to the area where the object is detected or the area of ​​interest.

6. In paragraph 5, The above judgment process is, A computer-implemented method for determining whether the object has collapsed based on whether the calculated ratio falls within a predetermined numerical range.

7. In paragraph 5, The above judgment process is, A computer-implemented method for determining whether an object has fallen based on a change pattern of a ratio calculated for each of a plurality of frames.

8. In paragraph 7, The above generating process is, Generating the virtual region of interest based on the region where the object is detected in each of the plurality of frames, A computer-implemented method for maintaining at least one of the size and position of a virtual region of interest generated in a frame preceding the target frame, if any one of the plurality of frames satisfies a predefined rule.

9. In paragraph 8, A computer-implemented method wherein the above predefined rule is that the aspect ratio of the area where the object is detected is smaller than a preset threshold aspect ratio.

10. A memory for storing instructions; and at least one processor, At least one processor executes the instructions, Detect objects from images, Based on the area where the above object is detected, a virtual area of ​​interest is created, A device that determines whether the object is taking a predetermined type of action based on the area of ​​the overlapping area, which is the area where the object is detected and the virtual area of ​​interest overlap.

11. A computer-readable recording medium having stored thereon a command, wherein the command, when executed by the computer, causes the computer to: The process of detecting objects from an image; A process of creating a virtual region of interest based on the region where the object is detected; and A computer-readable, non-transitory recording medium that causes a process of determining whether the object is taking a predetermined type of action based on the area of ​​an overlapping area, which is an area where the object is detected and the virtual area of ​​interest overlap.

Images