Object detection method using nested hierarchical structure analysis

Abstract

The present invention may provide an object detection method using nested hierarchical structure analysis, the method comprising the steps of: detecting a target object from an image by using an object detection model using artificial intelligence; acquiring object information about the target object; identifying an inclusion relationship between target objects from the object information through nested hierarchical structure analysis; and analyzing the object through the object information and the inclusion relationship between configured target objects. According to the above, it is possible to more accurately detect whether a worker is wearing a safety helmet or whether a motorcycle driver is wearing a helmet by defining target objects as a parent object and a child object, respectively, and analyzing a nested hierarchical structure of the parent object and the child object.

Description

Object detection method using nested hierarchy analysis

[0001] The present invention relates to an object detection method, and more particularly, to an object detection method using nested hierarchical structure analysis, which can more accurately detect whether a worker is wearing a safety helmet by analyzing the nested hierarchical structure of each object of a worker and a safety helmet.

[0002] Motorcycles, commonly referred to as "motorcycles," are powered by internal combustion engines and boast high speeds. However, they lack a body designed to protect the driver or passengers. Consequently, even minor accidents like collisions or rollovers frequently result in serious injury or death for the driver or passengers. Furthermore, construction sites for apartments and other buildings present numerous hazards, including accidents caused by falling objects like bricks, building materials, and tools, and the potential for unexpected hazards. Therefore, workers are required to wear safety gear.

[0003] In particular, since head injuries in accidents often lead to serious injury or death, helmets or safety helmets worn on the head are very important protective equipment.

[0004] However, a significant number of motorcyclists tend to avoid wearing helmets, and despite mandatory helmet wearing by law, they frequently ride without one. Meanwhile, despite mandatory helmet wearing at construction sites, accidents due to failure to wear one continue to occur.

[0005] Therefore, technologies to detect whether a motorcycle rider is wearing a helmet or a construction worker is wearing a hard hat are also being developed, given the importance of this.

[0006] However, conventional technologies for detecting whether a motorcycle helmet or safety helmet is worn have limitations in that they only detect objects independently in photos or videos, and thus cannot identify the inclusion relationship between objects. For this reason, although the driver and helmet, and the construction worker and safety helmet are each detected, there is no information that the helmet is included in the driver object or the safety helmet is included in the construction worker object. In particular, when two people ride a motorcycle, the motorcycle and the two drivers are detected, but there is no information on the relationship between the two people riding the motorcycle, making it difficult to accurately detect whether the motorcycle driver is wearing a helmet.

[0007] The purpose of the present invention is to provide an object detection method using nested hierarchical structure analysis, which can more accurately detect whether a worker is wearing a safety helmet by defining a worker and a safety helmet as a parent object and a child object, respectively, and analyzing their nested hierarchical structure.

[0008] The present invention can provide an object detection method using nested hierarchical structure analysis, characterized by including the steps of: detecting a target object from an image using an object detection model using artificial intelligence; obtaining object information about the target object; identifying an inclusion relationship between the target objects from the object information through nested hierarchical structure analysis; and analyzing the object using the object information and the inclusion relationship between the set target objects.

[0009] Here, the step of identifying the inclusion relationship of the target objects can identify the inclusion relationship between the parent object and the child object by defining the setting object as a parent object among the target objects and identifying the child object based on the parent object.

[0010] Furthermore, in the step of obtaining the object information, the object information may include the bounding box coordinates (x1, x2, y1, y2) of the parent object and the midpoint coordinates (x, y) of the child object.

[0011] The step of identifying the inclusion relationship of the above target objects can identify that the inclusion relationship exists by defining the target object as a child object of the parent object if the coordinates of the center point of the target object are located within the bounding box area of ​​the parent object by satisfying mathematical expression 1.

[0012] Mathematical formula 1

[0013] x1 < x < x2 and y1 < y < y2

[0014] Here, x1, x2, y1, y2 are the bounding box coordinates of the parent object, and x, y represent the midpoint coordinates of the child object.

[0015] The above target object includes a motorcycle driver, a helmet, and a human head (without a helmet), and the step of detecting the target object defines the motorcycle driver as a parent object from the detected target objects, and identifies the helmet and the human head by defining them as child objects based on the parent object, and the step of analyzing the object sets whether the motorcycle driver is wearing a helmet through object information of the target objects defined as the parent object and the child object, and if the child object is a helmet, the motorcycle driver is analyzed as wearing a helmet, and if the child object is a human head, the motorcycle driver is analyzed as not wearing a helmet.

[0016] In addition, the target object includes a worker, a safety helmet, and a human head, and the step of detecting the target object defines the worker as a parent object from the detected target objects, and identifies the worker by defining the safety helmet and the human head as child objects based on the parent object, and the step of analyzing the object sets whether the worker is wearing a safety helmet through object information of the target objects defined as the parent object and the child object, and if the child object is a safety helmet, the worker is analyzed as wearing a safety helmet, and if the child object is a human head, the worker is analyzed as not wearing a safety helmet.

[0017] The object detection method using nested hierarchical structure analysis according to the present invention can provide the effect of more accurately detecting whether a worker is wearing a safety helmet by defining each object as a parent object and a child object and analyzing the nested hierarchical structure thereof.

[0018] FIG. 1 is a flowchart illustrating an object detection method using nested hierarchical structure analysis according to an embodiment of the present invention.

[0019] FIG. 2 is a photograph showing a case where a target object in an image is identified in an object detection method using nested hierarchical structure analysis according to an embodiment of the present invention.

[0020] FIG. 3 is a diagram schematically showing the inclusion relationship conditions between parent objects and child objects in an object detection method using nested hierarchical structure analysis according to an embodiment of the present invention.

[0021] FIG. 4 is a flowchart showing a specific embodiment of an object detection method using nested hierarchical structure analysis according to an embodiment of the present invention.

[0022] FIG. 5 is a flowchart showing another specific embodiment of an object detection method using nested hierarchical structure analysis according to an embodiment of the present invention.

[0023] Hereinafter, a preferred embodiment of the present invention will be described with reference to the attached drawings to enable a more specific understanding of the present invention.

[0024] The present invention is susceptible to various modifications and embodiments. Specific embodiments are illustrated and described in detail in the drawings. However, this is not intended to limit the present invention to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present invention.

[0025] While terms such as "first" and "second" may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a "second component," and similarly, a second component may also be referred to as a "first component." The term "and / or" includes a combination of multiple related items described herein or any of multiple related items described herein.

[0026] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0027] The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0028] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and will not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

[0029] Hereinafter, with reference to the attached drawings, preferred embodiments of the present invention will be described in more detail. In order to facilitate an overall understanding in describing the present invention, identical reference numerals will be used for identical components in the drawings, and redundant descriptions of identical components will be omitted.

[0030] First, the object detection method using nested hierarchical structure analysis according to an embodiment of the present invention (hereinafter referred to as the “object detection method”) independently detects each object in an image including a photograph or video and identifies the inclusion relationship between the detected objects, thereby overcoming the limitation of not being able to identify the inclusion relationship between existing objects, and thus can accurately and effectively identify whether a worker is wearing a safety helmet or a motorcycle driver is wearing a helmet.

[0031] Referring to FIG. 1, a method for detecting an object according to an embodiment of the present invention is described. First, a target object can be detected from an image using an object detection model using artificial intelligence (S10).

[0032] Once a target object is detected (S20), object information about the target object can be obtained (S30). Here, the object information may include coordinates for the target object, based on the nested hierarchical structure analysis of the present invention.

[0033] When object information of target objects is obtained according to the above, the inclusion relationship of the target objects can be identified from the object information through nested hierarchical structure analysis (S40).

[0034] Here, the step of identifying the inclusion relationship of the target objects can identify the inclusion relationship between the parent object and the child object by defining the setting object as a parent object among the target objects and identifying the child object based on the parent object.

[0035] Meanwhile, at this time, the object information may include the bounding box coordinates (x1, x2, y1, y2) of the parent object and the midpoint coordinates (x, y) of the child object.

[0036] Accordingly, the inclusion relationship of the target objects can be identified by defining the target object as a child object of the parent object if the coordinates of the center point of the target object are located within the bounding box area of ​​the parent object, satisfying mathematical expression 1.

[0037] Mathematical formula 1

[0038] x1 < x < x2 and y1 < y < y2

[0039] Here, x1, x2, y1, y2 are the bounding box coordinates of the parent object, and x, y represent the midpoint coordinates of the child object.

[0040] Afterwards, the object can be analyzed through the object information and the inclusion relationship between the set target objects.

[0041] Meanwhile, the object detection method of the present invention can be applied to various fields, and as an example thereof, it can determine whether a worker is wearing a safety helmet or whether a motorcycle driver is wearing a helmet.

[0042] In the following, we will first examine the analysis of whether or not motorcycle drivers wear helmets.

[0043] First, the target object can be detected from the captured image using an object detection model.

[0044] Here, the target object is to determine whether a motorcycle driver is wearing a helmet, and may include a motorcycle driver, a helmet, and a human head (no helmet).

[0045] Figure 2 shows a state in which a motorcycle rider, a helmet, and a no-helmet are each independently detected within an image frame.

[0046] Here, the object detection model may be applied as an artificial intelligence model, such as YOLO, SSD, or R-CNN, and preferably, the identification of the target object is performed by applying the aforementioned YOLO model, but is not limited thereto.

[0047] Meanwhile, the target objects detected by the AI ​​model can be defined as parent and child objects. Accordingly, the motorcycle driver can be defined as the parent object among the detected target objects, and the helmet and human head can be defined as child objects based on this parent object for identification.

[0048] After identifying target objects within an image as described above, the inclusion relationship between each identified target object can be determined.

[0049] To this end, first, in the present invention, the inclusion relationship can be identified based on the coordinates of each object, and the inclusion relationship can be identified from the object information of each of the above-mentioned target objects.

[0050] Referring to FIG. 3, the object information may include bounding box coordinates (x1, x2, y1, y2) to secure area information in the case of the parent object, and may include midpoint coordinates (x, y) to determine whether the child object is included in the area of ​​the parent object.

[0051] When object information of each target object is obtained according to the above, the inclusion relationship of the target objects can be identified through object nesting hierarchical representation analysis.

[0052] In detail, the inclusion relationship between target objects can be identified by utilizing the size and position of the bounding box.

[0053] That is, if the center point coordinates of the target object are located within the bounding box area of ​​the parent object, the object is defined as a child object and identified as an inclusion relationship, and the child object can be identified as being in a child relationship with the parent object.

[0054] Mathematical expression 1 is a condition for identifying the inclusion relationship between these target objects. If Mathematical expression 1 is satisfied, the child object can be identified as being in a child relationship with the parent object.

[0055] Mathematical formula 1

[0056] x1 < x < x2 and y1 < y < y2

[0057] When the inclusion relationship of target objects is established as described above, it is possible to establish whether a motorcycle driver is wearing a helmet through object information of the target objects defined as parent objects and child objects.

[0058] That is, if the child object is a helmet, the motorcycle driver can be analyzed as wearing a helmet. On the other hand, if the child object is a human head, the motorcycle driver can be analyzed as not wearing a helmet.

[0059] FIG. 4 is a diagram illustrating an example of a helmet wearing detection method according to an embodiment of the present invention. Referring to the drawing, first, the driver, helmet, and no-helmet objects are independently detected in the image, object information for each object is obtained, and through object detection result processing, whether the object is included in the driver object can be confirmed.

[0060] At this time, the object detection result processing defines a function to input a list of detected objects and the bounding box of each Base (rider) object, and only checks if the detected object is an object within the Base (e.g., 'helmet' or 'no-helmet'), and checks whether the bounding box of the object overlaps with the bounding box of the Base, and if so, this object can be added to the list of related objects.

[0061] Afterwards, you can perform operations based on the hierarchy to determine whether a specific object is included. For example, checking whether a rider is not wearing a helmet can be done by receiving a map containing a list of objects related to the rider object above, determining whether there is a rider that includes 'no-helmet', and if a 'no-helmet' object is found, returning true immediately, and returning false if 'no-helmet' is not found after checking all riders.

[0062] Below, we will examine an example for determining whether a worker is wearing a safety helmet in addition to whether a motorcycle driver is wearing a helmet as mentioned above.

[0063] First, the detailed configuration for detecting whether the worker is wearing a safety helmet corresponds significantly to the configuration for detecting whether the motorcycle driver is wearing a helmet, and in the following, only the contents that are different from this will be examined in detail.

[0064] Referring to Fig. 5, the target object may first include a worker, a safety helmet, and a human head.

[0065] At this time, detection of the target object can be identified by defining the worker as a parent object from the detected target objects and defining the safety helmet and the human head as child objects based on the parent object.

[0066] Afterwards, the analysis of the above object can be performed by setting whether the worker is wearing a safety helmet through the object information of the target objects defined as parent objects and child objects. If the child object is a safety helmet, the worker can be analyzed as wearing a safety helmet.

[0067] On the other hand, if the child object is a human head, it can be analyzed that the worker is not wearing a safety helmet.

[0068] While the present invention has been described with reference to the embodiments illustrated in the drawings, these are merely exemplary, and those skilled in the art will appreciate that various modifications and equivalent alternative embodiments are possible. Therefore, the true scope of technical protection of the present invention should be determined by the technical spirit of the appended claims.

[0069] The present invention can be used in the traffic safety management and construction safety management industries.

Claims

1. A step of detecting a target object from an image using an object detection model using artificial intelligence; A step of obtaining object information for the above target object; A step of identifying the inclusion relationship of the target objects from the object information through nested hierarchical structure analysis; An object detection method using nested hierarchical structure analysis, characterized by including a step of analyzing an object through the object information and the inclusion relationship between the set target objects.

2. In paragraph 1, The step of identifying the inclusion relationship between the above target objects is: An object detection method using nested hierarchy analysis, characterized in that the target objects are defined as parent objects, child objects are identified based on the parent objects, and the inclusion relationship between the parent objects and the child objects is identified.

3. In paragraph 2, In the step of obtaining the above object information, the above object information is, An object detection method using nested hierarchy analysis, characterized in that it includes bounding box coordinates (x1, x2, y1, y2) of the parent object and the midpoint coordinates (x, y) of the child object.

4. In paragraph 2, The step of identifying the inclusion relationship between the above target objects is: An object detection method using nested hierarchy analysis, characterized in that if the coordinates of the center point of the target object are located within the bounding box area of ​​the parent object by satisfying mathematical expression 1, the target object is defined as a child object of the parent object and an inclusion relationship is identified. Mathematical Formula 1 x1 < x < x2 and y1 < y < y2 Here, x1, x2, y1, y2 are the bounding box coordinates of the parent object, and x, y represent the coordinates of the midpoint of the child object.

5. In paragraph 4, The above target object is, Including motorcyclists, helmets, and human heads (without helmets), The step of detecting the above target object is: From the detected target objects, the motorcycle driver is defined as the parent object, and the helmet and the human head are defined as child objects based on the parent object and identified. The steps for analyzing the above object are: An object detection method using nested hierarchy analysis, characterized in that it sets whether a motorcycle driver is wearing a helmet through object information of the target objects defined as parent objects and child objects, and if the child object is a helmet, it analyzes that the motorcycle driver is wearing a helmet, and if the child object is a human head, it analyzes that the motorcycle driver is not wearing a helmet.

6. In paragraph 4, The above target object is, Including workers, safety helmets, and human heads, The step of detecting the above target object is: From the detected target objects, the worker is defined as the parent object, and the safety helmet and the human head are defined as child objects based on the parent object and identified. The steps for analyzing the above object are: An object detection method using nested hierarchical structure analysis, characterized in that it sets whether a worker is wearing a safety helmet through object information of the target objects defined as parent objects and child objects, and if the child object is a safety helmet, it analyzes that the worker is wearing a safety helmet, and if the child object is a human head, it analyzes that the worker is not wearing a safety helmet.

Images