Method for determining whether safety equipment is fastened

Abstract

An embodiment of the present disclosure provides a method for monitoring whether safety equipment is fastened or not by a worker during work, in a captured image, which is performed by a computing device. In an embodiment, the method may comprise the steps of: receiving a first image captured at a first timepoint; acquiring the result of detecting multiple objects from the first image by using an artificial intelligence model; determining whether the worker has started working or not on the basis of the result of detection by a work determination module configured to determine whether the worker is working or not; acquiring the positions of the worker and the safety equipment on the basis of the result of detection by an object analysis module configured to determine the positions of the worker and the safety equipment among the multiple objects in response to the determination regarding whether the worker has started working or not; and determining whether the worker has fastened the safety equipment or not on the basis of the positions and whether the worker has started working or not by a safety equipment fastening determination module configured to determine whether the worker has fastened the safety equipment or not.

Description

Method for determining whether safety equipment is fastened

[0001] The present disclosure relates to the field of information processing, and more specifically, to a method for determining whether a worker is wearing safety equipment during work using an artificial intelligence model.

[0002] In modern society, with the advancement of artificial intelligence, systems utilizing artificial intelligence are being developed and utilized in various industries.

[0003] In industrial settings where worker safety is paramount, such as construction sites, construction sites, and logistics warehouses, systems that can monitor work sites or workers to ensure their safety are not threatened are in demand, incorporating artificial intelligence and image processing technologies.

[0004] Each work site has a manual to protect workers according to the characteristics of the work site, and previously, the supervision of whether the manual was followed was directly carried out by a person in the role of a supervisor.

[0005] However, it is somewhat difficult for a small number of people to continuously check in real time whether these manuals are being followed throughout the entire work site.

[0006] Accordingly, demand is increasing for monitoring systems that incorporate artificial intelligence to enable accurate supervision and management of work sites and workers through real-time footage captured at the work site.

[0007] The present disclosure is conceived in response to the aforementioned background technology, and has the purpose of efficiently using an artificial intelligence model to determine whether a worker is wearing safety equipment during work.

[0008] The technical problems of the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

[0009] In one embodiment of the present disclosure, a method for monitoring whether a worker is wearing safety equipment during work within a captured image, performed by a computing device, is disclosed.

[0010] In one embodiment, the method may include: receiving a first image captured at a first point in time; obtaining a detection result of detecting a plurality of objects from the first image using an artificial intelligence model; determining, by a work determination module that determines whether the worker is performing a task, whether the worker has started a task based on the detection result; obtaining, by an object analysis module that determines, in response to the determination of whether the task has started, the positions of the worker and the safety equipment among the plurality of objects, the positions of the worker and the safety equipment based on the detection result; and determining, by a safety equipment fastening determination module that determines, by a safety equipment fastening determination module that determines, whether the worker has fastened the safety equipment, whether the worker has fastened the safety equipment based on whether the task has started and the position.

[0011] In one embodiment, the work judgment module may calculate a first area corresponding to an area of ​​a predetermined reference area of ​​the first image that overlaps an area occupied by the worker in the first image, and compare the first area with a threshold area that is predetermined to correspond to a criterion for determining whether the worker starts or ends work, thereby determining whether the worker starts or ends work.

[0012] In one embodiment, determining whether the worker starts or ends work may be performed by determining, for a second image acquired at a second time point that is after the first time point, if a second area of ​​overlap between an area occupied by the worker in the second image and the reference area is different from the first area, and if the first area is less than the threshold area and the second area is greater than or equal to the threshold area, the second time point may be determined as the work start time point, and if the first area is greater than or equal to the threshold area and the second area is less than or equal to the threshold area, the second time point may be determined as the work end time point.

[0013] In one embodiment, the area occupied by the worker in the first image may be an area defined by a bounding box of an object corresponding to the worker or a segmentation result of an object corresponding to the worker in a detection result of detecting a plurality of objects in the first image.

[0014] In one embodiment, the object analysis module may use the detection result to obtain a first safety equipment position as the position of the safety equipment at the time when the worker's work begins, and a second safety equipment position corresponding to the position of the safety equipment after the time when the work begins. In addition, the safety equipment fastening judgment module may perform a primary judgment as to whether the safety equipment is fastened by comparing the first safety equipment position and the second safety equipment position.

[0015] In one embodiment, the object analysis module may obtain the position of the worker as a first worker position at the time when the worker's work begins, and obtain a second worker position corresponding to the position of the worker after the time when the worker's work begins. In addition, the first judgment as to whether the safety equipment is fastened may be performed based on the first safety equipment position, the first worker position, the second safety equipment position, and the second worker position.

[0016] In one embodiment, the first determination as to whether the safety equipment is fastened may be performed based on the results of calculating a first distance corresponding to the distance between the first safety equipment position and the second safety equipment position, a second distance corresponding to the distance between the first worker position and the second worker position, a third distance corresponding to the distance between the first safety equipment position and the first worker position, and a fourth distance corresponding to the distance between the second safety equipment position and the second worker position.

[0017] In one embodiment, the first worker position and the first safety equipment position may be positions corresponding to the first image at a first point in time. And the second worker position and the second safety equipment position may be positions corresponding to the second image at a second point in time that is after the first point in time.

[0018] In one embodiment, the safety equipment may include a safety block and a safety hook.

[0019] In one embodiment, the first safety equipment location may include a first-first safety equipment location and a first-second safety equipment location corresponding to the locations of the safety block and the safety hook, respectively.

[0020] In one embodiment, the second safety equipment location may include a 2-1 safety equipment location and a 2-2 safety equipment location.

[0021] In one embodiment, the primary determination of whether the safety equipment is fastened may be additionally performed based on the distance between the 1-1 safety equipment location and the 1-2 safety equipment location, and the distance between the 2-1 safety equipment location and the 2-2 safety equipment location.

[0022] In one embodiment, the step of determining whether the worker has fastened the safety equipment may include the step of determining whether the worker has fastened the safety equipment as a group unit of a plurality of images including the first image and at least one subsequent image following the first image.

[0023] In one embodiment, the step of determining whether the worker has fastened the safety equipment may include a step of determining whether the worker has fastened the safety equipment by using the primary judgments corresponding to each of the images included in the set of the plurality of images.

[0024] In one embodiment, the step of determining whether the worker has fastened the safety equipment may include the step of determining, as a majority value, a primary judgment value corresponding to the highest number of appearances among the primary judgments corresponding to each of the images included in the set of the plurality of images; and the step of using the majority value to determine, for the set of the plurality of images, whether the worker has fastened the safety equipment.

[0025] In one embodiment, the primary judgment value may be at least one of a first value determined based on whether the worker has started working; a second value determined when it is determined that the worker has fastened the safety equipment; and a third value determined when it is determined that the worker has not fastened the safety equipment.

[0026] In one embodiment, the method may further include a step of generating an event for notifying the worker to fasten the safety equipment from the time point at which the safety equipment is determined to be not fastened when the safety equipment fastening determination module determines that the safety equipment is not fastened; and a step of continuously generating the event by analyzing an image after the time point at which the safety equipment is determined to be not fastened until the worker is re-determined to have fastened the safety equipment.

[0027] In one embodiment of the present disclosure, a computer program stored in a computer-readable storage medium is disclosed.

[0028] In one embodiment, the computer program may include instructions for monitoring, by at least one processor of the computing device, whether a worker is wearing safety equipment during work within the captured image.

[0029] In one embodiment, the commands may include: a command for receiving a first image captured at a first point in time; a command for obtaining a detection result of detecting a plurality of objects from the first image using an artificial intelligence model; a command for determining, by a work determination module that determines whether the worker is performing a task, whether the worker has started a task based on the detection result; a command for obtaining, by an object analysis module that determines, in response to the determination of whether the task has started, the positions of the worker and the safety equipment among the plurality of objects based on the detection result; and a command for determining, by a safety equipment fastening determination module that determines, whether the worker has fastened the safety equipment, whether the worker has fastened the safety equipment based on whether the task has started and the position.

[0030] In one embodiment of the present disclosure, a computing device is disclosed for monitoring whether a worker is wearing safety equipment during work within a captured image.

[0031] In one embodiment, the computing device may include a processor; and memory.

[0032] In one embodiment, the processor receives a first image captured at a first point in time, obtains a detection result of detecting a plurality of objects from the first image using an artificial intelligence model, determines whether the worker is working, determines whether the worker has started working based on the detection result by a work determination module, and in response to the determination of whether the work has started, determines the positions of the worker and the safety equipment among the plurality of objects by an object analysis module, and obtains the positions of the worker and the safety equipment based on the detection result by a safety equipment fastening determination module that determines whether the worker has fastened the safety equipment, and determines whether the worker has fastened the safety equipment based on whether the work has started and the position.

[0033] A method according to one embodiment of the present disclosure can efficiently determine whether a worker is wearing safety equipment using an artificial intelligence model.

[0034] Further scope of the applicability of the present disclosure will become apparent from the detailed description below. However, since various modifications and variations within the spirit and scope of the present invention will become apparent to those skilled in the art, it should be understood that the detailed description and specific examples, such as preferred embodiments of the present invention, are given by way of example only.

[0035] Various aspects are now described with reference to the drawings, wherein like reference numerals are used to refer to similar components. In the following examples, for purposes of explanation, numerous specific details are set forth to provide a comprehensive understanding of one or more aspects. However, it will be apparent that such aspects may be practiced without these specific details.

[0036] FIG. 1 is a schematic diagram of a computing device for monitoring whether a worker is wearing safety equipment during work according to some embodiments of the present disclosure.

[0037] FIG. 2 is a schematic diagram of a system for monitoring whether a worker is wearing safety equipment during work according to some embodiments of the present disclosure.

[0038] FIG. 3 is a flowchart of a method for monitoring whether a worker is wearing safety equipment during work according to some embodiments of the present disclosure.

[0039] FIG. 4 is a schematic diagram illustrating an example of a method for monitoring whether a worker is wearing safety equipment during work according to some embodiments of the present disclosure.

[0040] FIG. 5 is a flowchart of another example of a method for monitoring whether a worker is wearing safety equipment during work according to some embodiments of the present disclosure.

[0041] FIG. 6 is an illustration of a method for monitoring whether a worker is wearing safety equipment during work according to some embodiments of the present disclosure.

[0042] FIG. 7 is another illustration of a method for monitoring whether a worker is wearing safety equipment while working, according to some embodiments of the present disclosure.

[0043] FIG. 8 is another illustration of a method for monitoring whether a worker is wearing safety equipment while working, according to some embodiments of the present disclosure.

[0044] FIG. 9 is a general schematic diagram of an artificial intelligence model in which embodiments of the contents of the present disclosure can be implemented.

[0045] FIG. 10 is a general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented.

[0046] Various embodiments and / or aspects are now disclosed with reference to the drawings. In the following description, for purposes of explanation, numerous specific details are set forth to provide a thorough understanding of one or more aspects. However, it will be apparent to one skilled in the art that such aspects may be practiced without these specific details. The following description and the accompanying drawings detail specific exemplary aspects of one or more aspects. However, these aspects are exemplary, and any of the various methods within the principles of various aspects may be utilized, and the description is intended to encompass all such aspects and their equivalents. Specifically, the terms "embodiment," "example," "aspect," and "example" as used herein are not intended to imply that any aspect or design described therein is preferred or advantageous over other aspects or designs.

[0047] Hereinafter, regardless of the drawing numbers, identical or similar components are assigned the same reference numerals, and redundant descriptions thereof are omitted. Furthermore, when describing the embodiments disclosed in this specification, if a detailed description of a related known technology is judged to obscure the gist of the embodiments disclosed in this specification, the detailed description thereof will be omitted. Furthermore, the attached drawings are only intended to facilitate easy understanding of the embodiments disclosed in this specification, and the technical concepts disclosed in this specification are not limited by the attached drawings.

[0048] In the present disclosure, terms expressed as N, such as first, second, or third, are used to distinguish at least one entity. For example, entities expressed as first and second may be the same or different from each other.

[0049] Although terms like "first" and "second" are used to describe various components, these components are not limited by these terms. These terms are merely used to distinguish one component from another. Therefore, it should be understood that a "first" component referred to below may also be a "second" component within the technical scope of the present invention.

[0050] Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used in their common sense to those of ordinary skill in the art to which the present invention pertains. Furthermore, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0051] Furthermore, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified or clear from context, "X employs A or B" is intended to mean either of the natural inclusive permutations. That is, if X employs A; X employs B; or X employs both A and B, "X employs A or B" can apply to any of these cases. Furthermore, the term "and / or" as used herein should be understood to refer to and include all possible combinations of one or more of the associated items listed.

[0052] Additionally, it should be understood that the terms "comprises" and / or "comprising" imply the presence of a given feature and / or component, but do not preclude the presence or addition of one or more other features, components, and / or groups thereof. Furthermore, unless otherwise specified or clear from context to refer to the singular form, the singular form in the specification and claims should generally be construed to mean "one or more."

[0053] Additionally, the terms “information” and “data” as used herein may often be used interchangeably.

[0054] The purpose and effects of the present disclosure, as well as the technical configurations for achieving them, will become clearer with reference to the embodiments described below in detail, along with the accompanying drawings. In describing the present disclosure, detailed descriptions of known functions or configurations will be omitted if they are deemed to unnecessarily obscure the gist of the disclosure. Furthermore, the terms described below are defined based on the functions of the present disclosure and may vary depending on the intent or custom of the user or operator.

[0055] However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various other forms. These embodiments are provided solely to ensure the completeness of the disclosure and to fully inform those skilled in the art of the disclosure of the scope of the disclosure. The disclosure is defined solely by the scope of the claims. Therefore, such definitions should be based on the contents of this specification as a whole.

[0056] Hereinafter, with reference to FIGS. 1 to 10, a method and examples for monitoring whether a worker is wearing safety equipment during work will be described.

[0057] FIG. 1 is a schematic diagram of a computing device for monitoring whether a worker is wearing safety equipment during work according to some embodiments of the present disclosure.

[0058] Referring to FIG. 1, a computing device (100) may include a processor (110) and a memory (120). However, the above-described components are not essential for implementing the computing device (100), and thus the computing device (100) may have more or fewer components than the components listed above.

[0059] In the present disclosure, the computing device (100) may be a typical server. Here, the server may be a system in which users share network resources, and may be a computing environment in which users can rent as much as they need and use it via a network at a desired time. Such a server-based system may include a deployment model such as a public cloud, a private cloud, a hybrid cloud, a community cloud, or a service model such as an infrastructure as a service (IaaS), a platform as a service (PaaS), or a software as a service (SaaS).

[0060] However, the computing device (100) of the present disclosure is not limited to such server-based systems, and may be implemented in a centralized or edge computing manner according to embodiments of the present disclosure. Additionally, depending on the implementation method, the computing device (100) may be implemented as a user terminal.

[0061] In one embodiment of the present disclosure, the processor (110) may typically include all types of devices and / or programs capable of processing the operations and data of the computing device (100). For example, it may refer to a data processing device built into hardware that has a physically structured circuit to perform a function expressed by a code or instruction included in the program. Examples of such data processing devices built into hardware include processing devices such as a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, an application-specific integrated circuit (ASIC), and a field programmable gate array (FPGA), but the scope of the present invention is not limited thereto.

[0062] For example, when monitoring whether a worker fastens safety equipment during work within a captured video, the computing device (100) may perform a process for monitoring whether safety equipment is fastened using programs stored in the memory (120) for monitoring whether safety equipment is fastened. That is, the processor (110) controls the entire process of monitoring whether a worker fastens safety equipment during work, including the above process, but is not limited thereto.

[0063] In a further embodiment of the present disclosure, the memory (120) may be included in another computing device (e.g., another server or another user terminal) separate from the computing device (100). In such a case, the computing device (100) may communicate with the other computing device and obtain desired data from the memory (120) included in the other computing device. For example, a server (not shown) that determines whether safety equipment including the memory (120) is fastened may exist separately from the computing device (100), and the computing device (100) may obtain from the server that determines whether safety equipment is fastened, which is necessary for performing the method according to embodiments of the present disclosure.

[0064] In one embodiment of the present disclosure, the memory (120) may include memory and / or a permanent storage medium. The memory may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk, but the scope of the present invention is not limited thereto.

[0065] In the present disclosure, a persistent storage medium may refer to a non-volatile storage medium capable of persistently storing any data, such as, for example, a magnetic disk, an optical disk, and a magneto-optical storage device, as well as a storage device based on flash memory and / or battery-backed memory. Such persistent storage medium may communicate with the processor (110) and memory (120) of the computing device (100) via various communication means, such as a communication unit. In a further embodiment, such persistent storage medium may be located outside the computing device (100) and may be capable of communicating with the computing device (100). According to one embodiment of the present disclosure, the persistent storage medium and the storage unit may be collectively referred to as memory (120). In a further embodiment, the persistent storage medium in the present specification may be used interchangeably with the memory (120).

[0066] In one embodiment of the present disclosure, the computing device (100) may further include an image acquisition module (not shown). In one embodiment, the image acquisition unit may include a camera that acquires images by photographing a work site where a worker is working.

[0067] In one embodiment, the image acquisition unit can acquire multiple images, rather than just one, at a specific point in time by installing multiple cameras at the work site. Furthermore, the computing device (100) can simultaneously analyze the multiple images to monitor whether safety equipment is fastened.

[0068] In one embodiment of the present disclosure, the computing device (100) may include a display module (not shown) and a speaker (not shown).

[0069] In one embodiment, the display unit and speaker can convey information to the worker by outputting an event output by the computing device (100), for example, a broadcast or notification sound guided by whether safety equipment is fastened.

[0070] FIG. 2 is a schematic diagram of a system for monitoring whether a worker is wearing safety equipment during work according to some embodiments of the present disclosure.

[0071] Referring to FIG. 2, the system (200) for monitoring whether the safety equipment of the present disclosure is fastened may include an object recognition module (210) including an artificial intelligence model.

[0072] In one embodiment, the system (200) may be a computing device (100) or may include at least one computing device (100) or may be one of the programs included in the computing device (100).

[0073] In one embodiment, the object recognition module (210) may receive an image acquired from an image capture unit and then recognize objects included in the input image using an artificial intelligence algorithm. The term "module" in the present disclosure refers to a computer-related entity, hardware, firmware, software, a combination of software and hardware, or an execution of software, and may be used interchangeably. For example, a module may be, but is not limited to, a procedure running on a processor, a processor, an object, a thread of execution, a program, an application, and / or a computing device. One or more modules may reside within a processor and / or a thread of execution. A module may be localized within a single computer. A single module may be distributed between two or more computers. Furthermore, these modules may be executed from various computer-readable media having various data structures stored therein. Modules may communicate via local and / or remote processes, for example, by signals having one or more data packets (e.g., data from one component interacting with another component in a local system, a distributed system, and / or data transmitted via signals to another system and / or over a network such as the Internet).

[0074] In one embodiment, the object recognition module (210) recognizes all objects included in the image, performs a process of inferring objects corresponding to work areas, workers, and safety equipment among all objects, and transmits information about the inferred work areas, workers, and safety equipment to the work judgment module (220) and / or the object analysis module (230).

[0075] The specific method by which the object recognition module (210) performs object recognition using an artificial intelligence algorithm will be described later in FIG. 9.

[0076] In one embodiment, the work judgment module (220) may use the inferred work area and worker information to determine whether the worker is working, including whether the worker has started, is in the process of working, or has finished working, and may transmit related information to the storage unit (250). In one embodiment, the work judgment module (220) may use the object recognition result or object detection result from the object recognition module (210) to determine the start time of the work and / or the end time of the work.

[0077] In one embodiment, the object analysis module (230) may obtain location information corresponding to the positions of the worker and safety equipment within the acquired image and transmit the information to the safety equipment fastening determination module (240) and / or analyze the location information and transmit information on whether the worker has fastened the safety equipment to the storage unit (250). For example, the object analysis module (230) may determine a first object corresponding to the worker and a second object corresponding to the safety equipment among the objects obtained from the object recognition module (210). As another example, the object analysis module (230) may determine the positions of the worker and the safety equipment within the image by determining the position information of the object corresponding to the worker and the object corresponding to the safety equipment included in the object recognition result or object detection result of the object recognition module (210). In addition, the object analysis module (230) may also determine whether the worker has fastened the safety equipment based on the determination of the positions of the objects and the safety equipment.

[0078] In one embodiment, the safety equipment fastening judgment module (240) may receive location information obtained from the object analysis module (230) and / or information stored in the storage unit (250) that primarily determines whether a worker is working and / or whether safety equipment is fastened.

[0079] In one embodiment, the object analysis module (230) calculates the x, y coordinate range according to a reference within the image of the worker and safety equipment and the horizontal length, vertical length and area calculated based on the x, y coordinate range, thereby obtaining the location and area of ​​the worker and safety equipment.

[0080] In one embodiment, the safety equipment fastening judgment module (240) can use the received information to make a final judgment on whether or not the worker has fastened the safety equipment.

[0081] In one embodiment, the storage (250) may be a separate memory (not shown) that is identical to or different from the memory (120) of the computing device (100).

[0082] In one embodiment, the storage unit (250) can sequentially accumulate information received from the task judgment module (220) and / or the object analysis module (230) according to the time of input to the storage unit (250) and store it in an array form.

[0083] In the present disclosure, a specific method for monitoring whether a worker fastens safety equipment during work by a system (200) is described below with reference to FIG. 3.

[0084] FIG. 3 is a flowchart of a method for monitoring whether a worker is wearing safety equipment during work according to some embodiments of the present disclosure.

[0085] Referring to FIG. 3, in step S100, the computing device (100) may receive a first image captured at a first point in time. In one embodiment, the first point in time may be a specific point in time, a predetermined time unit, and / or a frame of an image at a specific point in time.

[0086] And, in step S110, the computing device (100) can obtain a detection result of detecting a plurality of objects from the first image using an artificial intelligence model.

[0087] In one embodiment, the plurality of objects may be all objects included in the image that the artificial intelligence model can recognize, and the detection result may mean inferred information for all objects or inferred information for objects predefined by the user.

[0088] And, in step S120, the computing device (100) can determine whether the worker starts working based on the detection result obtained in step S110 by the work determination module (220) that determines whether the worker is working.

[0089] In one embodiment, a specific method for determining whether to start a task is described later with reference to FIG. 4.

[0090] FIG. 4 is a schematic diagram illustrating an example of a method for monitoring whether a worker is wearing safety equipment during work according to some embodiments of the present disclosure.

[0091] In one embodiment, the task judgment module (220) may define a reference area (310) of a predetermined reference area among the entire area (300) of the first image. In addition, the task judgment module (220) may calculate a first area (330) corresponding to the area of ​​an area overlapping the target area (320) occupied by the worker in the entire area (300). For example, an area corresponding to the worker (e.g., a rectangular bounding box) included in a detection result obtained through an artificial intelligence model may be compared with the reference area. The reference area may mean an area corresponding to a predetermined space for judging whether a task has started or ended.

[0092] In one embodiment, the work judgment module (220) can compare the first area (330) with a predetermined threshold area corresponding to a criterion for determining whether the worker starts or ends the work to determine whether the worker starts or ends the work.

[0093] In one embodiment, the reference area (310) and / or the target area (320) may be an area defined by a bounding box of an object corresponding to the work area and / or the worker or a segmentation result of an object corresponding to the work area and / or the worker in a detection result of detecting a plurality of objects in the first image.

[0094] In one embodiment, the area may mean the border occupied by the bounding box in the image and / or the pixel information corresponding within the border.

[0095] For example, the total area (300) of the first image may be 1920x1080 when expressed in pixels. In addition, the reference area (310) of the reference region may be 800x400, and the worker area (320) that the worker occupies in the total area (300) may be 600x300.

[0096] In one embodiment, the task judgment module (220) may obtain x, y coordinates occupied by the reference area (310) and x, y coordinates occupied by the target area (320) to calculate a first area (330) corresponding to the degree of overlap between the reference area (310) and the target area (320). In addition, the task judgment module (220) may obtain the degree of overlap based on each coordinate using the IoU (Intersection of Union) method.

[0097] In addition, the work judgment module (220) can determine that the worker is working and / or has started working when the first area (330) corresponding to the degree of overlap between the reference area (310) and the target area (320) is equal to or exceeds the threshold area.

[0098] In one embodiment, the reference area (310) may be an area corresponding to the work area, and the work area may be a predetermined area within the image.

[0099] In another embodiment, the work judgment module (220) can use the first area (330) to determine whether the worker starts and / or ends work, and can determine a specific time point for starting and / or ending work.

[0100] In one embodiment, the work judgment module (220) sets the area occupied by the worker in the second image as the target area (320) for the second image acquired at a second point in time that is after the first point in time, and if the second area (not shown) overlapping between the reference areas (310) is different from the first area (330), and if the first area (330) is less than a threshold area and the second area is greater than or equal to the threshold area, the second point in time can be determined as the work start point.

[0101] And, the work judgment module (220) can determine the second point in time as the work end point when the first area (330) is greater than or equal to the critical area and the second area is less than the critical area.

[0102] In one embodiment, the target area (320) may be a worker, safety equipment, or other object, and the IoU method may be used to detect the location of the object throughout the system (200).

[0103] For example, in response to a decision on whether to start work, an object analysis module (230) that determines the locations of workers and safety equipment among a plurality of objects can recognize workers and safety equipment from the detection results and obtain the locations of workers and safety equipment by calculating the area in the first image of the workers and safety equipment using the IoU method.

[0104] In one embodiment, a safety equipment fastening judgment module (240) that determines whether a worker has fastened safety equipment can determine whether a worker has fastened safety equipment based on whether work has started and the location of the worker and / or safety equipment.

[0105] The entire process of determining whether or not to fasten the worker's safety equipment is described below through Figures 5 to 8.

[0106] FIG. 5 is a flowchart of another example of a method for monitoring whether a worker is wearing safety equipment during work according to some embodiments of the present disclosure.

[0107] Referring to FIG. 5, in step S200, the system (200) may perform an analysis process to acquire an image and classify objects contained in the acquired image. In one embodiment, the analysis results obtained by performing the analysis process may correspond to workers and safety equipment contained in the image. The safety equipment may be further subdivided to include safety hooks, safety blocks, hard hats, safety belts, safety clothing, etc. Standards for safety equipment may be arbitrarily set by the user.

[0108] Then, in step S210, the work judgment module (220) can determine whether the analysis result obtained in step S100 includes a worker.

[0109] For example, if it is determined that there is no worker, in step S220, the system (200) can generate a first value corresponding to the information that there is no worker.

[0110] Alternatively, if it is determined that a worker is present, in step S230, the object analysis module (230) may use the detection results to determine whether the worker is wearing safety equipment. For example, if it is determined that safety equipment is worn, the system (200) may generate a second value corresponding to information indicating that the safety equipment is worn. Conversely, if it is determined that safety equipment is not worn, the system (200) may generate a third value corresponding to information indicating that the safety equipment is not worn.

[0111] In one embodiment, the first value, the second value, and the third value may be predetermined numbers, letters, symbols, and the like, for example, the first value, the second value, and the third value may be 0, 1, and 2, respectively.

[0112] In one embodiment, steps S210 to S250 may be a process for performing a primary judgment, which is a preliminary judgment performed before a final determination of whether or not safety equipment is installed. In one embodiment, the first value, the second value, and the third value may correspond to the results of the primary judgment.

[0113] In the next step S260, the system (200) can store the primary judgment results generated in steps S210 to S250 in the storage unit (250), and can accumulate and store them in the form of an array in the storage unit (250) each time the primary judgment results are generated under predetermined conditions.

[0114] In one embodiment, the predetermined condition may be a predetermined time unit, or frame unit.

[0115] And, in step S270, the system (200) determines whether the analysis process for the image exceeds a predetermined time unit or frame unit through the time or number of frames corresponding to the image, and if it determines that it does not exceed the predetermined time unit or frame unit, it returns to step S200 and repeats steps S200 to S270.

[0116] In one embodiment, if the system (200) determines that a predetermined time unit or frame unit has been exceeded, in step S280, the system (200) may, through the safety equipment fastening determination module (240), make a final determination result on whether the worker has fastened the safety equipment from the values ​​stored in the storage unit (250), and initialize the system by deleting the values ​​stored in the array in the storage unit (250). Then, the system (200) may re-acquire the array by performing the following steps again from step S200.

[0117] In one embodiment, the system (200) can determine whether the worker has fastened the safety equipment as a group of a plurality of images including the first image and the subsequent images by repeating steps S200 to S270 for the first image acquired at the first point in time and at least one subsequent image following the first image.

[0118] In one embodiment, the final judgment result corresponding to whether the worker is wearing safety equipment can be determined by using primary judgments corresponding to each of the images included in a set of multiple images to determine whether the worker is wearing safety equipment.

[0119] In one embodiment, the primary judgment is a process for pre-judging before determining whether safety equipment is fastened. Through the primary judgment, the computing device (100) can obtain information regarding the presence or absence of the worker in the image or whether work has begun, as well as information regarding whether the worker has fastened the safety equipment. For example, the primary judgment may include information indicating that the worker is not in the image or has not started work. Alternatively, the primary judgment may correspond to information indicating that the worker has fastened the safety equipment or information indicating that the worker has not fastened the safety equipment.

[0120] In one embodiment, the computing device (100) can obtain a final judgment and final result on whether the worker has fastened the safety equipment through a plurality of primary judgments.

[0121] In one embodiment, to determine whether a worker is wearing safety equipment, the system (200) may obtain primary judgments corresponding to each image included in a set of multiple images, and determine the primary judgment value corresponding to the highest number of appearances among the obtained primary judgments as a majority value. In addition, the system (200) may determine whether a worker is wearing safety equipment by using a value corresponding to the majority value in a predetermined time unit or a predetermined frame unit.

[0122] In one embodiment, the mainstream value may correspond to a final judgment obtained from a set containing a predefined number of primary judgments. For example, the mainstream value may correspond to the primary judgment with the highest number of occurrences among the predefined number of primary judgments.

[0123] For example, if the predefined number is 10, and within the set of primary judgments, there are 2 pieces of information indicating that the worker is not present, 3 pieces of information indicating that the safety equipment is not fastened, and 5 pieces of information indicating that the safety equipment is fastened, the mainstream value may be the information indicating that the safety equipment is fastened corresponding to 5 of the 10 primary judgments within the set.

[0124] In one embodiment, determining whether a predetermined time unit or frame unit has been exceeded may be determined by a separate timer (not shown) included in the system (200).

[0125] In one embodiment, the timer may determine how long to store an array containing a record of the initial judgment as to whether the safety equipment is engaged before producing a final judgment result.

[0126] For example, the timer can initialize the analysis time by using the current time value (T_NOW) as in the mathematical expression 1 below, storing the current time value as a previous time value variable (T_PRE), and setting the analysis time (Y), which corresponds to the time at which the first judgment on the image is performed, to 0.

[0127] [Mathematical Formula 1]

[0128] T_PRE=T_NOW,Y=0

[0129] In one embodiment, the timer may repeatedly extract images and / or frames, as in Equation 2, and update the analysis time using the analysis time value initialized immediately after extraction and the current time value. The timer may then store the current time value in a previous time value variable for calculating the next analysis time.

[0130] Mathematical expression 2 also uses the same variables as Mathematical expression 1, and may include a current time value (T_NOW) corresponding to the time at which the task is performed, a previous time value variable (T_PRE) that is updated after the current time value that occurs after the time at which the task is performed, and an analysis time (Y) corresponding to the time at which the first judgment is performed.

[0131] [Equation 2]

[0132] Y=Y+(T_NOW- T_PRE ),T_PRE=T_NOW

[0133] In one embodiment, the timer may correspond to program code using variables included in Equations 1 and 2.

[0134] By means of a timer, the computing device (100) can obtain the elapsed time since the analysis time was initialized.

[0135] In one embodiment, the computing device (100) can determine the time unit for performing the first judgment and the time unit for acquiring the image.

[0136] In one embodiment, an example of a process in which the system (200) monitors whether a worker is wearing safety equipment is described below through FIGS. 6 to 8.

[0137] FIG. 6 is an illustration of a method for monitoring whether a worker is wearing safety equipment during work according to some embodiments of the present disclosure.

[0138] FIG. 7 is another illustration of a method for monitoring whether a worker is wearing safety equipment while working, according to some embodiments of the present disclosure.

[0139] FIG. 8 is another illustration of a method for monitoring whether a worker is wearing safety equipment while working, according to some embodiments of the present disclosure.

[0140] Referring to FIG. 6, a first image (400) acquired at a first point in time is disclosed.

[0141] In the first image (400), the computing device (100) can obtain multiple bounding boxes corresponding to the work area (410), the worker (420), and the safety equipment (430-460).

[0142] In one embodiment, the safety equipment (430-460) may include a safety block (430), a safety hook (440), a safety belt (450), and a safety helmet (460).

[0143] In one embodiment, the computing device (100) can determine, through the work determination module (220), that the overlapping area between the worker (410) and the work area (400) is increasing, and accordingly determine that work is to begin.

[0144] In one embodiment, the computing device (100) records the point in time at which the task begins, and can obtain the position of the safety equipment (430-460) at the point in time at which the task begins as the first safety equipment position through the object analysis module (230). In addition, the computing device (100) can obtain the second safety equipment position corresponding to the position of the safety equipment (430-460) after the point in time at which the task begins.

[0145] In one embodiment, the object analysis module (230) and / or the safety equipment fastening judgment module (240) can perform a primary judgment on whether the safety equipment is fastened by comparing the first safety equipment location and the second safety equipment location.

[0146] In one embodiment, comparing the first safety equipment location and the second safety equipment location can be obtained by comparing the distance between the first safety equipment location and the second safety equipment location.

[0147] For example, if the coordinates of a pixel within the bounding box corresponding to the safety equipment (430-460) do not change based on a pixel, for example, a pixel determined to be the most central, the first safety equipment position and the second safety equipment position are determined to be the same, and since the distance between the first safety equipment position and the second safety equipment position does not change, the computing device (100) can determine that the safety equipment (430-460) is not engaged.

[0148] For example, if the position changes due to a change in the coordinates of a pixel based on one pixel within the bounding box corresponding to the safety equipment (430-460), the distance between the first safety equipment position and the second safety equipment position can be calculated, and if the calculated distance is greater than the threshold distance, it can be determined that the safety equipment (430-460) has been engaged.

[0149] In one embodiment, the distance can be obtained by calculating the coordinates of each of the first safety equipment location and the second safety equipment location using the Euclidean formula.

[0150] In another embodiment, if it is determined that the area of ​​the worker (420) overlapping the area of ​​the work area (410) is not greater than the critical area, the computing device (100) may generate a value of 0 and record and store it in an array format in the storage unit (250).

[0151] Referring to FIG. 7, after the work starts, the object analysis module (230) in the first image (500) can obtain the position of the worker (510) as the first worker position at the time when the work of the worker (510) starts, and can obtain the second worker position corresponding to the position of the worker (510) after the time when the work of the worker (510) starts.

[0152] In one embodiment, the computing device (100) can determine whether a worker is finished or working based on the first worker location and / or the second worker location.

[0153] For example, if the first worker position and the second worker position are the same for a preset period of time after the worker starts working (in this case, the areas corresponding to the first worker position and the second worker position may be the same), the computing device (100) may determine that the worker (510) has finished working. In the opposite case, the computing device (100) may determine that the worker (510) is working.

[0154] For example, if the safety equipment is a safety hook (520), if the distance between the first worker position and the first safety equipment position obtained at the first point in time and the distance between the second worker position and the second safety equipment position obtained at the second point in time are the same or if their respective overlapping areas are the same, the computing device (100) can determine that the worker (510) has finished working. In the opposite case, the computing device (100) can determine that the worker (510) is working.

[0155] In one embodiment, the first worker position and the first safety equipment position may be positions corresponding to a first image at a first point in time, and the second worker position and the second safety equipment position may be positions corresponding to a second image at a second point in time that is after the first point in time.

[0156] In one embodiment, the computing device (100) can determine a primary judgment as to whether the safety equipment is fastened by considering the first safety equipment location, the first worker location, the second safety equipment location, and the second worker location.

[0157] For example, the computing device (100) may obtain the distance between the first safety equipment location and the second safety equipment location as the first distance, and may obtain the distance between the first worker location and the second worker location as the second distance. Then, if the computing device (100) determines that the first distance and the second distance are equal within a predetermined error, the computing device (100) may determine that the worker (510) is fastening the safety equipment.

[0158] In another example, if the safety equipment to be determined whether or not it is fastened is safety equipment (530) attached to the body of a worker (510), the computing device (100) can determine the degree of overlap between the first worker position and the first safety equipment position, and if the degree of overlap between the second worker position and the second safety equipment position is within a predetermined error, it can be determined that the worker (510) is fastening the safety equipment.

[0159] In another example, the computing device (100) may obtain a distance between a first safety equipment location and a first worker location as a third distance, and a distance between a second safety equipment location and a second worker location as a fourth distance. Then, if the computing device (100) determines that the third distance and the fourth distance are equal within a predetermined error, the computing device (100) may determine that the worker (510) is fastening the safety equipment.

[0160] In one embodiment, when the safety equipment includes a safety block (520) and a safety hook (530), the computing device (100) can subdivide the first safety equipment position to obtain a first-first safety equipment position and a first-second safety equipment position corresponding to the positions of the safety block and the safety hook, respectively. Similarly, the computing device (100) can obtain a second-first safety equipment position and a second-second safety equipment position corresponding to the positions of the safety equipment, respectively, at a second point in time.

[0161] In one embodiment, the primary determination of whether the safety equipment is fastened may be additionally performed based on a fifth distance between the 1-1 safety equipment location and the 1-2 safety equipment location, a sixth distance between the 2-1 safety equipment location and the 2-2 safety equipment location, a seventh distance between the 1-1 safety equipment location and the 2-1 safety equipment location, and an eighth distance between the 1-2 safety equipment location and the 2-2 safety equipment location.

[0162] For example, the computing device may determine that the safety device is engaged if at least one of the fifth distance and the sixth distance exceeds a predetermined threshold distance or corresponds to a distance between the first worker position and the second worker position.

[0163] For another example, the computing device (100) may compare the seventh distance and the eighth distance and determine that the safety device is engaged if the seventh distance and the eighth distance are not equal.

[0164] In the present disclosure, the critical distance, critical area, critical range, and critical width are not fixed values, but may be values ​​set by the computing device (100) or the user of the computing device (100), and may be set differently depending on the situation. In addition, cases where the distance, area, and width are determined to be the same may include cases where they are the same within a margin of error.

[0165] In one embodiment, a method for determining whether a safety device is fastened when the safety device is not recognized in the image is described below with reference to FIG. 8.

[0166] In one embodiment, referring to FIG. 7, the first image (500) of the first time point includes a safety ring (530), but referring to FIG. 8, the second image (600) of the second time point includes a worker (610) and a safety block (620) but does not include a safety ring, and even then, the computing device (100) can determine whether safety equipment is fastened.

[0167] In the above case, the computing device (100) can obtain the first worker position, the 1-1 safety equipment position, and the 2-1 safety equipment position corresponding to the positions of the worker (510), the safety block (520), and the safety hook (530) included in the first point in time, respectively, and can obtain only the second worker position and the 1-2 safety equipment position corresponding to the positions of the worker (610) and the safety block (620) included in the second point in time.

[0168] And, the computing device (100) can perform the first judgment using only the first worker position, the 1-1 safety equipment position, the second worker position, and the 1-2 safety equipment position.

[0169] For example, the computing device (100) can determine whether the safety equipment is fastened by obtaining the distance between the first worker position and the 1-1 safety equipment position and comparing the distance between the second worker position and the 1-2 safety equipment position.

[0170] For example, if the distance between the first worker position and the 1-1 safety equipment position and the distance between the second worker position and the 1-2 safety equipment position are the same, it can be determined that the safety equipment is fastened.

[0171] In one embodiment, the primary judgment may mean a process in which the computing device (100) generates 0 and stores it in an array if the worker is not present, generates 1 and stores it in a storage unit (250) if the safety equipment is fastened, and generates 2 and stores it in a storage unit (250) if the safety equipment is not fastened.

[0172] In one embodiment, the primary judgment may be a process that is performed repeatedly for a predetermined period of time to generate an array containing a plurality of values ​​of 0, 1, and / or 2.

[0173] For example, an array may be a data structure configured in a storage unit (250) or memory (120). In one embodiment, the array may be in the form of a queue in which information to be stored is sequentially stored in a storage space, and may have a form such as [0, 1, 2, 2, ,2, 2, 1, 0].

[0174] In one embodiment, if the primary judgment exceeds a predetermined period of time, the computing device (100) may determine the value with the highest number of occurrences among 0, 1, and / or 2 contained in the array to make a final judgment as to whether the worker is wearing safety equipment.

[0175] For example, 0 may be information indicating that no worker is present in the work area or image, 1 may be information indicating that a worker is present and has fastened safety equipment after starting work and before ending work, and 2 may be information indicating that a worker is present and has not fastened safety equipment after starting work and before ending work.

[0176] In one embodiment, if the number of occurrences of at least two values ​​of 0, 1, and / or 2 is the same, the computing device (100) may perform the first judgment one more time to obtain a value and use it as the final judgment on whether or not to fasten the safety equipment.

[0177] In one embodiment, if the safety equipment fastening judgment module determines that the safety equipment is not fastened, the computing device (100) may generate an event notifying the worker to fasten the safety equipment from the time the safety equipment is determined to be not fastened. For example, the event may include a guidance screen and / or guidance voice informing the user that the safety equipment is not fastened and recommending that the safety equipment be fastened via a display module (not shown) and / or a speaker module (not shown).

[0178] In one embodiment, the computing device (100) can analyze footage from a point in time after the safety equipment is determined not to be fastened, and continuously generate events until the point in time when the worker is re-determined to have fastened the safety equipment.

[0179] FIG. 9 is a general schematic diagram of an artificial intelligence model in which embodiments of the contents of the present disclosure can be implemented.

[0180] In one embodiment, an artificial intelligence model included in a computing device (100) can detect objects from images using at least one of a bounding box technique, a polygon technique, a polyline technique, a point technique, a cuboid technique, and a segmentation technique, and perform an inference process on the objects to determine workers and safety equipment, etc.

[0181] In one embodiment, the AI ​​model performs the task of classifying what a recognized object corresponds to through a labeling technique, and can classify the classified object. For example, the labeling technique may include at least one of a bounding box labeling technique, a polygon labeling technique, a polyline labeling technique, a point labeling technique, a cuboid labeling technique, and a segmentation labeling technique.

[0182] Throughout this specification, the terms "artificial intelligence-based module or model," "artificial intelligence module or model," "computational model," "neural network," "network function," and "neural network" may be used interchangeably. A neural network may be comprised of a set of interconnected computational units, which may generally be referred to as nodes. These nodes may also be referred to as neurons. A neural network comprises at least one node. The nodes (or neurons) comprising the neural network may be interconnected by one or more links.

[0183] Before input data is input to the input layer of an artificial intelligence model, preprocessing techniques according to one embodiment of the present disclosure may be performed. These preprocessing techniques may include steps for reducing the size of the input data to a preset size or for deleting unnecessary data from the input data.

[0184] Within a neural network, one or more nodes connected via links can form a relationship between input nodes and output nodes. The concept of input nodes and output nodes is relative, meaning that any node that is in an output node relationship with one node can also be in an input node relationship with another node, and vice versa. As described above, the relationship between input nodes and output nodes can be created based on links. One input node can be connected to one or more output nodes via links, and vice versa.

[0185] In a relationship between input nodes and output nodes connected through a single link, the data of the output node can have its value determined based on the data input to the input node. Here, the link interconnecting the input nodes and output nodes can have a weight. The weight can be variable and can be varied by the user or an algorithm so that the neural network can perform a desired function. For example, when one or more input nodes are interconnected to one output node through each link, the output node can determine the output node value based on the values ​​input to the input nodes connected to the output node and the weight set on the link corresponding to each input node.

[0186] As described above, a neural network is a network in which one or more nodes are interconnected through one or more links, forming input and output node relationships within the network. The characteristics of a neural network can be determined based on the number of nodes and links within the network, the relationships between the nodes and links, and the weights assigned to each link. For example, if two neural networks have the same number of nodes and links but different weight values ​​for the links, the two neural networks can be perceived as different from each other.

[0187] A neural network can be composed of a set of one or more nodes. Some of the nodes constituting the neural network can form a layer based on their distances from the initial input node. For example, a set of nodes that are n distances from the initial input node can form n layers. The distance from the initial input node can be defined by the minimum number of links required to reach the node from the initial input node. However, this definition of a layer is arbitrary for illustrative purposes, and the order of layers within a neural network can be defined in a different way than described above. For example, a layer of nodes can also be defined by its distance from the final output node.

[0188] An initial input node may refer to one or more nodes within a neural network into which data is directly input without going through links with other nodes. Alternatively, within a neural network, it may refer to nodes that do not have other input nodes connected by links in the relationship between nodes based on links. Similarly, a final output node may refer to one or more nodes within a neural network that do not have output nodes in their relationship with other nodes. Furthermore, a hidden node may refer to nodes that constitute a neural network other than the initial input node and the final output node.

[0189] A neural network according to one embodiment of the present disclosure may be a neural network in which the number of nodes in an input layer may be the same as the number of nodes in an output layer, and the number of nodes decreases and then increases as it progresses from the input layer to the hidden layer. In addition, a neural network according to another embodiment of the present disclosure may be a neural network in which the number of nodes in an input layer may be less than the number of nodes in an output layer, and the number of nodes decreases as it progresses from the input layer to the hidden layer. In addition, a neural network according to another embodiment of the present disclosure may be a neural network in which the number of nodes in an input layer may be greater than the number of nodes in an output layer, and the number of nodes increases as it progresses from the input layer to the hidden layer. A neural network according to another embodiment of the present disclosure may be a neural network in a combined form of the neural networks described above.

[0190] A deep neural network (DNN) can refer to a neural network that includes multiple hidden layers in addition to input and output layers. Using a deep neural network, one can identify latent structures in data. That is, one can identify the latent structure of an image or photograph (e.g., what objects are in the photograph). A deep neural network can include a convolutional neural network (CNN), a recurrent neural network (RNN), an autoencoder, a generative adversarial network (GAN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a Q network, a U network, a Siamese network, a generative adversarial network (GAN), and the like. The description of the above-described deep neural network is merely an example, and the present disclosure is not limited thereto.

[0191] In one embodiment of the present disclosure, the network function may include an autoencoder. An autoencoder may be a type of artificial neural network that outputs output data similar to input data. The autoencoder may include at least one hidden layer, and an odd number of hidden layers may be arranged between input and output layers. The number of nodes in each layer may be reduced from the number of nodes in the input layer to an intermediate layer called a bottleneck layer (encoding), and then expanded symmetrically from the bottleneck layer to the output layer (symmetrical to the input layer). The autoencoder may perform nonlinear dimensionality reduction. The number of input layers and output layers may correspond to the dimensionality after preprocessing of the input data. In the autoencoder structure, the number of nodes in the hidden layer included in the encoder may have a structure in which the number of nodes decreases as it moves away from the input layer. The number of nodes in the bottleneck layer (the layer with the fewest nodes between the encoder and decoder) may be kept above a certain number (e.g., more than half of the input layer), as too few nodes may not transmit enough information.

[0192] In one embodiment, the neural network included in the artificial intelligence model may be trained using at least one of supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. Training a neural network may be a process of applying knowledge to the neural network to perform a specific operation. The neural network may be trained in a direction that minimizes output errors. Training a neural network involves repeatedly inputting training data into the neural network, calculating the neural network output and target error for the training data, and backpropagating the neural network error from the output layer to the input layer to update the weights of each node of the neural network in a direction that reduces the error. In the case of supervised learning, training data with the correct answer labeled for each training data is used (i.e., labeled training data), whereas in the case of unsupervised learning, the correct answer may not be labeled for each training data. That is, for example, in the case of supervised learning for data classification, the training data may be data in which each training data is labeled with a category. The labeled training data is input to a neural network, and an error can be calculated by comparing the output (category) of the neural network with the labels of the training data. As another example, in the case of unsupervised learning for data classification, an error can be calculated by comparing the input training data with the output of the neural network. The calculated error is backpropagated in the neural network in the backward direction (i.e., from the output layer to the input layer), and the connection weights of each node in each layer of the neural network can be updated according to the backpropagation. The amount of change in the connection weights of each node that are updated can be determined according to the learning rate.Neural net for input data.

[0193] The computation of work and backpropagation of errors can constitute a learning cycle (epoch). The learning rate can be applied differently depending on the number of iterations in the neural network's learning cycle. For example, a high learning rate can be used early in the learning process to quickly achieve a certain level of performance, thereby improving efficiency. A lower learning rate can be used later in the learning process to improve accuracy.

[0194] In neural network training, the training data can generally be a subset of the actual data (i.e., the data to be processed using the trained neural network), and therefore, there may be a learning cycle in which the error for the training data decreases but the error for the actual data increases. Overfitting is a phenomenon in which the error for the actual data increases due to excessive training on the training data. For example, a neural network trained on cats by showing it yellow cats will learn to recognize cats in colors other than yellow.

[0195] The phenomenon of failing to recognize a cat when seeing one could be a form of overfitting. Overfitting can increase the error rate of machine learning algorithms. Various optimization methods can be used to prevent overfitting. These include increasing the training data, regularization, dropout (inactivating some nodes in the network during the learning process), and the use of batch normalization layers.

[0196] By way of example and not limitation, the weights may include weights that vary during the neural network training process and / or weights that have completed neural network training. The weights that vary during the neural network training process may include weights at the start of the training cycle and / or weights that vary during the training cycle. The weights that have completed neural network training may include weights that have completed the training cycle. Accordingly, a data structure including the weights of a neural network may include a data structure including weights that vary during the neural network training process and / or weights that have completed neural network training. Therefore, the above-described weights and / or combinations of each weight are included in the data structure including the weights of a neural network. The above-described data structures are merely examples and the present disclosure is not limited thereto.

[0197] In one embodiment, a data structure including neural network weights may be stored in a computer-readable storage medium (e.g., memory, hard disk) after going through a serialization process. Serialization may be a process of converting a data structure into a form that can be stored on the same or a different computing device and later reconstructed and used. The computing device may serialize the data structure to transmit and receive data over a network. The serialized data structure including neural network weights may be reconstructed on the same or a different computing device through deserialization. The data structure including neural network weights is not limited to serialization. Furthermore, the data structure including neural network weights may include a data structure that increases computational efficiency while minimizing the use of computing device resources (e.g., a B-Tree, a Trie, an m-way search tree, an AVL tree, a Red-Black Tree in nonlinear data structures). The foregoing is merely an example, and the present disclosure is not limited thereto.

[0198] The data structure may include hyperparameters of a neural network. Furthermore, the data structure including the hyperparameters of the neural network may be stored on a computer-readable medium. The hyperparameters may be variables that can be varied by the user. The hyperparameters may include, for example, a learning rate, a cost function, the number of learning cycle repetitions, weight initialization (e.g., setting a range of weight values ​​to be subject to weight initialization), and the number of hidden units (e.g., the number of hidden layers, the number of nodes in the hidden layer). The above-described data structure is merely an example, and the present disclosure is not limited thereto.

[0199] FIG. 10 is a general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented.

[0200] Referring to FIG. 10, an exemplary environment for implementing various aspects of the present disclosure is illustrated, including a computer (1102), which includes a processing unit (1104), a system memory (1106), and a system bus (1108). The system bus (1108) connects system components, including but not limited to the system memory (1106), to the processing unit (1104). The processing unit (1104) may be any of a variety of commercially available processors. Dual processors and other multiprocessor architectures may also be utilized as the processing unit (1104).

[0201] The system bus (1108) may be any of several types of bus structures that may be additionally interconnected to a memory bus, a peripheral bus, and a local bus using any of a variety of commercial bus architectures. The system memory (1106) includes read-only memory (ROM) (1110) and random access memory (RAM) (1112). A basic input / output system (BIOS) is stored in non-volatile memory (1110), such as ROM, EPROM, or EEPROM, and includes basic routines that help transfer information between components within the computer (1102), such as during start-up. The RAM (1112) may also include high-speed RAM, such as static RAM, for caching data.

[0202] The computer (1102) also includes an internal hard disk drive (HDD) (1114) (e.g., EIDE, SATA) - which may also be configured for external use within a suitable chassis (not shown), a magnetic floppy disk drive (FDD) (1116) (e.g., for reading from or writing to a removable diskette (1118)), and an optical disk drive (1120) (e.g., for reading from or writing to a CD-ROM disk (1122) or other high-capacity optical media such as a DVD).

[0203] The hard disk drive (1114), the magnetic disk drive (1116), and the optical disk drive (1120) may be connected to the system bus (1108) by a hard disk drive interface (1124), a magnetic disk drive interface (1126), and an optical drive interface (1128), respectively. The interface (1124) for implementing an external drive includes at least one or both of Universal Serial Bus (USB) and IEEE 1394 interface technologies.

[0204] These drives and their associated computer-readable media provide nonvolatile storage of data, data structures, computer-executable instructions, and the like. In the case of a computer (1102), the drives and media correspond to storing any data in a suitable digital format. While the description of computer-readable media above refers to HDDs, removable magnetic disks, and removable optical media such as CDs or DVDs, those skilled in the art will recognize that the drives and their associated computer-readable media also include zip drives, magnetic cassettes, flash memory cards, cartridges, and the like.

[0205] It will be appreciated that other types of computer-readable media, such as a computer readable medium, may also be used in the exemplary operating environment, and that any such media may contain computer-executable instructions for performing the methods of the present disclosure.

[0206] A number of program modules, including an operating system (1130), one or more application programs (1132), other program modules (1134), and program data (1136), may be stored in the drive and RAM (1112). All or portions of the operating system, applications, modules, and / or data may also be cached in RAM (1112). It will be appreciated that the present disclosure may be implemented in various commercially available operating systems or combinations of operating systems.

[0207] A user may enter commands and information into the computer (1102) via one or more wired / wireless input devices, such as a keyboard (1138) and a pointing device such as a mouse (1140). Other input devices (not shown) may include a microphone, an IR remote control, a joystick, a game pad, a stylus pen, a touch screen, and the like. These and other input devices are often connected to the processing unit (1104) via an input device interface (1142) that is connected to the system bus (1108), but may be connected by other interfaces such as a parallel port, an IEEE 1394 serial port, a game port, a USB port, an IR interface, and the like.

[0208] A monitor (1144) or other type of display device is also connected to the system bus (1108) via an interface, such as a video adapter (1146). In addition to the monitor (1144), the computer typically includes other peripheral output devices (not shown), such as speakers, a printer, and so on.

[0209] The computer (1102) may operate in a networked environment using logical connections to one or more remote computers, such as remote computer(s) (1148), via wired and / or wireless communications. The remote computer(s) (1148) may be a workstation, a computing device computer, a router, a personal computer, a portable computer, a microprocessor-based entertainment device, a peer device, or other conventional network node, and generally include many or all of the components described for the computer (1102), although for simplicity, only the memory storage device (1150) is shown. The logical connections shown include wired / wireless connections to a local area network (LAN) (1152) and / or a larger network, such as a wide area network (WAN) (1154). Such LAN and WAN networking environments are common in offices and companies and facilitate enterprise-wide computer networks, such as intranets, all of which may be connected to a worldwide computer network, such as the Internet.

[0210] When used in a LAN networking environment, the computer (1102) is connected to a local network (1152) via a wired and / or wireless communication network interface or adapter (1156). The adapter (1156) may facilitate wired or wireless communications to the LAN (1152), which may also include a wireless access point installed therein for communicating with the wireless adapter (1156). When used in a WAN networking environment, the computer (1102) may include a modem (1158), be connected to a communications computing device on the WAN (1154), or have other means for establishing communications over the WAN (1154), such as via the Internet. The modem (1158), which may be internal or external and wired or wireless, is connected to the system bus (1108) via an input device interface (1142). In a networked environment, program modules or portions thereof described for the computer (1102) may be stored in a remote memory / storage device (1150). It will be appreciated that the network connections depicted are exemplary and other means of establishing a communications link between the computers may be used.

[0211] The computer (1102) operates to communicate with any wireless device or object that is arranged and operates in wireless communication, such as a printer, a scanner, a desktop and / or portable computer, a portable data assistant (PDA), a communication satellite, any equipment or location associated with a radio detectable tag, and a telephone. This includes at least Wi-Fi and Bluetooth wireless technologies. Accordingly, the communication may be performed in advance as in a conventional network.

[0212] It can be a defined structure or simply ad hoc communication between at least three devices.

[0213] Wi-Fi (Wireless Fidelity) enables connections to the Internet and other devices without wires. Wi-Fi is a wireless technology that allows devices, such as computers, to send and receive data anywhere within the coverage area of ​​a base station, both indoors and outdoors, similar to cell phones. Wi-Fi networks use wireless technologies called IEEE 802.11 (a, b, g, etc.) to provide secure, reliable, and high-speed wireless connections. Wi-Fi can be used to connect computers to each other, to the Internet, and to wired networks (using IEEE 802.3 or Ethernet). Wi-Fi networks can operate in the unlicensed 2.4 and 5 GHz radio bands, at data rates of, for example, 11 Mbps (802.11a) or 54 Mbps (802.11b), or in products that include both bands (dual-band).

[0214] Those skilled in the art will appreciate that information and signals may be represented using any of a variety of different technologies and techniques. For example, the data, instructions, commands, information, signals, bits, symbols, and chips referenced in the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0215] Those skilled in the art will appreciate that the various illustrative logical blocks, modules, processors, means, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, various forms of programs or design code (referred to herein, for convenience, as software), or a combination of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

[0216] The various embodiments presented herein can be implemented as a method, apparatus, or article of manufacture using standard programming and / or engineering techniques. The term article of manufacture includes a computer program, carrier, or media accessible from any computer-readable storage device. For example, computer-readable storage media include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips, etc.), optical disks (e.g., CDs, DVDs, etc.), smart cards, and flash memory devices (e.g., EEPROMs, cards, sticks, key drives, etc.). Furthermore, the various storage media presented herein include one or more devices and / or other machine-readable media for storing information.

[0217] It should be understood that the specific order or hierarchy of steps in the presented processes is merely an example of exemplary approaches. It should be understood that the specific order or hierarchy of steps in the processes may be rearranged within the scope of the present disclosure based on design priorities. The appended method claims provide elements of various steps in a sample order, but are not intended to be limited to the specific order or hierarchy presented.

[0218] The description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the embodiments set forth herein, but is to be construed in the broadest scope consistent with the principles and novel features disclosed herein. For example, components described as being single may be implemented in a distributed manner, and similarly, components described as being distributed may be implemented in a combined manner.

[0219] The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention.

[0220] Although the present disclosure has been described above as being generally implemented by a computing device, those skilled in the art will appreciate that the present disclosure may be implemented in combination with computer-executable instructions and / or other program modules that may be executed on one or more computers and / or as a combination of hardware and software.

[0221] Generally, program modules include routines, programs, components, data structures, and the like that perform particular tasks or implement particular abstract data types. Furthermore, those skilled in the art will appreciate that the methods of the present disclosure can be implemented with other computer system configurations, including single-processor or multiprocessor computer systems, minicomputers, mainframe computers, as well as personal computers, handheld computing devices, microprocessor-based or programmable consumer electronics, and the like, each of which may be operatively connected to one or more associated devices.

[0222] The described embodiments of the present disclosure can also be practiced in distributed computing environments, where certain tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

[0223] Computers typically include a variety of computer-readable media. Any computer-accessible media can be computer-readable media, including volatile and nonvolatile media, transitory and non-transitory media, removable and non-removable media. By way of example, and not limitation, computer-readable media can include computer-readable storage media and computer-readable transmission media. Computer-readable storage media includes volatile and nonvolatile media, transitory and non-transitory media, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital video disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be accessed by a computer and used to store the desired information.

[0224] Computer-readable transmission media typically includes any information delivery media that embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism. The term modulated data signal means a signal that has one or more of its characteristics set or changed so as to encode information in the signal. By way of example, and not limitation, computer-readable transmission media includes wired media, such as a wired network or direct-wired connection, and wireless media, such as acoustic, RF, infrared, or other wireless media. Combinations of any of the above are also intended to be included within the scope of computer-readable transmission media.

[0225] The present invention described above can be implemented as computer-readable code on a medium in which a program is recorded. Computer-readable media include all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include hard disk drives (HDDs), solid-state disks (SSDs), silicon disk drives (SDDs), ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, etc., and also include media implemented in the form of carrier waves (e.g., transmission via the Internet). In addition, the computer may include a processor of a terminal. Therefore, the above detailed description should not be construed as limiting in all respects, but should be considered as illustrative. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are intended to be included in the scope of the present invention.

Claims

1. A method for monitoring whether a worker is wearing safety equipment during work in a captured image performed by a computing device. A step of receiving a first image captured at a first point in time; A step of obtaining detection results by detecting multiple objects from the first image using an artificial intelligence model; A step of determining whether the worker has started working based on the detection result, by a work judgment module that determines whether the worker is working; In response to the decision on whether to start the above work, a step of obtaining the positions of the worker and the safety equipment based on the detection result by an object analysis module that determines the positions of the worker and the safety equipment among the plurality of objects; and A step of determining whether the worker has fastened the safety equipment based on whether the work has started and the location, by a safety equipment fastening judgment module that determines whether the worker has fastened the safety equipment; Including, method.

2. In paragraph 1, The above work judgment module, Calculating a first area corresponding to the area of ​​the area that overlaps the area occupied by the worker in the first image with the reference area of ​​the predetermined reference area of ​​the first image, and Comparing the first width with a threshold width determined in advance to correspond to a criterion for determining whether the worker starts or ends work, to determine whether the worker starts or ends work. method.

3. In paragraph 2, Determining whether the above worker starts or ends work: In the case where the second area of ​​overlap between the area occupied by the worker in the second image and the reference area is different from the first area for the second image acquired at the second point in time after the first point in time, If the first area is less than the critical area and the second area is greater than the critical area, the second point in time is determined as the work start point, and If the first area is greater than or equal to the critical area and the second area is less than or equal to the critical area, the second point in time is determined as the end point of the work. method.

4. In paragraph 2, The area occupied by the above worker in the above first image is, In the detection result of detecting multiple objects in the first image above, the area defined by the bounding box of the object corresponding to the worker or the segmentation result of the object corresponding to the worker, method.

5. In paragraph 1, The above object analysis module, Using the above detection results, the position of the safety equipment at the time when the worker's work starts is obtained as the first safety equipment position, and the second safety equipment position corresponding to the position of the safety equipment after the time when the work starts is obtained, and The above safety equipment fastening judgment module is, By comparing the first safety equipment position and the second safety equipment position, a primary judgment is made as to whether the safety equipment is fastened. method.

6. In paragraph 5, The above object analysis module, Obtaining the position of the worker as a first worker position at the time when the work of the worker starts, and obtaining a second worker position corresponding to the position of the worker after the time when the work of the worker starts, and The first judgment regarding whether or not the above safety equipment is installed is: performed based on the first safety equipment position, the first worker position, the second safety equipment position and the second worker position. method.

7. In paragraph 6, The first judgment regarding whether or not the above safety equipment is installed is: It is performed based on the result of calculating a first distance corresponding to the interval between the first safety equipment position and the second safety equipment position, a second distance corresponding to the interval between the first worker position and the second worker position, a third distance corresponding to the interval between the first safety equipment position and the first worker position, and a fourth distance corresponding to the interval between the second safety equipment position and the second worker position. method.

8. In paragraph 7, The above first worker position and the above first safety equipment position are positions corresponding to the first image at the first point in time, and The above second worker position and the above second safety equipment position are positions corresponding to the second image at the second point in time after the first point in time. method.

9. In paragraph 5, The above safety equipment includes safety blocks and safety hooks. The above first safety equipment position includes a first-first safety equipment position and a first-second safety equipment position corresponding to the positions of the safety block and the safety hook, respectively, The above second safety equipment location includes the 2-1 safety equipment location and the 2-2 safety equipment location. method.

10. In paragraph 9, The primary judgment regarding whether the above safety equipment is installed or not is: The distance between the above 1-1 safety equipment position and the above 1-2 safety equipment position, and additionally performed based on the distance between the above 2-1 safety equipment location and the above 2-2 safety equipment location, method.

11. In paragraph 5, The step of determining whether the above worker has fastened the above safety equipment is as follows: A step of determining whether the worker has fastened the safety equipment, as a group unit of a plurality of images including the first image and at least one subsequent image following the first image; Including, method.

12. In paragraph 11, The step of determining whether the above worker has fastened the above safety equipment is as follows: A step of determining whether the worker has fastened the safety equipment by using the primary judgments corresponding to each of the images included in the set of the plurality of images; Including, method.

13. In paragraph 12, The step of determining whether the above worker has fastened the above safety equipment is as follows: A step of determining a primary judgment value corresponding to the highest number of appearances among the primary judgments corresponding to each of the images included in the set of the plurality of images as a majority value; and A step of using the above mainstream value to determine whether the worker has fastened the safety equipment for the set of the plurality of images; Including, method.

14. In paragraph 13, The above primary judgment values ​​are, A first value determined based on whether the above worker has started working; A second value determined when it is determined that the above worker has fastened the above safety equipment; and A third value determined when it is determined that the above worker has not fastened the above safety equipment; At least one of them, method.

15. In paragraph 1, If it is determined from the above safety equipment fastening judgment module that the above safety equipment is not fastened, A step for generating an event to notify the worker to fasten the safety equipment from the time it is determined that the safety equipment is not fastened; and A step of continuously generating the event by analyzing the video after the point in time when it is determined that the safety equipment is not fastened, until the point in time when it is re-determined that the worker has fastened the safety equipment; Including more, method.

16. A computer program stored in a computer-readable storage medium, wherein the computer program includes commands for monitoring, by at least one processor of a computing device, whether a worker is wearing safety equipment during work in a captured image. The above commands are, A command to receive a first image captured at a first point in time; A command for obtaining a detection result of detecting a plurality of objects from the first image using an artificial intelligence model; A command to determine whether the worker starts working based on the detection result, by a work judgment module that determines whether the worker is working; In response to the decision on whether to start the above work, a command for obtaining the positions of the worker and the safety equipment based on the detection result by an object analysis module that determines the positions of the worker and the safety equipment among the plurality of objects; and A command to determine whether the worker has fastened the safety equipment based on whether the work has started and the location, by a safety equipment fastening judgment module that determines whether the worker has fastened the safety equipment; Including, program.

17. A computing device for monitoring whether a worker is wearing safety equipment during work within a filmed video. processor; and memory; Including, The above processor, Receive the first image captured at the first point in time, Using an artificial intelligence model, a detection result is obtained by detecting multiple objects from the first image, By the work judgment module that determines whether the above worker is working or not, Based on the above detection results, it is determined whether the worker should start working, In response to the decision on whether to start the above work, the positions of the worker and the safety equipment are obtained based on the detection result by an object analysis module that determines the positions of the worker and the safety equipment among the plurality of objects, and, By a safety equipment fastening judgment module that determines whether the worker has fastened the safety equipment, based on whether the work has started and the location, it is determined whether the worker has fastened the safety equipment. Computing device.

Images