Automated Barcode-Based Personnel Safety Compliance System
An automated PPE monitoring system using barcodes and image sensors addresses the challenge of ensuring PPE compliance, reducing workplace injuries by accurately tracking PPE use and adapting to various environments.
Patent Information
- Application Number
- JP2022574678
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2020-06-05
- Filing Date
- 2021-06-04
- Publication Date
- 2025-10-22
- Estimated Expiration
- 2041-06-04
AI Technical Summary
Existing systems fail to accurately and continuously monitor the use of personal protective equipment (PPE) in hazardous environments, leading to a significant number of workplace injuries due to improper use of safety gear.
An automated system using barcodes and image sensors to detect and track PPE items, coupled with a probabilistic algorithm to ensure compliance with safety regulations, which includes first and second image sensors capturing overlapping fields of view, a central controller, and a database to manage user and barcode information, enabling real-time monitoring and safety actions.
The system effectively monitors PPE use, reducing workplace injuries by ensuring compliance with safety regulations, minimizing false alarms, and adapting to dynamic workspaces without wearable sensors.
Smart Images

Figure 0007758357000001 
Figure 0007758357000002 
Figure 0007758357000003
Abstract
Description
[Technical Field]
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Patent Application No. 63 / 035,298, filed June 5, 2020, the entire contents of which are incorporated herein by reference.
[0002] The present disclosure relates to systems and processes for providing safety to personnel operating in hazardous environments, and more particularly to systems and processes applicable to environments requiring the use of protective clothing and articles. [Background technology]
[0003] The following paragraphs are provided as background to the present disclosure, but are not an admission that anything discussed herein is prior art or part of the knowledge of those skilled in the art.
[0004] In the United States, approximately 900,000 work-related eye injuries and 750,000 work-related hand injuries requiring medical treatment are reported annually. It is estimated that in 80% of reported cases, injuries are sustained by individuals who are not wearing appropriate protective clothing or items, such as safety helmets, gloves, face shields, and boots (see, for example, http: / / www.preventblindness.org / ten-ways-prevent-eye-injuries-work and http: / / www.ishn.com / articles / 94029-drive-home-the-value-of-gloves--hand-injuries-send-a-million-workers-to-ers-each-year). Beyond the obvious personal impact resulting from such injuries, there are significant medical costs associated with treating them. Therefore, it is highly desirable to reduce the number of reported workplace injuries by improving the preventative safety measures taken by workplace personnel when working in hazardous work environments.
[0005] Therefore, there is a need in the art for systems and processes for reducing injuries to people working in hazardous environments. An automated system that can accurately and continuously monitor the use of personal protective equipment and can be easily implemented in a wide variety of workspaces is particularly desirable. Summary of the Invention
[0006] The following paragraphs are intended to introduce the reader to the more detailed description that follows, but are not intended to define or limit the claimed subject matter of the present disclosure.
[0007] In one broad aspect, the present disclosure relates to systems and processes for monitoring the use of personal protective equipment. Accordingly, the present disclosure, in at least one aspect, in at least one implementation, provides a system for automatically monitoring the use of personal protective equipment by workers in a workspace, the system comprising: a plurality of barcodes, each barcode associated with a unique PPE item and including object information identifying the associated PPE item and user information for a given worker using the PPE item; first and second image sensors installable within the workspace, the first and second image sensors being spaced apart from one another and positioned to cover different first and second fields of view, respectively, the first and second fields of view overlapping within a given area within the workspace, and the first and second image sensors adapted to collectively capture a first plurality of images of the given area; a central controller coupled to the first and second image sensors, the central controller including at least one processor and a memory element including a database configured to store a plurality of barcodes and user information such that one or more barcodes are linked with user information for a given worker; the at least one processor and the first and second image sensors together; (i) detecting a human-like object in a first plurality of images; (ii) identifying a second plurality of images from the first plurality of images, the second plurality of images including the detected human-like object; (iii) detecting a first barcode object in at least one of the second plurality of images, the first barcode object being associated with the detected human-like object; (iv) identifying a first barcode corresponding to the detected first barcode object; (v) performing a search in a database using the first barcode to identify user information linked thereto and additional barcodes linked to the identified user information; (vi) detecting additional barcode objects in the second plurality of images corresponding to the identified additional barcodes; and (vii) applying a probabilistic algorithm to calculate a probability that a worker corresponding to the detected human-like object is wearing each personal protective equipment item associated with the first detected barcode and the additional identified barcode in accordance with safety regulations applicable to the workspace, wherein the probabilistic algorithm is applied to determine a prevalence of each barcode object corresponding to the first detected barcode and the additional barcode in the second plurality of images; (viii) performing a safety action for each of the first detected barcode and additional barcode objects having a calculated probability lower than a predetermined probability threshold; The present invention provides a system configured to:
[0008] In at least one implementation, any of the barcode objects can be detected in defined image regions within the second plurality of images, each defined image region being constructed to include a detected human-shaped object.
[0009] In at least one implementation, the first plurality of images may include a first set of temporally consecutive images captured by a first image sensor during a given time period and a second set of temporally consecutive images captured by a second image sensor during a given time period.
[0010] In at least one implementation, the given period of time can be from about 5 to about 60 seconds.
[0011] In at least one implementation, the first and second sets of images each include at least 10 images.
[0012] In at least one implementation, the first and second image sensors are spaced and angled such that an intersection between a first axis and a second axis extending centrally through the fields of view of the first and second image sensors, respectively, forms an angle in the range of about 15 degrees to about 175 degrees, or about 205 degrees to about 345 degrees.
[0013] In at least one implementation, the angle can range from about 30 degrees to about 150 degrees, or from about 210 degrees to about 330 degrees.
[0014] In at least one implementation, a human-shaped object may be detected by applying a human-shaped object image analysis algorithm to the first plurality of images.
[0015] In at least one implementation, the barcode object may be detected by applying a barcode object image analysis algorithm to the second plurality of images.
[0016] In at least one implementation, the defined image region can be constructed using a frame having an image boundary that encompasses the entirety of the detected human-shaped object within the frame, and the image boundary is formed such that there is no contact between the detected human-shaped object and the image boundary.
[0017] In at least one implementation, the image boundary may correspond to a distance of about 0.5 meters to about 3 meters away from a human-shaped object detected within the image region.
[0018] In at least one implementation, a central controller is coupled to an input device, the input device configured to receive user entry in the form of a query barcode, the central controller and the first and second image sensors are further configured to detect a plurality of barcode objects, read barcodes corresponding to the detected barcode objects in each of the first and second plurality of images, and store the read barcodes in a second database configured to store the plurality of read barcodes and images together with the time the image was captured by the first and / or second image sensors by using a link relationship such that one or more read barcodes are linked to one of the images and the time the image was captured by the first and / or second image sensors, and the central controller is further configured to determine when there is a match between the query barcode and one of the read stored barcodes.
[0019] In at least one implementation, the central controller can be coupled to an output device, and the central controller is further configured to provide an output to the output device when the query barcode is identical to the detected read barcode.
[0020] In at least one implementation, the output may include a time indicator corresponding to the time an image linked to a stored barcode matching the query barcode was captured by the first and / or second image sensor.
[0021] In at least one implementation, the output may include images linked to stored barcodes that match the query barcode.
[0022] In at least one implementation, the output may include a time indicator indicating the time an image linked to a stored barcode that is identical to the query barcode was captured by the first and / or second image sensors, and / or the output may include an image linked to a stored barcode that is identical to the query barcode, the time indicator or linked image corresponding to the most recent time a match between the query barcode and a read stored barcode was identified relative to the time the query barcode was entered.
[0023] In at least one implementation, the input device may be further configured to receive user input of a query time period defined by a first time and a second time, and the central controller is configured to detect a plurality of barcode objects and their corresponding barcodes in a first and second plurality of images captured by the first and / or second image sensors during the query time period.
[0024] In at least one implementation, the system may further include a plurality of barcodes associated with objects other than personal protective items, the barcodes including object information identifying the objects.
[0025] In a further aspect, the present disclosure, in at least one implementation, provides an automated process for monitoring use of personal protective equipment by workers in a workspace, the process comprising: acquiring a first plurality of images using first and second image sensors locatable within the workspace, the first and second image sensors being spaced apart from one another and positioned to cover different first and second fields of view, respectively, wherein the first and second fields of view overlap within a given area within the workspace; Detecting a humanoid object of a given worker in a first plurality of images; identifying a second plurality of images from the first plurality of images, the second plurality of images including the detected human-like object; Detecting a first barcode object in at least one of the second plurality of images, the first barcode object being associated with the detected human-shaped object; identifying a first barcode corresponding to the detected first barcode object; performing a search in a database using the first barcode to identify user information linked thereto and additional barcodes linked to the identified user information, the database storing a plurality of barcodes and user information using linking relationships such that one or more barcodes are linked to user information of a worker; detecting additional barcode objects in the second plurality of images corresponding to the identified additional barcodes; applying a probabilistic algorithm to calculate a probability that a given worker corresponding to the detected human-like object is wearing each PPE item associated with the first detected barcode and the additional identified barcode in accordance with safety regulations applicable to the workspace, wherein the probabilistic algorithm is applied to determine a prevalence of each barcode object corresponding to the first detected barcode and the additional barcode in the second plurality of images; performing a safety action for each of the first detected barcode and additional barcode objects having a calculated probability lower than a predetermined probability threshold; Includes.
[0026] In at least one implementation, any of the barcode objects can be detected in defined image regions within the second plurality of images, each defined image region being constructed to include a detected human-shaped object.
[0027] In at least one implementation, the first plurality of images may include a first set of temporally consecutive images captured by a first image sensor during a given time period and a second set of temporally consecutive images captured by a second image sensor during a given time period.
[0028] In at least one implementation, the given period of time can be from about 5 to about 60 seconds.
[0029] In at least one implementation, the first and second sets of images can each include at least 10 images.
[0030] In at least one implementation, the first and second image sensors are spaced and angled such that an intersection between a first axis and a second axis extending centrally through the fields of view of the first and second image sensors, respectively, forms an angle in the range of about 15 degrees to about 175 degrees, or about 205 degrees to about 345 degrees.
[0031] In at least one implementation, the angle can range from about 30 degrees to about 150 degrees, or from about 210 degrees to about 330 degrees.
[0032] In at least one implementation, a human-shaped object may be detected by applying a human-shaped object image analysis algorithm to the first plurality of images.
[0033] In at least one implementation, the barcode object may be detected by applying a barcode object image analysis algorithm to the second plurality of images.
[0034] In at least one implementation, the defined image region may be constructed using a frame having an image boundary that encompasses the entirety of the detected human-shaped object within the frame, and the image boundary is formed such that there is no contact between the detected human-shaped object and the image boundary.
[0035] In at least one implementation, the image boundary may correspond to a distance of about 0.5 meters to about 3 meters away from a human-shaped object detected within the image region.
[0036] In at least one implementation, a central controller may be coupled to an input device, the input device configured to receive a user entry including a query barcode, the central controller and the first and second image sensors further configured to read a plurality of barcode objects and their corresponding barcodes in each of the first and second plurality of images, and store the read barcodes in a second database configured to store the plurality of read barcodes and images together with the time a given image was captured by the first and / or second image sensors by using a linking relationship such that one or more read barcodes are linked to the image in which the one or more read barcodes were obtained and the time the image was captured by the first and / or second image sensors, and the central controller further configured to determine whether there is a match between the query barcode and one of the read stored barcodes.
[0037] In at least one implementation, the central controller can be coupled to an output device, and the central controller is configured to provide an output to the output device when the query barcode is identical to the read stored barcode.
[0038] In at least one implementation, the output may include a time indicator corresponding to the time an image linked to a barcode that is identical to the query barcode was captured by the first and / or second image sensor.
[0039] In at least one implementation, the output may include an image that is identical to the query barcode and includes a barcode captured by the first and / or second image sensors.
[0040] In at least one implementation, the output may include a time indicator indicating the time an image linked to the barcode matching the query barcode was captured by the first and / or second image sensors, and / or the output may include an image linked to the barcode matching the query barcode, the time indicator or image corresponding to the most recent time of a match between the query barcode and one of the stored barcodes that was read relative to the time the query barcode was received from the user.
[0041] In at least one implementation, the input device may be further configured to receive input from a user for a query time period defined by a first time and a second time, and the central controller is configured to detect a plurality of barcode objects and their corresponding barcodes in a set of images captured by the first and / or second image sensors during the query time period.
[0042] In at least one implementation, the system may further include a plurality of barcodes associated with objects other than personal protective items, the barcodes including object information identifying the objects.
[0043] Other features and advantages of the present disclosure will become apparent from the following detailed description. It should be understood, however, that the detailed description, while indicating certain exemplary implementations of the present disclosure, is given by way of example only, since various changes and modifications within the spirit and scope of the present disclosure will become apparent to those skilled in the art from the detailed description. [Brief explanation of the drawings]
[0044] The present disclosure is described in the paragraphs provided below, by way of example, with reference to the accompanying drawings. The figures provided herein are provided for a better understanding of exemplary implementations and to more clearly show how various implementations may be practiced. The figures are not intended to limit the present disclosure.
[0045] [Figure 1] FIG. 1 is a schematic diagram of a system for monitoring the use of personal protective equipment, according to an exemplary implementation of the present disclosure. [Figure 2] 1 is a flowchart of a process for monitoring the use of personal protective equipment according to an exemplary embodiment of the present disclosure. [Figure 3] FIG. 10 is a schematic diagram of another system for monitoring the use of personal protective equipment, according to an alternative exemplary implementation of the present disclosure. [Figure 4] FIG. 10 is a schematic diagram of another system for monitoring the use of personal protective equipment, according to an alternative exemplary implementation of the present disclosure. [Figure 5A] FIG. 1 is a schematic diagram of an electronic device for inclusion in an electronic access control system for monitoring the use of personal protective equipment, according to an exemplary implementation of the present disclosure. [Figure 5B] FIG. 1 is a schematic diagram of a gate control system for inclusion in an electronic access control system for monitoring the use of personal protective equipment, according to an exemplary implementation of the present disclosure. [Figure 5C] FIG. 1 is a schematic diagram of another gate control system for inclusion in an electronic access control system for monitoring the use of personal protective equipment, according to an exemplary implementation of the present disclosure. [Figure 6A] FIG. 2 is an exemplary diagram of an image of a workspace. [Figure 6B] 6B is an exemplary diagram of another image of the workspace, representing a portion of the image of FIG. 6A. [Figure 7] 7A, 7C, and 7E are captured by a first sensor, and the corresponding images shown in FIGS. 7B, 7D, and 7F are captured approximately simultaneously by a second sensor, according to an exemplary implementation of the present disclosure. [Figure 8]FIG. 8B is an exemplary diagram of another multiple temporally consecutive images of an area within a workspace, in which the images shown in FIGS. 8A, 8C, and 8E are captured by a first sensor, and the corresponding images shown in FIGS. 8B, 8D, and 8F are captured approximately simultaneously by a second sensor, according to an exemplary implementation of the present disclosure. [Figure 9] 1 is a schematic overhead view of a workspace and first and second sensors disposed therein, according to various exemplary implementations of the present disclosure. FIG. [Figure 10] FIG. 10 is a schematic diagram of another system for monitoring the use of personal protective equipment, according to an alternative exemplary implementation of the present disclosure. [Figure 11] 1 is a flowchart of a process for tracking the use of personal protective equipment and non-personal protective equipment, according to an alternative exemplary implementation of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION
[0046] Various systems and processes are described below to provide examples of implementations or implementations of each claimed subject matter. Any implementations described below do not limit any claimed subject matter, and any claimed subject matter may cover methods, systems, devices, assemblies, processes, or apparatuses other than those described below. The claimed subject matter is not limited to systems or processes having all of the features of any one system, method, device, apparatus, assembly, or process described herein, or to features common to multiple or all systems, methods, devices, apparatus, assemblies, or processes described herein. A system or process described herein may not be an implementation or implementation of any claimed subject matter. Any subject matter disclosed in the systems or processes described herein that is not claimed herein may be the subject of another means of protection, e.g., a continuing patent application, and the applicant, inventor, or owner does not intend to abandon, disclaim, or make available to the public any such subject matter by its disclosure herein.
[0047] As used in this specification and claims, singular forms such as "a," "an," and "the" include plural references, and vice versa, unless the context clearly dictates otherwise. Throughout this specification, unless otherwise indicated, the terms "comprise," "comprises," and "comprising" are used inclusively rather than exclusively, such that a stated integer or group of integers may include one or more other unstated integers or groups of integers.
[0048] The term "or" is inclusive unless modified, for example, by "either."
[0049] When ranges are used herein, such as geometric parameters or exemplary distances, all combinations and subcombinations of ranges, as well as specific implementations therein, are intended to be included. Other than in the operating examples, or where otherwise indicated, all numbers expressing quantities of ingredients or reaction conditions used herein should be understood to be modified in all instances by the term "about." The term "about," when referring to a number or numerical range, means that the referenced number or numerical range is an approximation within acceptable experimental variability (or statistical experimental error); thus, the number or numerical range may vary by 1% to 15% of the referenced number or numerical range, as readily recognized by the context. Furthermore, any range of values described herein is intended to specifically include the limits of the range and any intermediate values or subranges within the given range, and all such intermediate values and subranges are individually and specifically disclosed (e.g., a range of 1 to 5 includes 1, 1.5, 2, 2.75, 3, 3.90, 4, and 5). Similarly, other terms of degree, such as "substantially," "about," and the like, used herein, refer to a reasonable amount of deviation from the modified term such that the end result is not significantly altered. These terms of degree should be interpreted as including deviations from the modified term if they do not negate the meaning of the modified term.
[0050] Unless otherwise defined, scientific and technical terms used in connection with any explicit phrase described herein shall have the meaning commonly understood by those skilled in the art. The terminology used herein is for the purpose of describing particular implementations and is not intended to limit the scope of the teachings herein, which is defined solely by the claims.
[0051] All publications, patents, and patent applications are herein incorporated by reference in their entirety to the same extent as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated by reference in its entirety. definition
[0052] The terms "automated system" or "system," as used interchangeably herein, refer to a device, or configuration of multiple devices, that includes one or more electronic processing elements, such as hardware logic including one or more processors, application specific integrated circuits (ASICs), graphics processing units (GPUs), and field programmable gate arrays (FPGAs) that can execute non-compiled or compiled (i.e., machine-executable) instructions; such devices include, but are not limited to, any personal computer, desktop computer, handheld computer, laptop computer, tablet computer, mobile phone computer, smartphone computer, or other suitable electronic device or devices.
[0053] Some example implementations of the systems, devices, or methods described by the teachings herein may be implemented as a combination of hardware or software. For example, some of the implementations described herein may be implemented, at least in part, by using one or more computer programs running on one or more programmable devices (i.e., computing devices), each including at least one processing element and at least one data storage element (including volatile and non-volatile memory). These devices may also have at least one input device and at least one output device, as defined herein.
[0054] It should also be noted that some elements used to implement at least some of the implementations described herein may be implemented via software written in a high-level procedural language, such as object-oriented programming. TM, Visual Basic, Fortran, C, C++, or any other suitable programming language, and may include modules or classes as known to those skilled in the art of object-oriented programming. Alternatively, or in addition, some of these elements that are implemented via software may be written in assembly language, machine language, or firmware, as appropriate.
[0055] At least a portion of the software program used to perform at least one of the implementations described herein may be stored in a storage medium (e.g., a computer-readable medium, such as, but not limited to, a read-only memory (ROM), a magnetic disk, an optical disk, etc.) or a device readable by a general-purpose or special-purpose programmable device. The software program code, when read by at least one processor of the programmable device, configures the at least one processor to operate in a new, specific, predefined manner to perform at least one of the methods described herein.
[0056] Furthermore, at least some of the programs associated with the systems and methods of the implementations described herein may be distributed in a computer program product that includes a computer-readable medium bearing computer-usable instructions, such as program code or program instructions, for one or more processors. The program code may be pre-installed or embedded during manufacture and / or installed later as an update to an already deployed computing system. The medium may be provided in various forms, including, but not limited to, non-transitory forms such as, for example, one or more diskettes, compact discs, tapes, chips, Universal Serial Bus (USB) keys, external hard drives, magnetic and electronic media storage, tablet (e.g., iPad) or smartphone (e.g., iPhone) apps, etc. In alternative implementations, the medium may be transitory in nature, such as, but not limited to, wired transmissions, satellite transmissions, Internet transmissions (e.g., downloads), media, and digital and analog signals. The computer-usable instructions may also be in various formats, including compiled and non-compiled code.
[0057] In one aspect, the present disclosure includes a computer-readable medium comprising a plurality of instructions executable on a processing unit of a device to adapt the device to implement a process for monitoring use of personal protective equipment according to any of the process implementations described herein.
[0058] As used herein, the term "barcode" refers to any item or image that includes optically decodable or identifiable indicia. Common types of barcodes include, for example, linear barcodes, also called one-dimensional (1D) barcodes, which represent information by varying the width, height, and / or spacing of multiple parallel lines within the barcode. Another common type of barcode includes matrix barcodes, also called two-dimensional (2D) barcodes, which represent information using geometric patterns such as rectangles, dots, hexagons, or other geometric patterns spaced in two dimensions.
[0059] As used herein, the term "barcode object" refers to an image of a barcode that corresponds to the actual barcode.
[0060] The term "coupled" as used herein can have several different meanings depending on the context in which the term is used. For example, the term coupled can have a mechanical or electrical meaning depending on the context in which it is used, i.e., whether describing a physical layout or, as the case may be, describing the transmission of data. For example, depending on the context, the term coupled can indicate that two elements or devices may be directly physically or electrically connected to each other, or may be connected to each other through one or more intermediate elements or devices, such as, but not limited to, through physical or electrical elements, such as wires, inactive circuit elements (e.g., resistors), etc.
[0061] As used herein, the phrase "performing a safety action" refers to an automated system or a component of an automated system that performs an action to increase the safety of users of the system. This includes immediate short-term actions, such as issuing a warning signal, or longer-term actions, such as providing a safety report or safety training-related information.
[0062] As used herein, the term "humanoid" refers to a shape that uniquely corresponds to the shape of a person or a part of a person, such as a face or hand, for example.
[0063] As used herein, the term "humanoid object" refers to an image of a person that corresponds to a real person, or an image of a part of a person that corresponds to a part of a real person, such as a face or hand. Generally, in the context of this disclosure, humanoid objects may be present in an image, such as a workplace image. Generally, a workspace image may include one or more humanoid objects and / or one or more non-humanoid objects. For example, the image corresponds to reality, such as a workplace image corresponding to a real workplace.
[0064] The term "input device," as used herein, refers to any user-operable device used to input information, including, but not limited to, one or more of a terminal, a touch screen, a keyboard, a mouse, a mouse pad, a tracker ball, a joystick, a microphone, a voice recognition system, a light pen, a camera, a data input device such as a barcode reader or magnetic ink character recognition device, a sensor, or any other computing unit capable of receiving input data. An input device may include a TV, or a two-dimensional display such as a liquid crystal display (LCD), or a light-emitting diode (LED)-backlit display, a mobile phone display, or a display capable of receiving input from a user, for example, by way of a touch screen.
[0065] A user according to this specification may be any user or operator, including, for example, any safety officer, or work site operator or manager.
[0066] The term "non-humanoid" refers to shapes that correspond to any or all shapes other than shapes that uniquely correspond to a person or part of a person.
[0067] The term "non-humanoid object" refers to an image whose shape corresponds to a real object, excluding a real person or part thereof.
[0068] The term "output device," as used herein, refers to any device used to output information, including, but not limited to, one or more of a display terminal, a screen, a printer (e.g., laser, inkjet, dot matrix), a plotter or other hardcopy output device, a speaker, headphones, an electronic storage device, a buzzer or vibrator, a wireless or other communication device capable of communicating with another device, or any other computing unit. Output devices may also include two-dimensional displays, such as televisions, LCD, or LED-backlit displays, and / or mobile phone displays, capable of providing output data in a form viewable by a user.
[0069] As used herein, the term "input / output device" refers to any device that can be used as both an input device and an output device by including both input and output capabilities in the device.
[0070] The terms "personal protective equipment" or "PPE" are used interchangeably herein and refer to any equipment, implement, or article capable of reducing the risk of injury or physical damage to a person, e.g., eye injury, hand injury, foot injury, and other forms of physical harm, as a result of incidents, accidents, and / or hazards, including workplace incidents. PPE includes protective articles such as, but not limited to, safety helmets, safety gloves, safety glasses or goggles, safety shoes, face shields or masks, hearing protection devices, protective vests or jackets, safety suits or gowns, gas tanks and / or respirators, and safety boots. PPE also includes devices used for protection, such as, but not limited to, radiation dosimetry devices, noise measurement devices, and / or light intensity measurement devices for measuring light in the visible or other wavelength ranges. These terms are further intended to include protective equipment worn in workplaces where there is a risk of contamination of work objects due to direct human contact with the work objects, such as protective articles and implements used in, for example, electronics manufacturing or pharmaceutical or biologics manufacturing. General implementation of the system
[0071] As previously mentioned, the present disclosure relates to systems and processes for monitoring the use of personal protective equipment. The automated systems and processes can be implemented in a manner that promotes strong compliance with applicable safety rules and guidelines in a workspace, such as a hazardous work environment. The systems can be configured to accurately, continuously, or periodically monitor and verify whether a person is wearing one or more personal protective equipment items while present in the hazardous work environment. The systems of the present disclosure can simultaneously monitor multiple people, e.g., 10 or 20 people, in a workspace, each wearing multiple personal protective equipment items, e.g., 5 or 10 such items.
[0072] If the system detects a PPE item that is not being worn by the person, the system can execute a safety action, such as issuing a safety alert. The disclosed system is sensitive enough to easily detect separation between a user and a PPE item over a relatively short time period (e.g., 10 seconds) and / or physical separation between a user and the PPE over a relatively short distance (e.g., 1 meter). Therefore, the system disclosed herein is suitable for flexible use in many environments, including dynamic or temporary workspaces such as construction sites. The process and system are user-worn, i.e., the system does not involve the use of wearable sensors, such as photoelectric or pressure sensors, to monitor whether the PPE is being worn by the user. Therefore, the system is not susceptible to sensor malfunctions, or to slight adjustments that a user wearing the PPE may make from time to time, or to external factors such as weather, all of which can interfere with the function of wearable sensors, potentially resulting in the generation of false alarms. The system also does not require custom fitting, as may be required for wearable sensor-based systems. These and other beneficial aspects make the systems disclosed herein useful in preventing the occurrence of work-related injuries.
[0073] Broadly speaking, a system for automatically monitoring the use of one or more pieces of personal protective equipment includes a plurality of barcodes, each barcode associated with an item of personal protective equipment and user information for a user of the item of personal protective equipment. An image sensor is installed in a workspace and captures images of the workspace in which users wearing the personal protective equipment are present. The system is configured to detect human-like objects (corresponding to users of the personal protective equipment) and barcode objects (corresponding to the barcodes) in the images. Based on the detected human-like objects and barcode objects, a probabilistic algorithm is used to calculate the probability that any user in the image was wearing all personal protective equipment in accordance with safety rules applicable to the workspace at the time the image was captured. The system is further configured to execute a safety action when the calculated probability is below a predetermined probability threshold.
[0074] Selected embodiments will now be described with reference to the drawings.
[0075] As a general overview, Figures 1, 3, 4, 5A, 5B, and 5C are schematic diagrams of various hardware components and system configurations according to an exemplary implementation of the system of the present disclosure. Figure 2 is a flowchart diagram of an exemplary embodiment of the automated process of the present disclosure. Figures 6A, 6B, 7A, 7B, 7C, 7D, 7E, 7F, 8A, 8B, 8C, 8D, 8E, and 8F are exemplary diagrams of images captured by an image sensor according to an exemplary implementation of the system of the present disclosure. Figures 9A, 9B, 9C, and 9D are schematic diagrams of an exemplary image sensor installation according to an exemplary implementation of the system of the present disclosure. Figure 10 is a schematic diagram of various hardware components and configurations of a system according to an alternative exemplary implementation, and Figures 11A and 11B are flowchart diagrams of an alternative exemplary embodiment of the automated process of the present disclosure.
[0076] Referring initially to FIG. 1 , the present disclosure provides, in an exemplary implementation, a system 100 for use in conjunction with personal protective equipment. The system 100 enables automated monitoring of the use of multiple pieces of personal protective equipment by multiple workers in a workspace. The multiple pieces of personal protective equipment are illustrated by safety helmets 110a and 110b and safety vests 107a and 107b. It should be understood that the safety helmets 110a and 110b and the safety vests 107a and 107b represent examples of personal protective equipment. Other examples of personal protective equipment include, but are not limited to, at least one of safety gloves, safety shoes, safety suits, safety goggles, safety masks, and hearing protection devices, for example, all of which are not shown but may additionally or alternatively be detected and monitored by the system 100 disclosed herein. Each of the personal protective equipment 107a, 107b, 110a, and 110b is individually, permanently, or removably tagged with a barcode 109a, 109b, 115a, and 115b, respectively, by adhering or attaching the barcode 109a, 109b, 115a, and 115b to the personal protective equipment 107a, 107b, 110a, and 110b, respectively. The barcodes 109a, 109b, 115a, and 115b in various implementations can be 1D or 2D barcodes, such as QR Code, Data Matrix, Maxicode, or PDF417, or a combination of 1D and 2D barcodes. It should be noted that individual barcodes are typically selected to be unique for each PPE item, such that not only different classes of PPE (e.g., classes of safety gloves, safety masks), but also individual PPE items within a given class of PPE (e.g., multiple safety gloves) have unique barcodes. Thus, the image of the barcode can be said to represent object information of the PPE item that allows separate identification of a particular PPE item, e.g., a particular safety helmet, or a particular safety vest, e.g., safety vest 107a and safety vest 107b. A 2D barcode is shown in FIG. 1 by way of example only.However, 1D barcodes may be less sensitive to image distortion than 2D barcodes, and therefore may be preferred in implementations involving physical attachment of the barcode to a curved or curved surface of the protective equipment, or in implementations involving less sensitive or fewer sensors. Alternatively, 2D barcodes may be preferred in implementations where it is desirable to include more information in the barcode. Furthermore, in implementations where image data is recorded, as described further below, it may be easier to correct acquired image data that has some erroneous information from the 2D barcode, given the fact that the 2D barcode contains additional information that can be used to reconstruct any erroneous information in the acquired image data.
[0077] Continuing to refer to FIG. 1 , system 100 is installed within workspace 155 defined by workspace perimeter 156, and workers 105a and 105b working within workspace 155 are required to wear respective safety helmets 110a and 110b including barcodes 115a and 115b, respectively, and respective safety vests 107a and 107b including barcodes 109a and 109b, respectively, in accordance with applicable safety regulations for workspace 155. Note that the uniqueness of barcodes 109a, 109b, 115a, and 115b allows for differentiation between worker 105a and worker 105b. In some implementations, workspace 155 may be a permanent workspace, such as a warehouse, manufacturing facility, or laboratory. In other implementations, workspace 155 may be a temporary workspace, such as a construction site. In still other implementations, the workspace 155 may be a relatively small space around an individual piece of hazardous equipment, such as, for example, the space around a forklift, a circular saw, or the like.
[0078] System 100 further includes first and second image sensors 120 and 125, respectively, spaced apart in workspace 155. Furthermore, first and second image sensors 120 and 125 are angled (as described below with reference to FIGS. 9A-9D ) to enable image sensors 120 and 125 to capture images of an area 159 within workspace 155 from first and second angles of view (see double-headed arrows ar-a and ar-b, and ar1 and ar2 for image sensors 120 and 125, respectively, in FIG. 9A , which indicate the range of possible angles at which image sensor 120a1 can be selected for mounting image sensor 120a1). Furthermore, first and second image sensors 120 and 125 are configured to continuously or periodically capture temporally successive images of area 159 within workspace 155.
[0079] In some implementations, the first and second image sensors 120 and 125 may be cameras, e.g., Internet Protocol (IP) cameras with or without image processing capabilities, including cameras with a shutter speed of at least about 1 / 60 second to enable motion blur control, capable of recording an image stream having a resolution of, e.g., about 1920 x 1080 pixels, at a sampling rate of, e.g., about 3-5 frames per second, or capable of recording two image streams, e.g., a first image stream having a resolution of, e.g., about 1920 x 1080 pixels, at a sampling rate of, e.g., about 3-5 frames per second, and a second image stream having a resolution of, e.g., about 1,280 x 720 pixels, at a sampling rate of, e.g., 12-24 frames per second. Cameras providing two image streams can be used to perform image processing analysis of the first image stream and recording of the second image stream, as described further below. Image sensors 120 and 125 may further be selected to be operable within a range of temperatures likely to occur within workspace 155, for example, -30°C to +50°C.
[0080] In various implementations, image sensors that may be used may further include at least one of a digital camera (still or video), a light detection and ranging (LIDAR)-based sensor, or a time-of-flight (ToF)-based sensor, for example. ToF-based sensors may be useful in implementing the first or second sensors 120 and 125 because they enable spatial resolution of the distance between the camera and the object being imaged in three dimensions for each point in the image, thereby enhancing the ability to distinguish between human-like objects and their shadows. Image sensors that may be used may further include at least one of infrared (IR)-based sensors, including, for example, but not limited to, a passive IR (PIR) sensor, a deep IR-based sensor, or a ToF IR-based sensor. In this regard, deep IR-based sensors may be useful in implementing the first and second sensors 120 and 125 because they may enable identification of human-like objects in thermal images based on the detection and evaluation of temperature patterns in the thermal images. In particular, deep IR-based sensors can be used to distinguish an image of a human-shaped object displaying a temperature of approximately 37°C from the surrounding environment or objects within it that have a different temperature.
[0081] 1 , central controller 145, implemented by a computer server, controls the operation of the entire system 100 and is operably coupled to network 130, such as, for example, a wireless network or local area network (LAN). Central controller 145 is also operably coupled to first and second image sensors 120 and 125, respectively, via network 130. Central controller 145 includes a central processing unit (CPU) 146 having one or more processors or logic hardware, memory components, and input and output ports or communication buses that enable communicative coupling to peripheral input and output devices, such as input devices 140 (i.e., keyboards and / or touch-sensitive screens) or scanners (not shown). Central controller 145 is further configured to provide output data to output devices, such as printer 150, or to access and update databases or files in a data store (not shown). The data store may be part of central controller 145 or may be a separate hardware element.
[0082] The central controller 145 may be controlled by an operator via the input devices 140. In this manner, the central controller 145 may be configured to provide or track information related to the system 100, such as, but not limited to, user-related information, personal protective equipment-related information, worksite-related information, or barcode-related information.
[0083] In some implementations, the signaling component represented by the mobile phone 135 can be used as an input device.
[0084] According to one aspect of the present disclosure, the input data includes user information for a given user of the personal protective equipment 107a, 107b, 110a, and 110b. Thus, the central processing unit 146 can receive user information, such as the names, phone numbers, dates of birth, and / or user IDs of the workers 105a and 105b, and store such information in a database. The user information can be included in barcodes. Thus, for example, barcodes 109a and 115a can include the user ID of worker 105a, and barcodes 109b and 115b can include the user ID of worker 105b. The database is designed so that, for a given user, the user's personal information and all barcodes associated with all PPE worn by the user are all linked together in the database, such that when the database is searched using one of these barcodes, the personal information of the worker using the PPE item associated with the barcode and all other barcodes linked to the same personal information can be identified in the database. The database can be a relational database.
[0085] Further, according to an aspect of the present specification, the input data may include safety rules applicable to the workspace 155. As used herein, a safety rule includes safety-related parameters applicable to the workspace and / or the personal protective equipment. For example, a safety rule may include a safety rule time and / or a safety rule distance, as well as a safety rule action, and may be defined such that if a user moves away from the personal protective equipment, for example, by more than 2 meters (i.e., the safety rule distance) for a time period greater than 30 seconds (i.e., the safety rule time), the system executes the safety rule action associated with the safety rule, for example, to generate and emit a signal. Executable safety rule actions include, but are not limited to, warning actions (e.g., generating an audible signal, visual, or tactile warning) and operational actions (e.g., communicating with the central processing unit 146, sending an email, or logging data). The safety rule may be stored as safety rule data in the memory of the controller 145 or in a data store, both accessible by the central processing unit 146. Accordingly, the central processing unit 146 is configured to receive safety rules applicable to the workspace 155 via the input device 140 or to access safety rule data from a memory. For example, the central processing unit 146 is configured to receive a safety rule in the workspace 155 that a worker present in the workspace 155 must wear a safety helmet and a safety vest. Accordingly, the central processing unit 146 is configured to perform monitoring such that when the worker 105a or 105b is present in the workspace 155, the worker 105a is required to wear the safety vest 107a (including the barcode 109a) and the safety helmet 110a (including the barcode 115a), and the worker 105b is required to wear the safety vest 107b (including the barcode 109b) and the safety helmet 110b (including the barcode 115b).
[0086] Referring again to the first and second image sensors 120 and 125, respectively, the first and second image sensors 120 and 125 are spaced apart and angled to capture images of the area 159 within the workspace 155. As a result, the images captured by the first and second image sensors 120 and 125 correspond, at least in part, to the area 159 within the workspace 155. However, it should be noted that due to the different angles, the images captured by the first and second image sensors 120 and 125 may reveal or obscure different elements present in the area 159 of the workspace 155. Thus, for example, the image captured by image sensor 125 may only show a portion of worker 105a, as other portions of worker 105a may be obscured by the image of worker 105b. However, image sensor 120 captures a different image of worker 105a due to its different field of view of the area 159 compared to image sensor 125.
[0087] 9A-9D, schematic top views of workspaces 155a (FIG. 9A), 155a1 (FIG. 9B), 155a2 (FIG. 9C), and 155a3 (FIG. 9D) are shown, each defined as having north (N), south (S), west (W), and east (E) walls. Note that in the exemplary implementation shown in FIGS. 9A-9D, the workspaces are defined by walls. In other implementations, one or more of such walls may not be present, and the location of the walls may be thought of as a non-physical perimeter or boundary beyond which another portion of the workspace may be located. Image sensor 125a1 is mounted on the east (E) wall to capture images of workspaces 155a, 155a1, 155a2, and 155a3 with a field of view 158a directly toward the west (W) wall. Image sensor 120a is installed on the north (N) wall, but in other implementations, it may be installed on the east (E), west (W), or south (S) wall (not shown). Furthermore, image sensor 120a is installed such that angle (a) can be selected to vary in different implementations, as indicated by arrows (ar1) and (ar2). Because angle (a) is formed by intersecting axes that extend centrally through the fields of view of image sensors 125a1 and 120a3, it may be referred to as a selected angle based on the placement of image sensors 120a1 and 125a2. Thus, as an example, as shown in FIG. 9B, image sensors 125a1 and 120a2 can be installed such that angle (a) is selected to be 90 degrees. Angle (a), here indicated as a1, extends centrally through fields of view 158a and 157a2 of image sensors 125a1 and 120a2, respectively, and is formed by axes of interest b1 and c1. 9C, image sensor 125a1 and image sensor 120a3 may be positioned such that angle (a) is selected to be 45 degrees. Angle (a) is shown here as a2 and is formed by intersecting axes b2 and c2 that extend centrally through fields of view 158a and 157a3 of image sensors 125a1 and 120a3, respectively.9D, image sensors 125a1 and 120a4 can be positioned such that angle (a) is selected to be 110 degrees. Angle (a) is shown here as a3 and is formed by intersecting axes b3 and c3 that extend centrally through fields of view 158a and 157a4 of image sensors 125a1 and 120a4, respectively.
[0088] 1 and applying the general principles illustrated in Figures 9A-9D thereto, it is apparent that in different implementations, image sensors 120 and 125 may be mounted to be angled at various angles independently selected within angles ar-a and ar-b of image sensors 120 and 125, respectively, provided that overlap between respective fields of view 157 and 158 is created as shown in Figures 9A-9D. The selected angle may be formed by the intersection of a first axis and a second axis (not shown in Figure 1 to avoid cluttering the figure), the first axis extending centrally through field of view 157 of image sensor 120 and the second axis extending centrally through field of view 158 of image sensor 125. In preferred implementations, the angle is selected to be in the ranges of 15 degrees to 175 degrees and 205 degrees to 345 degrees, and when the angle is selected to be 205 degrees to 305 degrees, the image sensor 120 is mounted on the east (E) or south (S) wall (not shown). In even more preferred implementations, the angle is selected to be in the ranges of 30 degrees to 150 degrees and 220 degrees to 330 degrees, and when the angle is selected to be 220 degrees to 330 degrees, the image sensor 120 is mounted on the east (E) or south (S) wall (not shown). The inventors have found that angles selected within these ranges are particularly effective in operating the system 100.
[0089] In one embodiment, the first and second image sensors 120 and 125 are configured together with the central processing unit 146 to analyze the images captured by the image sensors 120 and 125 by obtaining captured images and applying a human-shaped object image analysis algorithm to the captured images to detect whether any human-shaped objects are present in each of the captured images, and by applying a barcode object image analysis algorithm to the captured images to detect whether any barcode objects are present in each of the captured images. As used herein, the phrase "first and second image sensors configured with a central processing unit" is intended to refer to operably coupled hardware and software components that can collectively execute several algorithms, including human-like object image analysis algorithms and barcode object image analysis algorithms; that is, a portion of the algorithm can be executed by the first sensor, a portion of the algorithm can be executed by the second sensor, a portion of the algorithm can be executed by the central processing unit, or a portion of the algorithm can be executed by only one of the image sensors 120 and 125 and the central processing unit 146, or an algorithm can be executed by only the central processing unit 146. Thus, the portions of the algorithms implemented by the image sensors 120 and 125 and the central processing unit 146 may vary, and the hardware and software components are operably coupled such that they can execute the algorithms together, by themselves, or in any combination of hardware that includes only one or two of the sensors and the central processing unit. In some implementations, an algorithm that can detect both human-like objects and barcode objects may be used.In different implementations, system 100 may be configured such that different portions of the hardware and software components are physically located within central processing unit 146 or within first and / or second image sensors 120 and 125. The relative distribution of hardware and software components between central processing unit 146 and first and / or second image sensors 120 and 125 may be adjusted as needed to accommodate different implementations. Thus, for example, in one implementation, application of a given image analysis algorithm may be performed by central processing unit 146 following transmission of a captured image by image sensors 120 and 125 and its receipt by central processing unit 146. In another implementation, application of a given image analysis algorithm may be performed by image sensors 120 and / or 125, with subsequent operations (described below) being performed by central processing unit 146.
[0090] The application of image analysis algorithms generally involves identifying one or more images of objects, particularly humanoid objects or barcode objects as described below, in images of workspace 155 captured by image sensors 120 and 125. Generally, this may be achieved by an object image analysis algorithm, such as a supervised learning algorithm, that is designed and trained using templates having humanoid features, e.g., human body shapes, or portions thereof, e.g., facial or hand features, or, in the case of barcode objects, using templates having features of barcode features, to identify one or more humanoid objects in the images.The object image analysis algorithms that can be used herein include, for example, those based on shape features (e.g., those based on Histogram of Oriented Gradients (HOG) (Dalal N. and B. Triggs. International Conference on Computer Vision & Pattern Recognition (CVPR '05), June 2005, San Diego, United States. pp. 886-893)) or Scale Invariant Feature Transform (SIFT) (D.G. Lowe., 1999, In: Computer vision, The proceedings of the seventh IEEE international conference on, volume 2, pp. 1150-1157)), appearance features (e.g., Haar features (Papageorgiou, C.P. et al. In: Computer vision, 1998, Sixth International Conference on, pp. 555-562. IEEE, 1998)), or motion features (e.g., those based on Histogram of Flow (HOF) (Dalal, N. et al., 2006, Human Detection Using Oriented Histograms of Flow and Appearance. In: Leonardis A., Bischof H., Pinz A. (eds) Computer Vision - ECCV 2006. ECCV 2006. Lecture Notes in Computer Science, vol 3952. Springer, Berlin, Heidelberg)), or a combination of these features.Classifiers (e.g., Viola, P. and M. Jones, 2001, Second International Workshop on Statistical and Computational Theories of Vision - Modeling, Learning, Computing and Sampling; Lienhart, R. and J. Maydt, Image Processing, 2002, Proceedings, International Conference on v. 1, pp. I-900 - I-903) or localizers (e.g., Blaschko, MB and CH Lampert, 2008, In: Computer Vision-ECCV, 2-15, Springer) are then used to identify objects, typically by running these classifiers or localizers on images or regions of images. Examples and further guidance regarding human-like object image analysis algorithms for identifying human-like objects can be found, for example, in Gavrila, D. M., 1999, Computer Vision and Image Understanding, 82-98; Wang, L. et al., 2003, Pattern Recognition, 36 285-601; U.S. Patent Nos. 9,256,781; 8,938,124; and 7,324,693.
[0091] 3, in some implementations, system 300 may be implemented to include a computational neural network 320, i.e., a computational model that performs calculations by applying one or more functions or approximations thereof to one or more inputs. Computational neural network 320 may be designed to classify input data obtained from captured images provided by first or second image sensors (120, 125) or by a database containing recorded image data (described below) to improve the ability of system 300 to accurately identify human-like objects and barcode objects in captured images of workspace 155 (described below). In this sense, computational neural network 320 functions as an image analysis algorithm capable of detecting both human-like objects and barcode objects. Thus, using images from the first or second image sensors (120, 125), the computational neural network 320 may be configured to develop improvements in, for example, proper identification / detection of humanoid objects (105a, 105b) within acquired images, addressing complexities such as, for example, variations in body posture, variations in body size, occlusion of body parts by other body parts or by non-humanoid objects or by body parts of other humanoid objects, etc. Additionally, using images from the first or second image sensors (120, 125), the computational neural network 320 may be configured to develop improvements in identification of barcodes (115a, 115b), addressing complexities such as, for example, image distortion resulting from the barcode being recorded from various angled viewpoints, for example, partial occlusion of the barcode by an object or body part, or curvature or warping of the barcode. One or both of these improvements may be implemented by periodically or continuously training the neural network 320 to improve its performance and reduce the occurrence of false detections of human-like objects in captured images and errors in detecting corresponding barcodes.
[0092] Generally, the computational neural network 320 can be configured to include multiple neurons or nodes configured to calculate outputs based on weights assigned to inputs, and can be arranged in layers, such as an input layer, an output layer, and at least one hidden layer. Input data to the computational network 320 can be provided by the first or second sensors (120, 125) or by a database containing image data recorded by the recording device 170 and stored in a data store as recorded images. The output of the computational network 320 can be provided to the central controller 145. At least one hidden layer of the computational network 320 can operate to perform a classification function, for example, comparing a newly identified estimated humanoid pose in an image with a series of images of humanoid objects to accurately classify the humanoid objects in the series of images. Thus, the output can include any detected humanoid objects and / or barcode objects in the image data. These objects are represented by coordinates within the image that are processed by the computational network 320, which may represent the boundaries of these objects within the image.Examples of suitable neural networks that can be used include You Only Look Once (YOLO) (Redmon, J. et al., 2016, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779-788), and neural networks based on Region-based Convolutional Neural Networks (R-CNN) (Girshick R. et al., 2014, IEEE Conference on Computer Vision and Pattern Recognition, pp. 580-587), faster R-CNN (Ren, S. et al., 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence 39 (6), pp. 1137-1349 (code at: https: / / github.com / ShaoqingRen / faster_rcnn; and https: / / github.com / rbgirshick / py-faster-rcnn), or Mask R-CNN (He K. et al. 2017 IEEE Conference on Computer Vision and Pattern recognition, pp. 2980-2988 (code at: https: / / github.com / facebookresearch / detectron2).
[0093] 6A and 6B, an exemplary image 601 of workspace 655 is shown. Image 601 may be, for example, an image captured by image sensor 120. Visible within image 601 are humanoid objects 105b-i (corresponding to actual workers, see e.g., worker 105a in FIG. 1 ), safety helmet objects 110b-i (corresponding to actual safety helmets, see e.g., safety helmet 110 in FIG. 1 ), and barcode objects 115b-i (corresponding to actual barcodes, see e.g., barcode 115 in FIG. 1 ). Also visible in FIG. 6A are various non-humanoid objects 610-i, 615-i, 633-i, and 620-i (corresponding to various actual non-humanoid objects, see e.g., wheelbarrow 133 in FIG. 1 ). In this example, the first image sensor 120, together with the central processing unit 146, is configured to execute a human-shaped object image analysis algorithm for the image 601, as described above, that enables it to identify human-shaped objects 105b-i in the image 601 of the workspace 655 while at the same time not identifying the non-human-shaped objects 610-i, 615-i, 620-i, and 633-i as human-shaped objects.
[0094] In one embodiment, the first and second image sensors 120 and 125, in conjunction with the central processing unit 146, are configured to select a second plurality of images from the first plurality of images, each of the images comprising the second plurality of images including at least one detected human-like object. Accordingly, the first and second image sensors 120 and 125, in conjunction with the central processing unit 146, are configured to execute a human-like object image analysis algorithm using a first plurality of captured images captured by both image sensors (120, 125), such as exemplary image 601 captured by image sensor 120. Some of these images may or may not include one or more human-like objects. For example, if an image is captured at a particular time and no worker is present within the field of view 157, no human-like object will be detected. The first and second image sensors 120 and 125, in conjunction with the central processing unit 146, are configured to select a second plurality of images from the first plurality of images, the second plurality of images including at least one detected human-like object.
[0095] In another embodiment, first and second image sensors 120 and 125, together with central processing unit 146, are configured to analyze the second plurality of images by applying an object image analysis algorithm, i.e., a barcode-object image analysis algorithm, which may be a barcode image analysis algorithm or a computational neural network 320, to images within the second plurality of images to detect multiple barcode objects. Generally, this can be accomplished by an object detection image algorithm, such as a supervised learning algorithm, designed and trained using a template having characteristics of the barcodes, or portions thereof. Thus, for example, first image sensor 120, together with central processing unit 146, is configured to perform image analysis on image 601 that enables identification of barcode objects 115b-i within image 601 of workspace 655. Algorithms and software that can be used in this regard include, for example, Zbar (http: / / zbar.sourceforge.net / ), ZXing (https: / / github.com / zxing / zxing), Quirc (https: / / github.com / dlbeer / quirc), and BoofCV (https: / / boofcv.org / index.php?title=Performance:QrCode).
[0096] It should be noted that, in general, identifying barcodes such as those shown in image 601, and even identifying specific, unique barcodes within an overall image where the barcode represents a relatively small portion of the overall image, can be difficult using known algorithms for image analysis, resulting in a variety of image analysis errors, such as other objects being misidentified as barcodes, barcodes not being identified despite their presence in the image, or an inability to link an identified barcode to a known, issued barcode, etc. However, as shown in image 625, this error rate can be significantly reduced by increasing the relative size of the barcode within the image, for example, by cropping initial image 601 or enlarging a portion of image 601 to generate image 625.
[0097] Thus, in some implementations, the first and second image sensors 120 and 125, together with the central processing unit 146, can be configured to execute a barcode object image analysis algorithm on selected regions of the image, particularly selected regions near a detected human-shaped object, to detect the barcode object. Thus, referring to FIG. 6B , an exemplary image region 625 representing a portion of the image 601 is shown. The image region 625 is selected for application of a 2D barcode object image analysis algorithm. This can be achieved, for example, by applying a framing or bounding algorithm, such as those described in U.S. Patent Nos. 9,256,781, 8,938,124, and 7,324,693, to the detected human-shaped object. The framing or bounding algorithm can create an image boundary within a region of the image such that the bounding region includes the human-shaped object. Thus, for example, an image boundary can be created within the image region to encompass the entire single human-shaped object, such that there is no contact between the human-shaped object and the image boundary. The image boundary may correspond, for example, to a distance of about 0.5 meters to about 3 meters from the actual worker corresponding to the human-like object. In one implementation, the image boundary may correspond to a distance of about 1 meter to about 2 meters from the human-like object within the image region. In contrast, image regions 635, 645, and 655, which contain images of non-human-like objects 610i, 615i, and 620i, respectively, are not selected for application to the framing algorithm and subsequent application of the barcode image analysis algorithm, because image regions 635, 645, and 655 are not classified as containing a detected human-like object. Within image region 625, human-like objects 105b-i are detected (corresponding to an actual worker, see, e.g., worker 105a in FIG. 1). After detecting the human-shaped object 105b-i, the first and second image sensors 120 and 125, together with the central processing unit 146, are then configured to execute a barcode object image analysis algorithm of the image region 625 that enables identification of the barcode object 115b-i within the image region 625.
[0098] Furthermore, upon identification of the barcode object image, a barcode reading algorithm may be applied to the barcode object image, and the barcode may be read as described below.
[0099] Thus, in at least one implementation, application of the framing algorithm results in a framed image region, as shown in Figures 6B, 7A-7F, and 8A-8F, where the framed image region is large enough to contain a single human-shaped object in its entirety, and the image boundary of the framed image region is formed such that there is no contact between the human-shaped object and the image boundary.
[0100] In at least one implementation, the image boundary can correspond to a distance of about 0.5 meters to about 3 meters, or about 1 meter to about 2 meters, away from the humanoid object 105a-i, 105b-i corresponding to the actual worker 105a or 105b.
[0101] 7A-7F and 8A-8F, FIGS. 7A-7F are illustrative diagrams of a series of temporally consecutive image regions of the human-shaped object 105b-i, and FIGS. 8A-8F are illustrative diagrams of a series of temporally consecutive image regions of the human-shaped object 105a-i. The consecutive image regions are determined while the human-shaped objects 105a-i and 105b-i are in the process of moving forward along lines 840-i and 740-i, respectively. For example, FIG. 7A represents an image region including the human-shaped object 105b-i that is determined earlier in time than an image region determined after the human-shaped object 105b-i shown in FIG. 7C. Human-shaped objects 105b-i are included in image regions 720a, 720b, and 720c of multiple images (not shown) including the human-shaped object captured by first image sensor 120, and in image regions 725a, 725b, and 725c of multiple images (not shown) including the human-shaped object captured by second image sensor 125. Similarly, with reference to Figures 8A-8F, human-shaped objects 105a-i are included in image regions 820a, 820b, and 820c of multiple images (not shown) including the human-shaped object captured by first sensor 120, and in image regions 825a, 825b, and 825c of multiple images (not shown) including the human-shaped object captured by second sensor 125.
[0102] Image sensors 120 and 125 are positioned such that each image sensor 120 and 125 can capture images from different angles relative to one another, such that barcodes and / or PPE are captured in images from at least one of image sensors 120 and 125. In this regard, note that image regions 725a, 725b, and 725c each include barcode 115b-i on safety helmet 110b-i, as captured by sensor 125. However, the same barcode 115b-i is not captured by sensor 120 due to the different angle of view. However, barcode 115a-i is not captured in image regions 825a, 825b, and 825c because safety helmet 110a-i is located within cabinet 860-i. For the same reason, the same barcode 115a-i is also not captured by the sensor 120 since the safety helmet 110a-i is located in the cabinet 860-i.
[0103] It is further noted that image regions 720a, 720b, and 720c, and 820a, 820b, and 820c each contain barcodes 109a-i and 109b-i, respectively, on safety vests 107a-i and 107b-i that are captured by image sensor 120. However, the same barcodes 835a-i and 735b-i are not captured by image sensor 125 due to different sensor angles.
[0104] System 100 is further configured to probabilistically determine whether worker 105b is wearing safety helmet 110b and safety vest 107b, and whether worker 105a is wearing safety helmet 110a and safety vest 107a, by using a probabilistic algorithm, as will be further described first with reference to Figures 7A-7F and then with reference to Figures 8A-8F.
[0105] 7A-7F, image regions 720a, 720b, and 720c are image regions from an image captured by image sensor 120. Image regions 725a, 725b, and 725c are image regions from an image captured by image sensor 125. In each of image regions 720a, 720b, 720c, 725a, 725b, and 725c, human-like objects 105b-i are detected using an image analysis algorithm adapted to detect human-like objects (i.e., a human-like object image analysis algorithm or a computational neural network capable of classifying human-like objects). Further, in each of image regions 725a, 725b, and 725c, barcode objects 115b-i are detected using an image analysis algorithm adapted to detect barcode objects (i.e., a barcode object image analysis algorithm or a computational neural network capable of classifying barcode objects). Central processing unit 146, together with first and second image sensors 120 and 125, is configured such that upon detecting a barcode object, e.g., barcode object 115b-i, in one of the image regions (e.g., in image region 725a), a barcode corresponding to the barcode object is read using a barcode reading algorithm to identify the worker to whom the corresponding barcode 115b is assigned, i.e., worker 105b. Barcode reading algorithms and software that may be used in this regard include, for example, the following: Zbar (http: / / zbar.sourceforge.net / ), ZXing (https: / / github.com / zxing / zxing), Quirc (https: / / github.com / dlbeer / quirc), and BoofCV (https: / / boofcv.org / index.php?title=Performance:QrCode).
[0106] This identification can be done automatically because the system 100 includes a database in which each worker's personal information is cross-referenced (i.e., linked) with all barcodes associated with PPE used by each worker, so that when the database is searched using the barcode, the personal information corresponding to the barcode can be identified, as previously described herein.
[0107] Additionally, central processing unit 146 is configured to be able to locate other barcodes assigned to worker 105b, including barcode 109b associated with safety vest 107b, along with first and second image sensors 120 and 125. This identification can be done automatically because system 100 includes a database that stores personal information for each worker and all barcodes associated with each worker's personal information, such that when the database is searched using a barcode (also known as a searched barcode), the personal information corresponding to the searched barcode and all other barcodes associated with the same personal information can be identified in the database. The central processing unit 146, together with the first and second image sensors 120 and 125, is further configured to evaluate each of the image regions 720a, 720b, 720c, 725a, 725b, and 725c for the presence of additional barcode objects 115b-i and 109b-i corresponding to the identified barcode objects 115b and 109b, and to apply a probabilistic algorithm based on the prevalence of each of the barcodes 115b and 109b corresponding to the additional barcode objects 115b-i and 109b-i within the image regions 720a, 720b, 720c, 725a, 725b, and 725c. The probabilistic algorithm further uses a predetermined probability threshold, which may vary in different implementations, to determine whether each of barcodes 115b and 109b corresponding to barcode objects 115b-i and 109b-i has actually been correctly detected in image regions 720a-720c and 725a-725c. Thus, for example, the predetermined probability threshold may be set so that the prevalence (the percentage of barcode objects per barcode in the total image region) is at least 33%. Thus, each of the barcodes assigned to worker 105b must be identified in at least 33% of image regions 720a, 720b, 720c, 725a, 725b, and 725c to ensure that the detected barcodes are not false positives and to account for the possibility that images captured from different angles may not contain these barcodes in all image regions.For example, barcode object 115b-i corresponding to retrieved barcode 115b is present in three of the six image regions, i.e., image regions 725a, 725b, and 725c. As another example, barcode object 109b-i corresponding to retrieved barcode 109b is present in three of the six image regions, i.e., image regions 720a, 720b, and 720c. Therefore, the probability value is 50%, which is higher than the predetermined probability threshold of 33%. Therefore, central processing unit 146, together with first and second image sensors 120 and 125, is configured to determine that worker 105b is likely wearing safety vest 107b and safety helmet 110b because the percentage of images of each of barcode objects 109b and 115b having a human-shaped object corresponding to worker 105b is determined to be greater than the predefined probability threshold.
[0108] 8A-8F, image regions 820a, 820b, and 820c are image regions from the image detected by image sensor 120, and image regions 825a, 825b, and 825c are image regions from the image detected by image sensor 125. In each of image regions 820a, 820b, 820c, 825a, 825b, and 825c, human-shaped object 105a-i is detected. Furthermore, in none of image regions 820a, 820b, 820c, 825a, 825b, and 825c, barcode object 115a-i is detected because safety helmet 110a-i is located in the cabinet corresponding to (non-human-shaped) image object 860-i. Upon detecting a barcode object, e.g., barcode object 109-i, in only one of the image regions (e.g., in image region 820a), central processing unit 146, in conjunction with first and second image sensors 120 and 125, is configured to read the barcode corresponding to barcode object 109-i and identify the worker to whom corresponding barcode 109b is assigned, i.e., worker 105a, by cross-referencing the worker's personal information with barcode 109b in the database. Furthermore, central processing unit 146, in conjunction with first and second image sensors 120 and 125, is configured to search the database for additional barcodes assigned to worker 105a, including, in this example, barcode 115a associated with safety helmet 110a. This identification can be done automatically because system 100 includes a database in which the personal information of a given worker is cross-referenced with all barcodes associated with PPE worn by that worker so that when the database is searched using a single barcode, additional barcodes having the same personal information as the personal information corresponding to that barcode can be identified within the database.The central processing unit 146, together with the first and second image sensors 120 and 125, is further configured to evaluate each of the image regions 820a, 820b, 820c, 825a, 825b, and 825c for the presence of barcode objects 115a-i and 109a-i corresponding to the identified additional barcodes 115a and 109a, and to apply a probabilistic algorithm based on the prevalence of the barcode objects 115a and 109a corresponding to the barcode objects 115a-i and 109a-i detected in the image regions 820a, 820b, 820c, 825a, 825b, and 825c. The probabilistic algorithm further includes a predetermined probability threshold, which may vary in different implementations. Thus, for example, the predetermined probability threshold may be set such that the prevalence (the proportion of the number of barcode objects per barcode in the total image region) is at least 33%. Therefore, each of the barcodes assigned to worker 105a must be identified in at least 33% of image regions 820a, 820b, 820c, 825a, 825b, and 825c. Barcode objects 109a-i corresponding to retrieved barcode 109b are present in three of the six image regions, namely, image regions 820a, 820b, and 820c. However, barcode objects 115a-i corresponding to retrieved barcode 115a are not present in any of the six image regions 820a, 820b, 820c, 825a, 825b, and 825c. Therefore, the determined probability value is 50% for barcode objects 109a-i associated with safety vest 107a, and the determined probability value is higher than the predetermined probability threshold. Therefore, the probability value is 0% for the barcode object 115a-i associated with the safety helmet 110a, and the determined probability value is lower than the predetermined probability threshold. Thus, the central processing unit 146, together with the first and second image sensors 120 and 125, is configured to determine that it is likely that the worker 105a is wearing the safety vest 107a and that it is likely that the worker 105a is not wearing the safety helmet 110a.
[0109] The foregoing represents an example based on image analysis of six image regions to detect two different barcodes. It should be apparent that the number of image regions, as well as the number of barcodes, can vary depending on the number of PPE items to be worn by workers in the workspace. Generally, the more image regions analyzed, the more accurate the results will be, especially when there is a change in the position of one or more objects within the image. At the same time, analysis of a larger number of image regions requires more computing power within central controller 145 and / or image sensors 120 and 125. The number of image regions that can be analyzed can vary. In some implementations, the number of image regions can be predefined to be from at least 10 images captured by each image sensor 120 and 125. Furthermore, the time period over which images are captured can range from about 5 seconds to about 60 seconds, for example, as controlled by a timer (not shown) that can be included in system 100. Thus, separation between a user and a PPE item for a certain amount of time (e.g., 10 seconds) and / or physical separation between a user and a PPE item over a relatively short distance (e.g., 1 meter) may be detected. For example, if the images shown in FIGS. 8A-8F correspond to an 8-second time period, separation would be detected by system 100 even if safety helmet 110a was placed in a cupboard (corresponding to 860-i) by worker 105a one second before the capture of the images shown in FIGS. 8A-8B and then put back on one second after the images shown in FIGS. 8F-8G were captured. Similarly, if the image region of FIGS. 8A-8F corresponds to a distance of approximately 0.5 meters away from a detected human-like object in the image region and safety helmet 110a was placed one meter away from worker 105a outside the image region (not in the cupboard), separation would be detected by system 100.
[0110] The central processing unit 146 is further configured to execute an associated safety action when the calculated probability value of the detected barcode object for a given PPE item is lower than a predetermined probability threshold, meaning that the worker does not have the given PPE item. Thus, for example, the central processing unit 146 can activate the signaling component indicated by the mobile phone 135 by issuing an alert signal. In other implementations, the signaling component can be another electronic device capable of issuing an alert signal, such as a tablet computer. The physical form of the alert signal can vary, for example, but not limited to, an optical signal, an audio signal, or a vibration signal, or a combination thereof, or any other detectable alert signal. In some implementations, a different strength of alert signal (i.e., a louder audio signal or a stronger vibration signal) can be issued depending on the amount of time the worker is not wearing the given PPE item. This amount of time can be determined based on the time interval over which images are captured and analyzed to probabilistically determine that the worker is without the given PPE. In general, the system may be configured to allow the central controller 145 to take safety action when there is a high probability that a worker is not wearing a given personal protective equipment item in accordance with safety regulations applicable to the workspace based on the calculated probability value.
[0111] Additionally, in some implementations, output data is provided to an output device as part of the execution of the safety behavior. Thus, the output data may be provided in the form of a safety report, an incident report, an accident report, etc. The output data and database (not shown) may include information about the PPE, e.g., first and last use time and / or date, information about individual users of the PPE, e.g., user identity and / or user location, information about the work site, information about barcodes used with the PPE, compliance observations, compliance history, safety violations, and safety actions taken, and the output data may be provided in a variety of formats, including, for example, a printed report, a photo format (e.g., JPEG, PNG, GIF), or a video format (e.g., MPEG, AVI, MP4). The output data can be used for a wide range of purposes, such as to develop strategies to improve the safety behavior of workers using personal protective equipment, to prevent or limit incidents or accidents or injuries to workers in hazardous work environments, or to assist in the investigation of incidents or accidents.
[0112] Referring again to FIG. 3 , in some implementations, a system according to the present disclosure can include an auxiliary data processing unit. Accordingly, the system 300 includes an auxiliary processing unit 180 configured to acquire auxiliary data from the first and second image sensors 120 and 125, where the auxiliary data can assist algorithmic image analysis, including metadata. The metadata can include, for example, sensor-related information, such as a sensor identifier (e.g., a unique identifier associated with the sensor 125 or the sensor 120), sensor positioning information, such as the position and / or relative angle of the image sensors 120 and 125, and timecode information, such as the time the image was captured, as well as other metadata. Such metadata can be useful for processing or for human review of the images. The central controller 145 is configured to acquire the auxiliary data from the auxiliary processing unit 180 via the network 130.
[0113] It should be noted that in some implementations, additional sensors may be included. Thus, for example, 2, 3, 4, 5, 6, 7, 8, 9, 10, or more sensors configured to continuously or periodically detect humanoid and barcode objects in a given workspace. Even more sensors may be installed, particularly in larger or more crowded workspaces. Most preferably, a sufficient number of sensors are installed to monitor substantially the entire workspace.
[0114] In some implementations, system 100 may further include a recording device 170, such as, for example, a network video recorder (NVR) or a digital video recorder (DVR), for recording image data, which is coupled to a data storage device. The data storage device may be included in central controller 145 or may be a separate data storage device, for example, included in recording device 170. Note that in some implementations, image data may be stored in a lower resolution image, for example, in Joint Photographics Expert Group (JPEG) format, compared to the image originally captured by image sensors 120 and 125, to limit storage space usage. In this regard, central processing unit 146 may be configured to convert the images captured by sensors 120 and 125 to a lower resolution format, such as JPEG format, before the images are recorded by recording device 170. The stored image data may be made available for analysis via central controller 145, including, for example, for human evaluation to assess, correct, adjust, verify, or improve the performance of system 100, or for analysis by computational neural networks, as further described below.
[0115] 4, 5A, and 5B, in some implementations, the system 400 may be configured to include an electronic access control system, i.e., a system capable of controlling access to the workspace 155 by, for example, personnel (105a, 105b). Accordingly, the electronic access control system 410 may be located at an enclosure perimeter 456 that defines the workspace 155 and separates the workspace 155 from an exterior 520. The enclosure perimeter 456 may be, for example, a fence, wall, or other barrier that generally prevents personnel from entering the workspace 155. In this example, the electronic access control system 410 includes a retractable gate 515 that provides personnel access to the workspace 155 under certain conditions via an entrance 530, and a gate control system 525 that includes a camera 550.
[0116] For example, in the implementation shown in FIGS. 5A-5B, a worker's access to workspace 155 is controlled by two keys: a first barcode key and a second biometric key.
[0117] 5A-5B, gate control system 525 is coupled to central control unit 145 via network 130. Gate control system 525 is further coupled to gate lock 565 and includes a processor (not shown) for controlling its operation and communicating with central controller 145. In order for an operator to open gate 515 and enter through entrance 530, in a first authentication step, the operator first presents a barcode, such as a 2D barcode, to the lens of camera 550. This barcode may also be referred to as a security or identification barcode. The barcode may be presented in any form, such as a card, an ID tag, or a barcode printed on a screen, such as a telephone screen. When a barcode is presented, gate control system 525 is configured to communicate with central control unit 145 to retrieve previously stored biometric data, such as facial recognition biometric data, that matches the presented barcode. In a second authentication step, gate control system 525 then records the worker's biometric data captured via camera 550 and communicates with central control unit 145 when the worker is positioned in front of gate 515 in (potentially marked) area 520b, which then attempts to match the recorded biometric data with the stored biometric data. If the stored biometric data matches the recorded biometric data, gate lock 565 is unlocked and the worker can pass through gate 515 and access workspace 155 via entrance 530. Conversely, if the stored biometric data does not match the recorded biometric data, the gate lock 565 remains locked and access to the workspace 155 is denied. It will therefore be apparent that opening the gate 515 requires the presentation of a bar code and verification of the biometric data in order for the worker to pass through the openable gate 515.
[0118] Gate control system 525 further includes light 545 to facilitate recording of biometric data via camera 550. For example, light 545 can be controlled to provide a desired level of illumination to make it easier for camera 550 to capture an image of the barcode and obtain biometric data from the worker.
[0119] Gate control system 525 further includes audio speaker 555 and microphone 565, which may be used to provide instructions to the worker regarding their position relative to the lens of camera 550, to enable proper capture of biometric data from the worker, to enable the worker to communicate with support or security staff, to provide audio instructions to the worker, and / or to record audio data.
[0120] 5C , in an alternative implementation, gate control system 525-b may include a processor (not shown) for controlling its operation and communicating with central controller 145, and may be configured to include a temperature detection device 580 coupled to central controller 145. Temperature detection device 580 is positioned and configured to detect the body temperature of a worker (not shown) located within presentation space 520b. Temperature detection device 580 may have a temperature sensor 581 that requires physical contact with the worker within presentation space 520b, for example, by physical contact between the worker's finger and temperature sensor 581. Instructions to the worker for establishing such contact may be provided by audio speaker 555. However, more preferably, temperature detection device 580 may include a temperature sensor 581 that enables remote temperature detection, i.e., a temperature sensor that does not require physical contact between the worker and temperature sensor 581, such as an infrared temperature scanning device that can operate at a distance of several inches from the worker's forehead or other sensing zone of the worker.
[0121] Additionally, in some implementations, the temperature detection device 580 can be installed so as to be fixed and positioned in a fixed location to enable temperature detection of a worker positioned within the presentation space 520b shown in Figure 5C. In other implementations, the temperature detection device 580 can be a portable device, including a handheld device, that can be operated by another person when the worker is positioned within the presentation space 520b.
[0122] The temperature detection device 580 may include any temperature scanner, thermometer, or other device for reading a human body temperature, including any temporal temperature scanner, i.e., a temperature scanner for detecting body temperature more or less continuously as a function of time. Temperature detection devices that can be used in accordance with the present invention include, for example, the temperature detection devices described in U.S. Patent No. 8,282,274.
[0123] Generally, the temperature detection device 580 can be configured to detect the body temperature of the worker in the presentation space 520b and then transmit the detected body temperature to the central controller 145. The central controller 145 can be configured to transmit a signal to unlock the electronic lock 565, such that when the worker's detected body temperature does not deviate from that of a healthy individual, e.g., when the worker's body temperature does not exceed an acceptable predefined body temperature range, such as a temperature range including temperatures of 37°C, 37.5°C, 38°C, or 38.5°C, the gate 515 is opened, allowing the worker access to the workspace 155. Conversely, when the worker's detected body temperature deviates from that of a healthy individual, e.g., exceeds 37°C, 37.5°C, 38°C, or 38.5°C, the central controller 145 is configured not to transmit a signal to the electronic lock 565, so that the worker remains locked even if he or she passes other security checks. In an alternative embodiment, communication with the central controller 145 is not required, and the gate control system 525-b and its processor may be configured to perform temperature checks and control the electronic lock 565. Thus, access to the workspace 155 may be restricted to workers who do not exhibit an elevated body temperature, for example, and only workers exhibiting a predefined body temperature range, e.g., approximately 36.5°C to approximately 38.5°C, may be permitted to enter the workspace 155. Furthermore, the central controller 145 may be configured to notify the worker in the presentation space 520b of the detected body temperature via the audio speaker 555. If a worker is denied access to the workspace 155 as a result of a detected abnormal body temperature, the worker may undergo further medical examination, if necessary. Thus, this exemplary implementation may be implemented to control the spread of a contagious disease that causes an elevated body temperature in a worker whose body temperature is not within a predefined body temperature range.
[0124] It should be noted that in some implementations, the temperature detection device 580 may be configured to be operable in conditions where substantial fluctuations in ambient temperature may occur, for example, due to changing or severe weather conditions. In this regard, the temperature detection device may be configured to compensate for fluctuations in ambient temperature. For example, if a worker arrives to enter a workspace in cold winter temperatures, the temperature detection device 580, in conjunction with the central controller 145, may be configured to compensate the detected body temperature upward. Similarly, if the worker is in warm summer temperatures, the temperature detection device 580, in conjunction with the central controller 145, may be configured to compensate the detected body temperature downward. Such compensation is particularly desirable when the temperature detection device measures / detects skin surface temperature.
[0125] It should be noted that the central controller 145 may be configured to enable the temperature detection device 580 to detect the worker's body temperature before the authentication step is performed, during the first and / or second authentication steps, or after the second authentication step is performed.
[0126] In some implementations, the temperature detection device 580 may be an infrared sensor configured to detect a thermal profile based on the thermal contour of a worker within the presentation space 520b. In such implementations, the detected thermal profile is transmitted to the central controller 145, which may be configured to send a signal to unlock the electronic lock 565, thus allowing the gate 515 to be opened and the worker to access the workspace 155 only when the detected thermal profile matches some characteristic or attribute of an actual worker within the presentation space 520b who does not have an abnormal temperature. Thus, for example, if the central controller 145 attempts to circumvent authentication during an authentication step requiring the worker's facial recognition biometric data by presenting an inanimate object, which may be, for example, a photograph of the face of a worker with permission to access the workspace, the detection of the inanimate object's thermal profile by the temperature detection device 580 will result in a detected thermal profile that is inconsistent with the presence of an actual worker, and as a result, the central controller 145 will not unlock the electronic lock 565.
[0127] Note that in some implementations, the unlocking of gate 515 is not automatic, but instead is performed by another person, such as a security guard. Thus, when a worker presents a barcode to camera 550, a previously recorded facial image corresponding to the barcode may be presented to the security guard, who may be remotely located. The security guard may then be required to verify that the worker seen via camera 550 corresponds to the previously recorded facial image before unlocking gate 515. Note that this authentication process allows a single security guard to control access by workers through multiple gates to a single work site, or through separate gates at multiple work sites.
[0128] The audio speaker 555 and microphone 565 may also be used to implement additional steps for providing access via the gate 515. Thus, for example, when comparing stored recorded biometric data, the gate control system 525 may prompt the submitting worker via the speaker 555 to close their eyes. The biometric system may then re-record the biometric data and compare the previously recorded biometric data (eyes open) with the newly recorded biometric data (eyes closed) to ensure that, except for the eyes being closed, the biometric data are identical. This additional step may prevent a worker from fraudulently submitting a facial image or photograph to the system and having the system use the image to attempt to obtain the recorded biometric data.
[0129] 4, the electronic access control system 410 further includes a recording device 170, such as an NVR or DVR, for example, to record audio and / or video of each worker entry or attempted entry into the workspace 155. The video recording can be used as needed, for example, to confirm worker entry when the first worker opens the gate 515, or to evaluate possible entry attempts by multiple workers.
[0130] Similarly, in at least one implementation, gate sensor 570 is used to verify that a worker has entered through gate 515 and that no other individuals have followed the first worker. In some implementations, gate sensor 570 can be a camera to detect the number of individuals passing through gate 515.
[0131] Note that in some implementations, camera lens 550 is positioned adjacent to gate 515 to enable recording of biometric data when the worker is positioned in area 520b in front of gate 515 and also to enable video recording of the worker upon entry through gate 515.
[0132] 10 , in some implementations, system 1000 may be configured to enable identification within workspace 155 of PPE and non-PPE objects present within workspace 155 using first and / or second multiple images captured by image sensors 120 and / or 125. In this regard, PPE, exemplified by safety vests 107a and 107b and safety helmets 110a and 110b, are separately tagged with barcodes 109a, 109b, 115a, and 115b. Additionally, non-PPE, exemplified by wheelbarrows 133 and 1033, are separately tagged with barcodes 1010 and 1020, respectively. The barcodes include object information that identifies the object; i.e., barcodes 1010 and 1020 include object information that enables objects 133 and 1033 to be identified as wheelbarrows. System 1000 is configured such that central processing device 1046 is coupled to input / output device 1030 to enable a user to input barcodes (e.g., barcodes 109a, 109b, 115a, 115b, 1010, and 1020) as query terms via user input providing input data. Thus, a user can provide input data, for example, by scanning a copy of barcode 109a, 109b, 115a, 115b, 1010, or 1020 using a barcode scanner (not shown), or by selecting barcode 109a, 109b, 115a, 115b, 1010, or 1020 in a database in which the same barcode has been entered. The database may be housed by the central controller 145 or may be provided by a separate data store accessible by the central controller 145. Image sensors 120 and / or 125, together with central processing unit 146, are further configured to detect barcode objects in the first and / or second plurality of images and read the corresponding barcodes as described in the previous embodiment.Thus, multiple barcode objects may be detected and barcodes corresponding to the barcode objects may be read in each image captured by image sensor 120 and / or 125. For example, barcodes 109a, 109b, 115a, 115b, 1010, and 1020 may be read in an image from the first or second plurality of images captured by image sensor 120 of workspace 155. The read barcodes may be stored in a relational database, the database implemented such that each read barcode is linked to the particular image in which it was read, for example, using unique metadata of the image acquired by auxiliary processing unit 180 to access it. Thus, a particular image of workspace 155 captured by image sensor 120 (referred to herein as "Image 1" for example) may be stored in a relational database, such as a relational database contained by central controller 145, along with the time Image 1 was taken (present in the metadata), for example, 11:01 AM on May 7, 2021, and barcodes 109a, 109b, 115a, 115b, 1010, and 1020.
[0133] In some implementations, the scanned barcode may be incorporated as part of the metadata with the image from which the barcode was scanned. It should be understood, therefore, that the metadata included with the scanned barcode along with the stored image may be stored to form a relational database linking the scanned barcode to the particular image from which it was scanned, as well as other metadata, such as the time the image was captured. As previously mentioned, in some implementations, the image may be stored using recording device 170 in a lower-resolution format. In this regard, it should be noted that, because scanning barcodes may be more difficult to perform on lower-resolution images, it is preferable to configure central processing unit 146 to detect and scan barcodes before proceeding with reducing resolution levels and image storage.
[0134] The central processing unit 146 is further configured, upon receiving the input query bar code, to compare the read bar codes in the relational database with the input query bar code, and, when a match is identified between one of the read bar codes and the query bar code, to provide an output to the input / output device 1030. The input / output device 1030 may be implemented as described above for the input and output devices 140 and 150. Thus, for example, when a user inputs barcode 1010 into input / output device 1030 as a query barcode, input / output device 1030 provides an output when a match is identified between one of the stored scanned barcodes in the relational database and the query barcode, and the output may be an image of workspace 155 captured by image sensor 120 or 125 linked to the matching barcode in the relational database, the image including a barcode object corresponding to the query barcode.
[0135] In some implementations, central processing unit 146 is configured to identify one image corresponding to a match (i.e., having a barcode object) between a stored barcode and a query barcode. For example, the match may correspond to a given image most recently captured by image sensor 120 or 125 relative to the time of input of the query barcode, the given image including the barcode object corresponding to the query barcode. In other implementations, central processing unit 146 is configured to identify up to five image matches, up to ten image matches, or up to fifty image matches going further back in time, or all image matches within a time period, such as from the most recently captured image relative to the time of input query barcode to all images captured in the preceding hour, day, or week.
[0136] The output provided by input / output device 1030 preferably includes a time indicator corresponding to the time each matched image was captured by image sensor 120 and / or 125, and may further include the matched images including the corresponding barcode object.
[0137] Thus, by way of example, system 1000 may be configured so that the output provided by input / output device 1030 includes (a) a time indicator corresponding to the time a given image was captured by image sensor 120 and included a barcode object that matches the query barcode, and (b) the given image including the corresponding barcode object at the time a match was identified. System 1000 may be further configured so that one match is identified, i.e., the match corresponding to the image most recently captured by image sensor 120 (including a read barcode object that corresponds to the query barcode) relative to the input time of the query barcode. For example, if a user inputs barcode 1010 as a query barcode into input / output device 1030 at 11:49 AM on May 7, 2021, the output may be the timestamp "May 7, 2021, 11:40 AM" and a copy of Image 1 (the last image before the time the query barcode was input, if the image has a barcode object that matches the barcode query), and the output may be shown on the screen of input / output device 1030. Thus, the user is notified by system 1000 that, at least as of May 7, 2021, 11:40 AM, pushcart 1033 is located in workspace 155, and the user can inspect Image 1 to confirm the presence of pushcart 1030. Similarly, if a user inputs barcode 109a as a query barcode into input / output device 1030 at 11:49 AM on May 7, 2021, the output may be the timestamp "May 7, 2021, 11:40 AM" and a copy of Image 1 (the last image before the time the query barcode was input, if the image has a barcode object that matches the barcode query), and the output may be shown on the screen of input / output device 1030. Thus, the user is notified by system 1000 that safety vest 109a, and most likely augmented work 105a, were located in workspace 155 at least as of 11:40 AM on May 7, 2021, and the user can inspect Image 1 to confirm the presence of safety vest 109a and worker 105a.
[0138] Thus, as another example, system 1000 may be configured such that the output provided by input / output device 1030 includes a time indicator corresponding to the time when N images were captured by image sensor 120, where N is an integer, and each of the N images includes a barcode object that matches the query barcode, and the system may be further configured to set N to a predetermined number, such as 5, so that the five most recently temporally acquired images that match are identified, and thus, in this example, the output includes an image match of the most recently captured image by image sensor 120 relative to the input time of the query barcode, including barcode objects that match the query barcode and the four consecutively previous image matches. For example, if a user inputs barcode 1010 as a query term into input / output device 1030 at 11:49 AM on May 7, 2021, the output may be the following time indicators: "May 7, 2021, 11:40 AM," "May 7, 2021, 11:30 AM," "May 7, 2021, 11:20 AM," "May 7, 2021, 11:10 AM," and "May 7, 2021, 11:00 AM." Thus, the user is notified by system 1000 that push cart 133 was located within workspace 155 at least on May 7, 2021, at 11:00 AM, and on May 7, 2021, at 11:40 AM, and at the referenced times therebetween.
[0139] It should now be apparent, therefore, that in this manner system 1000 generally enables a user to determine whether and when personal protective equipment items such as safety vests 107a, 107b, safety helmets 110a and 110b, and / or non-personal protective equipment, wheelbarrows 133 and 1033, were present in a workspace such as workspace 155.
[0140] In further implementations, the input / output device 1030 may be configured to receive, in addition to the query barcode, one or more query time periods in which a user desires to identify images having image objects, such as barcode objects, that match the query barcode. Thus, the query time period may be defined by a first and second time period; for example, a user may provide input to the input / output device 1030 such as "from 11:00 AM on May 7, 2021 to 1:15 PM on May 7, 2021," or a user may provide input to the input / output device 1030 such as "after 1:00 PM today." The central processing unit 146 is configured to analyze the user entry to determine the first and second times of the query time period. Upon receiving the query barcode, the central processing unit 146 is further configured to use the query time period to identify a set of images captured during the query time period, then analyze the set of images to determine those having barcode objects that match the input query barcode with the read barcodes stored in the relational database, for example, preferably by analyzing metadata including the identification of images associated with the read barcodes, and provide output in the form of matched images corresponding to images captured by sensors 120 and / or 125 within the query input time period and / or a time indicator for the time of the matched images.
[0141] Thus, as another example, system 1000 may be configured such that input / output device 1030 receives a query time period in which a user desires to identify any images containing a barcode object that were acquired during the query time period and that can be received by a query barcode provided by the user, and the output provided by input / output device 1030 may be a time indicator indicating one or more times of one or more matching images captured by image sensor 120 during the query time period. For example, if a user inputs barcode 1010 as a query term and "May 7, 2021 11:20 AM - May 7, 2021 11:55 AM" as a query time period into input / output device 1030, central processing unit 146 will attempt to identify matching images as described above, and upon finding a matching image, will be configured to generate an output that may include one of the following time indicators: "May 7, 2021 11:20 AM," "May 7, 2021 11:30 AM," "May 7, 2021 11:40 AM," and "May 7, 2021 11:50 AM." Thus, the user is informed by system 1000 that wheelbarrow 133 was located within workspace 155 at "May 7, 2021, 11:20 AM," "May 7, 2021, 11:30 AM," "May 7, 2021, 11:40 AM," and "May 7, 2021, 11:50 AM," and the user can reasonably conclude that wheelbarrow 133 was likely located within workspace 155 during the entire query time period.
[0142] In another aspect, in at least one implementation according to the teachings herein, the present disclosure also provides a process shown in FIG. 2. Accordingly, referring now to FIG. 2, the present disclosure includes a process 200 for monitoring the use of personal protective equipment in a workspace. Process 200 includes a first step 205 in which a worker is equipped with personal protective equipment that includes a barcode identifying the corresponding personal protective equipment and user information for the worker. The user information may be the worker's name, date of birth, phone number, and / or some other identifying information unique to the worker. The remainder of process 200 involves image capture and image analysis of the worker equipped with the PPE, and, if required, automatic execution of safety actions.
[0143] The second step 210 of process 200 may be initiated in a variety of ways, such as by the worker using the personal protective equipment initiating the authentication process by using a handheld device communicatively coupled to a central controller. In another implementation, the second step 210 may be initiated by the worker interacting with a scanner, such as a barcode scanner, or simply by the worker entering a workspace in which the first and second image sensors are located.
[0144] The process 200 further includes a third step 215 that includes capturing a first plurality of images using first and second image sensors located in the workspace.
[0145] Process 200 further includes a fourth step 220 that includes detecting human-like objects by applying a first image analysis algorithm, such as a human-like object image analysis algorithm or a computational neural network, to each of the captured images. The human-like object image analysis algorithm includes software instructions that configure the processing unit to classify and detect human-like objects in the first plurality of images, as described above.
[0146] Process 200 further includes fifth and sixth steps 225 and 230, which include determining whether at least one human-shaped object was detected in one of the first plurality of images (step 225) and selecting a second plurality of images from the first plurality of images, the second plurality of images including one or more detected human-shaped objects (step 230). In some examples, in step 225, no human-shaped object may be detected in the first plurality of images. For example, this may be when no worker is in the field of view 157 or 158 of image sensor 120 or 125, respectively (step 227). In such a situation, process 200 may then return to step 215 and capture a new image.
[0147] However, once a second plurality of images including one or more detected human-shaped objects is acquired, process 200 proceeds to a seventh step 235 which includes defining image regions each including one of the detected human-shaped objects in each of the second plurality of images. Techniques for defining image regions have been described above.
[0148] Process 200 then proceeds to an eighth step 240 which involves detecting barcode objects within the defined image regions by applying a second image analysis algorithm, such as a barcode object image analysis algorithm or computational neural network configured to classify barcode objects, to the second plurality of images, as described above. Alternatively, the use of defined image regions may be optional in other embodiments where a larger region of the entire image of the second plurality of images is analyzed to detect at least one barcode object.
[0149] The process 200 further includes a ninth step 245 that includes selecting the first detected barcode object from the detected barcode objects of the second plurality of images.
[0150] Process 200 further includes a tenth step 250 that includes performing a search using the first barcode, i.e., the barcode corresponding to the first detected barcode object selected in step 245. This search may be performed in a database or data store to find user information linked to the first barcode that identifies the worker corresponding to the detected human-shaped object in the defined image area of the second plurality of images.
[0151] Process 200 further includes an eleventh step 255 that includes retrieving additional barcodes and corresponding additional barcode objects that may be linked to the identified worker's located user information. These additional barcode objects may be associated with personal protective equipment items that the identified worker should wear while in the workspace.
[0152] The process 200 further includes a twelfth step 260 that includes applying a probabilistic algorithm to determine a prevalence of detected barcode objects in the defined image region of the second plurality of images, the detected barcode objects including the first detected barcode and an additional barcode linked to the user information found for the first detected barcode, and calculating, based on the prevalence, a probability that a user corresponding to the detected humanoid object is wearing each piece of personal protective equipment that should comply with safety rules applicable to the workspace, as described above. Thus, there is a probability value calculated for each barcode object.
[0153] The process 200 further includes a thirteenth step 265 that includes determining whether the calculated probability is below a predetermined probability threshold. If a given barcode object has a calculated probability that is lower than the predetermined probability threshold, this means that the worker is likely not wearing the PPE item associated with the given barcode object.
[0154] The process 200 further includes a fourteenth step 270 that includes performing a safety action for each barcode object with a calculated probability lower than a predetermined probability threshold. The safety action is based on safety rule data stored in a database or data store. Examples of safety actions are described above.
[0155] Note that steps 230-270 are repeated for each detected humanoid object in the first plurality of images that corresponds to a unique worker. Thus, for each worker in the first plurality of images, a probability calculation is performed for each barcode object associated with each worker.
[0156] Once process 200 is completed, by performing at least one safety action (step 270), or by not performing such an action, process 200 may be repeated beginning with step 215, such that process 200 is repeated for a different first plurality of images acquired at a subsequent time.
[0157] As previously described herein, the automated system of the present disclosure in one implementation may also be configured to enable identification of PPE and non-PPE present within the workspace using the first and second plurality of images. Accordingly, in another aspect, the present disclosure further provides the process shown in FIGS. 11A and 11B.
[0158] 11A, the present disclosure includes a process 1100 for identification within a workspace of PPE and non-PPE items present within the workspace. Process 1100 includes a first step 1105 in which a user provides an input query barcode. The remainder of process 1100 involves, for example, identifying matching barcodes in the first and / or second plurality of images captured by process 200 and providing output in the form of a time indicator of the time the image was captured if the image contains a matching barcode object and / or the output can be a matching image.
[0159] Process 1100 further includes a second step 1110 that includes matching the query barcode with one of the read stored barcodes. As previously described, a barcode object in an image captured by an image sensor may be identified, and then a barcode may be read for the identified barcode object. The read barcode may then be stored in a relational database that links the read barcode to the particular image captured by the image sensor that contains the barcode object from which the barcode was read. The query barcode received in step 1105 from user input may then be matched with the stored read barcodes. If a match is found, process 1100 proceeds to step 1120. If no match is found, process 1100 returns to step 1105.
[0160] Process 1100 further includes a third step 1115 that includes receiving an image corresponding to (i.e., linked to) a stored barcode that matches the query barcode and / or the time of the actual image itself. The time and / or image of the matching stored barcode may, for example, be the most recently captured image relative to the time the input query barcode was received in step 1105.
[0161] Process 1100 further includes a fourth step 1120 that includes providing output in the form of a time indicator corresponding to the time the image containing the barcode matching the query barcode was captured and / or the image itself. Process 1100 may then return to step 1105 to allow the user to provide further input in the form of a query barcode.
[0162] 11B, the present disclosure includes a further process 1101 for identifying PPE and non-PPE present within a workspace. Process 1101 includes a first step 1125 in which an input query barcode and a query time period defined by a first time and a second time are received via user input. The remainder of process 1101 involves identifying matching barcodes within the query time period, for example, in the first and / or second plurality of images captured by process 200, and providing output in the form of a time indicator including one or more times and / or one or more images including the query barcode.
[0163] Process 1101 further includes a second step 1130 that involves matching the query barcode with barcodes read and stored within the query time period. As previously described, a barcode object may be identified in an image captured by an image sensor, and then a barcode may be read from the identified barcode object. The read barcode may be stored in a relational database that links the read barcode to a particular image containing the barcode object corresponding to the read barcode. The query barcode received in step 1125 may be matched with the stored barcode. If a match is found, process 1101 proceeds to step 1135. If a match is not found, process 1101 returns to step 1125, at which point the user can provide further input in the form of another query barcode.
[0164] Process 1101 further includes a third step 1140 that includes receiving an image corresponding to (i.e., linked to) a stored barcode that matches the query barcode and / or the time of the actual image itself. The time and / or image of the matching stored barcode may, for example, be the most recently captured image relative to the time the input query barcode was received in step 1125.
[0165] Process 1101 further includes a fourth step 1140 that includes providing output in the form of a time indicator corresponding to the time an image containing a barcode matching the query barcode was captured and / or the image itself and the query time period. Process 1101 may then return to step 1125 to allow the user to provide further input in the form of a query barcode.
[0166] Thus, the automated process described above can monitor the use of personal protective equipment and non-personal protective equipment in a workspace.
[0167] While the applicant's teachings described herein are associated with various implementations for illustrative purposes, the applicant's teachings are not intended to be limited to such implementations. Rather, the applicant's teachings as described and illustrated herein encompass various alternatives, modifications, and equivalents without departing from the implementations described herein, the general scope of which is defined in the appended claims.
Claims
1. 1. A system for automatically monitoring the use of personal protective equipment by workers in a workspace, the system comprising: a plurality of barcodes, each barcode associated with a unique PPE item and including object information identifying the associated PPE item and user information for a given worker using the PPE item; first and second image sensors installable within the workspace, the first and second image sensors being spaced apart from one another and positioned to cover different first and second fields of view, respectively, the first and second fields of view overlapping within a given area within the workspace, the first and second image sensors being adapted to collectively capture a first plurality of images of the given area, the first plurality of images comprising a first set of temporally consecutive images captured by the first image sensor during a given time period and a second set of temporally consecutive images captured substantially simultaneously by the second image sensor during the given time period; a central controller coupled to the first and second image sensors, the central controller including at least one processor and a memory element including a database configured to store the plurality of barcodes and the user information such that one or more barcodes are linked with the user information for the given worker; Including, the at least one processor and the first and second image sensors together: (i) detecting a human-like object in the first plurality of images; (ii) identifying a second plurality of images from the first plurality of images, the second plurality of images including the detected human-like object, the second plurality of images including at least a portion of the first and second sets of temporally consecutive images; (iii) detecting a first barcode object in at least one of the second plurality of images, the first barcode object being associated with the detected human-like object; and (iv) identifying a first barcode corresponding to the detected first barcode object; (v) performing a search in the database using the first barcode to identify the user information linked thereto, which is then used to identify additional barcodes in the database linked to the identified user information; (vi) detecting additional barcode objects in the second plurality of images that correspond to the identified additional barcodes; and (vii) applying a probabilistic algorithm to calculate a probability that the worker corresponding to the detected human-shaped object is wearing each PPE item associated with the first detected barcode and the identified additional barcode in accordance with safety regulations applicable to the workspace, wherein the probabilistic algorithm is applied to determine a prevalence of each barcode object corresponding to the first detected barcode and the identified additional barcode from the database detected in each image from the second plurality of images; (viii) performing a safety action for each of the first detected barcode and additional barcode objects having a calculated probability lower than a predetermined probability threshold; A system that is configured to:
2. 2. The system of claim 1, wherein any of the barcode objects are detected in defined image regions within the second plurality of images, each defined image region being constructed to include the detected human-shaped object.
3. The system of claim 1 , wherein the given period of time is from about 5 seconds to about 60 seconds.
4. The system of claim 3 , wherein the first and second sets of images each include at least 10 images.
5. 2. The system of claim 1, wherein the first and second image sensors are spaced and angled such that an intersection between a first axis and a second axis extending centrally through the fields of view of the first image sensor and the second image sensor, respectively, forms an angle that spans between about 15 degrees and about 175 degrees, or between about 205 degrees and about 345 degrees.
6. The system of claim 5 , wherein the angle ranges between about 30 degrees and about 150 degrees, or between about 210 degrees and about 330 degrees.
7. 2. The system of claim 1, wherein the human-shaped object is detected by applying a human-shaped object image analysis algorithm to the first plurality of images, and / or the barcode object is detected by applying a barcode object image analysis algorithm to the second plurality of images.
8. 3. The system of claim 2, wherein the defined image region is constructed using the frame having an image boundary that encompasses the entirety of the detected human-shaped object within the frame, and the image boundary is formed such that there is no contact between the detected human-shaped object and the image boundary.
9. The system of claim 8 , wherein the image boundary corresponds to a distance of from about 0.5 meters to about 3 meters from the detected human-like object within the image region.
10. 2. The system of claim 1, wherein the central controller is coupled to an input device, the input device configured to receive user entries in the form of a query barcode; the central controller and the first and second image sensors are further configured to detect a plurality of barcode objects and read barcodes corresponding to the detected barcode objects in each of the first and second plurality of images; and store the read barcodes in a second database configured to store a plurality of read barcodes and images together with the time the image was captured by the first and / or second image sensors by using a linking relationship such that one or more read barcodes are linked to one of the images and the time the image was captured by the first and / or second image sensors; and the central controller is further configured to determine when there is a match between the query barcode and one of the read stored barcodes.
11. 11. The system of claim 10, wherein the central controller is coupled to an output device, and the central controller is configured to provide an output to the output device when the query barcode is identical to a read stored barcode.
12. 12. The system of claim 11, wherein the output includes a time indicator corresponding to the time the image linked to the stored barcode matching the query barcode was captured by the first and / or second image sensor.
13. The system of claim 11 , wherein the output includes the image linked to the stored barcode that matches the query barcode.
14. 12. The system of claim 11, wherein the output includes a time indicator indicating the time when the image linked to the stored barcode that is identical to the query barcode was captured by the first and / or second image sensors, and / or the output can include the image linked to the stored barcode that is identical to the query barcode, the time indicator or the linked image corresponding to the most recent time when a match between the query barcode and the read stored barcode was identified relative to the time when the query barcode was entered.
15. 11. The system of claim 10, wherein the input device is further configured to receive user input of a query time period defined by a first time and a second time, and the central controller is configured to detect the plurality of barcode objects and their corresponding barcodes in a set of images captured by the first and / or second image sensors during the query time period.
16. The system of claim 10 , wherein the system includes a plurality of barcodes associated with objects other than personal protective items, the barcodes including object information identifying the objects.
17. 1. An automated process for monitoring the use of personal protective equipment by workers in a workspace, the automated process comprising: acquiring a first plurality of images using first and second image sensors installable within the workspace, the first and second image sensors being spaced apart from one another and positioned to cover different first and second fields of view, respectively, the first and second fields of view overlapping within a given area within the workspace, the first plurality of images including a first set of temporally consecutive images captured by the first image sensor during a given time period and a second set of temporally consecutive images captured substantially simultaneously by the second image sensor during the given time period; Detecting a human-like object of a given worker in the first plurality of images; identifying a second plurality of images from the first plurality of images, the second plurality of images including the detected human-like object, the second plurality of images including at least a portion of the first and second sets of temporally consecutive images; detecting a first barcode object in at least one of the second plurality of images, the first barcode object being associated with the detected human-like object; identifying a first barcode corresponding to the detected first barcode object; performing a search in the database using the first barcode to identify user information linked thereto, which is then used to identify additional barcodes in the database linked to the identified user information, the database storing a plurality of barcodes and user information using linking relationships such that one or more barcodes are linked to the user information of the worker; detecting additional barcode objects in the second plurality of images corresponding to the identified additional barcodes; applying a probabilistic algorithm to calculate a probability that the given worker corresponding to the detected human-shaped object is wearing each PPE item associated with the first detected barcode and the identified additional barcode in accordance with safety regulations applicable to the workspace, wherein the probabilistic algorithm is applied to determine a prevalence of each barcode object corresponding to the first detected barcode and the identified additional barcode from the database detected in each image from the second plurality of images; performing a safety action for each of the first detected barcode and additional barcode objects having a calculated probability lower than a predetermined probability threshold; Automated processes, including:
18. 18. The automated process of claim 17, wherein any of the barcode objects are detected in defined image regions within the second plurality of images, each defined image region being constructed to include the detected human-shaped object.
19. 18. The automated process of claim 17, wherein the first and second image sensors are spaced and angled such that an intersection between a first axis and a second axis extending centrally through the fields of view of the first image sensor and the second image sensor, respectively, forms an angle that spans between about 15 degrees and about 175 degrees, or between about 205 degrees and about 345 degrees.
20. The automated process of claim 17, wherein a central controller is coupled to an input device, the input device configured to receive user entries in the form of a query barcode, the central controller and the first and second image sensors are further configured to detect a plurality of barcode objects and read their corresponding barcodes in each of the first and second plurality of images, and store the read barcodes in a second database configured to store a plurality of read barcodes and images together with the time at which the image was captured by the first and / or second image sensors by using a linking relationship such that one or more read barcodes are linked to the image from which the one or more read barcodes were obtained and the time at which the image was captured by the first and / or second image sensors, and the central controller is further configured to determine whether there is a match between the query barcode and one of the read stored barcodes.
Citation Information
Patent Citations
Systems and methods for monitoring personal protection equipment and promoting worker safety
US20140307076A1
Personal protective equipment system using optical articles for integrated monitoring, alerting, and predictive safety event avoidance
WO2018152475A1