Judgment device, judgment system, judgment method, program, and storage medium

Abstract

The present invention provides a judgment device, judgment system, judgment method, program, and storage medium that can suppress the occurrence of occupational accidents. [Solution] The determination device according to the embodiment detects people and objects from an image and acquires identification information of people read by a reading device. The determination device associates the detected person's identification information with the person's information registered in the database. The determination device identifies the detected object. The determination device extracts elements used to determine the level of danger from the location of the person calculated from the image, the location of the object calculated from the image, the person's information, and the object's information registered in the database. The determination device compares the extracted elements with the criteria for determining the level of danger, and if it determines that there is a danger, it transmits safety instructions to avoid the danger.

Description

Technical Field

[0001] Embodiments of the present invention relate to a determination device, a determination system, a determination method, a program, and a storage medium.

Background Art

[0002] At a work site, industrial accidents can occur. There is a need for technology to suppress the occurrence of industrial accidents.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The problem to be solved by the present invention is to provide a determination device, a determination system, a determination method, a program, and a storage medium that can suppress the occurrence of industrial accidents.

Means for Solving the Problems

[0005] The determination device according to the embodiment detects a person and an object from an image, and acquires identification information of the person read by a reading device. The determination device associates the detected person with the identification information and the information of the person registered in a database. The determination device identifies the detected object. The determination device extracts elements used for risk determination from the position of the person calculated from the image, the position of the object calculated from the image, the information of the person, and the information of the object registered in the database. The determination device compares the extracted elements with risk determination conditions, and when it is determined that there is a risk, transmits a safety instruction for avoiding the risk.

Brief Description of the Drawings

[0006] [Figure 1] Figure 1 is a schematic diagram showing the configuration of the determination system according to the embodiment. [Figure 2] Figure 2 is a schematic diagram showing the function of the determination device according to the embodiment. [Figure 3] Figure 3 is a schematic diagram showing an example of an image. [Figure 4] Figure 4 is a schematic diagram showing another example of an image. [Figure 5] Figure 5 is a table illustrating the data output by the detection unit. [Figure 6] Figure 6 is a diagram illustrating the processing performed by the tying mechanism. [Figure 7] Figure 7 is a diagram illustrating the processing performed by the extraction unit. [Figure 8] Figure 8 is a diagram illustrating the processing performed by the determination unit. [Figure 9] Figure 9 is a diagram illustrating the processing performed by the selection unit. [Figure 10] Figure 10 is a diagram illustrating the processing performed by the output unit. [Figure 11] Figure 11 is a schematic diagram showing an example of the output of a terminal device. [Figure 12] Figure 12 is a schematic diagram showing an example of the output of a terminal device. [Figure 13] Figure 13 is a flowchart showing the determination method according to the embodiment. [Figure 14] Figure 14 is a table illustrating the data output by the detection unit. [Figure 15] Figure 15 is a schematic diagram showing an example of the output of a terminal device. [Figure 16] Figure 16 is a schematic diagram showing the function of a determination device according to a modified embodiment. [Figure 17] Figure 17 is a table illustrating the disaster information that is registered. [Figure 18] Figure 18 is a diagram illustrating the update of the extracted element master data. [Figure 19] Figure 19 is a diagram illustrating the updating of the judgment condition master data. [Figure 20] FIG. 20 is a diagram for explaining the update of safety instruction master data. [Figure 21] FIG. 21 is a schematic diagram showing a hardware configuration. MODE FOR CARRYING OUT THE INVENTION

[0007] Embodiments of the present invention will be described below with reference to the drawings. In this specification and the respective drawings, the same reference numerals are given to the same elements as those already described, and detailed descriptions thereof are appropriately omitted.

[0008] In factories, warehouses, etc., objects are handled. For example, in a factory, a person uses production equipment to perform various operations. The operations include processing of members, assembly of semi-finished products or products, conveyance of members, semi-finished products, or products, etc. The production equipment includes devices for processing objects, carts for conveying objects, etc. As the production equipment, devices that can operate autonomously, such as an automated guided vehicle (AGV), an industrial robot, etc., may be used. Also, in a warehouse, a person brings in, conveys, or takes out goods. Here, members, semi-finished products, products, goods, production equipment, etc. are collectively referred to as "objects".

[0009] Accidents (industrial disasters) may occur at work sites such as factories and warehouses. An industrial disaster is a disaster caused by work, and refers to a situation where a worker has an accident at a location related to work. Industrial disasters are classified into various types, such as a person's fall, a person being卷入 by equipment, a collision between a person and an object, an object falling on a person, a fire, an electric shock, an explosion, contact between a person and a harmful substance, etc. Approximately half of these types are caused by the contact between a person and an object. Therefore, in order to avoid the occurrence of industrial disasters at a work site, it is effective to prevent accidents caused by the contact between a person and an object in advance. Embodiments of the present invention are used to suppress the occurrence of industrial disasters caused by the contact between a person and an object. Hereinafter, for the sake of simplicity of explanation, an industrial disaster will be simply referred to as a "disaster".

[0010] FIG. 1 is a schematic diagram showing the configuration of a determination system according to an embodiment. As shown in FIG. 1, the determination system 100 according to the embodiment includes an imaging device 10, a reading device 15, a determination device 20, a terminal device 30, and a storage device 40.

[0011] The imaging device 10 is installed at the work site and images the work site. The imaging device 10 may be provided at a high position so as to be able to image a wider range of the work site. For example, the imaging device 10 is attached to a wall, a pillar, a shelf, or a ceiling. The imaging device 10 transmits the acquired image to the determination device 20. Alternatively, the imaging device 10 may save the image in the storage device 40. In that case, the determination device 20 accesses the imaging device 10 to acquire the image.

[0012] The reading device 15 reads an identifier unique to each person. For example, the reading device 15 is provided at the entrance and exit of the work site. When a person enters or exits the work site, the person causes the reading device 15 to read an identifier associated with their own information. The identifier is a one-dimensional code (barcode), a two-dimensional code (QR code (registered trademark)), a Radio Frequency Identification (RFID) tag, an IC chip, or the like. For example, a person holds an employee ID with an identifier in front of the reading device 15. The reading device 15 reads the identifier and identifies the person entering the work site.

[0013] The reading device 15 may read biometric data. For example, the reading device 15 includes a camera and images a person's face. The reading device 15 inputs an image of the person's face into an identification model. The identification model classifies the face shown in the image. The reading device 15 identifies a person based on the classification result of the identification model. In addition to the face, the reading device 15 can acquire biometric data such as fingerprints, veins, irises, and voices, and may identify a person based on any of the biometric data.

[0014] When a person is identified by an identifier or biometric data, the reading device 15 acquires the identification information of that person. The identification information includes a unique character string (ID) for identifying an individual, a name, and the like.

[0015] The detection device 20 acquires images captured by the imaging device 10 and detects people and objects from the images. The storage device 40 stores various databases necessary for the processing of the detection device 20. The detection device 20 acquires information about the detected people and objects from the storage device 40. The detection device 20 uses the acquired information to determine whether there is a risk to people. If a risk is determined to exist, the detection device 20 sends a notification to the terminal device 30.

[0016] For example, multiple imaging devices 10 are installed at the work site. At least one imaging device 10 images the area where a reading device 15 is installed. As a result, when the reading device 15 reads a person's identification information, that person is detected from the image. The determination device 20 associates the identification information read by the reading device 15 with the detected person. Subsequently, the acquisition of images by the imaging devices 10 and the detection of people by the determination device 20 are repeated. In this way, the person is tracked while they are identified.

[0017] The terminal device 30 is carried by the worker. When the terminal device 30 receives a notification from the judgment device 20, it outputs a notification to the worker. For example, the notification may include information indicating a hazard or instructions to avoid the hazard, which are displayed on the display of the terminal device 30. Alternatively, the notification may include sound, light, vibration, etc., output from the terminal device 30.

[0018] Figure 2 is a schematic diagram showing the function of the determination device according to the embodiment. The invention according to the embodiment will be described in detail below with reference to specific examples. Here, an example of how hazards are determined for workers will be described. Hazards may be determined similarly for people other than workers (passersby, visitors, etc.).

[0019] As shown in Figure 2, the determination device 20 has the functions of a detection unit 21, a linking unit 22, an extraction unit 23, a determination unit 24, a selection unit 25, and an instruction generation unit 26. The storage device 40 stores worker master data 41, work master data 42, object master data 43, extraction element master data 44, determination condition master data 45, and safety instruction master data 46.

[0020] The detection unit 21 detects objects, including people and objects, from the acquired image. Artificial intelligence (AI) image recognition can be used for detecting people and objects. For example, a recognition model including a neural network is used for detecting people and objects. The image is input to the recognition model, and the recognition model outputs the detection results of people and objects from the image. To improve detection accuracy, it is preferable that the recognition model includes a convolutional neural network (CNN).

[0021] Detection distinguishes objects, including people and other objects, from other objects (such as floors, shelves, and desks). The detection unit 21 further performs posture detection on the detected objects. For posture detection, posture detection models such as OpenPose or DarkPose can be used. If a posture is detected, the object is determined to be a person. If no posture is detected, the object is determined to be something other than a person.

[0022] For the detection of people and objects, pattern matching may be used instead of recognition models and posture detection models. In pattern matching, the similarity between an object in an image and a pre-prepared template image is calculated. If a similarity of above a predetermined threshold is obtained between the object and the template image, the object in the image is determined to be the object in the template image.

[0023] Furthermore, the detection unit 21 calculates the positions of the detected people and objects. Preferably, the positions are expressed in three-dimensional coordinates. For example, the positions of people and objects are expressed in three-dimensional coordinates by associating the coordinates of each point in the imaging range of the imaging device 10 with pre-set three-dimensional coordinates. If the imaging ranges of multiple imaging devices 10 overlap, people or objects can be imaged from multiple directions. The three-dimensional coordinates of people and objects may be calculated from multiple images using known methods. If the imaging device 10 includes a depth sensor, the three-dimensional coordinates may be calculated using the depth measured by the imaging device 10 (the distance between the imaging device 10 and the target).

[0024] Through the process described above, the detection unit 21 generates a detection result 21a. The detection result 21a includes objects (people and things) detected from the image, the location of people, the location of things, etc.

[0025] The linking unit 22 performs tasks such as acquiring worker information identified by the reader 15 and linking the worker identified by the reader 15 with the worker detected by the detection unit 21. When acquiring worker information, the worker master data 41 and the work master data 42 are referenced. The worker master data 41 contains information about each worker. This information includes the worker's identification data (ID), name, and the terminal device they are carrying. The work master data 42 stores information about the work, information about the worker in charge of the work, the date and time of the work, and the safety equipment required for the work.

[0026] First, the linking unit 22 obtains the worker ID identified by the reading device 15. The linking unit 22 links the worker information identified by the reading device 15 with the worker detected by the detection unit 21. As a specific example, the linking unit 22 refers to the time when the worker was identified by the reading device 15. From the detection result 21a, the linking unit 22 extracts the worker who was closest to the reading device 15 at the time the worker was identified. The linking unit 22 determines that the extracted worker is the worker identified by the reading device 15 and links the information of those workers.

[0027] Next, the linking unit 22 accesses the worker master data 41 and obtains information such as the name of the identified worker and whether or not they have a terminal device. The linking unit 22 also accesses the work master data 42 and refers to the work assigned to the identified worker. The work master data 42 registers the safety equipment that should be worn for each work. The linking unit 22 obtains information on the safety equipment that should be worn for the work assigned to the identified worker. The identified worker can then be assumed to be wearing the safety equipment registered in the work master data 42.

[0028] Through the above process, worker information such as the worker's name, presence or absence of terminal equipment, and wearing of safety equipment is linked to the identified worker, and worker information 22a is generated.

[0029] The extraction unit 23 identifies the objects detected by the detection unit 21. Pattern matching, classification models, etc., can be used to identify objects. For example, in pattern matching, a template image is prepared for each object. The object in the template image with the highest similarity is identified as the object detected in the image. A classification model classifies the objects in the image into one of several classes. The extraction unit 23 inputs the image of the object into the classification model and obtains the classification result. The object in the image is identified as an object corresponding to the classified class.

[0030] The extraction unit 23 extracts information used for determining hazards from the information obtained by the processing of the detection unit 21, the information obtained by the processing of the linking unit 22, etc. At this time, the extraction unit 23 refers to the object master data 43 and the extraction element master data 44. The object master data 43 contains information about objects placed at the work site. The object information includes ID, weight, size, etc. The extraction element master data 44 registers elements that are extracted from the detection results 21a, worker information 22a, and the object information from the object master data 43, etc. For example, if the distance between a person and an object, the weight of the object, and the height of the object are used to determine hazards, the distance between a person and an object, the weight of the object, and the height of the object are registered in the extraction element master data 44 as elements to be extracted.

[0031] The extraction unit 23 extracts elements used to determine the level of risk from the detection result 21a, worker information 22a, and information contained in the object master data 43 for each object detected by the detection unit 21.

[0032] As an example, the determination of the danger posed by a falling object uses the horizontal distance between the person and the object, the weight of the object, and the height of the object. The height is the distance between the vertical position of the object and the floor. These elements are registered in the extraction element master data 44. The extraction unit 23 calculates the horizontal distance between the person and the object, and the height of the object, using the position of the object and the position of the person included in the detection result 21a, according to the information registered in the extraction element master data 44. The extraction unit 23 also obtains the weight of the object from the information registered in the object master data 43. For each detected object, the extraction unit 23 obtains the distance between the object and the person, the weight of the object, and the height of the object, and generates an extraction result 23a containing this information.

[0033] The determination unit 24 compares the extraction result 23a generated by the extraction unit 23 with the information registered in the determination condition master data 45. The determination condition master data 45 stores multiple types of disasters, determination conditions for each type, and components to be judged. If the information contained in the extraction result 23a satisfies any of the determination conditions, the determination unit 24 determines that there is a risk of disaster corresponding to that determination condition. The determination unit 24 generates a risk determination result 24a.

[0034] When the selection unit 25 obtains the judgment result 24a, it refers to the information in the safety instruction master data 46. The safety instruction master data 46 stores multiple types of disasters and safety instructions for each type. The safety instructions include procedures for preventing disasters. The selection unit 25 selects a safety instruction from the multiple safety instructions that corresponds to a disaster that has been determined to be dangerous. The selection unit 25 generates a selection result 25a that includes the selected safety instruction.

[0035] The instruction generation unit 26 generates a safety instruction 26a based on the worker information 22a and the selection result 25a from the selection unit 25. The instruction generation unit 26 transmits the generated safety instruction 26a to the terminal device 30. In addition to the safety instruction, the notification may also include the ID of the worker determined to be at risk of disaster, the time of determination, the type of disaster, etc.

[0036] Embodiments of the present invention will be described below with reference to more detailed examples.

[0037] Figure 3 is a schematic diagram showing an example of an image. The imaging device 10 images the work site and acquires an image 200, for example, as shown in Figure 3. The detection unit 21 detects people 201 and objects 211-216 from the image 200. At this time, objects of low risk, such as small objects, may be excluded from detection. For example, if a recognition model is used for detection, the recognition model may be pre-trained not to detect objects of low risk. Alternatively, after object detection, the identification process for small objects may be omitted. For example, the extraction unit 23 compares the size of the detected object with a preset threshold, and if the size is less than the threshold, the identification process is omitted. By omitting the detection or identification of small objects, the amount of computation can be reduced and processing can be sped up.

[0038] Figure 4 is a schematic diagram showing another example of an image. Similarly, another imaging device 10 images another work site and acquires the image 220 shown in Figure 4. The detection unit 21 detects people 221 and objects 231-236 from the image 220.

[0039] Figure 5 is a table illustrating the data output by the detection unit. The detection unit 21 assigns an object ID 21a1 to each object detected from images 200 and 220 and registers it. The detection unit 21 then determines whether each detected object is a person or an object. The detection unit 21 associates the determined type 21a2 with the object ID 21a1. The detection unit 21 also assigns a determination ID 21a3, which is assigned to each type, to each object ID 21a1.

[0040] Furthermore, the detection unit 21 calculates the positions of people and objects from the image. The detection unit 21 associates the calculated positions 21a4 with each object ID 21a1. The detection unit 21 may also extract features of the detected objects from the image. For example, the detection unit 21 extracts feature quantities for each object and associates feature data 21a5 containing those feature quantities with each object ID 21a1.

[0041] Based on the above, a detection result 21a is generated, which includes object ID 21a1, type 21a2, discrimination ID 21a3, position 21a4, and feature data 21a5, as shown in Figure 5.

[0042] Figure 6 is a diagram illustrating the processing performed by the tying mechanism. The linking unit 22 acquires the detection result 21a and the reading result 15a from the reading device 15. The reading result 15a includes the worker ID 15a1 of the identified worker and the reading device ID 15a2 for identifying the reading device 15. The linking unit 22 links the information of the reading result 15a with the information of the detection result 21a.

[0043] The linking unit 22 further refers to the information registered in the worker master data 41 and the work master data 42. The worker master data 41 includes the worker ID 41a, name 41b, terminal device 41c, transmission means 41d for the terminal device, and worker image 41e. The worker ID 41a is the ID of each worker who may be present at the work site. The name 41b is the name of each worker. The terminal device 41c indicates whether each worker possesses a terminal device. The transmission means 41d indicates the means for transmitting information to the terminal device. The worker image 41e is the file name of the worker's image. The worker image 41e may be used for pattern matching.

[0044] The work master data 42 includes worker ID 42a, date 42b, assigned process 42c, and safety equipment wearing information 42d. The work master data 42a indicates the ID of the worker assigned to the work. The date 42b indicates the date on which the work is performed. The assigned process 42c is the name of the process of the work to be performed. The wearing information 42d indicates the safety equipment that the worker should wear during the work.

[0045] The linking unit 22 queries the worker ID 15a1 read by the reader 15 against the worker ID 41a registered in the worker master data 41. The linking unit 22 retrieves the information of the queryed worker ID 41a from the worker master data 41. The linking unit 22 also queries the identified person's worker ID 41a against the worker ID 42a registered in the work master data 42. The linking unit 22 retrieves the information of the work performed by the queryed worker ID 42a from the work master data 42.

[0046] The linking unit 22 obtains safety equipment wearing information 42d for the queried work from the work master data 42. The linking unit 22 generates worker information 22a from the detection result 21a, worker master data 41, and work master data 42.

[0047] The worker information 22a includes a discrimination ID 22a1, worker ID 22a2, location 22a3, name 22a4, terminal device 22a5, communication means 22a6, and safety equipment 22a7. The discrimination ID 22a1 is a discrimination ID assigned by the detection unit 21. The worker ID 22a2 is a worker ID registered in the worker master data 41 and the work master data 42. The location 22a3 is the location of the worker included in the detection result 21a. The name 22a4, terminal device 22a5, and communication means 22a6 correspond to the information included in the worker master data 41. The safety equipment 22a7 indicates the safety equipment worn by the worker and is based on the information registered in the work master data 42.

[0048] Figure 7 is a diagram illustrating the processing performed by the extraction unit. The extraction unit 23 identifies the objects detected by the detection unit 21. The extraction unit 23 also extracts elements used to determine the risk of disaster from the detection results 21a, worker information 22a, and object master data 43. Elements are extracted for each identified object.

[0049] In the example shown in Figure 7, the object master data 43 includes the component ID 43a, weight 43b, size 43c, angle 43d, and image 43e. The component ID 43a is the ID of the object (component). The weight 43b and size 43c are the weight and size of the object, respectively. The angle 43d is the angle of inclination of the object relative to the surface on which it is placed. The image 43e is an image of the object. The image 43e is used for object identification.

[0050] The extraction element master data 44 includes extraction elements 44a, units 44b, and extraction methods 44c. Extraction elements 44a indicate the elements (information) to be extracted. Units 44b are the units of the elements to be extracted. Extraction methods 44c indicate the method for extracting the elements. The extraction unit 23 extracts the elements defined in extraction elements 44a from the detection results 21a, worker information 22a, and object master data 43 using the method defined by extraction methods 44c.

[0051] In the example shown in Figure 7, information is registered in the extraction element master data 44 so that the ID of the worker closest to the object, the height of the object, the distance between the person and the object, the weight of the object, and whether or not safety equipment is being worn are extracted. The distance between the person and the object is related to the risk of the person coming into contact with the object. The height and weight of the object are related to the risk to the person if the object falls. The presence or absence of safety equipment is related to the risk to the person in the event of an object falling or contact between the person and the object. In this way, the extraction element master data 44 defines the information used to determine the risk of disaster.

[0052] The extraction unit 23 processes the data to generate the extraction result 23a. The extraction result 23a includes a discrimination ID 23a1, a component ID 23a2, and extraction elements 23a3. The discrimination ID 23a1 corresponds to the information contained in the detection result 21a. The component ID 23a2 corresponds to the information contained in the object master data 43. The extraction elements 23a3 represent each element extracted according to the information registered in the extraction element master data 44. In the illustrated example, the extraction elements 23a3 include worker IDs within a 1m radius of the object, the height of the object, the weight of the object, the distance between the person and the object, and information on the wearing of safety equipment.

[0053] Figure 7 shows an example of how each element is extracted for object ID "X0001". Similarly, for object IDs "X0002" to "X0006" and "X0011" to "X0016" shown in Figure 5, the elements defined in the extraction element master data 44 are extracted.

[0054] Figure 8 is a diagram illustrating the processing performed by the determination unit. The determination unit 24 determines the presence or absence of a hazard based on the extraction result 23a and the determination condition master data 45. The determination condition master data 45 includes a hazard ID 45a, a target for determination 45b, a determination condition 45c, a determination content 45d, and a type of hazard 45e. The hazard ID 45a is an ID assigned to each type of hazard to be determined. The target for determination 45b indicates the object to be determined. Identification information of the object is registered as the object to be determined. The determination condition 45c is a condition used to determine the hazard. The determination content 45d indicates the content of the hazard when the condition of determination condition 45c is met. The type of hazard 45e indicates the type of hazard that may occur when the determination condition 45c is met.

[0055] The determination unit 24 determines whether the information contained in the extraction result 23a satisfies any of the conditions registered in the determination condition master data 45. In the example shown in Figure 8, component ID "M0001" is defined as being subject to hazard ID "R0001". Also, component ID "M0005" is defined as being subject to hazard ID "R0002". The determination unit 24 determines whether the elements of component ID "M0001" contained in the extraction result 23a satisfy the determination condition 45c for hazard ID "R0001". The determination unit 24 also determines whether the elements of component ID "M0005" contained in the extraction result 23a satisfy the determination condition 45c for hazard ID "R0002".

[0056] The determination unit 24 generates a determination result 24a by comparing the elements included in the extraction result 23a with the determination condition master data 45. The determination result 24a includes the discrimination ID 24a1, the component ID 24a2, the worker ID 24a3 within a 1m radius of the object, the hazard ID 24a4, the type of hazard 24a5, and the determination result 24a6. The discrimination ID 24a1, the component ID 24a2, and the worker ID 24a3 within a 1m radius of the object correspond to the information included in the extraction result 23a. The hazard ID 24a4 and type 24a5 correspond to the information included in the determination condition master data 45. The determination result 24a6 shows the result of comparing each element of the extracted element 23a3 with the determination condition 45c.

[0057] In the example shown in Figure 8, the distance included in the extracted element 23a3 for component ID "M0001" satisfies the judgment condition 45c. Therefore, component ID "M0001" is determined to be at risk of contact with hazardous equipment. Similarly, the height, weight, and distance included in the extracted element 23a3 for component ID "M0005" satisfy the judgment condition 45c. Therefore, component ID "M0005" is determined to be at risk of injury from falling objects from a height.

[0058] Figure 9 is a diagram illustrating the processing performed by the selection unit. The selection unit 25 selects a safety instruction for the worker from the information registered in the safety instruction master data 46 based on the judgment result 24a. The safety instruction includes instructions to prevent accidents. In the example shown in Figure 9, the safety instruction master data 46 includes the type of hazard 46a, the type of accident 46b, the selection conditions 46c, the instruction content 46d, and the target person 46e. The safety instruction master data 46 indicates the type of hazard to which the safety instruction applies. The type of accident 46b indicates the type of accident that may occur depending on the type of hazard 46a. The selection conditions 46c are the conditions for selecting the safety instruction. The instruction content 46d indicates the specific content of the safety instruction. The target person 46e indicates the person to whom the safety instruction is communicated.

[0059] The selection unit 25 determines whether the judgment result included in the judgment result 24a satisfies any of the selection conditions 46c. If the judgment result satisfies any of the selection conditions 46c, the selection unit 25 selects the instruction content 46d corresponding to that selection condition 46c as the output safety instruction. If the judgment result 24a does not include any result that has been determined to be dangerous, no safety instruction is selected.

[0060] In the illustrated example, the judgment result 24a indicates that there is a risk of contact with dangerous equipment and injury from falling objects from a height. These judgment results satisfy the selection criteria 46c for "contact with dangerous equipment" and the selection criteria 46c for "injury from falling objects from a height". The selection unit 25 selects the instruction content 46d corresponding to "contact with dangerous equipment": "move away from the equipment" and "wear work gloves". The selection unit 25 also selects the instruction content 46d corresponding to "injury from falling objects from a height": "move objects that are at a height". The selection unit 25 generates the selection result 25a, which includes the selected safety instructions.

[0061] The selection result 25a includes the discrimination ID 25a1, component ID 25a2, worker ID 25a3, type of hazard 25a4, type of accident 25a5, instruction content 25a6, and target person 25a7. The discrimination ID 25a1 is the discrimination ID of the object that was determined to be hazardous. The discrimination ID 25a1, component ID 25a2, worker ID 25a3, and type of hazard 25a4 correspond to the information contained in the judgment result 24a. The type 25a5, instruction content 25a6, and target person 25a7 correspond to the information contained in the safety instruction master data 46.

[0062] Figure 10 is a diagram illustrating the processing performed by the output unit. The instruction generation unit 26 generates a safety instruction 26a based on the selection result 25a. At this time, the target to which the safety instruction will be sent is determined based on the worker ID 25a3 and worker information 22a included in the selection result 25a.

[0063] For example, as shown in Figure 10, the safety instruction 26a includes a discrimination ID 26a1, a worker ID 26a2, a name 26a3, a terminal device 26a4, a transmission means 26a5, a type of hazard 26a6, instruction content 26a7, and a target person 26a8. The discrimination ID 26a1 corresponds to the information included in the selection result 25a. The worker ID 26a2 corresponds to the information included in the worker information 22a and the selection result 25a. The name 26a3, terminal device 26a4, and transmission means 26a5 correspond to the information included in the worker information 22a. The type of hazard 26a6, instruction content 26a7, and target person 26a8 correspond to the information included in the selection result 25a. The instruction generation unit 26 transmits the generated safety instruction 26a to the terminal device 30.

[0064] Figures 11 and 12 are schematic diagrams showing an example of the output of a terminal device. For example, as shown in Figures 11 and 12, the terminal device 30 is a smartphone. The instructions included in the safety instructions 26a are displayed on the smartphone screen. In the example shown in Figure 11, safety instructions to avoid "contact with hazardous equipment" are displayed on the terminal device 30. In the example shown in Figure 12, safety instructions to avoid "injury from falling objects from heights" are displayed on the terminal device 30. The terminal device 30 may be other smart devices such as tablets, smartwatches, smart glasses, or PCs. The instructions in the safety instructions 26a may be output by voice instead of, or in addition to, a display. Light, sound, or vibration corresponding to the instructions may also be output from the terminal device 30.

[0065] Figure 13 is a flowchart showing the determination method according to the embodiment. In the determination method M according to the embodiment, the detection unit 21 acquires an image captured by the imaging device 10 (step S0). The detection unit 21 detects people and objects from the image (step S1). The linking unit 22 acquires the reading result from the reading device 15 and links worker information to the person detected by the detection unit 21 (step S2). The extraction unit 23 identifies the object detected from the image and extracts elements used to determine the risk (step S3). The determination unit 24 compares the extraction result with the risk determination conditions and determines whether or not there is a risk of disaster (step S4). The selection unit 25 selects a safety instruction corresponding to the disaster that has been determined to be dangerous based on the determination result (step S5). The instruction generation unit 26 generates a safety instruction based on the selection result (step S6) and transmits the safety instruction to the terminal device 30 (step S7). The terminal device 30 outputs a notification to the target person (step S8).

[0066] The determination method M shown in Figure 13 is executed repeatedly each time an image is acquired by the imaging device 10. For example, the determination method M is executed in response to image acquisition, and a safety instruction is output. This makes it possible to determine the presence or absence of danger in real time and prevent disasters from occurring.

[0067] The advantages of the embodiment will be explained. As mentioned above, accidents can occur at work sites. For example, accidents such as cuts, abrasions, pinching, and entanglement can occur due to unintentional contact with production equipment. Accidents such as flying or falling objects can occur due to components placed at high positions. Other accidents related to objects include contact between people and hazardous substances, electric shock, and collisions between people and objects. Generally, efforts are made to prevent accidents by establishing rules at work sites and ensuring compliance with those rules. However, there is a need for technologies that can further reduce the occurrence of accidents.

[0068] In this embodiment of the present invention, the determination device 20 detects people and objects from an image. The determination device 20 also acquires identification information of people read by the reading device 15. The determination device 20 associates the detected person's identification information with the person's information registered in the worker master data 41 and generates worker information 22a. The determination device 20 also identifies the detected object and acquires information about the object registered in the object master data 43. The determination device 20 extracts elements used for determining hazard from the person's position calculated from the image, the object's position calculated from the image, the associated person's information, and the acquired object's information. The determination device 20 compares the extracted elements with the hazard determination conditions and, if it determines that there is a hazard, transmits safety instructions to avoid the hazard.

[0069] For example, if an object poses a danger upon contact, and the distance between the person and the object is short, the device determines that there is a risk of contact. Based on this determination, the device 20 transmits safety instructions to avoid contact. If an object is placed at a high location, and it is heavy and the distance between the person and the object is short, the device determines that there is a risk of it falling. Based on this determination, the device 20 transmits safety instructions to avoid the object falling.

[0070] According to the embodiment, the risk of disaster can be automatically determined from an image, and if a risk is determined, a safety instruction is sent. For example, a person who receives a safety instruction can avoid the occurrence of a disaster by acting in accordance with that instruction.

[0071] In this embodiment, a person is identified by reading identification information using the reader 15. By referring to the worker master data 41 using the identification information, it is possible to determine whether the identified person owns the terminal device 30. If the person owns the terminal device, a safety instruction is sent to the terminal device 30. This allows safety instructions to be conveyed to the person more quickly.

[0072] The embodiment can be realized by installing an imaging device 10 and a reading device 15 at the work site and preparing various databases. For example, it is also possible to achieve the same processing as the embodiment by attaching identifiers to all objects and people present at the work site and managing their locations and information. However, in that case, attaching identifiers and building a management system for locations and information would require a lot of cost and effort. According to the embodiment, a system capable of suppressing the occurrence of disasters can be introduced more simply compared to such a method.

[0073] Furthermore, wearing safety equipment is important for avoiding accidents. In this embodiment, a person is identified, and information about the safety equipment they are wearing is obtained from the work master data 42. Therefore, the presence or absence of a risk can be determined depending on whether or not a person is wearing safety equipment. For example, if a worker who does not normally wear safety equipment moves to an area where wearing safety equipment is recommended, the risk can be determined, and the worker can be prompted to wear safety equipment.

[0074] When the detection unit 21 detects a person from an image, it may further detect the safety equipment that the person is wearing. A recognition model or pattern matching can be used to detect the safety equipment. The linking unit 22 obtains information on the safety equipment to be worn from the work master data 42. At this time, the linking unit 22 may also determine whether there is a shortage of the detected safety equipment compared to the safety equipment obtained from the work master data 42. If there is a shortage of the detected safety equipment compared to the safety equipment defined in the work master data 42, the instruction generation unit 26 may generate an instruction to wear the missing safety equipment as a safety instruction.

[0075] Figure 14 is a table illustrating the data output by the detection unit. Figure 15 is a schematic diagram showing an example of the output of the terminal device. When safety equipment is detected by the detection unit 21, for example as shown in Figure 14, a column 21a6 indicating the detected safety equipment is added to the detection result 21a for each person detected. When the linking unit 22 obtains safety equipment information from the work master data 42, it determines whether there are any deficiencies in the detected safety equipment compared to the acquired safety equipment.

[0076] In the examples shown in Figures 14 and 6, the worker with identification ID "A0001" is identified as the worker with worker ID "H0001". In the work master data 42, the worker with worker ID "H0001" is required to wear a work cap and safety shoes. The detection unit 21 detects that the worker is wearing a work cap and work gloves. In this case, the instruction generation unit 26 generates a safety instruction prompting the worker to wear the missing safety equipment and sends it to the terminal device 30. As shown in Figure 15, the terminal device 30 outputs the safety instruction to the worker.

[0077] According to the work master data 42, the worker with worker ID "H0001" is not required to wear work gloves. However, the detection unit 21 has detected that the worker is wearing work gloves. In this case, the stringing unit 22 may assume that the worker is wearing work gloves and generate worker information 22a.

[0078] (modified version) Figure 16 is a schematic diagram showing the function of a determination device according to a modified embodiment. In addition to the functions shown in Figure 2, the determination device 20 may also be equipped with the functions of a registration unit 27 and an update unit 28, as shown in Figure 16. The storage device 40 may also store disaster information master data 47.

[0079] The user inputs disaster information into the determination device 20 using the input device 20a. The registration unit 27 accesses the disaster information master data 47 and registers the input disaster information in the disaster information master data 47. When a disaster occurs at a work site, the circumstances, location, and cause of the disaster are registered in the disaster information master data 47. Near misses are also registered in the disaster information master data 47, including the circumstances, location, and cause. A near miss refers to a situation that did not result in a disaster but occurred just before it did. The disaster information master data 47 contains records of past disasters and near misses. The registration unit 27 can also retrieve the information registered in the disaster information master data 47 and display it on the display device 20b. The user can then check the cases registered in the disaster information master data 47 from the displayed information. Here, actual disasters and near misses are collectively referred to as "disasters."

[0080] When a new disaster is registered by the registration unit 27, the update unit 28 uses the information of that disaster to appropriately update the information contained in the extraction element master data 44, the judgment condition master data 45, and the safety instruction master data 46. Thereafter, the presence or absence of a risk is determined using the updated information in each master data.

[0081] Figure 17 is a table illustrating the disaster information that is registered. For example, as shown in Figure 17, the disaster information master data 47 includes registration ID 47a, type 47b, type of disaster 47c, location 47d, date and time of occurrence 47e, component ID 47f, storage height 47g, and situation 47h. The registration ID 47a is information for identifying the registered incident. The type 47b indicates the type of incident, showing whether the incident was an actual disaster or a near miss. The type of disaster 47c indicates the type of disaster in the incident. Location 47d and date and time of occurrence 47e indicate the location and date and time the disaster occurred, respectively. Component ID 47f and storage height 47g are identification information of the component involved in the disaster and the height at which the component was stored, respectively. The situation 47h is entered by the user, describing the circumstances at the time the incident occurred.

[0082] Figure 18 is a diagram illustrating the update of the extracted element master data. When the update unit 28 obtains the disaster information master data 47, it updates the extraction element master data 44. Specifically, the update unit 28 identifies elements to be extracted when determining the risk by performing text analysis on the situation 47h. For example, during text analysis, elements with a high degree of similarity to the disaster type 47c are identified as elements to be extracted when determining the risk. Keywords may be prepared in advance for each disaster type, and elements with a high degree of similarity to the keywords may be identified as elements to be extracted when determining the risk. In addition, the update unit 28 extracts information such as storage height 47g and situation 47h according to the disaster type 47c.

[0083] In the illustrated example, the elements related to the fall are identified as worker ID within a 1m radius of the object, the height of the object (z position), the distance between the person and the object, and the safety equipment. The identified elements are added to the extracted element master data 44.

[0084] Figure 19 is a diagram illustrating the updating of the judgment condition master data. When the extracted element master data 44 is updated, the update unit 28 updates the judgment condition master data 45 using the information from the disaster information master data 47 and the updated information from the extracted element master data 44. Specifically, the update unit 28 generates judgment conditions for the elements added to the extracted element master data 44 based on the information from the disaster information master data 47. In the example shown in Figure 19, worker ID, object height, distance between person and object, and safety equipment have been added to the extracted element master data 44. Regarding the object height, the update unit 28 generates a judgment condition based on the storage height 47g information such that it is determined to be dangerous if it is 3.5m or more. Regarding safety equipment, the update unit 28 generates a judgment condition based on the situation 47h information such that it is determined to be dangerous if a helmet is not worn. The generated judgment conditions are added to the judgment condition master data 45.

[0085] Figure 20 is a diagram illustrating the updating of safety instruction master data. When the judgment condition master data 45 is updated, the update unit 28 updates the safety instruction master data 46 using the information from the disaster information master data 47 and the updated information from the judgment condition master data 45. Specifically, the update unit 28 adds the satisfaction of the added judgment conditions to the judgment condition master data 45 as a selection condition 46c for safety instructions. The update unit 28 also generates instruction content 46d based on the added selection condition 46c. The instruction content 46d is determined so that the selection condition 46c is satisfied. The generated safety instructions are added to the safety instruction master data 46.

[0086] According to the modified configuration described above, the update unit 28 automatically updates the extraction element master data 44, the judgment condition master data 45, and the safety instruction master data 46 in response to the registration of a case in the disaster information master data 47. Therefore, the administrator of the judgment system 100 can reduce the effort required to update the database.

[0087] The types of accidents that may occur differ from one work site to another. Therefore, it is desirable to have separate databases for each work site. This would allow for an appropriate determination of the presence or absence of risks at each work site. On the other hand, preparing a separate database for each work site is time-consuming and difficult. According to a modified embodiment, the database can be updated based on cases accumulated at each work site. This makes it possible to determine the presence or absence of risks more appropriately at each work site.

[0088] Figure 21 is a schematic diagram representing the hardware configuration. As the determination device 20, for example, the computer 90 shown in Figure 21 is used. The computer 90 includes a CPU 91, ROM 92, RAM 93, storage device 94, input interface 95, output interface 96, and communication interface 97.

[0089] ROM92 stores programs that control the operation of computer 90. ROM92 contains the programs necessary for computer 90 to perform each of the processes described above. RAM93 functions as a memory area where the programs stored in ROM92 are loaded.

[0090] The CPU 91 includes processing circuits. The CPU 91 uses the RAM 93 as work memory and executes programs stored in at least one of the ROM 92 or the storage device 94. During program execution, the CPU 91 controls each component via the system bus 98 and performs various processes.

[0091] The storage device 94 stores data necessary for program execution and data obtained through program execution. The storage device 94 may also be used as the storage device 40.

[0092] The input interface (I / F) 95 can connect the computer 90 and the input device 95a. The input I / F 95 is, for example, a serial bus interface such as USB. The CPU 91 can read various data from the input device 20a via the input I / F 95. The input device 95a may be used as the input device 20a shown in Figure 16.

[0093] The output interface (I / F) 96 can connect the computer 90 to the output device 96a. The output I / F 96 is, for example, a video output interface such as Digital Visual Interface (DVI) or High-Definition Multimedia Interface (HDMI®). The CPU 91 can transmit data to the output device 96a via the output I / F 96 and display an image on the output device 96a. The output device 96a may be used as the display device 20b shown in Figure 16.

[0094] The communication interface (I / F) 97 allows the computer 90 to connect with a server 97a located outside the computer 90. The communication I / F 97 is, for example, a network card such as a LAN card. The CPU 91 can read various data from the server 97a via the communication I / F 97.

[0095] The storage device 94 includes one or more selected from Hard Disk Drives (HDDs) and Solid State Drives (SSDs). The input device 95a includes one or more selected from a mouse, keyboard, microphone (voice input), and touchpad. The output device 96a includes one or more selected from a monitor, projector, printer, and speaker. Devices that have the functions of both input device 95a and output device 96a, such as a touch panel, may also be used.

[0096] Each process performed by the determination device 20 may be implemented by a single computer 90, or by the cooperation of multiple computers 90.

[0097] The processing of the various data described above may be recorded as a program that can be executed by a computer on a magnetic disk (flexible disk and hard disk, etc.), an optical disk (CD-ROM, CD-R, CD-RW, DVD-ROM, DVD±R, DVD±RW, etc.), a semiconductor memory, or another non-transitory computer-readable storage medium.

[0098] For example, information recorded on a recording medium can be read by a computer (or embedded system). The recording format (storage format) of the recording medium is arbitrary. For example, a computer reads a program from the recording medium and has the CPU execute the instructions written in the program based on this program. In a computer, program acquisition (or reading) may be performed via a network.

[0099] Embodiments of the present invention include the following features. (Feature 1) The system detects people and objects from the image and acquires identification information of people read by the reader. The identified person is linked to the person's information registered in the database. Identify the detected object, From the position of the person calculated from the image, the position of the object calculated from the image, the information of the person, and the information of the object registered in the database, elements used to determine the level of danger are extracted. A determination device that compares the extracted elements with the criteria for determining hazard, and if it determines that there is a hazard, transmits safety instructions to avoid the hazard. (Feature 2) Referencing a database that stores multiple judgment conditions and multiple safety instructions corresponding to each of the multiple judgment conditions, The safety instruction corresponding to the satisfied determination condition is obtained, The determination device according to feature 1, which transmits the acquired safety instructions. (Feature 3) Referencing a database that stores the identification information of multiple people and the safety equipment worn by each of them, Information on the safety equipment worn by the identified person is obtained, The determination device according to feature 1 or 2, which uses the information of the safety equipment as the information of the person. (Feature 4) From the aforementioned image, the safety equipment worn by the person is further detected. The determination device according to feature 3, which determines whether or not there is a deficiency in the detected safety equipment with respect to the acquired safety equipment. (Feature 5) The information of the person includes information indicating whether or not the person owns a terminal device. A determination device according to any one of features 1 to 4, which transmits the safety instruction to the terminal device when the person possesses the terminal device. (Feature 6) The determination device according to any one of features 1 to 5, wherein the element includes one or more selected from the following: the horizontal distance between the position of the person and the position of the object, the height of the object, the weight of the object, and the safety equipment worn by the person. (Feature 7) Detect people and objects from the image, identify the objects, The presence or absence of danger is determined using the position of the person calculated from the image, the position of the object calculated from the image, and the information of the identified object. A determination device that, when it determines that there is a risk, selects a safety instruction to avoid the risk from a pre-registered database, A determination device that adds new safety instructions to the database using disaster information, including the type of work-related accident, the objects involved in the accident, and the circumstances of the accident. (Feature 8) From the location of the person, the location of the object, and the information of the object, elements used to determine the risk are extracted. The determination device according to feature 7, which compares the extracted elements with the criteria for determining hazard, and if it is determined that there is a hazard, selects and transmits the safety instruction. (Feature 9) The determination device according to feature 8, which adds new determination conditions and new elements to the database using the aforementioned disaster information. (Feature 10) A determination device described in any one of features 1 to 9, An imaging device that acquires the aforementioned image, A judgment system equipped with the following features. (Feature 11) On the computer, The system detects people and objects from the image and acquires identification information of people read by the reading device. The detected person is linked to the identification information and the person's information registered in the database. To identify the detected object, From the position of the person calculated from the image, the position of the object calculated from the image, the information of the person, and the information of the object registered in the database, elements used to determine the level of danger are extracted. The extracted elements are compared with the criteria for determining danger, and if a danger is determined, safety instructions to avoid the danger are transmitted. Judgment method. (Feature 12) A program that causes the computer to execute the determination method described in Feature 11. (Feature 13) A storage medium containing the program described in Feature 12.

[0100] According to the embodiments described above, a determination device, determination system, determination method, program, and storage medium are provided that can suppress the occurrence of occupational accidents.

[0101] In this specification, "or" indicates that "at least one" of the items listed in the text may be adopted.

[0102] Although several embodiments of the present invention have been illustrated above, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. Furthermore, the embodiments described above can be implemented in combination with each other. [Explanation of Symbols]

[0103] 10: Imaging device, 15: Reading device, 15a: Reading result, 20: Judgment device, 20a: Input device, 20b: Display device, 21: Detection unit, 21a: Detection result, 22: Linking unit, 22a: Worker information, 23: Extraction unit, 23a: Extraction result, 24: Judgment unit, 24a: Judgment result, 25: Selection unit, 25a: Selection result, 26: Instruction generation unit, 26a: Safety instruction, 27: Registration unit, 28: Update unit, 30: Terminal device, 40: Storage device, 41: Worker master data, 42: Work master data, 43: Item master data, 44: Extraction element master data, 45: Judgment condition master data, 46: Safety instruction master data, 47: Disaster information master data, 100: Judgment system, 200, 220: Image, M: Judgment method

Claims

1. The system detects people and objects from the image and acquires identification information of people read by the reader. The identified person is linked to the person's information registered in the database. Identify the detected object, From the position of the person calculated from the image, the position of the object calculated from the image, the information of the person, and the information of the object registered in the database, elements used to determine the level of danger are extracted. A determination device that compares the extracted elements with the criteria for determining hazard, and if it determines that there is a hazard, transmits safety instructions to avoid the hazard.

2. Referencing a database that stores multiple judgment conditions and multiple safety instructions corresponding to each of the multiple judgment conditions, The safety instruction corresponding to the satisfied determination condition is obtained, The determination device according to claim 1, which transmits the acquired safety instruction.

3. Referencing a database that stores the identification information of multiple people and the safety equipment worn by each of them, Information on the safety equipment worn by the identified person is obtained, The determination device according to claim 1, wherein the information of the safety equipment is used as the information of the person.

4. From the aforementioned image, the safety equipment worn by the person is further detected. The determination device according to claim 3, which determines whether or not there is a deficiency in the detected safety equipment with respect to the acquired safety equipment.

5. The information of the person includes information indicating whether or not the person owns a terminal device. The determination device according to claim 1, which transmits the safety instruction to the terminal device when the person possesses the terminal device.

6. The determination device according to claim 1, wherein the element includes one or more selected from the following: the horizontal distance between the position of the person and the position of the object, the height of the object, the weight of the object, and the safety equipment worn by the person.

7. Detect people and objects from the image, identify the objects, The presence or absence of danger is determined using the position of the person calculated from the image, the position of the object calculated from the image, and the information of the identified object. A determination device that, when it determines that there is a risk, selects a safety instruction to avoid the risk from a pre-registered database, A determination device that adds new safety instructions to the database using disaster information, including the type of work-related accident, the objects involved in the accident, and the circumstances of the accident.

8. From the location of the person, the location of the object, and the information of the object, elements used to determine the risk are extracted. The determination device according to claim 7, which compares the extracted elements with the criteria for determining danger, and if it is determined that there is a danger, selects and transmits the safety instruction.

9. The determination device according to claim 8, wherein new determination conditions and new elements are added to the database using the disaster information.

10. A determination device according to any one of claims 1 to 9, An imaging device that acquires the aforementioned image, A judgment system equipped with the following features.

11. On the computer, The system detects people and objects from the image and acquires identification information of people read by the reading device. The detected person is linked to the identification information and the person's information registered in the database. To identify the detected object, From the position of the person calculated from the image, the position of the object calculated from the image, the information of the person, and the information of the object registered in the database, elements used to determine the level of danger are extracted. The extracted elements are compared with the criteria for determining danger, and if a danger is determined, safety instructions to avoid the danger are transmitted. Judgment method.

12. A program that causes the computer to execute the determination method described in claim 11.

13. A storage medium storing the program described in claim 12.

Images