Mounting confirmation system, method, and program

Abstract

To provide a wearing checking system, a method, and a program, that can easily check a wearing status of wearables on people.SOLUTION: A wearing checking system includes: a photographing device 12 for photographing visitors; person detection means which performs image analysis to a photographed image obtained by the photographing device 12 for detecting a person within the photographed image; wearables detection means which performs the image analysis to the photographed image for detecting a wearable of the person; wearing determination means for determining a wearing status of the wearable based on detection results by the person detection means and the wearables detection means; and storage means for storing the wearing status determined by the wearing determination means and the photographed image as a checked result.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a wearing confirmation system, method, and program.

Background Art

[0002] Conventionally, when entering an area involving dangerous work such as a construction site or a factory, for the safety of workers, wearing safety equipment such as helmets is mandatory. When entering such a special area, technologies for confirming the wearing of helmets have also been developed (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the above conventional technology has problems that a person cannot enter if they do not have an identification card or the like, and it is difficult to determine whether a wearable such as a helmet is correctly worn on a person's head.

[0005] Therefore, an object of the present disclosure is to provide a wearing confirmation system, method, and program capable of easily confirming the wearing state of a wearable on a person.

Means for Solving the Problems

[0006] To solve the above problems, in the present disclosure, an imaging device that images an entrant, a person detection means that performs image analysis on the captured image acquired by the imaging device and detects a person in the captured image, a wearable detection means that performs image analysis on the captured image and detects a wearable of the person, Wearing state determination means for determining the wearing state of the wearable based on the detection results of the person detection means and the wearable detection means, Storage means for storing the wearing state determined by the wearing state determination means and the captured image as a confirmation result, A wearing confirmation system having the above is provided.

[0007] Also, the wearing confirmation system of the present disclosure, May have a display device that displays the captured image, the detection reach of the person by the person detection means, and the detection reach of the wearable by the wearable detection means.

[0008] Also, in the wearing confirmation system of the present disclosure, The detection reach may be calculated based on the number of captured images analyzed by image analysis within a predetermined time, with the number of captured images in which a person or a wearable is detected as the detection time, and calculated as the reach of the detection time with respect to a pre-set detection reach time.

[0009] Also, in the wearing confirmation system of the present disclosure, The display device may further display a message according to the processing state of the person detection means and the wearable detection means, and the processing rate of the captured image.

[0010] Also, the wearing confirmation system of the present disclosure, May include a control device that unlocks an automatic door or a key box based on the determination result of the wearing state determination means.

[0011] Also, the wearing confirmation system of the present disclosure, The person detection means may calculate a feature amount in the face portion or the eye portion of the person in the captured image, and detect the person based on the feature amount.

[0012] Also, in the wearing confirmation system of the present disclosure, The wearable detection means may detect a wearable in the captured image by comparing a template image of the wearable with a region of interest in the captured image.

[0013] Also, in the wearing confirmation system of the present disclosure, the wearing object detection means may include three modes: a normal mode using a single template image, a three-template mode using three template images of different sizes, and a random template mode using a magnified / reduced template image obtained by magnifying and reducing the original template image within a predetermined range.

[0014] Also, in the wearing confirmation system of the present disclosure, the template image may be an image with a predetermined mark attached to the wearing object.

[0015] Also, in the present disclosure, a wearing confirmation method in which a computer connected to a photographing device that photographs an entrant confirms the state of a wearing object, wherein the computer performs image analysis on the photographed image acquired by the photographing device, detects a person in the photographed image, performs image analysis on the photographed image, detects the wearing object of the person, a wearing determination means for determining the wearing state of the wearing object based on the detection result of the person and the detection result of the wearing object, provides a wearing confirmation method for storing the wearing state determined by the wearing determination means and the photographed image as a confirmation result.

[0016] Also, in the present disclosure, a computer connected to a photographing device that photographs an entrant is a person detection means for performing image analysis on the photographed image acquired by the photographing device and detecting a person in the photographed image, a wearing object detection means for performing image analysis on the photographed image and detecting the wearing object of the person, a wearing determination means for determining the wearing state of the wearing object based on the detection results of the person detection means and the wearing object detection means, a storage means for storing the wearing state determined by the wearing determination means and the photographed image as a confirmation result, Provide a program for causing it to function as.

Advantages of the Invention

[0017] According to the present disclosure, it is possible to provide a wearing confirmation system, method, and program capable of easily confirming the wearing state of a wearable on a person.

Brief Description of the Drawings

[0018]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Modes for Carrying Out the Invention

[0019] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the drawings. <1. System Configuration> FIG. 1 is a diagram showing the configuration of a wearing confirmation system according to an embodiment of the present disclosure. FIG. 2 is a schematic diagram showing the appearance of the wearing confirmation system according to an embodiment of the present disclosure. FIG. 3 is a schematic diagram of the wearing confirmation system viewed from the side direction of the entrance. In FIGS. 1 to 3, 10 is a wearing confirmation system, 11 is a processing device, 12 is a photographing device, 13 is a display device, 14 is a control device, 15 is an automatic door, 16 is a key box, and 17 is a tablet PC.

[0020] The processing device 11 is a device that performs various arithmetic processes in the wearing confirmation system 10, and is realized by incorporating a program for realizing the functions of the wearing confirmation system 10 into a computer having a CPU, a main memory, a non-volatile storage device, etc. The photographing device 12 is a photographing means for performing photographing and acquiring a photographed image as image data, and is realized by a general-purpose photographing device such as a CCD camera. The display device 13 is a display means for performing display based on the result processed by the processing device 11, and is realized by a general-purpose display device such as a liquid crystal display or an organic EL display. In the present embodiment, the processing device 11, the photographing device 12, and the display device 13 are built in one tablet PC 17.

[0021] The control device 14 is connected to the automatic door 15 and the key box 16, and is a device that transmits an unlocking signal to the automatic door 15 and the key box 16. The control device 14 is connected to the processing device 11, and transmits an unlocking signal to the automatic door 15 and the key box 16 based on a command from the processing device 11. In FIG. 3, the illustration of the control device 14 is omitted.

[0022] The automatic door 15 is a door that opens automatically. In this embodiment, based on the unlocking signal from the control device 14, it moves in the direction along the wall to open the entrance so that entry is possible. The key box 16 is a box into which a key is inserted. As the key, a physical key or a card key can be used. When the door is not an automatic door, the locked door is unlocked using the key taken out from the key box 16. In this embodiment, based on the unlocking signal from the control device 14, the key box is unlocked. As a result, the user can take out the key of the door from the key box. The automatic door 15 and the key box 16 are an example of an entrance mechanism. The entrance mechanism is not limited to the automatic door 15 and the key box 16 as long as it directly or indirectly opens the entrance based on the control by the control device 14.

[0023] Figure 4 is a functional block diagram showing the details of the processing device 11. In this embodiment, the processing device 11 is realized by incorporating a dedicated program into a general-purpose computer. As shown in Figure 4, the processing device 11 includes a person detection means 11a, an accessory detection means 11b, a wearing determination means 11c, and a confirmation result storage means 11d.

[0024] The person detection means 11a is a means for performing image analysis on the captured image acquired by the imaging device 12 to detect a person in the captured image, and is realized by the CPU reading and executing a program stored in a non-volatile storage device. The accessory detection means 11b is a means for performing image analysis on the captured image acquired by the imaging device 12 to detect an accessory worn by a person, and is realized by the CPU reading and executing a program stored in a non-volatile storage device. The wearing determination means 11c is a means for determining the wearing state of the accessory based on the detection results of the person detection means 11a and the accessory detection means 11b, and is realized by the CPU reading and executing a program stored in a non-volatile storage device. The confirmation result storage means 11d is a means for storing the wearing state determined by the wearing determination means 11c and the captured image as a confirmation result, and is realized by a predetermined storage area of a non-volatile storage device.

[0025] FIG. 5 is a diagram showing the display screen of the display device 13. In FIG. 5, 13a is a photographed image display area, 13b is a message display area, 13c is a reach display area, 13d is a date and time display area, and 13e is a processing information display area. As shown in FIG. 5, on the display screen of the display device 13, the photographed image is displayed in the photographed image display area 13a at the center of the screen. Actually, the photographed images taken at a predetermined rate (fps: frame (image) / second) are sequentially switched and displayed. The displayed photographed image may be at the analysis rate obtained by image analysis by the processing device 11. In the present embodiment, the photographed images are sequentially switched and displayed at the analysis rate. For example, when the shooting rate is 30 fps and the analysis rate is 20 fps, the switching and display are performed at a pace of 20 times per second.

[0026] A message for the visitors is displayed in the message display area 13b at the upper part of the screen of the display device 13. This message is obtained from the processing device 11 and displayed according to the processing by the processing device 11. The reach of the analysis by the processing device 11 is displayed in the reach display area 13c at the left part of the screen of the display device 13. The current date and time are displayed in the date and time display area 13d at the upper right part of the screen of the display device 13. Other detailed information processed by the processing device 11 is displayed in the processing information display area 13e at the lower right part of the screen of the display device 13. Information indicating units less than seconds of the date and time, the processing rate (fps: frame (image) / second) of the photographed image, etc. can be displayed in the processing information display area 13e. As the processing rate, there are a shooting rate which is the shooting rate and an analysis rate which is the rate for analyzing the photographed image. In the example of FIG. 5, “45.40” indicating units less than seconds of the date and time, 30.0 fps indicating the shooting rate, and 20.0 fps indicating the analysis rate are shown in the processing information display area 13e. As the processing rate, either one of the shooting rate and the analysis rate may be displayed.

[0027] <2. Processing Operations> Next, the processing operation of the wearing confirmation system according to the present embodiment will be described together with the wearing confirmation method according to the present embodiment. FIG. 6 is a state transition diagram showing the outline of the processing of the wearing confirmation system according to the present embodiment. The wearing confirmation system according to the present embodiment performs processing while transitioning to six states of S1 to S6 after startup. The wearable is not particularly limited as long as it is worn by a person. Here, the case where a helmet is used as an example of the wearable will be described.

[0028] First, when the wearing confirmation system is started up, the processing starts. Then, the imaging device 12 constantly takes images, and the image data obtained by the imaging is captured as a captured image. The captured image that has been captured is passed to the processing device 11 and predetermined processing is performed. The imaging device 12 takes images at a predetermined rate (fps: frames (images) / second), captures the captured images, and sends them to the processing device 11. Further, the processing device 11 passes the images acquired from the imaging device 12 to the display device 13. Then, the display device 13 displays the images received from the processing device 11 at any time. For example, when performing image analysis at an analysis rate of 20 fps (frames / second), the captured images are transmitted from the processing device 11 every 1 / 20 second and displayed on the screen of the display device 13. Therefore, it will be displayed like a moving image on the screen of the display device 13. The acquisition of the captured images by the imaging device 12 and the display of the captured images by the display device 13 are always performed during the operation of the wearing confirmation system regardless of the states of S1 to S6 shown in FIG. 6.

[0029] When the processing is started in the wearing confirmation system, first, it shifts to the standby state S1. In the standby state S1, the imaging device 12 constantly takes images, and the image data obtained by the imaging is captured as a captured image. Then, the processing device 11 performs processing such as analysis on the captured image that has been captured. The analysis rate of the captured image in the processing device 11 may be the same as or different from the imaging rate by the imaging device 12. For example, when the imaging rate is 30 fps as described above, the analysis rate in the processing device 11 may be the same 30 fps as the imaging rate, or the rate may be reduced to 20 fps.

[0030] In the standby state S1, the person detection means 11a in the processing device 11 performs a person detection process. When the person detection means 11a detects a person even once with a size equal to or larger than a predetermined size, it shifts to the person detection state S2. FIG. 7 is a diagram showing the state of the screen displayed on the display device 13. In the standby state S1, the display device 13 displays a screen as shown in FIG. 7(a). Since the standby state S1 is a state where no person has been detected, a message prompting the user to move is displayed in the message display area 13b. For example, as shown in FIG. 7(a), a message such as "Please wear a helmet and approach the camera." is displayed.

[0031] When no person is detected from the captured image, the person detection means 11a repeatedly performs a person detection process on the captured image newly captured by the imaging device 12. Details of the person detection process will be described later. During the standby state S1, a message prompting the user to approach, such as "Please wear a helmet and approach the camera.", continues to be displayed.

[0032] In the person detection state S2, the person detection means 11a continues the person detection process on the captured image. Specifically, the person detection means 11a calculates the person detection time PDT. The person detection time PDT is the time when a person is detected. Specifically, the person detection means 11a executes a process according to the following [Equation 1] to calculate the person detection time PDT.

[0033] [Equation 1] Person detection time PDT = Person detection count PDM × (1 / Analysis rate FPS)

[0034] In [Equation 1], the person detection count PDM is the value counted by the person detection means 11a of the number of times a person is detected. The analysis rate FPS is the number of captured images analyzed per second. As shown in [Equation 1], the person detection time PDT is calculated by multiplying this person detection count PDM by the reciprocal of the analysis rate FPS. Then, the person detection means 11a executes a process according to the following [Equation 2] to calculate the person detection reach PGL.

[0035] [Equation 2] Person detection reach PGL = Person detection time PDT / Person detection reach time PGT

[0036] In [Equation 2], the person detection reach time PGT is the time required for person detection set in advance. As shown in [Equation 2], by dividing the person detection time PDT calculated by executing the process according to [Equation 1] by the person detection reach time PGT, the person detection reach PGL is calculated. That is, the person detection reach is calculated by using the number of captured images in which a person is detected as the detection time based on the number of captured images analyzed for image analysis at a predetermined time, and is calculated as the reach of the person detection time with respect to the person detection reach time set in advance. In the example of [Equation 2], the person detection reach PGL is calculated as a value between 0 and 1. The person detection reach time PGT can be arbitrarily set. For example, it can be set to 1.5 seconds.

[0037] The processing device 11 transmits the person detection reach PGL calculated by executing the process according to [Equation 2] to the display device 13, and the display device 13 displays the person detection reach PGL in the reach display area 13c. Therefore, in the person detection state S2, the display device 13 will display a screen as shown in Fig. 7(b).

[0038] As shown in Fig. 7(b), in the reach display area 13c, the person detection reach PGL is displayed in the form of a bar extending in the vertical direction. In the reach display area 13c, inside the bar extending in the vertical direction, it is displayed in a state where it is filled from its lower end to a height proportional to the person detection reach PGL. Therefore, immediately after the transition to the person detection state S2, since the person detection reach PGL = 0, there is no filled portion inside the bar. And as the person detection reach PGL increases, the inside of the bar is filled from the bottom, and when the person detection reach PGL = 1, the entire inside of the bar is filled. The example of Fig. 7(b) shows the point when the person detection reach PGL is about 0.4.

[0039] Also, in the person detection state S2, the current situation such as "Detecting a person" is displayed in the message display area 13b. By this display, the user can be informed of the current situation. Then, by performing a bar display of the person detection reach PGL in the reach display area 13c, the entrant can confirm to what extent the person detection has progressed. For this reason, the entrant can recognize how long to wait in front of the imaging device 12.

[0040] In the person detection state S2, when it becomes more and more difficult for the processing device 11 to detect a person from the captured image, the person detection count PDM does not increase easily, so the person detection time PDT does not increase either. In such a case, it is not desirable to continue the detection process forever. Therefore, when the pre-set person detection limit time PRT has elapsed, the processing device 11 performs a process of returning to the standby state S1. That is, once the person detection time PDT and the person detection reach PGL are reset to 0, and when a person is detected next, the process will transition to the person detection state S2 and perform person detection. The person detection limit time PRT can be set arbitrarily, but for example, it can be set to 10 seconds.

[0041] In the person detection state S2, when the person detection reach PGL reaches 1, it is regarded that the person has been correctly detected, and the person detection is completed. When the person detection is completed, the processing device 11 transitions to the wearable waiting state S3. In the wearable waiting state S3, the wearable detection means 11b performs a wearable detection process on the captured image. And when the helmet, which is a wearable, is detected even once, the process transitions to the wearable detection state S4.

[0042] In the wearable waiting state S3, the display device 13 displays a screen as shown in FIG. 7(c). Since the wearable waiting state S3 is a state in which the worn object, the helmet, cannot be detected, a message for the user is displayed in the message display area 13b. For example, as shown in FIG. 7(c), a message such as "Please wear a helmet." is displayed. In the example of FIG. 7(c), a captured image of a state without a helmet is displayed in the captured image display area 13a. However, in the wearable waiting state S3, even if a captured image of a state with a helmet worn is used, a similar message is displayed.

[0043] When a wearable is not detected in the captured image, the imaging device 12 repeatedly performs a wearable detection process on the captured image newly captured. Details of the wearable detection process will be described later. During the helmet waiting state S3, a message prompting the user to wear a helmet, such as "Please wear a helmet.", is continuously displayed.

[0044] In the wearable detection state S4, the wearable detection means 11b continues the helmet detection process on the captured image. Specifically, the wearable detection means 11b calculates a wearable detection time HDT. The wearable detection time HDT is the time when the helmet, which is the wearable, is detected. Specifically, the wearable detection means 11b executes a process according to the following [Equation 3] to calculate the wearable detection time HDT.

[0045] [Equation 3] Wearable detection time HDT = Wearable detection count HDM × (1 / Analysis rate FPS)

[0046] In [Equation 3], the wearable detection count HDM is the value counted by the wearable detection means 11b of the number of times the helmet, which is the wearable, is detected. The analysis rate FPS is, as in the case of [Equation 1], the number of captured images analyzed per second. As shown in [Equation 3], the wearable detection time HDT is calculated by multiplying the wearable detection count HDM by the reciprocal of the analysis rate FPS. Then, the wearable detection means 11b executes a process according to the following [Equation 4] to calculate the wearable detection achievement level HGL.

[0047] [Equation 4] Wearing object detection arrival degree HGL = Wearing object detection time HDT / Wearing object detection arrival time HGT

[0048] In [Equation 4], the wearing object detection arrival time HGT is the time required for wearing object detection set in advance. As shown in [Equation 4], the wearing object detection arrival degree HGL is calculated by dividing the wearing object detection time HDT calculated by executing the process according to [Equation 3] by the wearing object detection arrival time HGT. That is, the wearing object detection arrival degree is calculated based on the number of captured images analyzed at a predetermined time, and the number of captured images in which the wearing object is detected is calculated as the wearing object detection time, and is calculated as the arrival degree of the wearing object detection time with respect to the wearing object detection arrival time set in advance. In the example of [Equation 4], the wearing object detection arrival degree HGL is calculated as a value between 0 and 1, similar to the person detection arrival degree PGL in [Equation 2]. The wearing object detection arrival time HGT can be arbitrarily set. For example, it can be set to 1.5 seconds.

[0049] The processing device 11 transmits the wearing object detection arrival degree HGL calculated by executing the process according to [Equation 4] to the display device 13, and the display device 13 displays the wearing object detection arrival degree HGL in the arrival degree display area 13c. Therefore, in the wearing object detection state S4, the display device 13 will display a screen as shown in FIG. 7(e).

[0050] As shown in FIG. 7(e), in the reach display area 13c, the wearable detection reach HGL is displayed in the form of a bar extending in the vertical direction. In the reach display area 13c, inside the bar extending in the vertical direction, it is displayed in a state where it is filled from its lower end to a height proportional to the wearable detection reach HGL. Therefore, immediately after transitioning to the wearable detection state S4, since the wearable detection reach HGL = 0, there is no filled portion inside the bar. And as the wearable detection reach HGL increases, the inside of the bar is filled from the bottom, and when the wearable detection reach HGL = 1, the entire inside of the bar is filled. The example of FIG. 7(e) shows the point when the wearable detection reach HGL is about 0.7.

[0051] Also, in the wearable detection state S4, in the message display area 13b, the current processing status such as "Detecting helmet" is displayed. By this display, the user can be informed of the current processing status. And by performing the bar display of the wearable detection reach HGL in the reach display area 13c, the user can confirm to what extent the wearable detection has progressed. For this reason, the user can recognize how long to wait in front of the imaging device 12.

[0052] In the wearable detection state S4, when it becomes more and more difficult for the wearable detection means 11b to detect the wearable from the captured image, since the wearable detection count HDM does not increase easily, the wearable detection time HDT does not increase either. In such a case, it is not desirable to continue the detection process forever. For this reason, when the pre-set wearable detection limit time HRT has elapsed, the wearable detection means 11b performs a process of returning to the standby state S1. That is, once the person detection time PDT, the person detection reach PGL, the wearable detection time HDT, and the wearable detection reach HGL are all reset to 0, and when a person is detected next, it will transition to the person detection state S2 and perform person detection. The wearable detection limit time HRT can be arbitrarily set, but for example, it can be set to 10 seconds.

[0053] In the wearable detection state S4, when the wearable detection progress degree HGL reaches 1, it is considered that the helmet as the wearable has been correctly detected, and the wearable detection is completed. When the wearable detection is completed, the wearing determination means 11c transitions to the wearable wearing state S5 on the assumption that the helmet as the wearable is correctly worn.

[0054] When transitioning to the wearable wearing state S5, the display device 13 switches the display in the message display area 13b based on an instruction from the processing device 11. For example, as shown in FIG. 7(d), a message such as "[OK] Helmet wearing has been confirmed. Unlock the key box." is displayed. Then, the processing device 11 acquires a full-screen image obtained by capturing the screen of the display device 13 immediately after transitioning to the wearable wearing state S5 as shown in FIG. 7(d). This full-screen image records the captured image in which person detection and wearable detection are completed and the date and time. The wearing determination means 11c stores the acquired full-screen image in the confirmation result storage means 11d. As a result, the face image and date and time of the person whose entry is permitted (normal entrant) are stored. In the present embodiment, as a confirmation result, the full-screen image of the display screen of the display device 13 is stored in the confirmation result storage means 11d. However, as long as the wearing status (admission permission), face image (captured image including the face), and date and time are recorded, they may be stored in a format other than the full-screen image.

[0055] In the wearable wearing state S5, subsequently, the processing device 11 transmits an unlocking signal to the control device 14. Specifically, when the determination result by the wearing determination means 11c is "correctly worn", the unlocking signal is transmitted to the control device 14. When receiving the unlocking signal from the processing device 11, the control device 14 further transmits the unlocking signal to the automatic door 15 or the key box 16. After transitioning to the wearable wearing state S5, if the preset wearable wearing limit time HST is exceeded, it shifts to the standby state S1. The wearable wearing limit time HST can be arbitrarily set. For example, it can be set to 15 seconds.

[0056] In the wearable waiting state S3, if the wearable detection time limit HWT is exceeded before the wearable detection is completed, the system will transition to the non-wearable state S6. The wearable waiting time limit HWT can be arbitrarily set. For example, it can be set to 5 seconds. When transitioning to the non-wearable state S6, the display device 13 switches the display in the message display area 13b based on an instruction from the processing device 11. For example, as shown in Fig. 7(f), a message such as "[NG] Helmet wearing could not be confirmed." is displayed. Then, the processing device 11 acquires a full-screen image that captures the screen of the display device 13 immediately after transitioning to the non-wearable state S6 as shown in Fig. 7(f). This full-screen image records the captured image in which the person detection was successful but the wearable detection was not, along with the date and time. The wearing determination means 11c saves the acquired full-screen image in the confirmation result storage means 11d. As a result, the face image and date and time of those whose entry was not permitted (violators, outsiders) will be stored.

[0057] After transitioning to the non-wearable state S6, if the pre-set non-wearable time limit NHT is exceeded, the system will transition to the waiting state S1. That is, after transitioning to the non-wearable state S6, when the non-wearable time limit NHT has elapsed, the system will transition to the waiting state S1. The non-wearable time limit NHT can be arbitrarily set. For example, it can be set to 15 seconds. In the waiting state S1, the person detection means 11a repeatedly performs person detection processing on the acquired captured image.

[0058] <3. Details of Person Detection Processing> As described above, in the standby state S1 and the person detection state S2, the person detection means 11a performs person detection processing on the captured image. Here, the details of the person detection processing by the person detection means 11a will be described. As the person detection processing, various techniques can be used as long as they are techniques for detecting a person from an image. In the present embodiment, feature amounts in the face portion or eye portion of a person in the captured image are calculated, and a person is detected based on the calculated feature amounts. That is, a method based on feature amounts is used. As the method based on feature amounts, object detection using a Haar feature-based cascade classifier can be used. In the present embodiment, the person detection means 11a includes two modes: a face detection mode for performing face detection and an eye detection mode for detecting eyes, and either one can be selected by setting. FIG. 8 is a diagram for explaining the details of person detection. FIG. 8(a) corresponds to the face detection mode, and FIG. 8(b) corresponds to the eye detection mode.

[0059] First, the face detection mode will be described. The face detection mode is a mode for detecting a person using the features of the entire face. As shown in FIG. 8(a), in the face detection mode, one target region Ff is set in the captured image. Then, the face is detected using the features of the face included in the target region Ff.

[0060] Next, the eye detection mode will be described. The eye detection mode is a mode for detecting a person using only the features of the eyes. As shown in FIGS. 8(b) and 8(c), in the eye detection mode, a total of two target regions Fe are set in the captured image corresponding to each eye. Then, detection is performed using the features of the eyes included in the target region Fe. In the case of the eye detection mode, as shown in the captured image of FIG. 8(b), when the entire face is exposed, of course, but as shown in the captured image of FIG. 8(c), even when a mask is worn, a person can be detected.

[0061] <4. Details of Helmet Detection Processing> As described above, in the standby state S1 and the attached object detection state S4, the attached object detection means 11b performs an attached object detection process on the captured image. Here, the details of the attached object detection process will be described. As the attached object detection process, various techniques can be used as long as they detect an attached object such as a helmet from an image. In this embodiment, a template matching method is used. That is, in this embodiment, if the attached object is a helmet, an image of only the helmet is prepared as a template image, and the helmet is detected by comparing and matching with this template image.

[0062] As a template matching method, a method of comparing a template image with an attention area of ​​a captured image can be used. In this embodiment, three modes are provided: a normal mode, a three-template mode, and a random template mode, and any one of them can be selected by setting. FIG. 9 is a diagram for explaining the details of attachment detection. FIG. 9(a) corresponds to the normal mode, FIG. 9(b) corresponds to the three-template mode, and FIG. 9(c) corresponds to the random template mode.

[0063] First, the normal mode will be described. The normal mode is a mode in which one template image, i.e., a single template image, is set in advance, and detection is performed by matching with the template image. As shown in FIG. 9(a), in the normal mode, one target area Fm is set in the captured image. Then, the image included in the target area Fm is compared with the template image.

[0064] Next, the 3-template mode will be described. The 3-template mode is a mode in which three template images of different sizes are preset, and when it matches any of the three template images, it is considered detected. In this case, three template images with a scale factor of 1 (equal magnification), less than 1, and greater than 1 are used. The template image with a scale factor less than 1 and the template image with a scale factor greater than 1 can be created by reducing and enlarging the template image with a scale factor of 1. In the 3-template mode, as shown in Fig. 9(a), the target area Fm is set within the captured image, and the image included in the target area Fm is compared with the template image with a scale factor of 1.

[0065] Furthermore, in the 3-template mode, as shown in Fig. 9(b), the target area Fs is set within the captured image, and the image included in the target area Fs is compared with the reduced template image. Also, as shown in Fig. 9(b), the target area Fl is set within the captured image, and the image included in the target area Fl is compared with the enlarged template image. The reduction rate and the enlargement rate can be set as appropriate. For example, a template image reduced by a factor of 0.7 and a template image enlarged by a factor of 1.2 are created. This 3-template mode can tolerate a certain distance between the imaging device 12 and the detection target.

[0066] The random template mode will be described. The random template mode randomly selects a value within the range of two set magnification factors (less than 1 and greater than 1) each time detection is performed, and uses the selected value to generate a scaled template image (simply referred to as the "template image" in Fig. 9(c)) obtained by enlarging and reducing the template image with a scale factor of 1 (the original template image). When it matches the scaled template image, it is considered detected. As shown in Fig. 9(c), for example, a scaled template image is generated within the range from a magnification factor of 0.7 to a magnification factor of 1.2, and compared with the captured image. This random template mode can also tolerate a certain distance between the imaging device 12 and the detection target.

[0067] In this embodiment, an example in which a helmet is used as the wearable has been described. However, it is possible to detect even a predetermined logo attached to the front of the helmet. FIG. 10 is a diagram for explaining an example of detecting a logo attached to the front of a helmet. In this case, an image of a helmet with a logo attached to the front side is prepared as a template. Then, using this template, a comparison is made with the captured image, and in addition to the shape of the helmet, entry can be permitted when the logos match. For example, as the logo, a logo mark indicating a specific company can be used. FIG. 10 shows an example in which a logo "ABC" is attached to the front side of the helmet. In this case, the wearable detection means 11b performs wearable detection in the wearable detection state S4, and when the logos match, it transmits an unlocking signal to the control device 14 assuming that the person wearing the legitimate helmet. In this way, by preparing an image of a helmet with a logo attached to the front side as a template image, only a person wearing a helmet whose use is permitted will be allowed to enter, and unauthorized entry by outsiders can be prevented.

[0068] <5. Inversion of the captured image> The display device 13 may display the captured image as it is in the captured image display area 13a without changing the orientation of the captured image, or may display it in a horizontally flipped state. The normal display of displaying it as it is and the inverted display of displaying it in a horizontally flipped state can be switched by setting. By performing the inverted display of displaying it in a horizontally flipped state, the user can confirm his / her wearing state in the same feeling as looking in a mirror.

[0069] As described above, the preferred embodiments of the present disclosure have been described. However, the present disclosure is not limited to the above embodiments, and various modifications are possible. For example, in the above embodiment, a computer is used as the processing device 11, and the CPU executes a dedicated program stored in the storage device to realize the person detection means, the wearable detection means, the wearing determination means, and the confirmation result storage means. However, each of these means may be incorporated into hardware as an arithmetic circuit.

Description of reference numerals

[0070] 11 ··· Processing device 11a ··· Person detection means 11b ··· Wearing item detection means 11c ··· Wearing determination means 11d ··· Confirmation result memory means 12 ··· Photographing device 13 ··· Display device 14 ··· Control device 15 ··· Automatic door 16 ··· Key box 17 ··· Tablet PC

Claims

1. A photographing device for photographing entrants, person detection means for performing image analysis on the photographed image acquired by the photographing device and detecting a person in the photographed image, clothing detection means for performing image analysis on the photographed image and detecting the clothing worn by the person, wearing determination means for determining the wearing state of the clothing based on the detection results of the person detection means and the clothing detection means, storage means for storing the wearing state determined by the wearing determination means and the photographed image as a confirmation result, having, the person detection means calculates a feature amount in the eye portion of the person in the photographed image and detects a person based on the feature amount, displaying the detection reach of the person by the person detection means and the detection reach of the clothing by the clothing detection means, The detection reach is calculated based on the number of photographed images analyzed for a predetermined time, and the number of photographed images in which a person or clothing is detected is used as the detection time, and is calculated as the reach of the detection time with respect to a preset detection reach time. A wearing confirmation system.

2. The display device further displays a message according to the processing state of the person detection means and the clothing detection means, and the processing rate of the photographed image. The wearing confirmation system according to Claim 1.

3. The wearing confirmation system according to Claim 1 or Claim 2, further comprising a control device for unlocking an automatic door or a key box based on the determination result of the wearing determination means.

4. The clothing detection means detects the clothing in the photographed image by comparing the attention area of the photographed image with the template image of the clothing. The wearing confirmation system according to any one of Claims 1 to 3.

5. The clothing detection means includes three modes: a normal mode using a single template image, a three-template mode using three template images of different sizes, and a random template mode using a scaled template image obtained by enlarging and reducing the original template image within a predetermined range. The wearing confirmation system according to Claim 4.

6. The template image is an image with a predetermined mark attached to the clothing. The wearing confirmation system according to Claim 4 or Claim 5.

7. A wearing confirmation method in which a computer connected to a photographing device for photographing entrants checks the state of clothing, wherein the computer, Perform image analysis on the captured image acquired by the imaging device, calculate the feature amount in the eye portion of the person in the captured image, and detect the person in the captured image based on the feature amount. Perform image analysis on the captured image and detect the accessories worn by the person. Determine the wearing state of the accessory based on the detection result of the person and the detection result of the accessory. Store the determined wearing state and the captured image as a confirmation result. Display the detection reach of the person and the detection reach of the accessory. The detection reach is calculated based on the number of captured images analyzed by image analysis within a predetermined time. The number of captured images in which a person or an accessory is detected is used as the detection time, and it is calculated as the reach of the detection time with respect to a preset detection reach time. A wearing confirmation method.

8. A computer connected to an imaging device that captures an entrant. A person detection means that performs image analysis on the captured image acquired by the imaging device, calculates the feature amount in the eye portion of the person in the captured image, and detects the person in the captured image based on the feature amount. An accessory detection means that performs image analysis on the captured image and detects the accessories worn by the person. A wearing determination means that determines the wearing state of the accessory based on the detection results of the person detection means and the accessory detection means. A storage means that stores the wearing state determined by the wearing determination means and the captured image as a confirmation result. Based on the number of captured images analyzed by image analysis within a predetermined time, calculate the number of captured images in which a person is detected as the first detection time, and calculate the detection reach of the person as the reach of the first detection time with respect to a preset detection reach time. Based on the number of captured images analyzed by image analysis within a predetermined time, calculate the number of captured images in which an accessory is detected as the second detection time, and calculate the detection reach of the accessory as the reach of the second detection time with respect to a preset detection reach time. A display means for displaying. A program for causing it to function as such.

Images