Conference Room Surveillance System
The conference room monitoring system addresses the risk of infectious diseases by tracking occupancy and proximity, alerting users to adjust their behavior to reduce transmission risks.
Patent Information
- Application Number
- JP2020174270
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-10-15
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2040-10-15
AI Technical Summary
Conventional monitoring systems fail to reduce the risk of infectious diseases in conference rooms during outbreaks, as they do not account for overcrowding or close proximity of individuals.
A conference room monitoring system that uses imaging units to track the number of people and their proximity within defined sections, alerting on overcrowding, close contact, and mask usage, with a control unit to manage notifications and adjust seating arrangements.
The system effectively reduces the risk of infectious disease transmission by alerting occupants to adjust their behavior, such as reducing crowding and mask usage, thereby minimizing infection risk.
Smart Images

Figure 0007577504000001 
Figure 0007577504000002 
Figure 0007577504000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a conference room monitoring system that monitors a conference room by using captured images. [Background technology]
[0002] Conventionally, there have been surveillance systems that detect specific individuals by analyzing surveillance camera images. For example, the system described in Patent Document 1 detects a target person by analyzing camera images and provides information to the target person. Incidentally, during times of infectious disease epidemics, gathering multiple people in a conference room or the like may increase the risk of infection among those people. However, conventional surveillance systems were not able to reduce the risk of infection in conference rooms and other locations. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2015-2477 A Summary of the Invention [Problem to be solved by the invention]
[0004] An object of the present invention is to provide a surveillance system that can reduce the risk of infection in a conference room or the like. [Means for solving the problem]
[0005] One aspect of the present invention includes an imaging unit that captures captured images of a conference room defined so that one or more people are positioned in each of a plurality of sections, and a control unit, and the control unit controls the imaging unit to continuously capture captured images of the conference room, extract images of people from the captured images, determine a state of the conference room based on the extracted images of people, and, if an abnormality is determined, perform a notification process to notify the user of the abnormality. The control unit extracts a head image, which is an image of a person, from the captured image, associates the head image with a partition based on the image size of the head image, and when one partition is associated with two or more head images, determines that two or more people are present in one partition and performs the notification process.The conference room monitoring device is configured to have the following features. Effect of the Invention
[0006] According to the present invention, it is possible to provide a surveillance system that can reduce the risk of infection in a conference room or the like. [Brief description of the drawings]
[0007] [Figure 1] FIG. 2 is a diagram showing a floor plan of a conference room (Room No. 1) in the first embodiment, a monitor, and a monitoring device. [Diagram 2] FIG. 1 is a block diagram of a conference room monitoring system according to a first embodiment. [Diagram 3] FIG. 2 illustrates a management terminal according to the first embodiment. [Figure 4] 4A and 4B are diagrams illustrating an allowable time-capacity table and an area-face image initial setting table stored in the monitoring device of the first embodiment. [Diagram 5] 3 is a diagram illustrating an image analysis history storage unit stored in the monitoring device of the first embodiment. FIG. [Figure 6] 4 is a diagram illustrating a section-face image monitoring table stored in the monitoring device of the first embodiment. FIG. [Figure 7] 4A to 4C are diagrams illustrating an allowable time-capacity table and a reservation information storage unit stored in the server of the first embodiment. [Figure 8] 4 is a flowchart of an initial setting process according to the first embodiment. [Figure 9] 4 is a flowchart of a monitoring process according to the first embodiment. [Figure 10] FIG. 13 is a diagram showing a captured image of the conference room (Room 1) in the first embodiment under normal conditions. [Figure 11] FIG. 13 is a plan view of the conference room (Room 1) in the first embodiment when an abnormality occurs. [Figure 12] FIG. 13 is a diagram showing a captured image when an abnormality occurs in the conference room (Room 1) in the first embodiment. [Figure 13] 4 is a flowchart of a notification process according to the first embodiment. [Figure 14]4 is a flowchart of a schedule management process according to the first embodiment. [Figure 15] 11 is a flowchart of a reservation postponement process according to the first embodiment. [Figure 16] 11 is a flowchart of a conference time extension process according to the first embodiment. [Figure 17] FIG. 13 is a diagram showing a floor plan of a conference room in a second embodiment. [Figure 18] FIG. 13 is a diagram illustrating a monitoring screen according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0008] (Embodiment) Hereinafter, an embodiment of the present invention will be described with reference to the drawings. (First embodiment) FIG. 1 is a diagram showing a plan view of a conference room 1 (Room No. 1) of the first embodiment, a monitor 2, and a monitoring device 40. As shown in FIG. FIG. 2 is a block diagram of the conference room monitoring system 10 of the first embodiment. FIG. 3 is a diagram illustrating the management terminal 11 of the first embodiment. FIG. 4 is a diagram for explaining the permissible time-capacity table 45b and the section-face image initial setting table 45c stored in the monitoring device 40 of the first embodiment. FIG. 5 is a diagram illustrating the image analysis history storage unit 45d stored in the monitoring device 40 of the first embodiment. FIG. 6 is a diagram illustrating the section-face image monitoring table 45e stored in the monitoring device 40 of the first embodiment. FIG. 7 is a diagram for explaining the permissible time-capacity table 65b and the reservation information storage unit 65h stored in the server 60 of the first embodiment.
[0009] The conference room monitoring system 10 reduces the risk of person-to-person transmission of infectious diseases by monitoring people staying in a conference room. In this embodiment, a system relating to multiple conference rooms 1 in the same facility will be described. As shown in FIG. 2 etc., the conference room monitoring system 10 of the embodiment relates to three conference rooms 1 (rooms 1 to 3) as an example.
[0010] As shown in Fig. 1(A), the plan view of conference room 1 (Room 1) in the embodiment is a rectangle that is elongated in the vertical direction Y of Fig. 1(A), and a rectangular table 3 that is elongated in the vertical direction Y is placed in the center of conference room 1 (Room 1). The capacity of conference room 1 (Room 1) is eight people. On the left and right of table 3, sections L1 to L4 and R1 to R4 are defined, each of which can accommodate four people. The capacity of conference room 1 (rooms 1 to 3) varies from conference room to conference room depending on the size of the room, the tables 3, etc., and is 8, 10, and 12 people, respectively. In the following explanation, conference room 1 (room 1) is mainly used as an example. Note that the conference room monitoring system 10 can handle various conference rooms with capacities other than 8, 10, and 12 people or seating arrangements different from those shown in FIG. 1(A) by using initial settings described later.
[0011] The seating arrangement may be such that, for example, in a configuration in which each seat comprises one desk and one chair, the seats all face in one direction (a so-called school formation), and in this configuration, the seats may be arranged in two or more rows (for example, three rows (right, center, left), etc.). In addition, this system is not limited to conference rooms, but can be applied to any space where multiple people are located in a designated area and talking, such as classrooms, rooms used for meetings, reception rooms, etc.
[0012] As shown in FIG. 1, a monitor 2 may be installed in a conference room 1 . The monitor 2 is, for example, a large display device that can be viewed by everyone in the conference room 1, and may be a device that is widely available on the market. The monitor 2 is, for example, fixed to the wall of the conference room 1. The monitor 2 is used, for example, when the conference room 1 is in use, to display the screen of a terminal owned by the user.
[0013] (Configuration of the conference room monitoring system 10) As shown in FIG. 2, the conference room monitoring system 10 includes a management terminal 11, a user terminal 12, a router 13, a room system 30, and a server 60. The management terminal 11, the router 13, and the room system 30 constitute a system for the facility in which the conference room 1 is installed. The management terminal 11, user terminal 12, router 13, and room system 30 are connected to a communication network 15 within the facility such as a LAN (which may include a wireless LAN in part), and can communicate as necessary. The conference room monitoring system 10 also uses a cloud computing system for processing abnormality notifications for the conference room 1. Each device 40, 50, 59 of the facility's room system 30 and the server 60 can communicate as necessary via a communication network 16 such as the Internet.
[0014] In the embodiment, a computer refers to an electronic calculator equipped with a storage device, a control device, an arithmetic device, etc., and each of the devices 11, 12, 40, 50, 59, 60 constituting the present system is equipped with a storage unit, a control unit, etc., and is included in the concept of a computer. Also, each of the devices 11, 12, 40, 50, 59, 60 is not limited to being composed of a single electronic calculator, and may be composed of multiple electronic calculators as necessary. The storage unit is a storage device such as a hard disk or a semiconductor memory device that stores programs, information, etc. required for the operation of each of the devices 11, 12, 40, 50, 59, and 60. The control unit is a device for performing calculations required for the operation of each of the devices 11, 12, 40, 50, 59, and 60, and for comprehensively controlling each of the devices 11, 12, 40, 50, 59, and 60. The control unit is composed of, for example, a CPU (Central Processing Unit) etc. The control unit realizes various functions of the embodiment by appropriately reading and executing various programs stored in the storage unit.
[0015] The management terminal 11 is a terminal operated by a manager of the conference room 1. The management terminal 11 may be a computer dedicated to the system or a general-purpose personal computer (desktop type, laptop type, tablet type, etc.). The management terminal 11 is mainly used to perform initial settings of the monitoring device 40. The management terminal 11 includes an operation unit 11a, a display unit 11b, a storage unit 11c, and a control unit 11d. As shown in FIGS. 2 and 3, the operation unit 11a is an input device such as a mouse and a keyboard. The display unit 11b is a display device such as a liquid crystal display device.
[0016] The user terminal 12 is a terminal operated by a user of the conference room 1. The user terminal 12 is, for example, a personal computer similar to the management terminal 11, or a multi-function mobile phone. The user terminal 12 is mainly used when reserving the use of the conference room 1 by starting an installed application or connecting to a web server. Note that the user terminal 12 owned by an administrator or the like may have the functions of the management terminal 11. The user terminal 12 includes an operation unit 12a, a display unit 12b, a storage unit 12c, and a control unit 12d. The operation unit 12 a and the display unit 12 b are similar members to the operation unit 11 a and the display unit 11 b of the management terminal 11 .
[0017] (Room System 30) The room system 30 is a system made up of devices arranged in each conference room 1. The room system 30 includes a monitoring device 40 (imaging device), a smart speaker 50 (audio output device), and a door tablet 59. These devices 40, 50, and 59 arranged in the same conference room 1 are associated with each other by being paired in advance. These devices 40, 50, and 59 can communicate with each other as necessary via a communication network 15 within the facility. The monitoring device 40 is a terminal that controls and processes the main functions of the conference room monitoring system 10. The monitoring device 40 can be any of various computers with a built-in camera 43 and a computer to which the camera 43 is connected. In the embodiment, an example is described in which the monitoring device 40 is a computer with a built-in camera 43. In the embodiment, an example is shown in which the monitoring device 40 does not have a dedicated input device or display device and is operated by the user terminal 12 or the like, but is not limited to this, and the monitoring device 40 itself may be equipped with an input device, a display device, etc. For example, the monitoring device 40 and the monitor 2 may be a single monitor-integrated computer (for example, Surface Hub (registered trademark) (sold by Microsoft Japan Co., Ltd.), etc.). 1(B), the monitoring device 40 is attached directly below the monitor 2. The mounting form of the monitoring device 40 is not limited, and for example, double-sided tape, mounting fixtures, etc. can be used. Furthermore, the placement of the monitoring device 40 is not limited to directly below the monitor 2, and it is sufficient that all sections are included within the shooting angle of view θ43 of the camera 43. For example, the monitoring device 40 may be attached directly above the monitor 2, or may be placed at a position away from the monitor 2.
[0018] As shown in FIGS. 1 and 2, the monitoring device 40 includes a communication unit 42, a camera 43 (imaging unit), a storage unit 45, and a control unit 46. The communication unit 42 is an interface for communicating various types of information with external devices, and includes, for example, a cable connection terminal for a wired LAN, a communication device for a wireless LAN, and the like. The camera 43 is an imaging device that captures an image of the inside of the conference room. The camera 43 is arranged so that eight sections are included within the imaging angle of view θ43. In the embodiment, the camera 43 is arranged so that the left and right sections are symmetrical in the captured image.
[0019] The memory unit 45 has a monitoring program 45a, which is a program related to the main functions of the conference room monitoring system 10, an allowable time-capacity table 45b, an area-face image initial setting table 45c, an image analysis history memory unit 45d, and an area-face image monitoring table 45e, and also temporarily stores information required for various processing. The monitoring program 45a may be stored in advance when the conference room monitoring system 10 is installed, or may be downloaded from a digital content distribution server or the like. The allowable time-capacity table 45b, the section-face image initial setting table 45c, and the image analysis history storage unit 45d are created by the initial setting process (see FIG. 8). The image analysis history storage unit 45d and the section-face image monitoring table 45e are created by the monitoring process (see FIG. 9). The details of these tables will be described later. The control unit 46 performs various controls and processes related to the operation of the conference room monitoring system 10 .
[0020] The smart speaker 50 is a speaker device also called an AI speaker. The smart speaker 50 includes a microphone 51 (audio input / output unit), a communication unit 52, a speaker 53 (audio output unit), a storage unit 55, and a control unit 56. The smart speaker 50 has an AI assistant function that supports interactive voice operation, and is a device that can be operated by recognizing voice with the built-in microphone 51. This allows users in the conference room to conveniently operate the smart speaker 50 and the monitoring device 40 by voice. The communication unit 52, like the communication unit 42, is a device for communicating with an external device.
[0021] The door tablet 59 is a tablet computer or the like installed at the door 4 of the conference room 1 . The display unit 59b of the door tablet 59 displays information about the usage status and schedule of the conference room 1.
[0022] (Server 60) The server 60 is a server that provides a cloud computing service. The server 60 is a concept that includes a server operated in a serverless computing system. The server 60 includes a storage unit 65 and a control unit 66 . In the embodiment, an example is shown in which server 60 handles both the process related to monitoring of conference room 1 (i.e., the process related to abnormalities in conference room 1) and the process related to reservations for conference room 1, but these processes may be performed by independent servers. In this case, the server related to monitoring of conference room 1 and the server related to reservations for conference room 1 may communicate with each other as necessary.
[0023] The memory unit 65 has an allowed time-capacity table 65b, an area-face image initial setting table 65c, an image analysis history memory unit 65d (analysis result memory unit), an area-face image monitoring table 65e, an alarm voice memory unit 65g, and a reservation information memory unit 65h. 7(A), the allowable time-capacity table 65b stores the capacity and the allowable time in association with each other for all the conference rooms 1 (Rooms 1 to 3). The information stored in the allowable time-capacity table 65b is the same as the allowable time-capacity table 45b stored in the monitoring device 40 of each conference room 1 of each room system 30. Although not shown in the figure, the section-face image initial setting table 65c, image analysis history memory unit 65d, and section-face image monitoring table 65e similarly store the same information as the section-face image initial setting table 45c, image analysis history memory unit 45d, and section-face image monitoring table 45e stored in the monitoring device 40 of each room system 30.
[0024] The notification voice storage unit 65g stores a notification voice for notifying an abnormality in the conference room 1. As shown in FIG. 7B, the reservation information storage unit 65h stores reservation information for each conference room 1. The reservation information is information that associates the room number (1 to 3) of the conference room 1, the reservation start time, the reservation end time, and the reservation ID. The reservation ID is identification information for each piece of reservation information. Although a detailed description will be omitted, the control unit 46 of the server 60 accepts a reservation from the user terminal 12 or the like, assigns a reservation ID to the reservation information, and stores it in the reservation information storage unit 65h. The configuration in which the server 60 manages reservation information for multiple conference rooms 1 may be such that, for example, by using a scheduler using a web browser or a schedule management application, the reservation information for the conference rooms 1 can be managed (such as registering reservation information) from a user terminal 12, etc.
[0025] (Operation of the conference room monitoring system 10) The operation of the conference room monitoring system 10 will now be described. FIG. 8 is a flowchart of the initial setting process according to the first embodiment. FIG. 9 is a flowchart of the monitoring process according to the first embodiment. FIG. 10 is a diagram showing a captured image of the conference room 1 (room 1) in the first embodiment under normal conditions. FIG. 11 is a plan view of the conference room 1 (room 1) in the first embodiment when an abnormality occurs. FIG. 12 is a diagram showing a captured image when an abnormality occurs in the conference room 1 (room 1) in the first embodiment. FIG. 13 is a flowchart of the notification process according to the first embodiment. It should be noted that the text information and the like in the captured images 71 and 72 shown in FIGS. 10 and 12 are displayed for the sake of convenience. Moreover, the operation of the conference room monitoring system 10 will be mainly described with reference to conference room 1 (room 1), and the room number will be omitted as appropriate.
[0026] (Initial setting process) As a prerequisite for using the conference room monitoring system 10, the manager of the conference room 1 or the like performs initial settings of the monitoring device 40 by operating the management terminal 11. In addition, the manager of the conference room 1 or the like has people sit in chairs arranged in the section of the conference room 1 during the initial settings. In the embodiment, eight people, which is the capacity of conference room 1, sit in chairs arranged in sections L1 to L4 and R1 to R4. The capacity here refers to the capacity set by the administrator of the conference room, etc., in system operation. The capacity in system operation can be set lower than that in normal use (when the conference room is used when precautions against the risk of infection of infectious diseases are not necessary) in order to increase the distance between people, and can be set to, for example, 50% or less of that in normal use. Each figure illustrates the layout of the conference room corresponding to the capacity in system operation. 3, at the time of initial setting, an initial setting screen 11f is displayed on the display unit 11b of the management terminal 11. The manager of the conference room 1 or the like operates the operation unit 23 in accordance with the initial setting screen 11f.
[0027] In S1, the control unit 46 of the monitoring device 40 receives input of initial setting information. In this case, the monitoring device 40 acquires the input information of the management terminal 11 via the server 60 . As shown in FIG. 3, in the initial setting process, the control unit 11d of the management terminal 11 displays an initial setting screen 11f. The initial setting screen 11f is a screen for accepting input of the room number 11g of the conference room 1 in which the room system 30 for which initial setting is to be performed is installed, the allowable conference time 11h, and the capacity 11i. The allowable time is the time that the conference room can continue. The capacity of the conference room 1 is equal to the number of partitions. The allowable time and the capacity of the conference room 1 can be determined, for example, by the manager of the conference room 1, taking into consideration the size of the conference room 1, etc.
[0028] FIG. 3 shows an example in which the room number 11g, the allowable time 11h, and the capacity 11i are input as Room 1, 60 minutes, and 8 people, respectively. When the manager of the conference room 1 decides on the input information, he or she simply operates the decision button 11j. In response to this, the control unit 11d of the management terminal 11 transmits to the server 60 the input information on the room number, the allowable time, and the capacity. When the control unit 66 of the server 60 receives the input information of the room number, the allowable time, and the capacity, the control unit 66 stores this information in the allowable time-capacity table 65b. Then, the control unit 66 transmits this input information to the monitoring device 40. When the control unit 46 of the monitoring device 40 receives the input information of the room number, the allowable time, and the capacity, it stores this information in the allowable time-capacity table 45b (see FIG. 4).
[0029] The control unit 66 of the server 60 receives the initial settings of the room number, the allowable time, and the capacity for each of the three conference rooms 1 (Rooms 1 to 3). As shown in the permissible time vs. capacity table 65b in FIG. 7(A), the control unit 66 of the server 60 stores the capacity and permissible time in association with each other for the three conference rooms 1 (Rooms 1 to 3). As shown in the permissible time-capacity table 45b, the control unit 46 of the monitoring device 40 installed in each conference room 1 stores the capacity and permissible time of each conference room 1 in association with each other.
[0030] In S2, the control unit 46 of the monitoring device 40 acquires a captured image of the conference room 1 by controlling the camera 43 in response to receiving input information on the permitted time and the capacity from the server 60 (S1). In S3, the control unit 46 of the monitoring device 40 performs image recognition processing on the acquired captured image to extract a face image (head image) that is an image of a person's face. A known method can be used for the image recognition processing, and for example, after detecting a person by extracting a moving area from the captured image, the person's face can be further recognized based on images of facial parts (eyes, nose, mouth, etc.). In this embodiment, eight face images FL1 to FL4 and FR1 to FR4 (see FIG. 10) are extracted from the captured image 71. Note that the captured image at the time of initial setting is different from the captured image 71 at the time of monitoring processing (normal state) described later, but in the embodiment, it will be appropriately used for the description.
[0031] In S4, the control unit 46 of the monitoring device 40 performs a process of associating the eight face images FL1 to FL4, FR1 to FR4 with the sections L1 to L4, R1 to R4. In this case, the control unit 46 divides the eight face images FL1 to FL4 and FR1 to FR4 in the captured image 71 into left face images FL1 to FL4 and right face images FR1 to FR4 according to the left and right positions. Then, the control unit 46 acquires the image sizes of the facial images FL1-FL4, FR1-FR4 based on the captured image 71 of the camera 43. The image size can be expressed, for example, in units of pixels or the like, as the maximum outer dimensions (vertical and horizontal sizes) of the facial images FL1-FL4, FR1-FR4. As shown in the section-face image initial setting table 45c in Fig. 4(D), in this embodiment, for convenience of explanation, the "length x width" of the maximum outer shape of the largest face image FL1 is set to "100 x 100", and the "width x height" of the maximum outer shape of the smallest face image FL4 is set to "25 x 25" (hereinafter, the description of "width x height" will be omitted, and it may be simply referred to as "25 x 25" or the like). Then, based on these ratios and the ratios of the image sizes of the face images FL1 to FL4, the image sizes of the face images FL2 and FL3 are calculated and obtained as "37.5 x 37.5" and "75 x 75", respectively. Similarly, the control unit 46 also acquires the image sizes of the face images FR1 to FR4. 1(A) and the like, the sections L1 to L4 and the sections R1 to R4 are at the same position in the vertical direction Y (the optical axis direction of the camera 43). Therefore, two corresponding face images on the left and right (for example, face images FL1, FR1) usually have the same size. Therefore, the control unit 46 may omit acquiring the face images FR1 to FR4 and set the image size of the face images FR1 to FR4 to the same image size as the face images FL1 to FL4.
[0032] 10 etc., the size of the face image becomes smaller as the distance between the person and the camera 43 becomes smaller. Also, face images FL1-FL4 of people present in sections L1-L4 on the left side are positioned on the left side in the captured image 71, while face images FR1-FR4 of people present in sections R1-R4 on the right side are positioned on the right side in the captured image 71. For this reason, in the section-face image initial setting table 45c, face images FL1, FL2, FL3, and FL4 located on the left side of the captured image 71 are arranged in sections L1, L2, L3, and L4 of the section-face image initial setting table 45c in descending order of size. Similarly, in the section-face image initial setting table 45c, face images FR1, FR2, FR3, and FR4 on the right side are arranged in the order of sections R1, R2, R3, and R4. This allows the section-face image initial setting table 45c to store information corresponding to the arrangement of people in the conference room 1. Through the above process, the sections L1 to L4, R1 to R4 and the face images FL1 to FL4, FR1 to FR4 are associated with each other based on the image size and left and right position.
[0033] In S5, the control unit 46 of the monitoring device 40 sets a threshold value. As shown in the section-face image initial setting table 45c in FIG. 4(D), the control unit 46 calculates the median size between the sizes of the face image, and sets this as the threshold value. In the example of Fig. 4(D), the maximum value 112.5 of the largest face image FL1 and the minimum value 12.5 of the smallest face image FL4 are also calculated by taking other thresholds into consideration. In addition, for the sake of simplicity, the sizes of the left and right face images and the vertical and horizontal sizes are set to the same values. The threshold value is used in the close contact determination process described later.
[0034] With this, the control unit 46 ends the initial setting process. The control unit 46 creates an allowable time-capacity table 45 b and a section-face image initial setting table 45 c by initial setting, and stores this information in the storage unit 45 .
[0035] In principle, the initial settings are not required to be performed after the conference room monitoring system 10 is installed. However, if the layout of the sections in the conference room is changed due to a change in the layout of the tables 3, the capacity of the room is changed, or the layout of the monitoring device 40 is changed, the initial settings will need to be performed again.
[0036] (Monitoring process) The monitoring process is a process performed when a conference is held in the conference room 1, and is mainly performed by the monitoring device 40. As shown in Fig. 9, in S10, the control unit 46 of the monitoring device 40 judges whether or not a conference has started. This judgment can be made by performing a process similar to "process of acquiring a captured image and extracting human images, face images, etc. from the captured image" (see S2 and S3 in Fig. 8), and judging that a conference has started if a human image, etc. is extracted. In other words, this is because if people are present in the conference room 1 due to the start of the conference, a human image, etc. will be extracted from the captured image. If the control unit 46 determines that a conference has started (S10: YES), the process proceeds to S11. On the other hand, if the control unit 46 determines that a conference has not started (S10: NO), the process repeats the process of S10.
[0037] In S11, similarly to S2 and S3, the control unit 46 of the monitoring device 40 controls the camera 43 to obtain a captured image of the conference room 1 and extracts human images, facial images, etc. Then, the control unit 46 obtains the position and size of each facial image in the captured image.
[0038] (Counting people (obtaining crowd information)) In S12, the control unit 46 of the monitoring device 40 counts the number of people present in the conference room by analyzing the captured image acquired in S11, similar to S3 at the time of initial setting. If the counted number of people is greater than the capacity of the conference room 1, it can be regarded as a state in which multiple people are crowded in the conference room 1. The control unit 46 stores information on the number of people present in the conference room in the image analysis history storage unit 45d. Although not shown in the figure, the control unit 46 of the monitoring device 40 transmits information on the counted number of people to the server 60, and upon receiving this, the control unit 66 of the server 60 stores it in the image analysis history storage unit 65d.
[0039] (Creating the area-face image monitoring table 45e (obtaining close contact information)) In S13, the control unit 46 of the monitoring device 40 performs a process of associating the extracted face images with the sections, thereby creating a section-face image monitoring table 45e, and stores this in the storage unit 45. Although not shown in the figure, the control unit 46 of the monitoring device 40 transmits information on the section-face image monitoring table 45e to the server 60, and upon receiving this, the control unit 66 of the server 60 stores it in the storage unit 65 as the section-face image monitoring table 65e.
[0040] In this process, the control unit 46 selects a partition corresponding to the size of the face image by referring to the thresholds in the partition-face image initial setting table 45c. The section-face image monitoring table 45e (D10300) in FIG. 6(A) is created based on the captured image 71 shown in FIG. For example, the face image FL3 extracted from the captured image 71 has an image size of "40 x 40". This image size is within the range of the image size of the face image in section L3 in the section-face image initial setting table 45c, "(37.5 to 62.5) x (37.5 to 62.5)". Therefore, the control unit 46 associates this face image FL3 with the section L3. Similarly, the control unit 46 associates the other left face images FL1, FL2, and FL4 with the sections L1, L2, and L4, respectively. Similarly, the control unit 46 associates the four face images FR1 to FR4 on the right side with the face images FR1 to FR4, respectively, based on the image sizes. Through such processing, the control unit 46 creates a section-face image monitoring table 45e (D10300) and stores it in the storage unit 45.
[0041] On the other hand, the section-face image monitoring table 45e (D11050) in FIG. 6(B) is created based on the captured image 72 shown in FIG. 12, with the people in the conference room being arranged as shown in FIG. In this example, the facial image FL3 extracted from the captured image 72 has an image size of "65 x 65". This image size is within the range of the image size of the facial image in section L2 in the section-facial image initial setting table 45c, "(62.5 to 87.5) x (62.5 to 87.5)". Therefore, the control unit 46 associates this facial image FL2 with section L3. Furthermore, the facial image FL2 has an image size of 70×71, which is also within the range of the image size of the facial image in section L2 in the section-facial image initial setting table 45c, that is, (62.5 to 87.5)×(62.5 to 87.5). Therefore, the control unit 46 also associates this facial image FL2 with section L2. Therefore, the section-face image monitoring table 45e (D11050) stores information that the two face images FL2 and FL3 are associated with the same section L2. Here, a state in which two or more face images are associated with one section means that two or more people are present in one section. Therefore, this state can be regarded as a state in which two or more people are close to each other (for example, the distance between people is within 1 to 2 m, etc.).
[0042] (Acquisition of information on mask non-wearing) In S14, the control unit 46 of the monitoring device 40 analyzes the features of each face image acquired in S11 to determine whether or not a mask image is included in the face image. There are various methods for this determination, for example, a method in which a plurality of mask images are stored in the storage unit 45 and the degree of match between the features of each face image and the mask image is determined. Here, if a mask image is not included, it can be assumed that the person corresponding to the mask image is not wearing a mask (a state in which the person is not wearing a mask). As shown in FIG. 4(D), the control unit 46 stores information on the section associated with the face image in which a mask is not being worn in the image analysis history storage unit 45d. Although not shown in the figure, the control unit 46 of the monitoring device 40 transmits information about the section corresponding to the face image in which a mask is not being worn to the server 60, and upon receiving this, the control unit 66 of the server 60 stores it in the memory unit 65.
[0043] In S15, the control unit 46 of the monitoring device 40 determines whether one minute has passed since the previous determination in S15 was YES. If the control unit 46 determines that one minute has passed (S15: YES), the control unit 46 proceeds to S16, whereas if the control unit 46 determines that one minute has not passed (S15: NO), the control unit 46 repeats the process from S11.
[0044] By the process of S15, the control unit 46 of the monitoring device 40 can store information on the number of people in the conference room 1 for one minute, information on the section-face image monitoring table, and information on people not wearing masks. The control unit 46 of the monitoring device 40 performs a series of processes from S11 to S15 (i.e., one loop time) in 10 seconds. That is, the control unit 46 performs the process of S15 at 10-second intervals. Therefore, the control unit 46 obtains six records in one minute. For the sake of convenience, the image analysis history storage unit 45d in FIG. 5 shows an example in which the first record is acquired at exactly 00 seconds in the time unit of seconds, and the sixth record is acquired at exactly 50 seconds in the time unit of seconds. Therefore, when the time reaches 50 seconds (S15: YES), the control unit 46 of the monitoring device 40 proceeds to S16. The monitoring device 40 also transmits the information acquired in S11 to S14 to the server 60. For this reason, the server 60 associates information similar to that in the image analysis history storage unit 45d with the room number of the conference room 1, and stores the information in the image analysis history storage unit 65d.
[0045] By the above process, the process from S16 onwards is repeated every minute after the conference starts. The process up to S15 and the process from S16 onwards may be realized by continuous serial processing, or may be operated independently and realized by asynchronous processing via a database.
[0046] (Crowding determination process) In S16, the control unit 46 of the monitoring device 40 judges whether or not the state is crowded by referring to the image analysis history storage unit 45d. In this case, the control unit 46 judges whether or not the six counted numbers acquired in one minute exceed the capacity of the conference room 1. If the counted number of people exceeds the capacity of the conference room 1 (S16: YES), the control unit 46 proceeds to S16a, but if it determines that the number of people does not exceed the capacity (S16: NO), the control unit 46 proceeds to S17. For example, in the record from 10:30:00 to 10:30:50 corresponding to the scenes shown in FIG. 1(A) and FIG. 10, the number of people counted in six places is eight, which is less than the capacity of the conference room 1 (S16: NO), so the control unit 46 proceeds to S17. On the other hand, in the period from 11:05:00 to 11:05:50 corresponding to the scenes shown in FIGS. 11 and 12, the number of people counted in five records is nine, which exceeds the capacity of conference room 1 (S16: YES), so that control unit 46 proceeds to S16a.
[0047] In S16a, the control unit 46 of the monitoring device 40 performs a crowded state notification process. The control unit 46 performs the crowded state notification process in accordance with the notification process (see FIG. 13). Details will be described later, but in the crowded state notification process, the smart speaker 50 outputs a voice message such as "The capacity has been exceeded and the state is crowded." Thereafter, the control unit 46 proceeds to S20.
[0048] (Closeness determination process) In S17, the control unit 46 of the monitoring device 40 judges whether or not the close contact state exists by referring to the image analysis history storage unit 45d. In this case, the control unit 46 judges whether or not the close contact state exists in which two or more face images are associated with one section by further referring to the section-face image monitoring table 45e based on the image analysis history storage unit 45d. Then, it judges whether or not 30% or more of the six records are in the close contact state. By setting a ratio such as 30% in this manner, the monitoring device 40 can accurately determine whether or not a close contact state exists even if there is a processing error when acquiring the close contact information (S13 above), and can also ignore close contact states that exist for a short period of time. This ratio may be set by an administrator or the like as appropriate. If the control unit 46 determines that the close contact state exists (S17: YES), the process proceeds to S17a. On the other hand, if the control unit 46 determines that the close contact state does not exist (S17: NO), the process proceeds to S18.
[0049] Here, for example, when 10:30:50 has elapsed, the control unit 46 determines the close contact state based on six records at the times of 10:30:10, 10:30:20, . . . , and 10:30:50. In the captured image 71 at 10:30:00 in FIG. 10 and the section-face image monitoring table 45e (D10300) in FIG. 6(A), face images FL1-FL4, FR1-FR4 correspond to eight sections L1-L4, R1-R4, respectively. Therefore, the record at 10:30:50 is not a close contact state. Although not shown, the other five records are similarly not a close contact state. Therefore, the control unit 46 determines that the close contact state is not present because less than 30% of the records are in a close contact state.
[0050] Furthermore, for example, when 11:05:50 has elapsed, the control unit 46 determines the close contact state based on six records at the times of 11:05:10, 11:05:20, . . . , and 11:05:50. At the time of 11:05:10, as shown in the captured image 72 in Fig. 12 and the section-face image monitoring table 45e (D11050) in Fig. 6(B), two face images FL2 and FL3 correspond to section L2 (S17: YES), so it is a close state. Although not shown, it is assumed that the other five records are not in a close state. In this case, the control unit 46 determines that it is a close state because the close state records are 30% or more.
[0051] In a meeting, for example, two or more people may become closer to each other when two or more people are sharing and viewing a document, when an emotional discussion is taking place, etc. The control unit 46 can determine that the two or more people have become closer to each other in such a situation by the process of S17.
[0052] In S17a, the control unit 46 of the monitoring device 40 performs a close contact state notification process. Although details will be described later, in the close contact state notification process, a voice such as "People are in a close contact state" is output from the smart speaker 50. After that, the control unit 46 proceeds to S20.
[0053] (Mask not worn) In S18, the control unit 46 of the monitoring device 40 determines whether or not a mask is not being worn by referring to the image analysis history storage unit 45d. In this case, the control unit 46 determines that a mask is not being worn when the rate of mask non-wearing exceeds 70%, that is, when the percentage of records of non-mask wearing exceeds 70% among the six records acquired in one minute. By setting a ratio such as 70% in this manner, similar to S16 above, it is possible to accurately determine whether a mask is not being worn, and also to ignore a short period of time when a mask is not being worn. If the control unit 46 determines that the mask is not being worn (S18: YES), the process proceeds to S18a. On the other hand, if the control unit 46 determines that the mask is not being worn (S18: NO), the process proceeds to S19.
[0054] For example, when 10:30:50 has passed, the control unit 46 determines the mask-not-wearing state based on the six records at 10:30:10, 10:30:20, ..., 10:30:50. Of these six records, five records are not mask-not-wearing, and one record (10:30:30) is mask-not-wearing, so the mask-not-wearing rate is about 17%. Therefore, the control unit 46 determines that the mask is not not-wearing state when 10:30:50 has passed (S18: NO).
[0055] Furthermore, for example, when 11:05:50 has elapsed, the control unit 46 determines the mask-not-wearing state based on six records at 11:05:10, 11:05:20, ..., 11:05:50. Of these six records, one record (11:05:20) is not a mask-not-wearing state, and five records (10:30:30) are mask-not-wearing states, so the mask-not-wearing rate is approximately 83%. Therefore, the control unit 46 determines that the mask is not-wearing state when 111:05:50 has elapsed (S18: YES).
[0056] In S18a, the control unit 46 of the monitoring device 40 performs a mask non-wearing notification process. Although details will be described later, in the mask-not-wearing notification process, a voice such as "There is someone in the conference room who is not wearing a mask" is output from the smart speaker 50. After that, the control unit 46 proceeds to S20.
[0057] (Time exceeded judgment) In S19, the control unit 46 of the monitoring device 40 compares the time that has elapsed since the start of the conference with the allowed time of 60 minutes. The starting point of the elapsed time can be set appropriately according to the specifications, etc., and can be set, for example, to the time when the face image of the first person is detected, the start time of the scheduled conference, etc. If there is an infected person among the people in the conference room, continuing the conference beyond the allowed time increases the risk of infection to other people in the conference room. If the control unit 46 determines that the elapsed time of the conference has exceeded the allowed time (S19: YES), the process proceeds to S19a. On the other hand, if the control unit 46 determines that the elapsed time of the conference has not exceeded the allowed time (S19: NO), the process proceeds to S20. For example, in the meeting scene in Fig. 10, the elapsed time of the meeting is 30 minutes, which is within the allowable time of 60 minutes (S19: NO), so the control unit 46 proceeds to S20. On the other hand, in the meeting scene in Fig. 12, the elapsed time of the meeting is 65 minutes, which is 5 minutes longer than the allowable time of 60 minutes (S19: YES), so the control unit 46 proceeds to S19a.
[0058] In S19a, the control unit 46 of the monitoring device 40 performs a conference time expiration notification process. Although details will be described later, in the conference time exceeding notification process, a voice such as "The conference time has exceeded" is output from the smart speaker 50. After that, the control unit 46 proceeds to S20.
[0059] (Decision on end of meeting) In S17, the control unit 46 of the monitoring device 40 judges whether the conference has ended. This judgment can be made by performing the "process of extracting human images, etc. from captured images" (see S2 and S3 in FIG. 8), and judging that the conference has ended if no human images, etc. are extracted. In other words, this is because if the conference room 1 is vacant due to the end of the conference, no human images, etc. are extracted from the captured images. If the control unit 46 determines that the conference has ended (S20: YES), it repeats the process from S10. On the other hand, if the control unit 46 determines that the conference has not ended (S17: NO), it repeats the process from S11. The control unit 46 waits for the start of the next conference by repeating the process from S10 (S17: YES). Also, the control unit 46 continues to monitor the inside of the conference room of the ongoing conference by repeating the process from S11 (S17: NO).
[0060] (Notification processing) As shown in FIG. 13, in S30, the control unit 46 of the monitoring device 40 transmits information about the analysis result (i.e., that an abnormality has been determined in the conference room (either "S16: YES", "S17: YES", "S18: NO", or "S19: YES")) to the server 60 in a manner that allows the four types of abnormality to be identified and in a form that does not include imaging information. The information about the type of abnormality may be an identification signal corresponding to the type of abnormality, or may be audio text information corresponding to the type of abnormality (for example, if the capacity is exceeded (S16: YES), text information such as "The conference room is over capacity and is crowded").
[0061] In S31, when the control unit 66 of the server 60 receives information on the type of abnormality, it stores information on the analysis result converted into voice, as well as information on the notification voice corresponding to the type of abnormality (analysis result voice information), in the notification voice memory unit 65g. In a specification in which the information on the abnormality type is an identification signal or the like, the storage unit 65 may store the identification signal and the notification sound in the form of a table in association with each other. In this case, the control unit 66 may read out the notification sound corresponding to the identification signal received from the monitoring device 40 by referring to the information in the storage unit 65, and store this in the notification sound storage unit 65g. In a specification in which the information on the abnormality type is text information of a sound corresponding to the abnormality type, the control unit 66 may convert the text information into the notification sound, and store this in the notification sound storage unit 65g.
[0062] In S32, the control unit 66 of the server 60 transmits to the monitoring device 40 the URI (analysis result storage location information) of the storage location of the notification voice stored in S31, that is, information on the storage location of the analysis result on the network. In S33, in response to receiving the URI information from the server 60, the control unit 46 of the monitoring device 40 transmits the URI information to the smart speaker 50.
[0063] In S34, in response to receiving the URI information from the monitoring device 40, the control unit 56 of the smart speaker 50 requests the server 60 to transmit the information of the notification sound stored in the URI. In S35, the control unit 66 of the server 60 reads out the information of the notification sound stored in S31 in response to a request from the smart speaker 50, and transmits this to the smart speaker 50. In S36, when the control unit 56 of the smart speaker 50 receives the notification sound information from the server 60, it outputs it from the speaker 53. For example, in the crowded state notification process, the smart speaker 50 outputs a voice message such as "The capacity has been exceeded and the place is crowded." Thereafter, the control unit 46 returns to the monitoring process (S20) of FIG.
[0064] Through the above process, the conference room monitoring system 10 can report the risk of infection from the smart speaker 50 when the monitoring device 40 detects an abnormality in the conference room. Although the conference room monitoring system 10 uses a cloud computing configuration, the room system 30 in the facility transmits the result of the abnormality determination to the server 60, but does not transmit information on the captured image. This allows the conference room monitoring system 10 to improve the safety of personal information of users of the conference room 1, information on the conference content, etc.
[0065] Furthermore, if there is an abnormality in the crowded determination process, close contact determination process, mask wearing determination process, or time exceeded determination process, a warning (audio) indicating the abnormality is output from the smart speaker 50. By hearing these warnings, people in the conference room can take measures such as reducing the number of people in the conference room to below capacity, separating people who are in close contact with each other, and encouraging people to wear masks. Furthermore, if the conference time exceeds the allowed time, people in the conference room can take measures such as ending the conference as soon as possible. If abnormal conditions persist in the crowded determination process, close contact determination process, mask wearing determination process, and time exceeded determination process due to the repeated processing from S11 onwards, the warning continues to be output from the smart speaker 50. People in the conference room can reduce the risk of infection by taking the above-mentioned measures to stop the warning from being continuously output.
[0066] As described above, the conference room monitoring system 10 of this embodiment warns that there is a high risk of infection by analyzing captured images in the conference room 1. In response to this warning, people in the conference room 1 can take measures to reduce the risk of infection.
[0067] In addition, a user in the conference room 1 can check the situation of the conference room 1 at any time. In this case, the user can input a command such as "Tell me the current situation" to the smart speaker 50 by voice. In response to this, the smart speaker 50 requests the image analysis history from the server 60. In response to the request from the smart speaker 50, the server 60 reads the latest image analysis history from the image analysis history storage unit 65d, converts it into voice information, and transmits it to the smart speaker 50. The smart speaker 50 can then output this voice information from the speaker 53. The content of the voice can be, for example, "Currently, Room 1 is being used by three people, and one person has been detected as not wearing a mask. No close contact has been detected."
[0068] (Schedule management process) In addition to the basic processing described above, the conference room monitoring system 10 can also perform processes such as schedule management. FIG. 14 is a flowchart of the schedule management process according to the first embodiment. FIG. 15 is a flowchart of the reservation postponement process according to the first embodiment. FIG. 16 is a flowchart of the conference time extension process in the first embodiment.
[0069] 14, in S40, the control unit 46 of the monitoring device 40 communicates with the management terminal 11 to obtain reservation information for the next conference in the conference room 1 in which the monitoring device 40 is installed. Note that, although not shown in FIG. 14, the control unit 66 of the server 60 reads out the next reservation information for this conference room 1 from the reservation information storage unit 65h in response to a request from the monitoring device 40 and transmits it to the monitoring device 40.
[0070] In S41, the control unit 46 of the monitoring device 40 determines whether or not the conference room 1 is occupied at the reservation start time, that is, whether or not there is anyone in the conference room. This determination can be made by performing "processing for extracting face images, etc. from captured images" (see S3 and S11 in FIG. 8). If the control unit 46 determines that someone is present in the conference room 1 (S41: YES), the process proceeds to S42. On the other hand, if the control unit 46 determines that someone is absent (S41: NO), the process proceeds to S41a.
[0071] In S41a, the control unit 46 of the monitoring device 40 determines whether or not 15 minutes have passed since the reservation start time, that is, whether or not a state in which the conference has not started (S41a: NO) has continued for 15 minutes. If the control unit 46 determines that 15 minutes have passed since the reservation start time (S41a: YES), it proceeds to S41b, but if it determines that 15 minutes have not passed (S41a: NO), it repeats the process from S41. The 15 minutes is the allowable time from the reservation start time until the user of the conference room 1 actually starts the conference. The allowable time may be changed as appropriate according to the operational status of the conference room 1 by the administrator of the conference room monitoring system 10 operating the management device. In S41b, the control unit 46 of the monitoring device 40 performs reservation postponement processing (see FIG. 14), and then repeats the processing from S40. The reservation postponement processing will be described in detail later.
[0072] In S42, the control unit 46 of the monitoring device 40 transmits to the door tablet 59 presence information which is information indicating that the conference room 1 is occupied. In S43, the door tablet 59, in response to receiving the presence information, displays a message (presence display) such as "in use" indicating that someone is present in the conference room 1 on the display unit 59b, as shown in FIG. 1(C).
[0073] In S44, the control unit 46 of the monitoring device 40 judges whether or not there is no one in the conference room 1 (a state in which no one is present). This judgment may be performed in the same manner as in S41. Through this S44, the control unit 46 can judge that after someone becomes present in the conference room because the conference has started (S41: YES), they become absent because the conference has ended. If the control unit 46 determines that no one is present in the conference room 1 (S44: YES), the process proceeds to S45. On the other hand, if the control unit 46 determines that no one is present in the conference room 1 (S44: NO), the process proceeds to S44a.
[0074] In S44a, the control unit 46 of the monitoring device 40 determines whether the reservation end time has passed. By this S44a, the control unit 46 can determine whether a person is present in the conference room 1 after the reservation end time has passed. If the control unit 46 determines that the reservation end time has passed (S44a: YES), the process proceeds to S44b. On the other hand, if the control unit 46 determines that the reservation end time has not passed (S44a: NO), the process repeats the process from S44. In S44b, the control unit 46 of the monitoring device 40 performs a conference time extension process (see FIG. 15). The conference time extension process will be described in detail later.
[0075] In S45, the control unit 46 of the monitoring device 40 acquires from the server 60 reservation information for the next conference. In S46, the control unit 46 of the monitoring device 40 transmits the reservation start time of the next conference to the door tablet 59. After that, the control unit 46 repeats the process from S41. Note that the control unit 46 repeats the process from S41 based on the reservation start time of the next conference. In S47, when the door tablet 59 receives the reservation start time of the next conference from the management device, the door tablet 59 displays this start time on the display unit 59b. The display content may be, for example, "The start time of the next conference is 15:00." The door tablet 59 may acquire the reservation end time of the next conference from the monitoring device 40 in addition to the reservation start time of the next conference, and display this reservation end time on the display unit 59b.
[0076] (Reservation postponement process (S41b)) 15, in S50, the control unit 46 of the monitoring device 40 notifies the user terminal 12 that has reserved the conference room 1 that the reserved start time of the conference has passed. Although detailed explanation will be omitted, the control unit 46 transmits information indicating that the time has passed to the user terminal 12, for example, by email or a schedule management application. When using the schedule management application, the control unit 46 cooperates with the server 60 to transmit information indicating that the time has passed to the user terminal 12. The control unit 12d of the user terminal 12 that has reserved the conference room 1 accepts a selection operation from the user as to whether or not to cancel the reservation of the conference, and transmits operation information as to whether or not to cancel the reservation to the server 60. Thereafter, the control unit 66 of the server 60 transmits the operation information of the user terminal 12 to the monitoring device 40.
[0077] In S51, if the operation information of the user terminal 12 received from the server 60 is a reservation cancellation (S51: YES), the control unit 46 of the monitoring device 40 proceeds to S51a, and if it is not a reservation cancellation (S51: NO), the control unit 46 proceeds to S52. In S51a and S51b, the control unit 46 of the monitoring device 40 and the control unit 66 of the server 60 communicate with each other to perform the process of canceling the reservation of the conference. That is, the control unit 46 of the monitoring device 40 requests the control unit 66 of the server 60 to cancel the reservation of the conference, and in response, the control unit 66 of the server 60 deletes the reservation information of this conference from the reservation information storage unit 65h. This allows the user of conference room 1 to cancel the reservation of conference room 1 if there is a reason that prevents the conference from starting even after the reserved start time of the conference.
[0078] In S52, the control unit 46 of the monitoring device 40 requests the server 60 to postpone the reservation information of this conference, that is, to postpone the reservation start time and reservation end time. In S53, in response to a request from the monitoring device 40, the control unit 66 of the server 60 refers to the reservation information storage unit 65h to determine whether the interval between this reservation information (i.e., the reservation end time) and the next reservation information (i.e., the reservation start time) is 15 minutes or more. If the interval between the two is 15 minutes or more, the control unit 66 of the server 60 postpones the reservation information by 15 minutes and updates the reservation information storage unit 65h. Note that since the reservation start time has already passed, the control unit 66 of the server 60 may postpone only the reservation end time and may or may not postpone the reservation start time. On the other hand, if the interval between the two is less than 15 minutes, the control unit 66 of the server 60 maintains the reservation information without making any changes. In S54, the control unit 66 of the server 60 transmits to the monitoring device 40 the result of the determination in S53 (information on whether the reservation should be postponed or maintained).
[0079] In S55, the control unit 46 of the monitoring device 40 transmits the determination result of S53 to the user terminal 12 in the same manner as in S50. Although not shown in the figure, when the control unit 12d of the user terminal 12 receives this determination result, it notifies the user of the determination result by displaying it on the display unit 12b or the like.
[0080] After completing the processes of S51a and S55, the control unit 46 proceeds to S40 (see FIG. 14). The processes from S40 onward are performed based on the reservation information of the next conference following the cancellation of the reservation, the postponed reservation information, or the maintained reservation information.
[0081] In this way, the user of conference room 1 can easily postpone or cancel the conference even if the reserved start time of the conference has passed.
[0082] (Meeting time extension process (S44b)) As shown in FIG. 16, in S60, the control unit 46 of the monitoring device 40 requests the server 60 to extend the conference time of the conference, that is, to postpone the scheduled end time. In S61, in response to a request from the monitoring device 40, the control unit 66 of the server 60 refers to the reservation information storage unit 65h to determine whether the interval between the current time and the next reservation information (i.e., the reservation start time) is 15 minutes or more. If the interval between the two is 15 minutes or more, the control unit 66 of the server 60 extends the conference time of the reservation information by 15 minutes, that is, extends the reservation end time by 15 minutes, and updates the reservation information. On the other hand, if the interval between the two is less than 15 minutes, the control unit 66 of the server 60 maintains the conference time in the reservation information without making any changes.
[0083] In S62, the control unit 66 of the server 60 transmits to the monitoring device 40 the determination result of S62 (information on whether the conference time is to be extended or maintained). In S63, the control unit 46 of the monitoring device 40 transmits the determination result of S62 to the user terminal 12 in a manner similar to that of S55. Although not shown in the figure, when the control unit 12d of the user terminal 12 receives this determination result, it notifies the user of the determination result by displaying it on a display or the like.
[0084] After completing the process of S63, the control unit 46 proceeds to S44 (see FIG. 14). The process from S44 onward is performed based on the reservation end time corresponding to the extended conference time or the reservation end time corresponding to the maintained conference time.
[0085] In this way, if there is sufficient time before the start of the next conference, the user of the conference room 1 can extend the conference time without any operation. However, the monitoring device 40 may extend the conference time in response to a response from the user terminal 12 after inquiring of whether or not to extend the conference time. In this case, too, the operation for extending the conference time is easy.
[0086] Second embodiment Next, a second embodiment of the present invention will be described. In the following description and drawings, parts that perform the same functions as in the first embodiment described above are appropriately given the same reference numerals or the same reference numerals with the same suffixes (last two digits) as appropriate, and duplicated descriptions are appropriately omitted. FIG. 17 is a diagram showing a plan view of the conference room 1 in the second embodiment. As shown in FIG. 17, in the conference room monitoring system 210 of this embodiment, the distance between the monitoring device 240 and the sections L1, R1 is short, so that the sections L1, R1 are located outside the imaging angle of view θ243 of the camera 243. Since sections L2 to L4 and R2 to R4 are located within the shooting angle of view θ243 of the camera 243, the control unit of the monitoring device 240 can process the facial images of people present in sections L2 to L4 and R2 to R4 in the same manner as in the first embodiment.
[0087] The control unit of the monitoring device 240 tracks the facial images of people who enter and leave the conference room 1, and when the person moves outside the imaging angle of view θ243, determines that the person is present in the sections L1 and R1. For example, person ML1 enters conference room 1 through the entrance and then moves to section L1. In this case, the control unit of the monitoring device 240 extracts the face image of person ML1 from the captured image of conference room 1 and determines that the face image of person ML1 is located on the left side within the captured image. The control unit of the monitoring device 240 then tracks the face image of person ML1, and when person ML1 moves out of the captured image, determines that person ML1 is present within section L1. The control unit of the monitoring device 240 also makes similar determinations for person MR1 and section R1.
[0088] The control unit of the monitoring device 240 performs the above-mentioned processing for the sections L1 and R1, thereby enabling the same monitoring processing as in the first embodiment. That is, in the congestion determination process, the number of people included in the captured image is counted, and the number of people who have moved into sections L1 and R1 is added to this number.
[0089] In the close contact determination process, the control unit of the monitoring device 240 tracks the facial images of people moving into section L1, and if there are two or more of them, it can determine that two or more people are present in section L1. The control unit of the monitoring device 240 can make a similar determination for section R1. To determine whether a mask is being worn, the control unit of monitoring device 240 not only determines whether the facial images of people in sections L2-L4, and R2-R4 contain mask images, but also determines whether the facial images of people moving into sections L1 and R1 contain mask images.
[0090] In this way, the conference room monitoring system 210 of this embodiment can perform monitoring processing for the conference room 1 even when a person is present outside the imaging angle of view θ243 of the camera 243.
[0091] Third embodiment Next, a third embodiment of the present invention will be described. FIG. 18 is a diagram illustrating a monitoring screen 375 according to the third embodiment. The monitoring screen 375 is created by the server 60 based on the most recent information in the image analysis history storage unit 65d in order to display the most recent situation in the conference room 1. 18, the monitoring screen 375 displays the room number of conference room 1, and also displays a schematic diagram 375a of conference room 1 and a schematic diagram 375b of people in the conference room in plan view. In addition, in the schematic diagram of a user not wearing a mask, a "X" is displayed within the schematic diagram of the mask (see schematic diagram 375b of section R1).
[0092] The management terminal 11 etc. can obtain information on the monitoring screen 375 by making a request to the server 60 and display it on the display unit 11b etc. This allows the manager of conference room 1 to check the status of conference room 1 almost in real time.
[0093] Although the embodiment of the present invention has been described above, the present invention is not limited to the above-mentioned embodiment, and various modifications and changes are possible, such as the modified forms described below, which are also within the technical scope of the present invention. Furthermore, the effects described in the embodiment are merely a list of the most preferable effects resulting from the present invention, and the effects of the present invention are not limited to those described in the embodiment. Note that the configurations of the above-mentioned embodiment and the modified forms described below can be used only in part, or in appropriate combination, but detailed description will be omitted.
[0094] (Variations) (1) In the embodiment, the monitoring device extracts a face image as a head image from a captured image, but is not limited to this. For example, the monitoring device may extract an image including a face image, and may extract an image of the entire head, for example. (2) In the embodiment, the monitoring device does not display detailed analysis information, but the present invention is not limited to this. The monitoring device may display a captured image including detailed analysis information, such as that shown in Fig. 10 and Fig. 12, on a display unit of a monitor, a management terminal, or the like.
[0095] (3) In the embodiment, the conference room monitoring system performs the crowded state notification process when the conference room is crowded (see S16 and S16a in FIG. 9), but the present invention is not limited to this. If the conference room is crowded (S16: YES), the conference room monitoring system may guide the user to another conference room. For example, in the reservation ID 1001 in the reservation information storage unit 65h in Fig. 7(B), a conference will be held from 10:00 to 11:00 in conference room No. 1, which has a capacity of 8 people. On the other hand, no conference will be held in conference room No. 2, which has a capacity of 10 people, during this time. Therefore, when a conference room reserved with reservation ID 1001 is being held and the conference room is crowded (S16: YES), the monitoring device transmits information indicating that the conference room has been determined to be abnormal (over-capacity information) (see S30 in FIG. 13), inquiring of the server about the availability of other conference rooms that can accommodate the number of people present in the conference room currently being used, and in response, the server refers to the reservation information storage unit 65h. In this case as well, the server transmits information to the smart speaker that conference room No. 2 is available, and the smart speaker outputs a sound indicating that conference room No. 2, which has a capacity of 10 people, is available.
[0096] (4) The reservation information may include information about a remote conference, etc. In this case, when it is determined that the conference has started (see S10 in FIG. 9), the monitoring device may connect the camera device for the remote conference to the other party's information terminal. In this case, the smart speaker may output a voice such as "Connecting to the other party's information terminal."
[0097] (5) In the embodiment, the monitoring device has been described as an example in which a captured image of the inside of a conference room is captured by one camera, but this is not limiting. The monitoring device may capture two or more captured images of the inside of a conference room by two or more cameras with different imaging ranges. In this case, the monitoring device can monitor a conference room with a larger area. In this case, the monitoring device may monitor the individual analysis information of each captured image by integrating them, or may combine two or more captured images into one captured image like a panoramic image and analyze the captured image. Furthermore, by connecting the wide-angle camera device and the monitoring device via a communication network within the facility, it may be possible to monitor a conference room having a larger area.
[0098] (6) When the control unit of the monitoring device performs an alarm process for each judgment (which may be for all judgments or for any arbitrary judgment), the control unit of the monitoring device may store the imaging information at the judgment timing in the memory unit. In this case, the operator can take various measures by analyzing the imaging information stored in the storage unit. For example, if the system makes an incorrect judgment, the operator can improve the accuracy of the judgment by adjusting the settings. Also, for example, if an abnormality is repeatedly judged to be present for a specific user of the conference room, the operator can instruct the user to be more vigilant about infectious diseases. In order to smoothly identify such users, it is preferable that the control unit of the monitoring device stores the imaging information in association with the user of the conference room.
[0099] (7) In the embodiment, the control unit immediately proceeds to the next process after determining whether the conference has ended (S20), but this is not limited to this. After the notification process of one type of abnormality, the control unit may skip the notification process even if it determines that any type is abnormal in the determination process (S16 to S19) until a certain time has passed, and may execute the notification process regarding a new abnormality after the certain time has passed. In this way, the conference room monitoring system can reliably notify people in the conference room of the one type of abnormality (violation details) until the certain time has passed in response to the determination of the one type of abnormality.
[0100] (8) In the embodiment, the abnormality determination process (S16 to S19) in the conference room is performed in the order of crowded, close contact, wearing a mask, and time exceeded, but this is not limited to the above order. These orders may be changed as appropriate depending on the specifications, etc., and may be set as appropriate. For example, the abnormality determination process may be performed in the order of wearing a mask, crowded, close contact, and time exceeded.
[0101] (9) In the embodiment, the close contact determination process determines that a close contact state occurs when two or more people (face images) are present in one section. However, the present invention is not limited to this. In the close contact determination process, for example, if it is determined that there are people in adjacent sections, it may be determined that there is a close contact state. Also, if two or more people (face images) are present in one section, it may be determined that the state is a high closeness state, and if it is determined that there is a person in each of the adjacent sections, it may be determined that the state is a low closeness state. This allows the degree of closeness to be confirmed in more detail. Furthermore, the administrator of the conference room may be able to set such criteria as appropriate depending on the infectious disease alert level. [Explanation of symbols]
[0102] 1: Conference room 2: Monitor 4: Door 10,210: Conference room monitoring system 11: Management terminal 12: User terminal 30: Room System 40,240: Monitoring equipment 43,243: Camera 50: Smart speaker 53: Speaker 59: Door tablet 60: Server
Claims
1. an imaging unit for capturing images of a conference room defined so that one or more persons are positioned in each of a plurality of sections; A control unit. The control unit is By controlling the imaging unit, the imaging unit continuously acquires images of the conference room; Extracting images of people from the captured images; The state of the conference room is judged based on the extracted image of the person, and if it is judged that an abnormality exists, a notification process is performed to notify the user of the abnormality. The control unit is A head image, which is an image of a person, is extracted from the captured image, and the head image is associated with a partition based on an image size of the head image; When two or more head images are associated with one section, it is determined that two or more people are present in the section, and the notification process is performed.
13. A conference room monitoring device comprising:
2. The control unit counts images of people extracted from a captured image, and performs the notification process when the count number exceeds the capacity of the conference room.
2. The conference room monitoring device according to claim 1.
3. The control unit performs the notification process when it determines that a head image, which is an image of a person extracted from a captured image, does not include a mask.
3. The conference room monitoring device according to claim 1 or 2.
Citation Information
Patent Citations
Method for monitoring wearing of mask by vertical ladder passengers for epidemic prevention and control
CN111382719A
Information processing apparatus, information processing system, and information processing method
JP2015002477A
Device, management system, and program
JP2019079408A
