Image analysis apparatus and image analysis method

Abstract

To reduce work burdens of various kinds of work related to in-car image analysis.SOLUTION: An image analysis device 30 is an image analysis device connected to an elevator control device that controls an operation of an elevator and an imaging device in a car of the elevator, and includes an image processing unit 34 that performs image processing using an image of the inside of the car captured by the imaging device, and an image processing setting unit 34 that outputs a command for executing at least one operation of the elevator to the elevator control device and registers, for each operation, setting information for performing image processing using an image of the inside of the car captured after the operation is executed in a setting information holding unit that holds the setting information.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an image analysis device and an image analysis method. [Background technology]

[0002] Conventionally, image analysis is performed using images of the interior of an elevator car captured by a camera installed in the car. Based on the results of the image analysis, elevator control, such as passing through intermediate floors when the car is full, the ascending / descending speed when the car is unoccupied, and the opening and closing of the car doors, is performed. To maximize the performance of image analysis, appropriate settings related to image analysis must be configured depending on conditions such as the presence or absence of passengers in the car and the camera's installation position and orientation. However, when image analysis settings are manually configured by an operator, this requires a significant workload and may not result in optimal settings. Furthermore, after configuration, periodic checks for performance degradation and adjustments to resolve performance degradation must be performed. This periodic manual configuration also poses a significant workload. Furthermore, when the car is moved, external light from the window may cause fluctuations in the area to be masked in the captured image. Therefore, there is a need to automatically configure the mask area for image processing in accordance with the fluctuations in the area to be masked in the captured image. Therefore, a technology for automatically configuring the mask area in accordance with the fluctuations in the area to be masked in the captured image due to external light has been proposed, for example, as described in Patent Document 1.

[0003] Patent Document 1 states that "the image processing unit is configured to perform mask area setting control when a special car call is registered, and as the mask area setting control, the image processing unit sequentially acquires frame images from the output of the imaging means while the car is moving in response to the special car call, sets the frame image acquired at a predetermined time as a reference frame image, and after setting the reference frame image, each time a frame image is acquired at a predetermined interval, extracts a composite variation area, which is an area different from the reference frame image in an image obtained by superimposing frame images acquired since the reference frame image was acquired, based on the frame images acquired so far, calculates the increase in the composite variation area extracted this time relative to the composite variation area extracted last time and stores this in a data storage unit, and when the calculated increase in the composite variation area is below a predetermined threshold for a predetermined number of consecutive times, performs control to set the last extracted composite variation area as a mask area." [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-119461 Summary of the Invention [Problem to be solved by the invention]

[0005] As described above, a technology has been proposed for automatically setting a mask area for image processing that may fluctuate due to factors such as external light coming through windows when the car is moving. However, there are various tasks related to car interior image analysis other than the task of setting a mask area that may fluctuate due to external light, as described in Patent Document 1. For example, there are various tasks related to car interior image analysis, such as setting an exclusion area to be excluded in the person detection process inside the car, checking the quality of images captured by a camera inside the car, and checking the installation conditions of the camera inside the car. Unless these tasks are automated, the problem of a heavy workload cannot be resolved. However, Patent Document 1 does not mention automating these tasks.

[0006] The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to reduce the workload of various tasks related to the analysis of images inside the car. [Means for solving the problem]

[0007] The image analysis device of the present invention is an image analysis device connected to an elevator control device that controls the operation of an elevator, and an image processing unit that performs image processing using images of the inside of the elevator car captured by the imaging device, and an image processing setting unit that outputs commands to the elevator control device to execute at least one or more operations of the elevator, and registers, for each operation, setting information for performing image processing using images of the inside of the elevator captured after the operation is executed in a setting information storage unit that stores the setting information. The image analysis device described above is one aspect of the present invention, and an image analysis method reflecting one aspect of the present invention is configured in the same manner as the image analysis device described above. [Effects of the Invention]

[0008] According to the present invention having the above configuration, it is possible to reduce the workload of various tasks relating to the analysis of images inside the car. Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram illustrating an example of the configuration of an image analysis system according to an embodiment of the present invention. [Figure 2] 1 is a diagram illustrating an example of the functional configuration of an image analysis system according to an embodiment of the present invention. [Figure 3] FIG. 2 is a diagram illustrating an example of the hardware configuration of a computer that constitutes each device of the image analysis system according to one embodiment of the present invention. [Figure 4] 1 is a flowchart showing the steps of a first example of processing for automatically setting a human-detection excluded region in an image analyzing device according to an embodiment of the present invention. [Figure 5] 10A and 10B are diagrams for explaining extraction of exclusion region candidates in an image analysis device according to an embodiment of the present invention. [Figure 6] 10A and 10B are diagrams for explaining edge correction of exclusion area candidates in an image analyzing device according to an embodiment of the present invention. [Figure 7] FIG. 10 is a diagram showing an example of a human detection exclusion area setting screen in the worker terminal device according to the embodiment of the present invention. [Figure 8] 10 is a flowchart showing the procedure of Example 2 of the human-detection exclusion region automatic setting process in the image analyzing device according to one embodiment of the present invention. [Figure 9] 1 is a flowchart showing the procedure of a first example of processing for automatically checking the camera position and orientation and image quality in an image analysis device according to an embodiment of the present invention. [Figure 10] 10A and 10B are diagrams for explaining camera position determination and image quality determination using a test image in an image analysis device according to an embodiment of the present invention. [Figure 11] 10 is a flowchart showing the procedure of a second example of processing for automatically checking the camera position and orientation and image quality in an image analysis device according to an embodiment of the present invention. [Figure 12] 10 is a flowchart illustrating an example of a procedure for automatically checking settings for detecting a person inside a car in an image analyzing device according to an embodiment of the present invention. [Figure 13] 10A and 10B are diagrams for explaining a specific example of checking the setting for detecting a person in a car in the image analyzing device according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functions or configurations are designated by the same reference numerals, and redundant description will be omitted.

[0011] <One embodiment> [Image analysis system configuration example] First, a configuration example of an image analysis system 1 according to one embodiment of the present invention will be described. Fig. 1 is a diagram showing a configuration example of the image analysis system 1 according to this embodiment. As shown in Fig. 1, the image analysis system 1 includes a camera 11 in a car 10, a display device 12 in the car 10, an elevator control device 20, an image analysis device 30, and an operator terminal device 40. The image analysis device 30 is connected to each of the camera 11, the elevator control device 20, and the operator terminal device 40.

[0012] The camera 11 is an example of an imaging device, and is installed in a position where it can capture images of the inside of the car 10, such as the display device 12 and the car door, and captures images of the inside of the car to monitor the situation inside the car 10. The camera 11 outputs the captured images of the inside of the car to the image analysis device 30. In the following description, the camera 11 captures an image when it receives a command to capture an image and outputs the image, but the camera 11 may also output the captured image when it receives a command to capture an image while continuously capturing images (acquiring video). Furthermore, if an image of the inside of the car is required for control processing in the elevator control device 20, the camera 11 may output the image of the inside of the car to the elevator control device 20.

[0013] The elevator control device 20 is disposed in the elevator shaft of the car 10, on the ceiling of the car 10, or the like, and controls each operation of the elevator, including the car 10. The operations of the elevator include, for example, raising and lowering the car 10, opening and closing the doors of the car 10, landing the car 10 at the destination floor, and changing the display content on the display device 12 inside the car. During initial setup or maintenance of the elevator, the elevator control device 20 controls the opening and closing of the doors of the car 10, changing the display content on the display device 12 inside the car, and the like, in accordance with instructions from the image analysis device 30. The elevator control device 20 is also connected to a control center that remotely monitors the elevator via a network (not shown), and transmits elevator operation status, abnormality notifications, and the like to the control center.

[0014] The image analysis device 30 is disposed in the elevator shaft of the car 10, on the ceiling of the car 10, or the like, and may be disposed in the same position as the elevator control device 20. The image analysis device 30 acquires images of the inside of the car from the camera 11, and performs a process of automatically setting an area to be excluded in the person detection process using the images of the inside of the car (a process of automatically setting a person detection exclusion area, which will be described later). The image analysis device 30 also performs a process of automatically checking the position and orientation of the camera 11 inside the car and the quality of the image captured by the camera 11 using the images of the inside of the car (a process of automatically checking a camera position and image quality, which will be described later). The image analysis device 30 also performs a process of automatically checking the person detection settings inside the car 10 using the images of the inside of the car (a process of automatically checking a person detection setting inside the car, which will be described later). In addition, when executing each of the above processes, the image analysis device 30 outputs commands to the elevator control device 20, such as commands to open / close the door or change the display content of the display device 12 inside the car, as necessary, and transmits the display content to be displayed on the worker terminal device 40 to the worker terminal device 40.

[0015] The worker terminal device 40 is a mobile terminal device carried by a worker, such as a smartphone, tablet, or laptop (Personal Computer). The worker terminal device 40 displays an operation screen (see FIGS. 7 and 13 described below) that the worker operates when performing various tasks related to the elevator. The worker terminal device 40 also transmits input information input by the worker to the image analyzing device 30 via a network (not shown).

[0016] [Example of functional configuration of image analysis system] Next, an example of the functional configuration of the image analysis system 1 according to this embodiment will be described. Fig. 2 is a diagram showing an example of the functional configuration of the image analysis system 1 according to this embodiment.

[0017] (Functional configuration of elevator control device 20) As shown in Fig. 2, the elevator control device 20 includes a door opening / closing control unit 21, a lifting / lowering control unit 22, and an in-car display device control unit 23. Note that Fig. 2 only shows the components of the elevator control device 20 according to this embodiment, and illustration and description of the other components are omitted. The door opening / closing control unit 21 controls the opening and closing operation of the door of the car 10. The lifting / lowering control unit 22 controls the lifting and lowering operation of the car 10. The in-car display device control unit 23 controls the display operation of the display device 12 inside the car.

[0018] (Functional configuration of image analysis device 30) As shown in FIG. 2 , the image analysis device 30 includes a camera IF (Interface) unit 31, an elevator control IF unit 32, an image processing setting unit 33, an image processing unit 34, and an image processing setting information storage unit 35. The image processing setting unit 33 is connected to each of the camera IF unit 31, the elevator control IF unit 32, the image processing unit 34, and the image processing setting information storage unit 35 so as to be able to transmit and receive data therefrom. The image processing setting unit 33 is also connected to an operator terminal device 40 via a network (not shown) so as to be able to transmit and receive data therefrom. The image processing unit 34 is also connected to each of the camera IF unit 31, the elevator control IF unit 32, and the image processing setting unit 33 so as to be able to transmit and receive data therefrom. The image processing unit 34 is also connected to the image processing setting information storage unit 35 so as to be able to acquire information data from the image processing setting information storage unit 35.

[0019] The camera IF unit 31 acquires an image of the inside of the car from the camera 11, and outputs the acquired image of the inside of the car to the image processing setting unit 33 and the image processing unit . The elevator control IF unit 32 outputs an operation command from the image processing setting unit 33 or the image processing unit 34 to the elevator control device 20.

[0020] The image processing setting unit 33 outputs a command to the elevator control device 20 to execute at least one or more operations of the elevator. Furthermore, for each of at least one or more operations of the elevator, the image processing setting unit 33 registers, in a setting information storage unit (image processing setting information storage unit 35) that stores setting information for performing image processing, an image of the inside of the car captured after the operation is executed.

[0021] Specifically, the image processing setting unit 33 extracts a differential image between an image of the inside of the elevator car captured after the elevator car door is closed and an image of the inside of the elevator car captured after the elevator car door is opened, as a candidate exclusion area for person detection image processing in the image processing unit. The image processing setting unit 33 also corrects edges of the candidate exclusion area and transmits the corrected candidate exclusion area to the worker terminal device 40 or the display device 12 in the elevator car. The image processing setting unit 33 also registers the candidate exclusion area in the setting information storage unit (image processing setting information storage unit 35) as a person detection exclusion area for person detection image processing in accordance with setting instructions input by the worker via the worker terminal device 40 or the display device 12 in the elevator car. The above-described process is Example 1 of the person detection exclusion area automatic setting process described below, and detailed processing procedures will be described with reference to FIG. 4 described below.

[0022] Furthermore, if the deviation between the extracted exclusion area candidate and the person-detection exclusion area already registered in the setting information storage unit (image processing setting information storage unit 35) satisfies a predetermined condition (greater than or equal to a first threshold value and less than a second threshold value, which will be described later), the image processing setting unit 33 updates the person-detection exclusion area with the exclusion area candidate. The above-described processing is included in Example 2 of the person-detection exclusion area automatic setting processing, which will be described later, and the detailed processing procedure will be described with reference to FIG.

[0023] Furthermore, the image processing setting unit 33 performs a comparison process of the image inside the car captured after the operation of displaying a pre-registered test image on the display device 12 inside the car has been executed, with pre-registered correct image information associated with the test image, and checks for deviations in the position and orientation of the imaging device and deterioration in the quality of the captured image by the imaging device based on the results of the comparison process. The above-mentioned process is included in Examples 1 and 2 of the automatic camera position setting and automatic image quality checking process described below, and detailed processing procedures will be described in Figures 9 and 11 described below.

[0024] Furthermore, the image processing setting unit 33 transmits work instructions to the worker to the terminal device 40, acquires images of the inside of the car between the time when it receives a work start signal from the terminal device 40 and the time when it receives a work end signal, performs a comparison process of the person detection results detected by the image processing unit 34 using the acquired images of the inside of the car with correct person detection results that are linked to the work instructions and registered in advance, and confirms the accuracy of the person detection processing settings in the image processing unit 34 based on the results of the comparison process. The above-mentioned process is, in other words, an automatic confirmation process of the person detection settings inside the car, which will be described later, and the detailed processing procedure will be described later with reference to FIG. 12.

[0025] The image processing unit 34 performs image processing using the image of the inside of the car input from the camera 11. The image processing unit 34 acquires setting information for performing image processing from the image processing setting information storage unit 35, and performs image processing such as person detection processing in accordance with the setting information.

[0026] The image processing setting information storage unit 35 is an example of a setting information storage unit, and stores (stores) setting information for various image processes in the image processing unit 34. The image processing setting information includes, for example, an exclusion area for in-car person detection processing and detection position information. The image processing setting information storage unit 35 also stores pre-registered test images and pre-registered correct image information linked to the test images, which are used in the automatic camera position setting and automatic image quality confirmation processing in the image processing setting unit 33. The image processing setting information storage unit 35 also stores pre-registered correct person detection results linked to each elevator operation included in the automatic confirmation processing of in-car person detection settings in the image processing setting unit 33.

[0027] [Example of hardware configuration of computers that make up each device in the image analysis system] Next, we will explain an example of the hardware configuration of the computer that constitutes each device in the image analysis system 1. Fig. 3 is a diagram showing an example of the hardware configuration of the computer 50 that constitutes each device in the image analysis system 1 according to this embodiment. The computer 50 is an example of hardware used as a computer that can operate as the elevator control device 20 and the image analysis device 30.

[0028] The computer 50 includes a CPU (Central Processing Unit) 51, a ROM (Read Only Memory) 52, and a RAM (Random Access Memory) 53, each connected to a bus 54. The computer 50 further includes a non-volatile storage 55 and a network interface 56.

[0029] The CPU 51 reads out program code of software that realizes each function according to this embodiment from the ROM 52, loads it into the RAM 53, and executes it. Variables, parameters, etc. that are generated during the calculation processing of the CPU 51 are temporarily written to the RAM 53, and these variables, parameters, etc. are read out as appropriate by the CPU 51. However, an MPU (Micro Processing Unit) may be used instead of the CPU 51.

[0030] The nonvolatile storage 55 can realize the functions of the image processing setting information holding unit 35, and may be, for example, a hard disk drive (HDD), a solid state drive (SSD), a flexible disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, or a nonvolatile memory. In addition to an operating system (OS) and various parameters, programs for operating the computer 50 are recorded in the nonvolatile storage 55. The ROM 52 and the nonvolatile storage 55 record programs, data, etc. necessary for the CPU 51 to operate, and are used as an example of a computer-readable non-transitory storage medium that stores programs executed by the computer 50.

[0031] The network interface 56 may be, for example, a NIC (Network Interface Card), and various data can be transmitted and received between devices via a LAN (Local Area Network), dedicated line, etc. connected to the terminal of the NIC.

[0032] [Example 1 of automatic setting of human detection exclusion area] Next, the procedure of Example 1 of the automatic human-detection exclusion area setting process in the image analyzing device 30 will be described. FIG. 4 is a flowchart showing the procedure of Example 1 of the automatic human-detection exclusion area setting process in the image analyzing device 30 according to this embodiment. Example 1 of the automatic human-detection exclusion area setting process is an example of the automatic human-detection exclusion area setting process when a worker is present during initial setup of the elevator. When the worker performs an operation to start setting the human-detection exclusion area on the worker terminal device 40, the process described below starts.

[0033] First, the image processing setting unit 33 outputs a command to close the door of the car 10 to the elevator control device 20 via the elevator control IF unit (step S100). In accordance with the command to close the door of the car 10, the elevator control device 20 controls the door of the car 10 to close.

[0034] Next, the image processing setting unit 33 acquires a first camera image via the camera IF unit 31 (step S101). In this process, the image processing setting unit 33 outputs a command to capture an image of the inside of the car to the camera 11 via the camera IF unit 31. Then, the camera 11 captures an image (first camera image) of the inside of the car with the door closed, and outputs the first camera image to the image processing setting unit 33 via the camera IF unit 31.

[0035] Next, the image processing setting unit 33 outputs a command to open the door of the car 10 to the elevator control device 20 via the elevator control IF unit (step S102). The elevator control device 20 performs control to open the door of the car 10 in accordance with the command to open the door of the car 10.

[0036] Next, the image processing setting unit 33 acquires a second camera image via the camera IF unit 31 (step S103). In this process, the image processing setting unit 33 outputs a command to capture an image of the inside of the car to the camera 11 via the camera IF unit 31. Then, the camera 11 captures an image (second camera image) of the inside of the car with the door open, and outputs the second camera image to the image processing setting unit 33 via the camera IF unit 31.

[0037] Next, the image processing setting unit 33 extracts a difference region between the first camera image and the second camera image as an exclusion region candidate (step S104). Figure 5 is a diagram for explaining the extraction of exclusion region candidates in the image analyzing device 30 according to this embodiment. The image processing setting unit 33 extracts a difference region between the first camera image (a) and the second camera image (b) shown in Figure 5, i.e., the gray region in the figure, as an exclusion region candidate.

[0038] Next, the image processing setting unit 33 corrects the edges of the extracted exclusion area candidate (step S105). The edges of the exclusion area candidate may be misaligned due to the intrusion of external light, etc. (see the area surrounded by the dashed-dotted line in Figure 6). In this process, the image processing setting unit 33 corrects the edges of the exclusion area candidate in the second camera image using the image of the outer edge of the door in the first camera image. Figure 6 is a diagram for explaining edge correction of the exclusion area candidate in the image analysis device 30 according to this embodiment. As shown in Figure 6, the edges of the exclusion area candidate (a) before correction are misaligned due to the intrusion of external light (see the area surrounded by the dashed-dotted line). As a result of the correction by the image processing setting unit 33, the edges of the exclusion area candidate (b) after correction are accurate, as shown in the figure. This process can improve the accuracy of the extracted exclusion area candidate.

[0039] Next, the image processing setting unit 33 transmits a command to display the exclusion area candidates and an image indicating the exclusion area candidates to the worker terminal device 40 (step S106). In accordance with the command to display the exclusion area candidates, the worker terminal device 40 displays the image indicating the exclusion area candidates on a human detection exclusion area setting screen. Fig. 7 is a diagram showing an example of the human detection exclusion area setting screen in the worker terminal device 40 according to this embodiment.

[0040] As shown in FIG. 7 , the person-detection exclusion area setting screen U1 displays an image showing exclusion area candidates (gray areas). A message indicating that the person-detection exclusion area has been set, such as "Are you sure you want to set the person-detection exclusion area as follows?", is displayed at the top of the image. An "OK" button and a "Reset" button are provided at the bottom of the image. When the "OK" button is pressed, an instruction to set the exclusion area candidate as the person-detection exclusion area is transmitted to the image processing setting unit 33. When the "Reset" button is pressed, an instruction to reset the exclusion area candidate is transmitted to the image processing setting unit 33. The person-detection exclusion area setting screen U1 shown in FIG. 7 may be displayed on the display device 12 in the car 10. In this case, the image processing setting unit 33 transmits a command to display the exclusion area candidate and an image showing the exclusion area candidate to the elevator control device 20 via the elevator control IF unit 32. The car display device control unit 23 of the elevator control device 20 controls the person-detection exclusion area setting screen U1 to be displayed on the display device 12 in the car.

[0041] Next, the image processing setting unit 33 determines whether to set the exclusion area candidate as a human detection exclusion area (step S107). In this process, if the instruction to set the human detection exclusion area from the worker terminal device 40 is to set the exclusion area candidate as a human detection exclusion area, the determination in step S107 is YES. On the other hand, if the instruction to set the human detection exclusion area from the worker terminal device 40 is to reset the exclusion area candidate, the determination in step S107 is NO.

[0042] In step S107, if it is determined that the exclusion area candidate should be set as a human detection exclusion area (YES determination in step S107), the image processing setting unit 33 sets the exclusion area candidate as a human detection exclusion area (step S108). In this process, the image processing setting unit 33 registers the exclusion area candidate as a human detection exclusion area in the image processing setting information holding unit 35.

[0043] On the other hand, in step S107, if the image processing setting unit 33 determines that the exclusion area candidate should not be set as a person detection exclusion area (NO determination in step S107), it transmits a message indicating that manual setting is required to the worker terminal device 40 (step S109). The worker terminal device 40 displays the message received from the image processing setting unit 33. Note that the message indicating that manual setting is required may be displayed on the display device 12 in the car 10.

[0044] After the processing of step S108 or step S109, the human detection exclusion area automatic setting processing ends.

[0045] [Example 2 of automatic setting of human detection exclusion area in image analysis device] Next, the procedure of Example 2 of the automatic human-detection exclusion area setting process in the image analyzing device 30 will be described. FIG. 8 is a flowchart showing the procedure of Example 2 of the automatic human-detection exclusion area setting process in the image analyzing device 30 according to this embodiment. Example 2 of the automatic human-detection exclusion area setting process is an example of the automatic human-detection exclusion area setting process when no workers are present during elevator maintenance. The process described below starts when the current time reaches a preset maintenance time. The maintenance time is set, for example, to a time period when elevators are not used much, such as at night.

[0046] First, the image processing setting unit 33 checks the state of the elevator (car 10) (step S200). In this process, the image processing setting unit 33 acquires information on the open / closed state of the door of the car 10 from the elevator control device 20 via the elevator control IF unit.

[0047] Next, the image processing setting unit 33 determines whether or not the door of the car 10 is closed (step S201).

[0048] In step S201, when the image processing setting unit 33 determines that the door of the car 10 is open (NO determination in step S201), the person detection exclusion area automatic setting process ends.

[0049] On the other hand, if the image processing setting unit 33 determines in step S201 that the door of the car 10 is closed (YES determination in step S201), it acquires a first camera image via the camera IF unit 31 and repeatedly performs person detection (step S202). In this process, the image processing setting unit 33 outputs a command to the camera 11 via the camera IF unit 31 to capture images of the interior of the car at a predetermined time interval. The camera 11 then captures a first camera image at the predetermined time interval and outputs the first camera image to the image processing unit 34 via the camera IF unit 31. The image processing unit 34 also repeatedly performs person detection processing using the images of the interior of the car acquired at the predetermined time interval. Here, the predetermined time interval is a time unit smaller than the fixed time in step S203 described below and is set in advance. The image processing unit 34 outputs the number of detected people to the image processing setting unit 33.

[0050] Next, the image processing setting unit 33 determines whether the number of people detected within a certain period of time is 0 and there are no car calls (step S203). Here, the certain period of time is set in advance, and may be, for example, 10 minutes. Car call information is acquired from the elevator control device 20.

[0051] In step S203, if the image processing setting unit 33 determines that the number of people detected within the certain time is not 0 or that there is a car call (NO determination in step S203), the person detection exclusion area automatic setting process ends.

[0052] On the other hand, in step S203, if the image processing setting unit 33 determines that the number of people detected within the certain time period is 0 and that there are no car calls (YES determination in step S203), a maintenance mode is activated (step S204). Also, in this process, the image processing setting unit 33 outputs a command to disable car calls to the elevator control device 20 via the elevator control IF unit. Then, the elevator control device 20 performs control to display a message that indicates that car calls are disabled on the display devices of all halls.

[0053] Next, the image processing setting unit 33 outputs a command to open the door of the car 10 to the elevator control device 20 via the elevator control IF unit (step S205). The elevator control device 20 performs control to open the door of the car 10 in accordance with the command to open the door of the car 10.

[0054] Next, the image processing setting unit 33 acquires a second camera image via the camera IF unit 31 (step S206). In this process, the image processing setting unit 33 outputs a command to capture an image of the inside of the car to the camera 11 via the camera IF unit 31. Then, the camera 11 captures a second camera image and outputs the second camera image to the image processing setting unit 33 via the camera IF unit 31.

[0055] Next, the image processing setting unit 33 extracts a difference region between the first camera image and the second camera image as an exclusion region candidate (step S207).

[0056] Next, the image processing setting unit 33 corrects the edges of the extracted exclusion area candidate (step S208).

[0057] Next, the image processing setting unit 33 determines whether the deviation between the extracted exclusion area candidate and the registered human-detection exclusion area is less than a first threshold (step S209). Here, the first threshold is a deviation threshold when the deviation between the exclusion area candidate and the registered human-detection exclusion area requires updating of the registered human-detection exclusion area, and is set in advance. In other words, if the deviation between the exclusion area candidate and the registered human-detection exclusion area is less than the first threshold, updating of the registered human-detection exclusion area is not necessary.

[0058] In step S209, if the image processing setting unit 33 determines that the deviation between the extracted exclusion area candidate and the registered human detection exclusion area is less than the first threshold (YES determination in step S209), it performs the processing of step S214 described below.

[0059] On the other hand, if it is determined in step S209 that the deviation between the exclusion area candidate and the registered human-detection exclusion area is equal to or greater than the first threshold (NO determination in step S209), the image processing setting unit 33 determines whether the deviation is less than a second threshold (step S210). Here, the second threshold is a deviation threshold at which the deviation between the exclusion area candidate and the registered human-detection exclusion area requires manual adjustment of the position and orientation of the camera, and is set in advance.

[0060] In step S211, if the image processing setting unit 33 determines that the deviation between the exclusion area candidate and the registered human-detection exclusion area is less than the second threshold (YES determination in step S210), it updates the registered human-detection exclusion area with the exclusion area candidate (step S211). That is, if the deviation between the extracted exclusion area candidate and the human-detection exclusion area registered in the image processing setting information storage unit 35 satisfies a predetermined condition (not less than the first threshold and less than the second threshold), the image processing setting unit 33 updates the human-detection exclusion area with the exclusion area candidate.

[0061] On the other hand, in step S210, if the image processing setting unit 33 determines that the deviation between the candidate exclusion area and the registered human detection exclusion area is equal to or greater than the second threshold (NO determination in step S210), it notifies the control center (not shown) that the exclusion area setting has failed (step S212).

[0062] Next, the image processing setting unit 33 transmits a message indicating that the setting of the person detection exclusion area is currently stopped to the worker terminal device 40 (step S213). The worker terminal device 40 displays the message received from the image processing setting unit 33. Note that the message indicating that the setting of the person detection exclusion area is currently stopped may be displayed on the display device 12 in the car 10.

[0063] If the determination in step S209 is YES, after the processing of step S211 or after the processing of step S213, the image processing setting unit 33 ends the maintenance mode (step S214). Also, in this processing, the image processing setting unit 33 outputs a command to the elevator control device 20 via the elevator control IF unit to clear the display of the message indicating that car calls are not allowed at all landings. In accordance with the command from the image processing setting unit 33, the elevator control device 20 clears the display of the message indicating that car calls are not allowed at all landings.

[0064] If the determination in step S201 is NO, if the determination in step S203 is NO, or after the processing in step S214, the human detection exclusion area automatic setting processing ends.

[0065] In Example 2 of the automatic setting process for the person detection exclusion area, instead of "switching to a maintenance mode in which car calls are not accepted," there is also a method of "terminating the process and restoring the settings to the original state when a car call is received." With this method, the process does not switch to the maintenance mode between steps S204 and S214, and when a car call is received, the settings are restored and the process transitions to "end."

[0066] [Example 1 of automatic camera position and posture and image quality verification process] Next, the procedure of Example 1 of the automatic camera position setting and automatic image quality confirmation processing in the image analyzing device 30 will be described. Fig. 9 is a flowchart showing the procedure of Example 1 of the automatic camera position setting and automatic image quality confirmation processing in the image analyzing device 30 according to this embodiment. Example 1 of the automatic camera position and attitude and image quality confirmation processing is an example of the automatic camera position and attitude and image quality confirmation processing when a worker is present during initial elevator configuration. When the worker performs an operation on the worker terminal device 40 to start automatic confirmation of the camera position and attitude and image quality, the processing described below starts.

[0067] First, the image processing setting unit 33 outputs a command to close the door of the car 10 to the elevator control device 20 via the elevator control IF unit (step S300). In accordance with the command to close the door of the car 10, the elevator control device 20 controls the door of the car 10 to close.

[0068] Next, the image processing setting unit 33 acquires a test image and correct image information associated with the test image from the image processing setting information storage unit 35 (step S301). Here, the test image is an image for generating correct image information that is a standard for judging the position and orientation of the camera 11 and the quality of the image inside the car captured by the camera 11, and any image may be used. The test image is registered in advance in the image processing setting information storage unit 35. In this embodiment, for example, an image of a star mark is used as the test image, as shown in FIG. 10 (described later). Furthermore, the correct image information of the test image includes position information (coordinate data) and pixel value information of the test image in the image inside the car captured under accurate imaging conditions when the test image is displayed on the display device 12 inside the car, and is registered in advance in the image processing setting information storage unit 35.

[0069] Next, the image processing setting unit 33 outputs the test image and a command to display the test image to the elevator control device 20 via the elevator control IF unit (step S302). The elevator control device 20 displays the test image on the display device 12 in the car 10 in accordance with the command from the image processing setting unit 33.

[0070] Next, the image processing setting unit 33 outputs an elevator up / down command to the elevator control device 20 and acquires one or more images of the inside of the car during the up / down operation period (step S303). In this process, the image processing setting unit 33 outputs the elevator up / down command to the elevator control device 20 via the elevator control IF unit 32. The elevator control device 20 controls the up / down operation of the car 10 in accordance with the command from the image processing setting unit 33. The image processing setting unit 33 also outputs a command to the camera 11 via the camera IF unit 31 to capture one or more images of the inside of the car while the car 10 is rising or falling. The camera 11 captures one or more images of the inside of the car in accordance with the command from the image processing setting unit 33 and outputs the images to the image processing setting unit 33.

[0071] Next, the image processing setting unit 33 compares the test image in the image of the inside of the car with the correct image information of the test image (step S304).

[0072] Here, using FIG. 10, camera position and attitude determination and image quality determination using test images will be described. Image P1 shown in FIG. 10 is an example of a normal car interior image, i.e., an image of the car interior captured under accurate imaging conditions. Image P2 is an example of a car interior image when the quality has deteriorated. Image T1 is an example of correct image information for the test image. Image T21 is a test image on the car interior image when the camera position and attitude has shifted. Image T22 is a test image on the car interior image when the image quality has deteriorated. As shown in FIG. 10, by comparing image T21 with image T1, it is possible to determine whether the camera position and attitude has shifted. Furthermore, it is possible to determine whether image quality has deteriorated by comparing image T22 with image T1.

[0073] Returning to step S305 in Fig. 9, the following description will be given. In the processing of step S305, the image processing setting unit 33 determines whether the deviation between the position information of the test image in the acquired car interior image and the position information of the test image included in the correct answer image information is equal to or greater than a predetermined positional deviation threshold. For example, the image processing setting unit 33 detects multiple positional deviations between each of the positional information of the test image in multiple car interior images and the positional information of the test image included in the correct answer image information, and if the average value of the multiple positional deviations is equal to or greater than the predetermined positional deviation threshold, the determination in step S305 is YES. On the other hand, if the average value of the multiple positional deviations is less than the predetermined positional deviation threshold, the determination in step S305 is NO.

[0074] In the process of step S305, if the image processing setting unit 33 determines that the positional deviation of the test image is equal to or greater than a predetermined positional deviation threshold (YES determination in step S305), the image processing setting unit 33 performs the process of step S306. In the process of step S306, the image processing setting unit 33 transmits a message indicating that the camera position and orientation are inappropriate to the worker terminal device 40 (step S306). The worker terminal device 40 displays the message received from the image processing setting unit 33. Note that the message indicating that the camera position is inappropriate may be displayed on the display device 12 in the car 10.

[0075] On the other hand, in the processing of step S305, if the image processing setting unit 33 determines that the positional deviation of the test image is less than the predetermined positional deviation threshold (NO judgment in step S305), it performs the processing of step S307. In the processing of step S307, the image processing setting unit 33 determines whether the difference between the pixel value of the test image in the acquired image inside the car and the pixel value of the test image included in the correct image information is equal to or greater than a predetermined image quality judgment threshold. In this processing, the image processing setting unit 33, for example, calculates multiple differences between each of the pixel values ​​of the test image in multiple images inside the car and the pixel value of the test image included in the correct image information, and if the average value of the multiple differences is equal to or greater than the predetermined image quality judgment threshold, the judgment in step S307 is YES. On the other hand, if the average value of the multiple differences is less than the predetermined image quality judgment threshold, the judgment in step S307 is NO.

[0076] In the process of step S307, if the image processing setting unit 33 determines that the difference in pixel values ​​of the test image is equal to or greater than a predetermined image quality judgment threshold (YES determination in step S307), the image processing setting unit 33 performs the process of step S308. In the process of step S308, the image processing setting unit 33 transmits a message indicating that the image quality is inappropriate to the worker terminal device 40 (step S308). The worker terminal device 40 displays the message received from the image processing setting unit 33. Note that the message indicating that the image quality is inappropriate may be displayed on the display device 12 in the car 10.

[0077] After the process of step S306, after the process of step S308, or if the determination in step S307 is NO, the automatic camera position setting and automatic image quality checking process ends.

[0078] [Example 2 of automatic camera position and posture and image quality verification process] Next, the procedure of Example 2 of the automatic checking process for camera position and orientation and image quality in the image analyzing device 30 will be described. FIG. 11 is a flowchart showing the procedure of Example 2 of the automatic checking process for camera position and orientation and image quality in the image analyzing device 30 according to this embodiment. Example 2 of the automatic checking process for camera position and orientation and image quality is an example of the automatic checking process for camera position and orientation and image quality when no worker is present during elevator maintenance. The process described below starts when the current time reaches a preset maintenance time. Note that steps S200 to S204 shown in FIG. 11 are the same as steps S200 to S204 shown in FIG. 8, and therefore redundant description will be omitted. Also, steps S301 to S308 shown in FIG. 11 are the same as steps S301 to S308 shown in FIG. 9, and therefore redundant description will be omitted.

[0079] 11, after the processing of step S306 or the processing of step S308, the image processing setting unit 33 transmits a message indicating that the automatic confirmation of the camera position and attitude and image quality is currently stopped to the worker terminal device 40 (step S409). The worker terminal device 40 displays the message received from the image processing setting unit 33. Note that a message indicating that the setting of the human detection exclusion area is currently stopped may be displayed on the display device 12 in the car 10.

[0080] If the determination in step S307 is NO, or after the processing of step S409, the image processing setting unit 33 ends the maintenance mode (step S214). Note that step S214 shown in Fig. 11 is the same as step S214 shown in Fig. 8, and therefore a duplicated description will be omitted.

[0081] If the determination in step S201 is NO, if the determination in step S203 is NO, or after the processing in step S214, the human detection exclusion area automatic setting processing ends.

[0082] In Example 2 of the automatic camera position and attitude and image quality check process, instead of "switching to a maintenance mode in which car calls are not accepted," there is also a method of "terminating the process and restoring the settings to the original state if a car call is received." With this method, the process does not switch to the maintenance mode between steps S204 and S214, and if a car call is received, the settings are restored and the process transitions to "end."

[0083] [Example of automatic checking process for person detection settings in car] Next, the steps of an example of the automatic confirmation process of the in-car person detection setting in the image analyzing device 30 will be described. Fig. 12 is a flowchart showing the steps of an example of the automatic confirmation process of the in-car person detection setting in the image analyzing device 30 according to this embodiment. The automatic confirmation process of the in-car person detection setting is a process that is performed when an operator is present during initial setup of the elevator. When the operator operates the operator terminal device 40 to start automatic confirmation of the in-car person detection setting, the process described below starts.

[0084] First, the image processing setting unit 33 acquires the work instruction content and the correct person detection result associated with the work instruction content from the image processing setting information storage unit 35 (step S500). Here, the work instruction content is information indicating the worker's actions for confirming the in-car person detection setting, such as information on entering or exiting the car, staying in a specified position and posture in the car, etc. The correct person detection result of the work instruction content is the person detection result that can be obtained after each action of the worker when in-car person detection is operating normally. For example, after a worker leaves the car 10 in accordance with the work instruction content, the correct person detection result associated with the worker's action is result information in which the number of detected people is 0. For example, if a worker enters the car 10 in accordance with the work instruction content and stays at a specified position, the correct person detection result associated with the worker's action includes coordinate information of the specified position and information in which the number of detected people is 1.

[0085] Next, the image processing setting unit 33 transmits the work instruction content and a command to display the work instruction content to the worker terminal device 40 (step S501). The worker terminal device 40 displays a message and / or video indicating the work instruction content in accordance with the instruction from the image processing setting unit 33. The message and / or video indicating the work instruction content may be displayed on the display device 12 inside the car. The worker acts in accordance with the displayed work instruction content.

[0086] When starting an action, the worker performs a work start operation on the worker terminal device 40, for example, by pressing the "Start" button in images P5 and P6 shown in FIG. 13 (described later). When the work start operation is performed, the worker terminal device 40 transmits a work start signal to the image processing setting unit 33. Similarly, when completing an action, the worker performs a work end operation on the worker terminal device 40, for example, by pressing the "End" button in images P5 and P6 shown in FIG. 13 (described later). When the work end operation is performed, the worker terminal device 40 transmits a work end signal to the image processing setting unit 33. Furthermore, the worker's work start and work end operations are performed, for example, for each of the multiple worker actions included in the work instruction content.

[0087] Next, the image processing setting unit 33 repeatedly executes the processing of steps S502 to S503 the number of times equal to the number of worker actions included in the work instruction content. In the processing of step S502, the image processing setting unit 33 acquires images of the inside of the car from when the work start signal is received until when the work end signal is received (step S502). In this processing, the number of images of the inside of the car to be acquired is arbitrary. In addition, the image processing setting unit 33 outputs the acquired images of the inside of the car to the image processing unit 34.

[0088] Next, the image processing unit 34 performs person detection on the image inside the car input from the image processing setting unit 33 (step S503). In this process, the image processing unit 34 outputs the detection result including the number of people detected and the coordinate values ​​of the detected people in the image inside the car to the image processing setting unit 33.

[0089] Next, the image processing setting unit 33 determines whether or not the person detection results corresponding to the multiple worker actions included in the work instruction content are the same as the correct person detection results for the registered work instruction content (step S504).

[0090] In the processing of step S504, if the image processing setting unit 33 determines that any one of the multiple person detection results corresponding to each of the multiple worker actions included in the work instruction content differs from the correct person detection result corresponding to that worker action included in the correct person detection result of the work instruction content (NO determination in step S504), the image processing setting unit 33 performs the processing of step S505. In the processing of step S505, the image processing setting unit 33 transmits a message indicating that the person detection setting is inappropriate to the worker terminal device 40. The worker terminal device 40 displays the message received from the image processing setting unit 33. Note that the message indicating that the person detection setting is inappropriate may be displayed on the display device 12 in the car 10.

[0091] If the determination in step S504 is YES, or after the processing of step S505, the automatic checking processing of the in-car person detection setting is terminated.

[0092] Next, a specific example of checking the car's person detection setting will be described using FIG. 13. FIG. 13 is a diagram for explaining a specific example of checking the car's person detection setting in the image analyzing device 30 according to this embodiment. Image P4 shown in FIG. 13 shows an image in which a worker is positioned in the car 10 and checking the work instruction content via the worker terminal device 40. Image P5 shows an example of a display of a first worker action included in the work instruction content displayed on the worker terminal device 40. Image P6 shows an example of a display of a second worker action included in the work instruction content displayed on the worker terminal device 40. Note that images P5 and P6 have the same screen configuration, so the screen configuration will be described below using image P5 as an example.

[0093] At the top of image P5, a message indicating the work instructions is displayed, for example, "Please stay at the circled position in the image below with the door open." Below the message, an image of the inside of the car is displayed, showing the circled position (stay position). At the bottom of the image, a "Start" button and an "End" button are displayed. The correct person detection result corresponding to this worker behavior is that the number of people detected at the circled position is 1. Furthermore, the correct person detection result corresponding to the worker behavior shown in image P6 is that the number of people detected from the circled position (stay position) in the image inside the car is 0.

[0094] Image P7 shows an image of the inside of the car after the worker action shown in image P5, captured by camera 11. The image area surrounded by a frame in image P7 indicates a person detected by image processing unit 24. As shown in image P7, one person is detected in the circled position, which is the same as the correct person detection result corresponding to the worker action shown in image P5.

[0095] Image P8 shows an image of the inside of the car after the worker action shown in image P6, captured by camera 11. No person was detected at the circle position in the image of the inside of the car shown in image P8 (the number of people detected was 0), which is the same as the correct person detection result corresponding to the worker action shown in image P6. Since the detection results in both images P7 and P8 are the same as the correct person detection result, it is determined that the in-car person detection setting is normal.

[0096] [effect] As described above, the image analysis device 30 according to this embodiment compares an image of the interior of the car with the door closed and an image of the interior of the car with the door open during initial elevator setup, and extracts the difference region as an exclusion region candidate. The image analysis device 30 then corrects the edges of the extracted exclusion region candidate and presents the corrected exclusion region candidate to the worker. The image analysis device 30 registers the person-detection exclusion region in response to instructions from the worker. During elevator maintenance, the image analysis device 30 automatically updates the person-detection exclusion region according to the level of deviation between the extracted exclusion region candidate and the registered person-detection exclusion region. The image analysis device 30 then displays a test image on the display device 12 inside the car, and automatically checks the camera position and orientation and image quality using the image of the interior of the car containing the test image and pre-registered correct image information. Furthermore, the image analysis device 30 presents the worker with work instructions for confirming the car's in-car person detection settings, and compares the person detection results using car interior images obtained after the worker follows the work instructions with pre-registered correct person detection results to confirm whether the car's in-car person detection settings are appropriate. The various processes performed by the image analysis device 30 described above significantly reduce the manual work required by workers in setting the person detection exclusion area, checking the camera position and orientation and image quality, and checking the car's in-car person detection settings. Therefore, the image analysis device 30 according to this embodiment can reduce the workload of various tasks related to car interior image analysis.

[0097] The present invention is not limited to the above-described embodiment, and it goes without saying that various other applications and modifications are possible without departing from the gist of the present invention as set forth in the claims. For example, the above-described embodiments have described in detail and specifically the configuration of an image analysis system in order to clearly explain the present invention, and are not necessarily limited to systems that include all of the described configurations. Furthermore, it is possible to replace part of the configuration of the embodiments described here with the configuration of other embodiments, and it is also possible to add the configuration of one embodiment to the configuration of another embodiment. Furthermore, it is also possible to add, delete, or replace part of the configuration of one embodiment with other configurations. In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines in the product. In reality, it can be assumed that almost all components are interconnected. [Explanation of symbols]

[0098] 1...image analysis system, 11...camera, 12...display device, 20...elevator control device, 21...door opening / closing control unit, 22...lifting / lowering control unit, 23...in-car display device control unit, 24...image processing unit, 30...image analysis device, 31...camera IF unit, 32...elevator control IF unit, 33...image processing setting unit, 34...image processing unit, 35...image processing setting information storage unit, 40...worker terminal device

Claims

1. An elevator control device that controls the operation of an elevator, and an image analysis device connected to an imaging device in the elevator car, an image processing unit that performs image processing using the image of the inside of the car captured by the imaging device; an image processing setting unit that outputs a command to the elevator control device to execute at least one operation of the elevator, and that registers, for each operation, setting information for performing the image processing in a setting information storage unit that stores the setting information, using an image of the inside of the car captured after the operation is executed. Image analysis device.

2. The image processing setting unit extracts a difference image between an image of the inside of the elevator car captured after the operation of closing the elevator car door is performed and an image of the inside of the elevator car captured after the operation of opening the elevator car door is performed as a candidate exclusion area for person detection image processing in the image processing unit. The image analysis device according to claim 1 .

3. The image processing setting unit corrects edges of the exclusion area candidate, and transmits the corrected exclusion area candidate to a terminal device held by a worker performing various tasks related to the elevator, or to a display device in the elevator car. The image analysis device according to claim 2 .

4. The image processing setting unit registers the exclusion area candidate in the setting information storage unit as a human detection exclusion area for the human detection image processing in accordance with a setting instruction input by the worker via the terminal device or the display device. The image analysis device according to claim 3 .

5. When a difference between the extracted exclusion area candidate and the human-detection exclusion area registered in the setting information storage unit satisfies a predetermined condition, the image processing setting unit updates the human-detection exclusion area with the exclusion area candidate. The image analysis device according to claim 4.

6. The image processing setting unit performs a comparison process between an image of the inside of the car captured after the operation of displaying a pre-registered test image on a display device inside the car is executed and correct image information associated with the test image and pre-registered, and checks for a deviation in position and orientation of the imaging device and deterioration in the quality of the captured image of the imaging device based on the result of the comparison process. The image analysis device according to claim 1 .

7. The image processing setting unit transmits work instruction contents to the worker to the terminal device, acquires images of the inside of the car from the time when a work start signal is received from the terminal device until a work end signal is received, performs a comparison process with a person detection result detected by the image processing unit using the acquired image of the inside of the car and a correct person detection result associated with the work instruction contents and registered in advance, and confirms the accuracy of the person detection processing setting in the image processing unit based on the result of the comparison process. The image analysis device according to claim 3 .

8. An elevator control device that controls the operation of an elevator, and an image analysis method in an image analysis device connected to an imaging device in the elevator car, comprising: performing image processing using the image of the inside of the car captured by the imaging device; outputting a command to the elevator control device to execute at least one operation of the elevator, and for each operation, registering setting information for performing the image processing in a setting information storage unit that stores the setting information using an image of the inside of the car captured after the operation is executed. Image analysis methods.

Images