Maintenance device, maintenance system, and maintenance program

The maintenance system uses a camera with adjustable lenses and drive information to align images for precise hoistway inspection, addressing alignment issues and reducing costs in elevator maintenance.

JP2024165375APending Publication Date: 2024-11-28HITACHI LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023081524
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-17
Publication Date
2024-11-28

AI Technical Summary

Technical Problem

Existing elevator hoistway inspection systems face challenges in accurately detecting small foreign objects and equipment changes due to difficulties in aligning camera shooting points, which can be exacerbated by aging or dirt accumulation, and require multiple alignment marks, increasing cost, especially in high-rise buildings.

Method used

A maintenance system that includes a camera with adjustable lenses, an information processing device, and a mobile terminal for analyzing image data in conjunction with elevator drive information, allowing precise alignment and inspection without the need for multiple alignment marks.

Benefits of technology

Enables high-accuracy, low-cost maintenance and inspection of elevator hoistways with reduced complexity by aligning camera shots using drive information, minimizing false detections and measurement errors.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024165375000001_ABST
    Figure 2024165375000001_ABST
Patent Text Reader

Abstract

To provide a maintenance device which enables highly accurate maintenance inspection of an elevator at low costs with a simpler structure.SOLUTION: The invention relates to a maintenance device which acquires image data of a hoistway which is imaged by an imaging device during maintenance inspection of an elevator. The maintenance device includes an image processing unit which extracts analysis image data to be analyzed in the maintenance inspection from the image data of the hoistway. The image processing unit acquires drive information regarding a travel state of a car of the elevator during the maintenance inspection and extracts the analysis image data from the image data of the hoistway based on the drive information.SELECTED DRAWING: Figure 4
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a maintenance device, a maintenance system, and a maintenance program. [Background technology]

[0002] Conventionally, various systems for inspecting elevator hoistways have been proposed (see, for example, Patent Document 1). Patent Document 1 proposes a system in which multiple cameras are installed inside the elevator hoistway, above and below the car, and the system compares current images of the hoistway taken by the cameras with images of the hoistway taken in the past to determine whether maintenance is required. When such a system is adopted, it becomes possible to automatically inspect for abnormalities even in buildings such as high-rise buildings where visual inspection is difficult. In addition, since the condition inside the hoistway can be confirmed without an operator visiting the site, the burden of maintenance work can be significantly reduced. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] European Patent Application Publication No. 3409629 Summary of the Invention [Problem to be solved by the invention]

[0004] As described above, a system has been proposed in the past that captures images of the interior of an elevator shaft with a camera to detect changes within the shaft. In such a system, for example, in order to accurately detect small foreign objects in the shaft or changes in the appearance of equipment, the shooting point (shooting position) of the camera needs to be precisely controlled. For example, the system disclosed in the above Patent Document 1 employs a method that uses a physical alignment mark formed by painting or the like to precisely align the shooting point of the current image of the shaft with the shooting point of the past image of the shaft.

[0005] However, in the system disclosed in Patent Document 1, for example, when the alignment mark itself deteriorates due to aging or accumulation of dirt, alignment itself becomes difficult. In addition, in the system disclosed in Patent Document 1, it is necessary to install multiple alignment marks corresponding to the installation positions of multiple cameras, and the installation costs are high, especially when the system is applied to a high-rise building. Therefore, in the conventional system for performing maintenance inspection by photographing the state inside the elevator hoistway with a camera, there is a demand for development of a technology that enables maintenance inspection with high accuracy, a simpler configuration, and low cost.

[0006] The present invention has been made to solve the above problems. An object of the present invention is to provide a technology for a maintenance device and a maintenance system that perform maintenance and inspection by photographing the state inside an elevator hoistway with a camera, which enables maintenance and inspection with high accuracy, a simpler configuration, and at low cost. [Means for solving the problem]

[0007] In order to solve the above problems, the maintenance device of the present invention is a maintenance device that acquires image data of a hoistway photographed by a photographing device during elevator maintenance and inspection, and includes an image processing unit that extracts analysis image data to be analyzed during the maintenance and inspection from the image data of the hoistway. The image processing unit also acquires drive information related to the running status of the elevator car during the maintenance and inspection, and extracts the analysis image data from the image data of the hoistway based on the drive information.

[0008] In order to solve the above problems, the maintenance system of the present invention is a maintenance system including an image capturing unit that captures image data of a hoistway during elevator maintenance and inspection, and further includes an image processing unit, an image analysis unit, and a display unit that displays the results of the analysis performed by the image analysis unit, which are included in the maintenance device of the present invention.

[0009] Furthermore, in order to achieve the above object, a maintenance program of the present invention is a maintenance program that installs and executes in the maintenance device the processing performed by an image processing unit included in the maintenance device of the present invention. Effect of the Invention

[0010] According to the present invention configured as described above, in a maintenance device and maintenance system that performs maintenance inspection by photographing the state inside an elevator hoistway with a camera, maintenance inspection can be performed with high accuracy, with a simpler configuration, and at low cost. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram of an elevator maintenance system according to an embodiment of the present invention. FIG. [Diagram 2] 1A to 1C are diagrams illustrating various configuration examples of an elevator drive monitoring unit that can be used in an elevator maintenance system according to an embodiment of the present invention. [Diagram 3] 1 is a hardware configuration diagram of an information processing device and a mobile terminal device that constitute an elevator maintenance system according to one embodiment of the present invention. FIG. [Figure 4] 1 is a functional block diagram of an elevator maintenance system according to an embodiment of the present invention. FIG. [Diagram 5] A diagram showing the relationship between the car drive pattern, the operation of photographing images of the elevator shaft, and the operation of extracting images in an elevator maintenance system according to one embodiment of the present invention. [Figure 6] FIG. 1 is a diagram for explaining a method for detecting foreign objects in a hoistway by an elevator maintenance system according to one embodiment of the present invention. [Figure 7] FIG. 1 is a diagram for explaining a method for measuring dimensions of devices and parts in a hoistway by an elevator maintenance system according to one embodiment of the present invention. [Figure 8] 5 is a flowchart showing the procedure of a first image analysis performed when a foreign object is detected in a hoistway by the elevator maintenance system according to one embodiment of the present invention. [Figure 9]10 is a flowchart showing the procedure of a second image analysis performed when a foreign object is detected in a hoistway by the elevator maintenance system in one embodiment of the present invention. [Figure 10] FIG. 13 is a diagram showing an example of an image analysis result of foreign object detection displayed on a display unit of an elevator maintenance system according to one embodiment of the present invention. [Figure 11] 11 is a diagram for explaining a method for calculating the amount of deviation between the shooting point of a current image acquired in an elevator maintenance system according to one embodiment of the present invention and that of a reference image. FIG. [Figure 12] FIG. 13 is a diagram showing an example of an image analysis result of dimensional measurement of devices and parts in a hoistway, displayed on a display unit of an elevator maintenance system according to one embodiment of the present invention. [Figure 13] FIG. 13 is a diagram showing an example of an image analysis result of foreign object detection displayed on the display unit of the elevator maintenance system according to one embodiment of the present invention. [Figure 14] FIG. 13 is a diagram showing a comparative example of image analysis results of foreign object detection displayed on the display unit of the elevator maintenance system according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] An elevator maintenance device and a maintenance system according to an embodiment of the present invention will be specifically described below with reference to the drawings. The degree of reduction in the load on maintenance workers obtained by the maintenance system of the present invention increases as the number of floors in a building increases. Therefore, the embodiment described below assumes that the elevator is installed in a building with six or more floors.

[0013] [Elevator maintenance system configuration] Fig. 1 is a block diagram of an elevator maintenance system according to one embodiment of the present invention. For the sake of simplicity, only components related to control of image capture, image analysis, display of analysis results, and the like, which are performed during maintenance and inspection of an elevator hoistway, are shown here.

[0014] 1, the maintenance system 10 includes a maintenance device 1, a mobile terminal device 2, and a drive monitoring unit 3 (drive information detection unit). The maintenance device 1 also includes a camera 4 (imaging unit, imaging device), a light 5, and an information processing device 6. The camera 4 and the light 5 may be newly installed as components of the maintenance device 1, or may be already installed in an elevator or a building.

[0015] In the maintenance system 10 of this embodiment, the information processing device 6 and the mobile terminal device 2 are connected by wireless and / or wired communication lines (the communication path may include a wide area communication network). The information processing device 6 and the drive monitoring unit 3, the information processing device 6 and the camera 4, and the information processing device 6 and the light 5 are also connected by wireless or wired communication lines.

[0016] The camera 4 captures images (video / still images) of the elevator hoistway. In this embodiment, the camera 4 is a camera equipped with a fisheye lens, but the present invention is not limited to this. For example, any lens with an angle of view (wide-angle lens) can be used as long as it can capture the entire state inside the hoistway. In addition, as described later, when performing dimensional measurement (see FIG. 7 described later) of various devices and parts installed in the hoistway 20 during maintenance inspection, it is preferable to be able to enlarge and capture the devices and parts to be measured. Therefore, as the lens of the camera 4, multiple types of lenses with different angles of view (focal lengths) may be provided, and a function may be provided to switch the lens to be used to a lens with an appropriate angle of view depending on the target item of the maintenance inspection (hereinafter referred to as the "maintenance inspection target"). In addition, as the lens of the camera 4, for example, a zoom lens with a changeable focal length may be used.

[0017] Furthermore, for example, a visible light camera, an infrared camera, etc. can be used as the camera 4. When a visible light camera is used as the camera 4, image data including multi-dimensional information of red (R), green (G), and blue (B) can be obtained, enabling more accurate judgment during maintenance and inspection.

[0018] Furthermore, the type of image (video or still image) captured by camera 4 can be changed as appropriate depending on the maintenance and inspection target. For example, when the maintenance and inspection target is detection of foreign objects in a hoistway, which will be described later, the images captured by camera 4 may be video or still images captured continuously. For example, when the maintenance and inspection target is measurement of dimensions of devices and parts in a hoistway, which will be described later, the images captured by camera 4 may be video or still images (single image). Furthermore, depending on the maintenance and inspection target, a configuration may be adopted in which a still image is extracted from the video captured by camera 4.

[0019] In this embodiment, the camera 4 is installed on the underside of the elevator car (see Figs. 2A and 2B described later), but it may be installed on the upper or side surface of the car. Also, the camera 4 does not necessarily have to be installed on the car side, and may be installed in a position suitable for photographing devices and parts in the elevator shaft that are the subject of maintenance and inspection, for example.

[0020] Light 5 is provided to illuminate the inside of the elevator shaft, which is usually a very dark environment. Light 5 is installed near camera 4. As light 5, a light that irradiates light suitable for the wavelength band of camera 4 to be used is used. If camera 4 is a visible light camera, a visible light light that can irradiate visible light is used, and if camera 4 is an infrared camera, an infrared light that can irradiate infrared light is used.

[0021] When a visible light camera is used as the camera 4, a visible light is used as the light 5, so if a vision glass is provided at the landing door, the light of the light 5 illuminating the inside of the hoistway during elevator maintenance and inspection work will also reach the building side. In this case, the appearance of the car running with the light 5 illuminating it may be observed by passengers, and from the passenger's perspective, it may seem strange that the car is running with the light 5 illuminating it. On the other hand, when an infrared camera is used as the camera 4, an infrared light is used as the light 5, so the light of the light 5 will not be observed by passengers during elevator maintenance and inspection work, and the above-mentioned situation can be avoided.

[0022] The information processing device 6 is configured with a processing device having a computing function and a communication function, such as a tablet, a notebook computer, etc. In this embodiment, the information processing device 6 is placed at a predetermined location in a building in which an elevator is installed, but may be installed in the elevator.

[0023] The information processing device 6 acquires image (video / still image) data captured by the camera 4, and information relating to the running status of the car detected by the drive monitoring unit 3 (hereinafter referred to as "drive information"). Then, based on the acquired drive information, the information processing device 6 extracts image data required for maintenance and inspection of the elevator shaft from the image data captured by the camera 4. That is, in the maintenance system 10 of this embodiment, image data of the elevator shaft required for maintenance and inspection is extracted in cooperation with the drive information of the car. The method of extracting this image data will be described in detail later with reference to the drawings.

[0024] Moreover, the information processing device 6 transmits the extracted image data to the mobile terminal device 2. Furthermore, when performing maintenance and inspection, the information processing device 6 transmits instruction signals to the camera 4 to start and end shooting, and transmits instruction signals to the light 5 to turn it on and off.

[0025] The mobile terminal device 2 is composed of a processing device with a computing function and a communication function, such as a smartphone, tablet, or laptop computer, which is carried or managed by an operator such as an elevator maintenance worker. The mobile terminal device 2 analyzes the image data of the elevator shaft transmitted from the maintenance device 1 and displays the analysis results on a screen. This enables the operator to check all maintenance and inspection items, such as the presence or absence of foreign objects in the elevator shaft and changes in the appearance of various devices and parts in the elevator shaft, on the terminal at hand without visiting the site. The method of analyzing the image data of the elevator shaft by the mobile terminal device 2 and the display mode of the analysis results will be described in detail later with reference to the drawings.

[0026] The drive monitoring unit 3 acquires various drive information of the elevator car, such as position information, acceleration information, speed information, etc. The drive information of the car acquired by the drive monitoring unit 3 is used to detect the extraction timing when the information processing device 6 extracts image data required for the maintenance and inspection of the elevator shaft from the image data captured by the camera 4.

[0027] As a first configuration mode of the drive monitoring unit 3, for example, a configuration in which a sensor box including a group of sensors capable of detecting the position, acceleration, speed, etc. of the car is newly installed in the car is considered. In such a configuration, the drive monitoring unit 3 is configured as a portable sensor unit, so that it becomes easy to add it to an elevator that is already installed, for example, and it becomes easy to introduce it into a maintenance system.

[0028] As a second configuration of the drive monitoring unit 3, for example, a configuration using a drive control system that is already installed in a building in which an elevator is installed and outputs control information such as a driving command to the car is considered. In this configuration, the drive information of the car is detected in cooperation with the control information such as a driving command to the car.

[0029] Furthermore, as a third configuration aspect of the drive monitoring unit 3, for example, a configuration using various sensors that are pre-installed in the elevator system or building is conceivable. For example, a configuration is conceivable in which PosiTector (registered trademark), a shielding plate, a floor correction sensor, a specific distance sensing device, a car position detection device, etc., capable of detecting the position information of the car 21, are used to obtain the car drive information. In the second and third configuration aspects of the drive monitoring unit 3 described above, since there is no need to add a new sensor to obtain the car drive information, the drive monitoring unit 3 can be configured at low cost.

[0030] Here, specific configuration examples of the first and second configuration aspects of the drive monitoring unit 3 described above will be described. Fig. 2A is a diagram showing the first configuration aspect of the drive monitoring unit 3, and Fig. 2B is a diagram showing the second configuration aspect of the drive monitoring unit 3. Figs. 2A and 2B show a state in which an elevator car 21 that can ascend and descend within a hoistway 20 is stopped at the height of a landing door 22 on the first floor. Also, in Figs. 2A and 2B, the light 5 is omitted from illustration in order to simplify the description.

[0031] In a first configuration of the drive monitoring unit 3, as shown in FIG. 2A, the camera 4 is attached to the lower surface of the car 21, and a sensor box 3a is attached to the upper surface of the car 21 as the drive monitoring unit 3. Various car drive monitoring sensors capable of measuring the drive state of the car 21 are provided in the sensor box 3a. Specifically, for example, an acceleration sensor capable of detecting the acceleration / deceleration of the car 21, an air pressure sensor capable of detecting the height of the car 21, a magnetic sensor capable of measuring the surrounding magnetic field and estimating the self-position, and the like are provided as car drive monitoring sensors in the sensor box 3a. Then, the detection signals detected by these car drive monitoring sensors are transmitted to the information processing device 6. In addition, the information processing device 6 that receives the detection signals extracts image data of the elevator shaft 20 required for maintenance and inspection based on the detection signals.

[0032] In the first configuration of the drive monitoring unit 3, for example, a method may be adopted in which an air pressure sensor and an acceleration sensor are combined, the acceleration sensor captures the acceleration and deceleration at the start of travel, and the air pressure sensor measures the position (height) of the car 21 while travelling. Also, for example, the height of the car 21 may be more accurately estimated using a system state estimation algorithm such as a Kalman filter or an αβ filter using multiple sensor data.

[0033] In a second configuration of the drive monitoring unit 3, as shown in Fig. 2B, a drive control system 3b that has already been installed in a building (hoistway, etc.) in which an elevator is installed and that outputs control information such as travel commands to the car 21 is used as the drive monitoring unit 3. Note that, in the second configuration of the drive monitoring unit 3, the camera 4 is also attached to the underside of the car 21. In the second configuration of the drive monitoring unit 3, the control information generated by the drive control system 3b is transmitted to the information processing device 6, and the information processing device 6 that has received the control information extracts image data of the hoistway 20 required for maintenance and inspection based on the control information.

[0034] [Hardware configuration of information processing device and mobile terminal device] In the maintenance system 10 of this embodiment, the information processing device 6 provided in the maintenance device 1 and the mobile terminal device 2 can be configured with an arithmetic processing device such as a computer device. Fig. 3 is a block diagram showing an example of the hardware configuration of a computer device 100 applicable as the information processing device 6 and the mobile terminal device 2.

[0035] The computer device 100 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, and a RAM (Random Access Memory) 103, all connected to a bus line 108. The computer device 100 also includes a non-volatile storage 104, a network I / F (interface) 105, an input unit 106, and a display unit 107, all connected to the bus line 108. Although not shown in Fig. 3, the computer device 100 also includes various interfaces used when executing input / output processing of various data (various information) between the computer device 100 and external devices.

[0036] The CPU 101 reads out program code of software for implementing various processing functions of the information processing device 6 and the mobile terminal device 2 from the ROM 102 to the RAM 103 and executes it. At this time, variables and parameters generated during the arithmetic processing are also temporarily written to the RAM 103.

[0037] The non-volatile storage 104 can be configured, for example, with a hard disk drive (HDD), a solid state drive (SSD), a flexible disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a non-volatile memory, etc. In addition to an operating system (OS) and various parameters, the non-volatile storage 104 stores programs for causing the computer device 100 to function as the information processing device 6 and the mobile terminal device 2. Note that information (data) such as programs, tables, and files that realize the various functions of the information processing device 6 and the mobile terminal device 2 may be stored in a recording medium such as an IC card, an SD card, or a DVD, other than the ROM 102 or the non-volatile storage 104.

[0038] The network I / F 105 is configured, for example, by a network interface card (NIC) or the like, and transmits and receives various data to and from each device connected via wireless communication.

[0039] The input unit 106 is composed of, for example, keys and buttons, generates an operation signal according to the operation content input by an operator, and supplies the operation signal to the CPU 101. The display unit 107 is composed of, for example, a liquid crystal panel, and displays characters, images, and the like on a screen. The display unit 107 may be composed of a touch panel, in which case the display unit 107 and the input unit 106 are integrally configured. Also, the information processing device 6 may not include the input unit 106 and / or the display unit 107.

[0040] [Relationship between the camera sampling rate and the drive monitoring unit sampling rate] As described above, in the maintenance system 10 of this embodiment, the information processing device 6 extracts image data required for the maintenance and inspection of the hoistway 20 from the image data captured by the camera 4 in cooperation with the drive information of the car 21 acquired by the drive monitoring unit 3. Then, based on the extracted image data, image analysis is performed on the maintenance and inspection target of the hoistway 20. Therefore, in order to perform the maintenance and inspection of the hoistway 20 with high accuracy based on the image data, it is preferable to control not only the shooting timing (shooting point) of the camera 4 but also the extraction timing of the image data with high accuracy.

[0041] For example, in detecting a foreign object in the elevator shaft 20 described later, for example, while the elevator car 21 is traveling, a video is taken by the camera 4, and a current image of the elevator shaft 20 obtained by the video is compared with a corresponding past image of the elevator shaft 20. In the following, the current image of the elevator shaft 20 to be compared is referred to as a "current image", and a past image of the elevator shaft 20 corresponding to the current image is referred to as a "reference image". In this case, in order to detect a foreign object in the elevator shaft 20 with high accuracy, it is necessary to improve the accuracy of the alignment between the shooting point (shooting position) of the current image and that of the reference image. Specifically, in detecting a foreign object in the elevator shaft 20, it is preferable to control so that there is no deviation between the shooting point of the current image and that of the reference image even for one frame. To realize this control, it is necessary to control the shooting point (shooting position) of the camera 4 with high accuracy of several centimeters or less (for example, 3 cm or less).

[0042] Therefore, in the maintenance system 10 of this embodiment, in order to control the shooting point of the camera 4 with high accuracy, the sampling rate (detection cycle of the sensor) of the drive monitoring unit 3 is set to be higher than the sampling rate (frame rate) of the camera 4. For example, since the frame rate of the camera 4 is generally about 30 fps, the sampling rate of the drive monitoring unit 3 is set to a value higher than 30 Hz. Note that in various analysis examples of this embodiment described later (FIGS. 5A, 5B, 10 to 14 described later), the frame rate of the camera 4 is set to 30 fps, and the sampling rate of the drive monitoring unit 3 is set to 200 Hz.

[0043] [Maintenance system functional block configuration] 4 is a functional block diagram of the maintenance system 10. Note that, in order to simplify the explanation, only the functional block section related to the control of the image capturing process, image extraction process, image analysis process, and analysis result display process of the elevator shaft 20, which are performed by the maintenance system 10 during maintenance and inspection, is shown here.

[0044] (1) Configuration of the information processing device The information processing device 6 in the maintenance device 1 has an image processing unit 30, a notification unit 33, and a storage unit 34, as shown in Fig. 4. The image processing unit 30 has a drive section detection unit 31 (extraction trigger signal generation unit) and a current image extraction unit 32 (image extraction unit). The storage unit 34 has a current drive information storage unit 34a and an original image storage unit 34b. Although not shown in Fig. 4, the information processing device 6 has a drive information receiving unit for receiving drive information (information on position, acceleration, etc.) of the car 21 transmitted from the drive monitoring unit 3, and an image receiving unit for receiving image data transmitted from the camera 4.

[0045] In the information processing device 6, in terms of processing functions, the drive section detection unit 31 is connected to the current image extraction unit 32 and the current drive information storage unit 34a, and the current image extraction unit 32 is connected to the original image storage unit 34b and the notification unit 33. The current drive information storage unit 34a is connected to the drive monitoring unit 3 via a drive information receiving unit (not shown) by a wireless or wired communication line, and the original image storage unit 34b is connected to the camera 4 via an image receiving unit (not shown) by a wireless or wired communication line. The notification unit 33 is connected to a receiving unit 41 (described later) in the mobile terminal device 2 by a wireless and / or wired communication line.

[0046] During maintenance and inspection, the drive section detection unit 31 first sequentially acquires drive information (information on position, acceleration, etc.) of the car 21 in operation, which is sequentially transmitted from the drive monitoring unit 3, in real time via the current drive information storage unit 34a. Next, based on the acquired drive information of the car 21, the drive section detection unit 31 generates a trigger signal indicating the extraction start timing and a trigger signal indicating the extraction end timing for extracting image data required for maintenance and inspection of the elevator shaft 20 from the image data being photographed. Then, the drive section detection unit 31 outputs the generated trigger signal indicating the extraction start timing (hereinafter referred to as the "start trigger") and the trigger signal indicating the extraction end timing (hereinafter referred to as the "end trigger") to the current image extraction unit 32 at the corresponding timings.

[0047] First, the current image extraction unit 32 sequentially acquires image (video / still image) data of the hoistway 20 sequentially transmitted from the camera 4 during maintenance inspection in real time via the original image storage unit 34b. Next, when a start trigger is input from the drive section detection unit 31, the current image extraction unit 32 starts an extraction process of image data (hereinafter referred to as "current image data") required for maintenance inspection of the hoistway 20 from the original image data. Note that this extraction process of the current image data continues until an end trigger is input from the drive section detection unit 31. Next, when an end trigger is input from the drive section detection unit 31, the current image extraction unit 32 ends the extraction process of the current image data. As a result, the current image data required for maintenance inspection of the hoistway 20 is extracted from the original image data. Then, the current image extraction unit 32 temporarily stores the current image data (analysis image data) in a storage unit for current image data (not shown) in the storage unit 34, and outputs the stored current image data to the notification unit 33. The current image data may be stored in the original image storage unit 34b.

[0048] The notification unit 33 is an interface that transmits the current image data (video / still image) of the elevator shaft 20 extracted by the current image extraction unit 32 to a receiving unit 41 (described later) in the mobile terminal device 2. In this embodiment, the notification unit 33 also transmits the drive information of the car 21 successively stored in the current drive information storage unit 34a during maintenance and inspection to the receiving unit 41 (described later) in the mobile terminal device 2.

[0049] The current drive information storage unit 34a sequentially stores in real time drive information (information on position, acceleration, etc.) of the car 21 in operation sequentially transmitted from the drive monitoring unit 3 during maintenance inspection. Also, the original image storage unit 34b sequentially stores in real time original image data (video / still image) of the elevator shaft 20 sequentially transmitted from the camera 4 during maintenance inspection.

[0050] Various processes of the image processing unit 30 (the driving section detection unit 31 and the current image extraction unit 32) described above are controlled by the CPU 101 in FIG. 3. Therefore, the image processing unit 30 (the driving section detection unit 31 and the current image extraction unit 32) are functionally included in the CPU 101 in FIG. 3. In addition, various processing functions of the image processing unit 30 (the driving section detection unit 31 and the current image extraction unit 32) described above are controlled by software, that is, by the processor (CPU 101) interpreting and executing a program for realizing each function. In addition, the storage unit 34 (the current driving information storage unit 34a and the original image storage unit 34b) is functionally included in the RAM 103 in FIG. 3, and the notification unit 33 is included in the network I / F 105 in FIG. 3. Note that some or all of the above-mentioned functional block units may be configured by hardware.

[0051] (2) Configuration of the mobile terminal device As shown in Fig. 4, the mobile terminal device 2 has a receiving unit 41, an image analysis unit 42, a display unit 43, an input unit 44, and a storage unit 45. The storage unit 45 has a current image storage unit 45a, a current driving information storage unit 45b, a reference image storage unit 45c, and a reference driving information storage unit 45d. Although not shown in Fig. 4, the mobile terminal device 2 has a reference image receiving unit that receives reference image data transmitted from a reference image database 46, and a reference driving information receiving unit that receives reference driving information transmitted from a reference driving information database 47. The storage unit 45 is functionally included in the RAM 103 in Fig. 3.

[0052] In the portable terminal device 2, the receiving unit 41 is connected to the storage unit 45 in terms of processing functions. The image analysis unit 42 is connected to the display unit 43, the input unit 44, and the storage unit 45 in terms of processing functions. The reference image storage unit 45c is connected to the reference image database 46 via a reference image receiving unit (not shown) by a wireless and / or wired communication line (the communication path may also include a wide area communication network). In addition, the reference drive information storage unit 45d is connected to the reference drive information database 47 via a reference drive information receiving unit (not shown) by a wireless and / or wired communication line.

[0053] The receiving unit 41 is an interface that receives current image data (video / still image) of the elevator 20 transmitted from the notification unit 33 in the information processing device 6, and drive information (information on position, acceleration, etc.) of the car 21 during maintenance and inspection. The receiving unit 41 outputs the received current image data (video / still image) to a current image storage unit 45a. The receiving unit 41 also outputs the received drive information of the car 21 to a current drive information storage unit 45b. The receiving unit 41 is included in the network I / F 105 in FIG. 3.

[0054] The image analysis unit 42 acquires various information (image data and driving information) required for various maintenance and inspection targets from the storage unit 45, and performs image analysis on the maintenance and inspection targets based on the acquired various information. Then, the image analysis unit 42 outputs the image analysis results of the maintenance and inspection targets to the display unit 43.

[0055] The various processes of the image analysis unit 42 described above are controlled by the CPU 101 in Fig. 3. Therefore, the image analysis unit 42 is functionally included in the CPU 101 in Fig. 3. The various processing functions of the image analysis unit 42 described above are controlled by software, that is, by the processor (CPU 101) interpreting and executing a program for realizing each function. Note that some or all of the various processing functions of the image analysis unit 42 may be configured by hardware.

[0056] 3, and displays the image analysis results of the maintenance and inspection target performed by the image analysis unit 42. The items of the image analysis results displayed on the display unit 43 vary depending on the type of maintenance and inspection target, and as described below, for example, a difference image between the current image and the reference image, an enlarged image of a specific part in the hoistway 20, the dimensions of the specific part, and the like are displayed on the display unit 43.

[0057] 3, and when an operator's operation of an operation means such as a key or a button is detected, the input unit 44 outputs an operation signal corresponding to the operation to the image analysis unit 42. When the display unit 43 (display unit 107 in FIG. 3) of the mobile terminal device 2 is configured as a touch panel, the display unit 43 and the input unit 44 are configured integrally.

[0058] Selection of the maintenance inspection target to be analyzed by the image analysis unit 42 and determination of the display contents of the image analysis results, such as which shooting position the image analysis results are to be displayed, are performed, for example, by an operator's input operation to the input unit 44. Specifically, for example, when the operator specifies a desired maintenance inspection target and the display contents of the image analysis results via the input unit 44, the specification signal is input to the image analysis unit 42. Then, the image analysis unit 42 performs image analysis of the specified maintenance inspection target, and controls the display unit 43 so that the result is displayed on the display unit 43 with the specified display contents.

[0059] The current image storage unit 45a stores the current image data (moving image / still image) received by the receiving unit 41. The current drive information storage unit 45b stores the drive information (information on the position, acceleration, etc.) of the car 21 received by the receiving unit 41.

[0060] The reference image storage unit 45c stores reference image data (video / still image) corresponding to the current image data (video / still image) stored in the current image storage unit 45a. The reference driving information storage unit 45d stores reference driving information (information on position, acceleration, etc.) corresponding to the driving information of the car 21 stored in the current driving information storage unit 45b (hereinafter referred to as "current driving information"). The reference image data (video / still image) stored in the reference image storage unit 45c is extracted from the reference image database 46, and the reference driving information stored in the reference driving information storage unit 45d is extracted from the reference driving information database 47.

[0061] The process of extracting reference image data corresponding to the current image data from the reference image database 46 and storing it in the reference image storage unit 45c is controlled by the image analysis unit 42 based on the current drive information (drive pattern). In addition, the process of extracting reference drive information corresponding to the current drive information from the reference drive information database 47 and storing it in the reference drive information storage unit 45d is also controlled by the image analysis unit 42 based on the current drive information (drive pattern).

[0062] (3) Configuration of the reference image database and the reference driving information database The reference image database 46 stores image data captured by the camera 4 in advance for each drive pattern when the car 21 is driven in all drive patterns assumed during maintenance and inspection in the elevator hoistway 20 in a normal state. The image data stored in the reference image database 46 is image data extracted from the original image data in the same manner as the method of extracting current image data, which is performed in the current image extraction unit 32 of the information processing device 6 and will be described later. The reference drive information database 47 stores drive information of the car 21 in advance for each drive pattern when the car 21 is driven in all drive patterns assumed during maintenance and inspection in the elevator hoistway 20 in a normal state. In grouping each data in each database, the data group for each drive pattern may be further subdivided by the shooting conditions (exposure conditions).

[0063] The various data stored in the reference image database 46 and the reference driving information database 47 may be obtained in advance by operating the elevator exclusively for obtaining these various data. Also, for example, the various data obtained when the elevator was in a normal state during past maintenance inspections may be stored in the reference image database 46 and the reference driving information database 47. Also, for example, the various data may be obtained when the elevator was in a normal state during normal operation in the past. Furthermore, the various data obtained by employing at least two of the three obtaining methods described above may be stored in the reference image database 46 and the reference driving information database 47.

[0064] Furthermore, between each reference image data stored in the reference image database 46 and each reference drive information stored in the reference drive information database 47, those having the same drive pattern of the car 21 are linked together. Note that the linking condition between the reference drive information and the reference image data may be that the drive pattern is the same and the shooting conditions (exposure conditions) of the elevator shaft 20 are the same.

[0065] The reference image database 46 and the reference driving information database 47 may be provided outside the maintenance system 10, or may be provided inside the maintenance system 10. In the former case, the reference image database 46 and the reference driving information database 47 may be provided, for example, at a predetermined location in a building, on an elevator manufacturer's server, in the cloud, etc. In the latter case, the reference image database 46 and the reference driving information database 47 may be provided, for example, in the non-volatile storage 104 in FIG. 3, or in a database unit provided separately from the maintenance device 1 and the mobile terminal device 2.

[0066] [Current image data extraction methods] Next, a method of extracting current image data from original image data, which is performed by the information processing device 6 (current image extraction unit 32) during maintenance and inspection, will be described. In the maintenance system 10 of this embodiment, two methods are provided as methods of extracting current image data.

[0067] Fig. 5A is a diagram showing the principle of a first extraction method of current image data performed by the current image extraction unit 32, and is a diagram showing the relationship between the drive pattern of the car 21, the operation of taking an image of the hoistway 20, and the operation of extracting the image, in the first extraction method. Fig. 5B is a diagram showing the principle of a second extraction method of current image data performed by the current image extraction unit 32, and is a diagram showing the relationship between the drive pattern of the car 21, the operation of taking an image of the hoistway 20, and the operation of extracting the image, in the second extraction method.

[0068] 5A and 5B show an example of maintenance inspection in which the car 21 is raised from the first floor to the twentieth floor in a 20-story building while the inside of the elevator shaft 20 is photographed by the camera 4. In this example, the frame rate of the camera 4 is 30 fps, and the sampling rate of the drive monitoring unit 3 (acceleration sensor) is 200 Hz. As described later, such driving operation of the car 21 and shooting operation of the camera 4 are performed, for example, when inspecting whether there is any foreign object or dropped object in the elevator shaft 20 (foreign object detection). A command to run the car 21 is issued from a drive control system 3b (see FIG. 2B) provided in the building.

[0069] The signal waveforms shown at the top of each of Fig. 5A and Fig. 5B are waveforms showing the change in acceleration of the car 21 from the start to the end of the maintenance inspection. This acceleration signal is detected by an acceleration sensor provided in the drive monitoring unit 3. In the acceleration signal, the acceleration rises once when the car 21 starts to ascend from the first floor, and then returns to 0 when the car shifts to constant speed operation. Then, when the car 21 reaches the 20th floor and stops, the acceleration falls once, and when the car 21 stops, the acceleration returns to 0.

[0070] 5A and 5B are signal waveforms of a start trigger output from the drive section detection unit 31 to the current image extraction unit 32. The drive section detection unit 31 detects the timing at which the car 21 starts to rise from the first floor (the timing at which it starts to move) and outputs a pulse-like trigger signal (start trigger) to the current image extraction unit 32. Therefore, in the signal waveform of the start trigger, the start trigger (pulse signal) is generated at the timing at which the acceleration signal first becomes high.

[0071] 5A and 5B are signal waveforms of an end trigger output from the drive section detection unit 31 to the current image extraction unit 32. The drive section detection unit 31 detects the timing at which the car 21 reaches the 20th floor and stops, and outputs a pulsed trigger signal (end trigger) to the current image extraction unit 32. Therefore, in the signal waveform of the end trigger, the acceleration signal becomes low once after the start trigger is output, and then the end trigger (pulse signal) is generated at the timing when the acceleration returns to 0.

[0072] In the first extraction method of the current image data performed by the current image extraction unit 32, as shown in Fig. 5A, the camera 4 takes pictures of the elevator shaft 20 from the start to the end of the maintenance and inspection work. That is, in the first extraction method, the timing of starting to take pictures of the elevator shaft 20 is before the timing when the car 21 starts to rise from the first floor (the timing of occurrence of the start trigger). Also, the timing of ending to take pictures of the elevator shaft 20 is after the timing when the car 21 reaches the 20th floor and stops (the timing of occurrence of the end trigger).

[0073] In the first extraction method of the current image data, the current image extraction unit 32 first starts the extraction operation of the current image data of the elevator shaft 20 in response to the input of a start trigger from the drive section detection unit 31. Next, the current image extraction unit 32 ends the extraction operation of the current image data of the elevator shaft 20 in response to the input of an end trigger from the drive section detection unit 31. In other words, the timing at which the extraction of the current image data starts is later than the timing at which the original image data is started to be captured, and the timing at which the extraction of the current image data ends is earlier than the timing at which the original image data is captured. Therefore, in the first extraction method, the capacity of the current image data extracted by the current image extraction unit 32 is smaller than the capacity of the original image data captured by the camera 4.

[0074] On the other hand, in the second extraction method of the current image data performed by the current image extraction unit 32, as shown in FIG. 5B, the start timing and end timing of the extraction operation of the current image data by the current image extraction unit 32 are made the same as those of the shooting operation of the original image data by the camera 4. Specifically, in the second extraction method, first, when the elevator car 21 starts to rise, the shooting operation of the camera 4 and the extraction operation of the current image extraction unit 32 are started simultaneously in response to the input of a start trigger from the drive section detection unit 31. Then, when the elevator car 21 stops, the shooting operation of the camera 4 and the extraction operation of the current image extraction unit 32 are ended simultaneously in response to the input of an end trigger from the drive section detection unit 31. Therefore, in the second extraction method, the capacity of the current image data extracted by the current image extraction unit 32 is the same as the capacity of the original image data shot by the camera 4.

[0075] As described above, the second extraction method of the current image data performed by the current image extraction unit 32 can reduce the volume of the original image data compared to the first extraction method. Therefore, in this embodiment, for example, the first or second extraction method is appropriately selected as the extraction method of the current image data by the current image extraction unit 32 in consideration of the memory capacity of the information processing device 6 (maintenance device 1) and the volume of the image data.

[0076] In the second extraction method, the operation timings of starting and ending shooting with the camera 4 and starting and ending extraction of the current image data are performed simultaneously, but in this case, there is a possibility that the operation timings of the two may be slightly different due to the influence of noise, etc. In this case, the number of frames of the current image data may differ slightly from that of the reference image data. However, in the first extraction method, the operation timings of starting and ending shooting with the camera 4 and starting and ending extraction of the current image data are different, so the influence of the noise described above can be reduced.

[0077] [Image analysis method used in the image analysis section] Next, the contents of the image analysis method performed by the image analysis unit 42 will be described. Here, detection of foreign objects in the elevator hoistway 20 and measurement of the overlapping margin of the door lock drive mechanism on each floor will be taken as examples of maintenance inspection targets, and the contents of the image analysis method performed by the image analysis unit 42 in each example will be described.

[0078] (1) Detection of foreign objects in elevators In detecting a foreign object in the hoistway 20, for example, the car 21 is raised from the first floor to the top floor, while the camera 4 takes an image of the state inside the hoistway 20, and an inspection is performed to see whether there is a foreign object or a dropped item inside the elevator hoistway 20. Note that in detecting a foreign object in the hoistway 20, for example, the car 21 may be lowered from the top floor to the first floor, while the camera 4 takes an image of the state inside the hoistway 20.

[0079] Fig. 6A is a schematic diagram of the first floor landing door 22 as seen from the landing side. Fig. 6B and Fig. 6C are schematic diagrams of images of the elevator shaft 20 captured by the camera 4 when the car 21 moves (rises) to a position slightly higher than the first floor landing door 22. Therefore, in each of Fig. 6B and Fig. 6C, the first floor landing door 22 and the pit (bottom) are also captured.

[0080] The image shown in FIG. 6B is an image when no foreign object 50 is present in the hoistway 20 (normal), and this image corresponds to the reference image acquired by the image analysis unit 42 from the reference image database 46. On the other hand, the image shown in FIG. 6C is an image of the hoistway 20 captured during a maintenance inspection currently being performed, and this image corresponds to the current image acquired by the image analysis unit 42 from the maintenance device 1. Note that the example shown in FIG. 6C illustrates an example in which multiple foreign objects 50 (triangle marks in the figure) are present in the hoistway 20. Also, the shooting point (shooting position) of the current image shown in FIG. 6C is the same as that of the reference image shown in FIG. 6B.

[0081] Then, the image analysis unit 42, which has acquired the reference image shown in Fig. 6B and the current image shown in Fig. 6C, calculates a difference image between the two. At this time, since the shooting point (shooting position) of the current image is the same as that of the reference image, only images of the multiple foreign bodies 50 are extracted in the difference image.

[0082] In order to realize highly accurate detection of foreign objects in the elevator shaft 20 by the image analysis unit 42, it is necessary to accurately align the shooting point (shooting position) of the current image with that of the reference image. As described above, in the maintenance system 10 of this embodiment, the maintenance device 1 acquires current image data required for maintenance inspection in coordination with the drive information of the car 21, specifically, based on a trigger signal generated by the drive section detection unit 31. Similarly to the current image data, the reference image data is also acquired in coordination with the drive information of the car 21.

[0083] As a result, in this embodiment, in principle, the number of frames of the current image data (moving image) to be compared by the image analysis unit 42 is the same as the number of frames of the reference image data (moving image). Also, the shooting point (shooting position) of the current image located at the first frame position in the current image data is the same as the shooting point of the reference image located at the first frame position in the reference image data. Furthermore, the shooting point (shooting position) of the current image located at a predetermined frame position from the beginning in the current image data is also the same as the shooting point of the reference image located at a predetermined frame position from the beginning in the reference image data.

[0084] That is, in principle, in this embodiment, the shooting point (shooting position) of a current image of a predetermined frame number in the current image data is the same as the shooting point of a reference image of a predetermined frame number in the reference image data. Even if there is some influence of noise, etc., the shooting point of a current image of a predetermined frame number in the current image data is very close to the shooting point of a reference image of a predetermined frame number in the reference image data. Therefore, in detecting a foreign object in the elevator shaft 20 by the image analysis unit 42, only the image of the foreign object 50 can be extracted with high accuracy by simply calculating the difference image between the current image and the reference image of the same frame number.

[0085] (2) Measurement of the engagement gap of the door lock drive mechanism on each floor In the maintenance system 10 of this embodiment, it is also possible to measure the dimensions of various devices, parts, etc. installed in the elevator shaft 20 by image analysis in the image analysis unit 42. Here, as one example, a case will be described in which the dimensions of the engagement allowance of the door lock drive mechanism on each floor are measured by image analysis in the image analysis unit 42 during maintenance inspection.

[0086] Normally, in an elevator system, a door lock drive mechanism provided in the hoistway 20 above the landing door 22 at each floor is always in a locked state when the landing door 22 is closed. When the car 21 arrives at each floor, the door lock drive mechanism is mechanically unlocked.

[0087] FIG. 7 is a schematic diagram of an image (current image) of the tip part (lock function part 60) of the door lock drive mechanism captured by the camera 4 installed on the underside of the car 21 during maintenance inspection. This image is captured at the timing when the car 21 moves to a predetermined position where the lock function part 60 of the door lock drive mechanism is within the shooting range (angle of view) of the camera 4 during maintenance inspection. This image is obtained by extracting only the image of the lock function part 60 of the door lock drive mechanism from the current image captured by the camera 4. The current image shown in FIG. 7 may be an image of a frame number corresponding to the shooting position of the lock function part 60 extracted from video data captured while the car 21 is moving. The current image shown in FIG. 7 may be a still image captured by stopping the car 21 when the car 21 moves to the shooting position of the lock function part 60.

[0088] 7, the lock function unit 60 of the door lock drive mechanism has a key portion 61 and a hook portion 62, and when locked (the door is closed), the key portion 61 engages with the hook portion 62 in the locked position. When locked, the key portion 61 is pulled by a coil spring (not shown) and is in the locked position, and the door panel is locked so as not to open.

[0089] Under normal circumstances (normal operation), in the locked position, the gap 63 between the key portion 61 and the hook portion 62 is about several millimeters, and when the hall door 22 opens, the key portion 61 opens smoothly and the lock is released. If the dimension of the gap 63 becomes narrower for some reason, when the door opens from a closed state, the key portion 61 and the hook portion 62 get caught, causing abnormal noise and abnormal vibration. In addition, if the dimension of the gap 63 becomes even narrower, the hall door 22 may not open in a closed state, in which case there is a risk that a user may be trapped inside the elevator car 21, and therefore it is necessary to prevent this from happening in advance.

[0090] In this embodiment, the image analysis unit 42 analyzes a current image of the lock function unit 60 as shown in FIG. 7, identifies (detects) the gap 63 (the hook portion 61 and the hook portion 62), and automatically measures the size of the gap 63.

[0091] As a method for detecting the key portion 61, the hook portion 62, and the gap 63 by the image analysis unit 42, for example, teacher data for detecting a large number of key portions 61 and hook portions 62 may be prepared in advance, and the key portion 61 and the hook portion 62 may be detected from the current image using machine learning. Also, for example, if it is known from the results of a preliminary study that there is a gap 63 in the central region of the current image shown in Fig. 7, image analysis may be performed only within the gap expected region 64 to detect the key portion 61, the hook portion 62, and the gap 63.

[0092] Furthermore, the following method can be adopted as a method for detecting the gap 63 and measuring the size by the image analysis unit 42. First, the image analysis unit 42 extracts the contours of the key portion 61 and the hook portion 62 by edge extraction processing such as the Canny method to identify the gap 63. Next, the image analysis unit 42 measures the number of pixels between the edge of the key portion 61 and the edge of the hook portion 62 that face each other in the gap 63, and sets the number of pixels as the size of the gap 63. When such a method is adopted, it is possible to perform dimensional measurement of elevator parts with high accuracy even in a site where the presence of various equipment structures and adhesion of dirt are expected, without preparing a large amount of teacher data. In addition, in this case, it is possible to replace dimensional measurement, which was previously performed by an operator visiting the site, with camera photography, thereby reducing the workload of the operator.

[0093] [Processing flow of image analysis (maintenance inspection) performed in the maintenance system] Next, an example of a process flow of image analysis (maintenance inspection) performed by the maintenance system 10 of this embodiment will be described. Here, a process flow from the start of maintenance inspection to display of the result of the maintenance inspection when detecting a foreign object in the hoistway 20 described in Fig. 6 will be described.

[0094] Here, the process flows for the current image extraction unit 32 extracting the current image data using the first extraction method (see FIG. 5A) and the second extraction method (see FIG. 5B) described above will be described. In the following, the former process flow will be referred to as the "first image analysis flow" and the latter process flow will be referred to as the "second image analysis flow." Furthermore, in the example described below, during maintenance inspection, an elevator car 21 is raised from the first floor to the twentieth floor in a 20-story building, and during that time, the state inside the hoistway 20 is photographed by the camera 4, and an inspection will be performed to see whether there is a foreign object 50 inside the hoistway 20.

[0095] (1) First image analysis flow Fig. 8 is a diagram showing a first image analysis flow when detecting a foreign object in the elevator shaft 20 by the maintenance system 10 of this embodiment. The first image analysis flow shown in Fig. 8 is started when, for example, an operator issues a maintenance inspection start instruction to the maintenance system 10. The maintenance inspection instruction is transmitted to the maintenance device 1 by, for example, wireless communication or wired communication from a mobile terminal device 2 owned by the operator.

[0096] First, when a maintenance inspection start instruction is issued to the maintenance system 10, the maintenance system 10 is started, and the information processing device 6 of the maintenance device 1 controls the camera 4 to start image (video) capture processing of the elevator shaft 20 (S1). In addition, in the processing of S1, the drive monitoring unit 3 also starts processing to acquire drive information of the car 21 during maintenance inspection.

[0097] The start timing of the photographing operation by the camera 4 is not limited to the above example. For example, the start timing of the photographing operation by the camera 4 may be set appropriately depending on the maintenance inspection target. Also, for example, the start timing of the photographing operation may be the timing when the lower part of a hatch door provided in an openable and closable manner at each landing in the elevator 20 is captured within the shooting range of the camera 4 after a maintenance inspection start instruction is issued to the maintenance system 10. Furthermore, for example, the start timing of the photographing operation may be the timing when the pit (bottom) of the elevator 20 is captured within the shooting range of the camera 4.

[0098] Next, the drive section detection unit 31 of the information processing device 6 detects the timing of the cage 21 starting to move (starting to rise) based on the drive information (acceleration information) of the cage 21 input from the drive monitoring unit 3 (S2). Also, in the process of S2, if the timing of the cage 21 starting to move (starting to rise) is detected, the drive section detection unit 31 outputs a start trigger to the current image extraction unit 32. Furthermore, in the process of S2, the current image extraction unit 32 starts the extraction process of image data in response to the input of the start trigger.

[0099] Next, the drive section detection unit 31 detects the stopping timing of the car 21 based on the drive information (acceleration information) of the car 21 input from the drive monitoring unit 3 (S3). Also, in the process of S3, if the stopping timing of the car 21 is detected, the drive section detection unit 31 outputs an end trigger to the current image extraction unit 32. Furthermore, in the process of S3, the current image extraction unit 32 ends the image data extraction process in response to the input of the end trigger. This allows the current image data to be analyzed in maintenance and inspection to be acquired.

[0100] Next, the information processing device 6 of the maintenance device 1 controls the camera 4 to end the image (video) capturing process of the elevator shaft 20 (S4). In addition, in the process of S4, the process of acquiring the drive information of the car 21 during maintenance inspection by the drive monitoring unit 3 is also ended.

[0101] Next, the current image extraction unit 32 saves the acquired current image data (S5). Next, the information processing device 6 (maintenance device 1) transmits the saved current image data to the mobile terminal device 2 (receiving unit 41) via the notification unit 33 (S6). The current image data transmitted to the mobile terminal device 2 (receiving unit 41) is stored in the current image storage unit 45a in the mobile terminal device 2.

[0102] Next, the image analysis unit 42 in the mobile terminal device 2 calculates the difference between the current image data received via the receiving unit 41 and the reference image data obtained from the reference image storage unit 45c and corresponding to the current image data (S7). In this process, a difference image between the current image and the reference image having the same frame number is generated for each frame number. That is, difference image data consisting of difference images of the same number of frames as the number of frames of the current image data and the reference image data is generated. The reference image data corresponding to the current image data is selected from the reference image database 46 and downloaded to the reference image storage unit 45c under the control of the image analysis unit 42.

[0103] Next, the image analysis unit 42 performs a noise removal process on each differential image included in the differential image data generated in the process of S7 (S8). Next, the image analysis unit 42 outputs the image analysis result including the differential image data that has been subjected to the noise removal process to the display unit 43 (S9). Through this process, the image analysis result (including the differential image data) of the maintenance inspection is displayed on the display unit 43. Then, after the image analysis result is displayed, the series of processes of the maintenance inspection described above is completed.

[0104] In the display process of the image analysis results in S9, the image analysis unit 42 displays various pieces of information on the analysis results including corresponding differential images, etc., on the display unit 43 in accordance with the display contents of the analysis results of the maintenance inspection target (detection of a foreign object in the hoistway 20) specified, for example, by the operator operating the input unit 44. A specific display example of the image analysis results when a foreign object is detected in the hoistway 20 will be described later with reference to the drawings.

[0105] (2) Second image analysis flow Fig. 9 is a diagram showing a second image analysis flow when the maintenance system 10 of this embodiment detects a foreign object in the hoistway 20. The second image analysis flow shown in Fig. 9 is started, like the first image analysis flow described above, when an instruction to start maintenance inspection is issued to the maintenance system 10 from an operator, for example.

[0106] First, when a maintenance inspection start instruction is issued to the maintenance system 10, the maintenance system 10 is started, and the drive monitoring unit 3 starts a process of acquiring drive information of the car 21 during maintenance inspection. Next, the drive section detection unit 31 of the information processing device 6 detects the timing of the car 21 starting to move (starting to rise) based on the drive information (acceleration information) of the car 21 input from the drive monitoring unit 3 (S11). Also, in the process of S11, if the timing of the car 21 starting to move (starting to rise) is detected, the drive section detection unit 31 outputs a start trigger to the current image extraction unit 32. Furthermore, in the process of S11, when the start trigger is input to the current image extraction unit 32, the information processing device 6 controls the camera 4 to start image (video) shooting process of the elevator shaft 20, and the current image extraction unit 32 starts extraction process of current image data.

[0107] Next, the drive section detection unit 31 detects the stop timing of the car 21 based on the drive information (acceleration information) of the car 21 input from the drive monitoring unit 3 (S12). Also, in the process of S12, if the stop timing of the car 21 is detected, the drive section detection unit 31 outputs an end trigger to the current image extraction unit 32. Furthermore, in the process of S12, when the end trigger is input to the current image extraction unit 32, the information processing device 6 controls the camera 4 to end the image (video) shooting process of the elevator shaft 20, and the current image extraction unit 32 ends the extraction process of the current image data.

[0108] Thereafter, in the second image analysis flow, the processes of S13 to S17 are performed in the same manner as the processes of S5 to S9 in the first image analysis flow described in Fig. 8, and the series of processes during the maintenance inspection described above is completed. Also, the display mode of various analysis results (including differential images, etc.) of the maintenance inspection in the second image analysis flow is the same as that in the first image analysis flow described above.

[0109] Although detailed description is omitted, when dimensional measurements of various devices and parts in the elevator shaft 20 described in FIG. 7 are performed during maintenance and inspection, only the contents of the processing of S7 and S8 in FIG. 8 and S15 and S16 in FIG. 9 change, and similar processing is performed in the other steps.

[0110] Specifically, instead of the processes of S7 and S8 in FIG. 8 and S15 and S16 in FIG. 9, the following process is performed. When measuring the dimensions of various devices and parts in the hoistway 20, the image analysis unit 42 first extracts a current image of a frame number corresponding to a shooting point designated by, for example, an operator operating the input unit 44 from the current image data (video) received via the receiving unit 41. Then, the image analysis unit 42 performs various analysis processes, such as the edge extraction process described above, on the extracted current image to calculate the dimensions of a desired portion of the device or part (gap 63 in the example of FIG. 7). Thereafter, the analysis results (dimensions, etc.) are output to and displayed on the display unit 43 in the same manner as the processes of S9 in FIG. 8 and S17 in FIG. 9. A specific display example of the image analysis results when measuring the dimensions of various devices and parts in the hoistway 20 will be described later with reference to the drawings.

[0111] [Example of image analysis results] Next, a display example of the image analysis result obtained by the maintenance system 10 of this embodiment during maintenance inspection will be described. Here, a display example of the image analysis result when detecting a foreign object in the elevator shaft 20 described in FIG. 6 and a display example of the image analysis result when measuring the dimension of the gap 63 of the door lock drive mechanism described in FIG. 7 will be described. In the example described below, the frame rate of the camera 4 is 30 fps, and the sampling rate of the drive monitoring unit 3 (acceleration sensor) is 200 Hz. Furthermore, in the example described below, an example will be described in which the display unit 43 of the mobile terminal device 2 is configured as a touch panel, that is, an example of a configuration in which the operator can directly touch the display screen of the display unit 43 and specify the display content of the image analysis result.

[0112] (1) Example of a foreign object detected in a hoistway FIG. 10 is a diagram showing an example of the image analysis result (inspection result) displayed on the display unit 43 of the mobile terminal device 2 when the maintenance system 10 detects a foreign object in the elevator shaft 20. In this maintenance inspection, the elevator car 21 is raised from the first floor to the twentieth floor in a 20-story building, and the state inside the elevator shaft 20 is photographed by the camera 4 during the elevation. This example also describes an example in which the current image extraction unit 32 in the maintenance device 1 extracts current image data from the original image data using the first extraction method shown in FIG. 5A. Furthermore, this example describes an example in which an acceleration sensor is included in the drive monitoring unit 3, and the detection signal is acquired as drive information of the elevator car 21.

[0113] A reference image, a current image, and a difference image are displayed in a horizontally arranged manner in the upper part of the display screen 43a of the display unit 43, as shown in Fig. 10. Note that, in accordance with the explanation of Fig. 6, the reference image, the current image, and the difference image are shown in schematic diagrams rather than as they are actually obtained.

[0114] The reference image displayed in Fig. 10 is a reference image of a predetermined frame number included in the reference image data that is the subject of difference processing by the image analysis unit 42 of the mobile terminal device 2. Moreover, the current image displayed in Fig. 10 is a current image of a predetermined frame number included in the current image data that is the subject of difference processing by the image analysis unit 42. In this embodiment, as described above, in principle, the reference image data and current image data that are the subject of difference processing by the image analysis unit 42 have the same number of frames, so the frame numbers of the reference image and current image displayed in Fig. 10 are also the same. Moreover, the difference image displayed in Fig. 10 is displayed in black and white to clearly indicate the presence or absence of a foreign object 50 (triangle mark in the figure).

[0115] When the reference image, the current image, and the difference image are displayed as described above, the difference area between the reference image and the current image is highlighted in the difference image. Therefore, the operator can quickly and easily confirm the change from the normal state captured in the past by simply looking at the difference image. In addition, the operator can easily confirm, for example, the type of foreign object 50 in the hoistway 20 or the structural change by referring to the current image and the reference image of the area where the difference occurred.

[0116] In addition, within the display screen 43a, a display block of the playback time of the reference image is disposed below the reference image, and a display block of the playback time of the current image is disposed below the current image.

[0117] The value displayed in the display block for the playback time of the reference image (0.505 seconds in the example of FIG. 10) is the elapsed time from the start of shooting of the reference image data. The value displayed in this display block for the playback time of the reference image is calculated based on the frame rate of the camera 4 and the frame number of the reference image being displayed. Meanwhile, the value displayed in the display block for the playback time of the current image (1.000 seconds in the example of FIG. 10) is the elapsed time from the start of shooting of the original image data. The value displayed in this display block for the playback time of the current image is calculated from the driving information (acceleration signal) of the car 21 acquired by the image analysis unit 42 and the playback time of the reference image, as described below.

[0118] Moreover, the display block of each playback time is a user interface that can be accessed (for numerical input) by the operator's touch operation on the display screen 43a. Therefore, in this embodiment, the numerical values ​​displayed in the display blocks of the playback times of the reference image and the current image can be changed on the display screen 43a by the operator's touch operation. When the numerical values ​​displayed in the display blocks of the playback times of the reference image and / or the current image are changed by the operator's touch operation, the reference image and / or the current image corresponding to the changed playback time are displayed. At this time, a difference image is generated again by the image analysis unit 42 based on the changed playback time, and the changed difference image is displayed on the display screen 43a. The change display process of the above-mentioned analysis result by the operator's touch operation is controlled by the image analysis unit 42 based on an instruction signal corresponding to the operator's touch operation.

[0119] In addition, on the display screen 43a, below the display blocks of the playback times of the reference image and the current image, a seek bar (playback bar) indicating the playback position of the current image being displayed in the current image data (moving image) is displayed.

[0120] The seek bar is a user interface that can be accessed (changed) by an operator's touch operation on the display screen 43a. Therefore, in this embodiment, the playback position of the current image can be changed on the seek bar by the operator's touch operation. When the playback position is changed by the operator's touch operation on the seek bar, the current image corresponding to the changed playback position is displayed. At this time, a difference image is generated again by the image analysis unit 42 based on the changed current image, and the changed difference image is displayed on the display screen 43a. The change display process of the analysis result by the operator's touch operation on the seek bar is controlled by the image analysis unit 42 based on an instruction signal corresponding to the operator's touch operation.

[0121] Moreover, in the display screen 43a, the left area in the lower area of ​​the seek bar displays acceleration data ("reference data" in the figure) included in the drive information of the car 21 when the reference image data was acquired. Also, in the display screen 43a, the left area in the lower area of ​​the seek bar displays acceleration data ("current data" in the figure) included in the drive information of the car 21 when the original image data was acquired. The "current data" is displayed below the "reference data".

[0122] In addition, in the display screen 43a, the acquisition date of the reference image data ("reference data date" in the figure) and the acquisition date of the current image data ("current data date" in the figure) are displayed in the right area below the seek bar.

[0123] The display (selection) blocks of "reference data date" and "current data date" are user interfaces that can be accessed (selected) by the operator's touch operation on the display screen 43a. Therefore, in this embodiment, the acquisition date of the reference image data and the acquisition date of the current image data can be changed on the display screen 43a by the operator's touch operation. When the acquisition date of the reference image data and / or the current image data is changed by the operator's touch operation on the display blocks of "reference data date" and / or "current data date", the reference image and / or the current image corresponding to the changed acquisition date are displayed. At this time, the image analysis unit 42 generates a difference image again based on the changed acquisition date of the reference image data and / or the current image data, and the changed difference image is displayed on the display screen 43a. Note that the frame numbers of the reference image and the current image displayed on the display screen 43a when the acquisition date is changed are the same as those before the change of the acquisition date. Note that the change display process of the analysis result by the operator's touch operation on the display blocks of "reference data date" and / or "current data date" is controlled by the image analysis unit 42 based on an instruction signal corresponding to the operator's touch operation.

[0124] In addition, in the display screen 43a, an "Adjust" button and a "Cancel Adjustment" button are displayed below the display blocks of "Reference Data Date" and "Current Data Date", respectively. Note that the "Adjust" button and the "Cancel Adjustment" button are user interfaces that can be accessed (pressed) by the operator's touch operation on the display screen 43a.

[0125] When the operator presses the "Execute Adjustment" button on the display screen 43a, the amount of deviation between the reference image data and the original image data, i.e., the amount of deviation (delay or advance) in the timing at which the car 21 starts moving, is calculated. This calculation process is performed by the image analysis unit 42 of the mobile terminal device 2 using acceleration data included in the drive information of the car 21 at the time the reference image data is acquired and acceleration data included in the drive information of the car 21 at the time the original image data is acquired.

[0126] The calculated deviation amount is then displayed on the indicator below the "Perform adjustment" button and the "Cancel adjustment" button ("Optimal time adjustment amount: +0.495 seconds" in the figure). At this time, the extraction time of the image data is also displayed within the indicator ("Extraction time range: 20.0 seconds" in the figure). The extraction time of the image data essentially corresponds to the running time of the elevator 21 during maintenance inspection, and is calculated from the frame rate of the camera 4 and the number of frames of the reference image data, and this calculation process is also controlled by the image analysis unit 42. Furthermore, when the operator presses the "Cancel adjustment" button on the display screen 43a, the numerical values ​​of the optimal time adjustment amount and the extraction time range that were displayed within the indicator disappear.

[0127] The display mode of the image analysis results of the detection of a foreign object in the elevator shaft 20 is not limited to the mode shown in Fig. 10. Some of the various display items (user interfaces) shown on the display screen 43a in Fig. 10 may be appropriately deleted, or other user interfaces may be appropriately added, as necessary. For example, in the display mode shown in Fig. 10, some operators may find it easier to use if only the bare minimum of functions are displayed, so display items such as the playback times of the reference image and the current image, acceleration data, optimal time adjustment amount, and extraction time range may be deleted.

[0128] Also, for example, when the noise of the acceleration sensor is large and the reference image data (video) and the current image data (image) are out of sync by a frame, a new user interface may be provided that allows easy manual fine adjustment of the playback frame. In this case, it is preferable that the user interface allows manual fine adjustment of the playback frame more easily than the display block of the playback time of the current image and the seek bar shown in Fig. 10. For example, it is preferable to provide a user interface that allows adjustment of the frame number of the current image in units of ±1, instead of the playback time.

[0129] Furthermore, information indicating the extent to which the differential regions are included in the entire differential image may be displayed for the operator to understand before the differential image is displayed. In this case, for example, the total area of ​​the differential regions included in the entire differential image and / or the ratio of the differential regions to the entire differential image may be calculated and displayed. Note that such information regarding the differential regions (total area, ratio) is calculated by the image analysis unit 42 of the mobile terminal device 2.

[0130] (2) Calculation method for the amount of deviation between reference image data and original image data (optimum time adjustment amount) Here, a calculation method of the deviation amount (optimum time adjustment amount) between the reference image data and the original image data displayed on the indicator in the display screen 43a of FIG. 10 will be described. FIG. 11A is a diagram showing an example of a signal waveform of vertical acceleration data at the start of the car 21 movement obtained when the original image data is acquired, where the vertical axis is acceleration and the horizontal axis is time. FIG. 11B is a diagram showing an example of a signal waveform of vertical acceleration data at the start of the car 21 movement obtained when the reference image data is acquired, where the vertical axis is acceleration and the horizontal axis is time. Also, FIG. 11C shows the change characteristic of the correlation coefficient between the signal waveform of the acceleration data shown in FIG. 11A and the signal waveform of the acceleration data shown in FIG. 11B, which is calculated using a cross-correlation function in order to estimate the deviation amount between them. In the change characteristic of the correlation coefficient shown in FIG. 11C, the vertical axis is the correlation coefficient and the horizontal axis is the lag amount. The lag amount shown in FIG. 11C is the number of data points indicating the delay amount of the signal waveform of the acceleration data shown in FIG. 11A (original image data) relative to the signal waveform of the acceleration data shown in FIG. 11B (reference image data).

[0131] In the example shown in Fig. 11C, the correlation coefficient reaches its maximum value at a lag of +99 points, so it can be seen that there is a lag of +99 points between the signal waveform of the acceleration data shown in Fig. 11A and the signal waveform of the acceleration data shown in Fig. 11B. That is, in this example, the signal waveform of the acceleration data shown in Fig. 11A (original image data) is delayed by a delay amount equivalent to a lag of +99 points relative to the signal waveform of the acceleration data shown in Fig. 11B (reference image data).

[0132] In this example, since the sampling rate of the acceleration sensor (drive monitoring unit 3) is 200 Hz, the lag amount of +99 points corresponds to a delay amount of +0.495 seconds (=99 data÷200 Hz). Therefore, this delay time is displayed as the optimal time adjustment amount displayed on the indicator in the display screen 43a of FIG. 10. Also, in this example, since the frame rate of the camera 4 is 30 fps, the number of frames equivalent to a delay amount of 0.495 seconds is 14.85 frames (=0.495 seconds×30 fps). In other words, the original image data is delayed by about 15 frames from the reference image data.

[0133] In addition, the 1.000 seconds displayed in the display block of the playback time of the current image on the display screen 43a shown in Figure 10 is the value obtained by adding the delay amount of 0.495 seconds calculated as described above to 0.505 seconds displayed in the display block of the playback time of the reference image.

[0134] (3) Example of dimensional measurement of equipment in elevator 12 is a diagram showing an example of the image analysis result (inspection result) displayed on the display unit 43 of the mobile terminal device 2 when the maintenance system 10 measures the size of the engagement allowance (gap 63 of the lock function unit 60) of the door lock drive mechanism on each floor. In this maintenance inspection, the car 21 is made to travel in a drive pattern in which it is raised from the first floor to the third floor and stopped at each floor, while the camera 4 photographs the state inside the elevator shaft 20. Note that, in measuring the size of the gap 63 of the lock function unit 60, the car 21 may be made to travel in a drive pattern in which it is lowered from the third floor to the first floor and stopped at each floor, while the camera 4 photographs the state inside the elevator shaft 20.

[0135] 12, an example is described in which the current image extraction unit 32 in the maintenance device 1 extracts current image data from the original image data using the first extraction method shown in FIG. 5A. Furthermore, this example describes an example in which the drive monitoring unit 3 includes not only an acceleration sensor but also an air pressure sensor, and the detection signals thereof are acquired as drive information of the car 21. That is, this example describes an example in which not only information on the timing of the start and stop of the car 21 obtained from the detection signal of the acceleration sensor, but also position (height) information of the car 21 obtained from the detection signal of the air pressure sensor is acquired as drive information of the car 21.

[0136] 12, an overall image of the lock function part 60 of the door lock drive mechanism for the second floor and an enlarged image showing the area around the gap 63 of the lock function part 60 are displayed side by side. Note that, in accordance with the explanation of FIG. 7, the overall image and the enlarged image of the lock function part 60 are shown in schematic diagrams rather than as they are actually obtained.

[0137] In addition, in the display screen 43a, the left area in the lower area of ​​the overall image and the enlarged image of the lock function unit 60 displays the driving information (time change characteristics of acceleration data and air pressure data) of the car 21 acquired during maintenance inspection. Note that the air pressure decreases as the car 21 rises, so in the air pressure data, the air pressure is highest on the first floor and lowest on the third floor. In addition, on the display screen 43a, a vertical bar displayed overlapping the time change characteristics of the acceleration data and the air pressure data indicates the floor number of the lock function unit 60 currently displayed in the overall image and the enlarged image. In the example shown in FIG. 12, in the time change characteristics of the acceleration data and the air pressure data, a vertical bar is displayed during the stop time on the second floor, so the image of the lock function unit 60 on the second floor is displayed in the overall image and the enlarged image.

[0138] Additionally, within the display screen 43a, various display (selection) blocks for the "Image Data Selection" item are displayed in the right-hand area within the lower area of ​​the entire image and the enlarged image. Specifically, the acquisition date of the current image data ("Data Acquisition Date" in the figure), the floor number ("Floor" in the figure), and a "Select" button are displayed. The display (selection) blocks for "Data Acquisition Date" and "Floor" and the "Select" button are user interfaces that can be accessed by the operator's touch operation on the display screen 43a.

[0139] When displaying an image of the lock function unit 60 for a floor desired by the operator, the operator first selects the acquisition date of the desired current image data and the desired floor number on the display blocks of "Data Acquisition Date" and "Floor" by touch operation. Next, when the operator presses the "Select" button, the entire image and enlarged image of the lock function unit 60 for the selected data acquisition date and floor number are displayed on the display screen 43a. At this time, the display position of the vertical bar on the time change characteristics of the acceleration data and the atmospheric pressure data automatically transitions to the stop time corresponding to the selected floor number.

[0140] Furthermore, in the display screen 43a, an "Adjust" button is displayed in the area below the "Select" button as an "Enlargement adjustment area adjustment" item. The "Adjust" button is a user interface that can be accessed (pressed) by an operator's touch operation on the display screen 43a. When the operator presses the "Adjust" button on the display screen 43a, the enlargement area can be moved by the operator's touch operation on the enlarged image.

[0141] Furthermore, in the display screen 43a, an indicator showing the size of the gap 63 ("Gap size: 2.9 mm" in the figure) and a "Gap measurement" button are displayed in the lower area of ​​the "Enlargement adjustment area adjustment" item. The "Gap measurement" button is a user interface that can be accessed (pressed) by the operator's touch operation on the display screen 43a.

[0142] When the operator presses the "Gap Measurement" button on the display screen 43a, the width of the portion corresponding to the gap 63 is automatically calculated by the image analysis unit 42, and the calculated gap dimension is displayed on the indicator. Then, the operator can quickly determine whether the dimension of the gap 63 is within a specified value range by looking at the dimension of the gap 63 displayed on the indicator.

[0143] The display mode of the image analysis result of the gap 63 of the lock function unit 60 is not limited to the mode shown in FIG. 12. Some of the various display items (user interfaces) shown on the display screen 43a of FIG. 12 may be appropriately deleted as necessary, or other user interfaces may be appropriately added. For example, the screen may be configured to display the entire image of the floor, an enlarged image, and gap dimensions all at once without displaying graphs of air pressure data, acceleration data, etc. Also, for example, a user interface may be added to enable a saving function for saving dimensional data obtained by measurement, a function for the operator to leave comments on the measurement results, etc.

[0144] In addition, in the above-mentioned measurement of the dimensions of various devices and parts in the elevator shaft 20, it is assumed that video data is acquired by the camera 4, but the present invention is not limited to this. For example, after the car 21 stops at each floor, one still image may be taken, and the still image data may be analyzed to measure the dimensions of various devices and parts in the elevator shaft 20. In the maintenance system 10 of this embodiment, since it is possible to take an image of the elevator shaft 20 in cooperation with the driving information (position information, etc.) of the car 21, such a measurement method can be easily realized. In addition, in this case, the capacity of the image data to be analyzed is smaller than when video data is acquired, so the pressure on the data capacity is reduced, and the processing load and memory load can be reduced.

[0145] 12, an example has been described in which the gap 63 of the lock function unit 60 is measured as a dimensional measurement of various devices and parts in the hoistway 20, but the present invention is not limited to this. In the maintenance system 10 of this embodiment, by photographing various other device locations and parts in the hoistway 20 in the same manner as in the above embodiment, it is also possible to measure the dimensions of these, and it becomes possible to inspect and diagnose the state and dimensions of each device and part on the display screen 43a.

[0146] [Various effects] As described above, in the maintenance system 10 of this embodiment, the current image data and the reference image data are acquired in cooperation with the driving information of the car 21. Therefore, in the maintenance system 10 of this embodiment, the number of frames of the current image data (video) to be compared by the image analysis unit 42 can be made the same as the number of frames of the reference image data (video). As a result, for example, in detecting a foreign object in the elevator shaft 20, it is possible to precisely extract only the image of the foreign object 50 by simply calculating the difference image between the current image and the reference image having the same frame number.

[0147] Therefore, when foreign objects in the elevator shaft 20 are detected by the maintenance system 10 of this embodiment, maintenance inspection can be performed with high accuracy and with a simpler configuration. Furthermore, since the positioning of the shooting points of the current image and the reference image can be performed based on the frame number, for example, it is not affected by deterioration over time, and maintenance costs are not required. In other words, the maintenance system 10 of this embodiment allows maintenance inspection to be performed at low cost.

[0148] Incidentally, in the above-mentioned detection of a foreign object in the elevator shaft 20, it is necessary to match the shooting point of the current image with that of the reference image with high accuracy, but according to verification by the present inventors, it has been found that the shooting point of the camera 4 needs to be controlled with high accuracy of less than a few centimeters (for example, less than 3 cm). Here, an example of the verification will be described. However, in the drawings of the verification example shown below, the analysis results actually obtained in the verification experiment are not shown as they are, but are shown as schematic diagrams to further emphasize the knowledge obtained.

[0149] 13A, 13B, and 13C are schematic diagrams of a reference image, a current image, and a difference image obtained when foreign object detection is performed (example) in the same manner as the foreign object detection in the hoistway 20 described in Fig. 10. Since these images are acquired by the maintenance system 10 of this embodiment, the alignment of the measurement points in the current image and the reference image is performed with high precision, and only foreign object 50 (triangle mark in the figure) is displayed in the difference image.

[0150] On the other hand, Figures 14A, 14B, and 14C respectively show the reference image, current image, and difference image shown in Figures 13A, 13B, and 13C, in which the current image is shifted by one frame (comparative example). In this case, a different difference region 51 (an arc-shaped region in the figure) is detected in addition to the foreign object 50 in the difference image. That is, in this case, not only the foreign object 50 but also the difference region 51 is erroneously detected as a foreign object.

[0151] If the frame rate of the camera 4 is 30 fps for a typical elevator speed (45 to 105 m / min), the elevator will move 1.7 cm to 5.8 cm during one frame. For this reason, in foreign object detection, even if the shooting point of the current image and the shooting point of the reference image differ by only a few centimeters, a change in a non-abnormal part will be detected as a foreign object (difference). The cause of the above-mentioned false detection is due to a slight deviation in the coordinates of each device captured within the shooting range, and if the false detection area increases in this way, there is a possibility that the operator will overlook a foreign object that should be found. It is difficult to completely eliminate the causes of the above-mentioned false detection, but in order to reduce their influence as much as possible, it is possible to apply an algorithm that removes them using morphological processing, etc.

[0152] Therefore, from the above verification experiment, it was found that in order to capture a small foreign object or change of about several pixels with good sensitivity by differential detection, it is necessary to align the shooting point of the reference image and that of the current image as much as possible, and it is also necessary to suppress the error of one frame. On the other hand, in the maintenance system 10 of the above-mentioned embodiment, it is possible to align the shooting points of the current image and the reference image with high accuracy, and it is clear from the verification results of the difference images shown in Figures 13C and 14C that the above problem can be solved.

[0153] Another possible solution to the above problem is to increase the sensitivity of the difference detection (for example, the frame rate of the camera 4, the image resolution, the number of pixels, etc.) in order to accurately detect small foreign objects in the elevator 20 or minute positional changes of equipment to within a few pixels. However, if the sensitivity of the difference detection is too high, there is a disadvantage that even non-abnormal locations may be erroneously detected as differences due to slight differences in the shooting points. In contrast, the maintenance system 10 of the above-mentioned embodiment can, in principle, align the shooting points of the current image and the reference image with high accuracy without increasing the sensitivity of the difference detection, thereby eliminating such disadvantages.

[0154] Furthermore, even when measuring the dimensions of various devices and parts in the elevator shaft 20 from image data captured by the camera 4, if the capturing point changes, the measurement results will have a large error. Specifically, since many parts and devices are exposed in the elevator shaft 20, other parts are likely to be captured within the angle of view of the camera 4. Therefore, if image processing such as edge extraction processing is performed on the entire captured image, erroneous detection will occur, in which edges of not only the maintenance inspection target but also other parts and devices are extracted.

[0155] One possible method for solving this problem is to extract from the entire image an image portion that includes only the parts to be inspected, so that parts and equipment other than those to be inspected are not included in the entire image, and then perform image analysis. However, even with this method, if the shooting point varies slightly, even by a few centimeters, due to variations in the shooting point that occur for each maintenance inspection, parts and equipment other than those to be inspected will appear in the extracted image.

[0156] In contrast, in the maintenance system 10 of this embodiment, the current image data is acquired in cooperation with the driving information of the car 21, so that the maintenance inspection target can be photographed at the same photographing point (photographing position) with good reproducibility for each maintenance inspection. That is, in the maintenance system 10 of this embodiment, the maintenance inspection target and the camera 4 can be photographed with the same positional relationship for each maintenance inspection, and the variation in the photographing point for each frame can be made very small. Therefore, in the maintenance system 10 of this embodiment, the measurement error of the dimensions of various devices and parts in the hoistway 20 can also be made very small, and highly accurate dimensional measurement can be performed. That is, in this embodiment, it is possible to detect the appearance changes of various devices and parts in the hoistway 20 with high accuracy. In addition, in this embodiment, the maintenance inspection target can be photographed at the same photographing point (photographing position) with good reproducibility for each maintenance inspection, so that the appearance changes of the maintenance inspection target can be easily confirmed even by visual inspection.

[0157] As described above, in the maintenance system 10 of this embodiment, in principle, the number of frames of the current image data (video) to be compared by the image analysis unit 42 can be made the same as the number of frames of the reference image data (video). In addition, in the maintenance system 10 of this embodiment, the current image data is acquired in conjunction with the driving information of the car 21, so that the variation in the shooting points of each frame for each maintenance inspection can be made very small. However, for example, if the noise of the acceleration sensor included in the driving monitoring unit 3 becomes large, it is possible that the extraction timing of the current image data may deviate from the desired timing. That is, if the noise of the acceleration sensor during maintenance inspection becomes large, for example, the number of frames of the current image data may differ from the number of frames of the reference image data, or the variation in the shooting points for each maintenance inspection may become somewhat large.

[0158] However, in the maintenance system 10 of this embodiment, a function is provided that allows the operator to manually change the current image to be analyzed on the display screen 43a of the display unit 43 of the mobile terminal device 2. Furthermore, in the maintenance system 10 of this embodiment, the image analysis unit 42 of the mobile terminal device 2 also has a function for calculating the amount of delay between the original image data and the reference image data based on the driving information (detection signal of the acceleration sensor) of the car 21. Therefore, even if there is an effect of noise from the acceleration sensor described above, by utilizing these functions, this effect can be easily eliminated and highly accurate maintenance inspection can be performed.

[0159] [Various variations] Although the maintenance system 10 and the maintenance device 1 according to one embodiment of the present invention have been described above, the present invention is not limited to these. For example, the following various modified examples can be adopted.

[0160] In the maintenance system 10 of the above embodiment, an example has been described in which the extraction process is started based on a start trigger and ended based on an end trigger in the extraction operation of the current image data and the reference image data (see Figs. 5A and 5B), but the present invention is not limited to this. For example, in the extraction operation of the current image data and the reference image data, the extraction process is started based on a start trigger, but the end timing of the extraction process may be the same as the timing of the end of shooting. In other words, it is not necessary to provide a configuration for ending the extraction process based on an end trigger. In this case, too, the shooting point (shooting position) of the first current image of the current image data and the shooting point of the first reference image of the reference image data can be aligned with high accuracy, so that the same effect as in the above embodiment can be obtained. In addition, in this case, the extraction process of the current image data is also simplified, so the processing load on the information processing device 6 is also reduced.

[0161] In addition, in the maintenance inspection such as detection of foreign objects in the elevator shaft 20 by the maintenance system 10 of the above embodiment, an example has been described in which the extraction period of the current image data and the reference image data is the period from when the car 21 starts moving to when it stops (see Figs. 5A and 5B). However, the present invention is not limited to this, and the extraction period of the current image data and the reference image data is appropriately changed depending on the maintenance inspection target, the drive pattern of the car 21, etc.

[0162] For example, in maintenance inspection such as detection of foreign objects in the elevator 20, the extraction period for the current image data and the reference image data may be the period during which the car 21 is operating at a constant speed. In this case, the drive section detection unit 31 detects the timing when the car 21 starts to move, the acceleration increases once, and then returns to 0, and outputs a start trigger, and thereafter detects the timing when the acceleration of the car 21 starts to decrease, and outputs an end trigger. In this case, the capacity of the current image data and the reference image data to be analyzed can be reduced compared to the above embodiment.

[0163] Furthermore, with regard to the photographing operation by the camera 4 in the maintenance system 10 of the embodiment, the following various aspects can be adopted, taking into consideration, for example, the object of maintenance and inspection, the volume of image data, and inspection accuracy.

[0164] For example, when video is captured by the camera 4, the frame rate of the video data does not necessarily have to be the maximum rate of the camera, and can be set appropriately according to, for example, the object of maintenance and inspection.

[0165] Also, the photographing operation by the camera 4 may be configured to continuously photograph still images at a predetermined interval while the car 21 is traveling. For example, when the maintenance inspection target is detection of foreign objects, fallen objects, etc. in the elevator shaft 20, a still image may be photographed each time the car 21 travels to the 1st floor, 1.5th floor, 2nd floor, 2.5th floor, etc., that is, once for every half floor. In this case, the photographing timing can be determined by the relationship between the distance between each floor of the elevator and the travel speed of the car 21. For example, if the floors are 5 m apart and the elevator (car 21) travels at a speed of 60 m per minute, then photographing once for every half floor can be performed by photographing once every 2.5 seconds. In this case, the amount of image data can be reduced compared to video shooting.

[0166] Furthermore, the elevator travel speed during the photographing operation (maintenance inspection) may be a general high-speed operation so that the entire inside of the hoistway 20 can be photographed in a short time, but may also be a low-speed operation used during elevator maintenance inspection in order to photograph a cleaner and clearer image. In this case, the accuracy of the maintenance inspection can be improved. Also, the elevator travel speed during maintenance inspection can be appropriately set according to the object of maintenance inspection. Therefore, depending on the object of maintenance inspection, an image of the hoistway 20 may be photographed with the car 21 completely stopped. For example, when an elevator user drops something, the dropped item often falls into the pit (bottom) of the hoistway 20 or gets stuck under the door. When such an event is the object of maintenance inspection, the car 21 may be moved to a location appropriate for the event and stopped, and then photographed, so that the area where the dropped items are likely to be dropped can be photographed with high resolution. In this case, since the car 21 is in a stopped state, the exposure time can be set long, and a clear image can be obtained even in the dark hoistway 20, thereby improving the accuracy of the maintenance inspection. In this case, the volume of the photographed image data can also be reduced.

[0167] In the maintenance system 10 of the above embodiment, a configuration example (see FIG. 4) in which the image processing unit 30 (the drive section detection unit 31 and the current image extraction unit 32) is provided in the information processing device 6 arranged on the building side has been described, but the present invention is not limited to this. For example, the image processing unit 30 (the drive section detection unit 31 and the current image extraction unit 32) may be provided in the mobile terminal device 2.

[0168] In addition, the above-mentioned embodiment examples are detailed and specific descriptions of the configuration of the device in order to easily explain the present invention, and are not necessarily limited to those having all of the described configurations. In addition, the position, size, shape, range, etc. of each component shown in the drawings, etc. may not represent the actual position, size, shape, range, etc. in order to facilitate understanding of the invention. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings, etc. Furthermore, the present invention may take on various other application examples and modified examples as long as they do not deviate from the gist of the present invention described in the claims. [Explanation of symbols]

[0169] 1...maintenance device, 2...mobile terminal device, 3...drive monitoring unit, 3a...sensor box, 3b...drive control system, 4...camera, 6...information processing unit, 10...maintenance system, 20...hoistway, 21...car, 30...image processing unit, 31...drive section detection unit, 32...current image extraction unit, 34, 45...storage unit, 34a, 45b...current drive information storage unit, 34b...original image storage unit, 42...image analysis unit, 43...display unit, 43a...display screen, 44...input unit, 45a...current image storage unit, 45c...reference image storage unit, 45d...reference drive information storage unit

Claims

1. A maintenance device that acquires image data of an elevator shaft photographed by an imaging device during elevator maintenance and inspection, an image processing unit that extracts analysis image data to be analyzed in maintenance inspection from the image data of the elevator shaft; The image processing unit acquires driving information regarding the running state of the elevator car during the maintenance inspection, and extracts the analysis image data from the image data of the elevator shaft based on the driving information. A maintenance device characterized by:

2. The image processing unit an extraction trigger signal generating unit that generates a start trigger signal indicating a start of extraction of the analysis image data based on the drive information and outputs the start trigger signal; an image extraction unit that starts extracting the image data for analysis based on the start trigger signal output from the extraction trigger signal generation unit; 2. The maintenance device according to claim 1.

3. the extraction trigger signal generation unit generates an end trigger signal indicating an end of extraction of the analysis image data, and outputs the end trigger signal; The image extraction unit terminates the extraction of the image data for analysis based on the termination trigger signal output from the extraction trigger signal generation unit.

3. The maintenance device according to claim 2.

4. The image data of the elevator shaft is video data, The timing to start extracting the analysis image data is after the timing to start capturing the image data of the elevator shaft, and the timing to end extracting the analysis image data is before the timing to end capturing the image data of the elevator shaft.

2. The maintenance device according to claim 1.

5. The image data of the elevator shaft is video data, The timing at which the extraction of the analysis image data starts is the same as the timing at which the image capture of the elevator shaft image data starts, and the timing at which the extraction of the analysis image data ends is the same as the timing at which the image capture of the elevator shaft image data ends.

2. The maintenance device according to claim 1.

6. In a maintenance system having a photographing unit that photographs image data of an elevator shaft during maintenance and inspection, an image processing unit that extracts analysis image data to be analyzed during maintenance inspection from the image data of the elevator shaft; an image analysis unit that analyzes a predetermined maintenance and inspection target based on the analysis image data extracted by the image processing unit; A display unit that displays the results of the analysis performed by the image analysis unit, The image processing unit acquires driving information regarding the running state of the elevator car during the maintenance inspection, and extracts the analysis image data from the image data of the elevator shaft based on the driving information. A maintenance system characterized by:

7. the image data captured by the imaging unit is video data, The frame rate of the imaging unit is lower than the sampling rate of the driving information.

7. The maintenance system according to claim 6.

8. a drive information detection unit having a sensor for detecting the drive information; 8. The maintenance system according to claim 7.

9. The image processing unit acquires, as the drive information, control information output from a drive control system that outputs control information including a travel command to the car.

7. The maintenance system according to claim 6.

10. The image processing unit, the image analysis unit, and the display unit are configured separately, The image data for analysis extracted by the image processing unit is transmitted to the image analysis unit by communication.

7. The maintenance system according to claim 6.

11. further comprising an input unit operable by an operator and outputting an operation signal corresponding to the operation by the operator; The display content of the analysis result displayed on the display unit is changed based on the operation signal corresponding to the content specified by the operator's operation on the input unit.

7. The maintenance system according to claim 6.

12. A maintenance program to be implemented and executed in a maintenance device that acquires image data of a hoistway captured by a photographing device during elevator maintenance and inspection, An image processing unit included in the maintenance device performs a process of acquiring drive information regarding a running state of the elevator car during the maintenance inspection; and a process by the image processing unit to extract, based on the drive information, image data for analysis to be analyzed in maintenance inspection from the image data of the elevator shaft. A maintenance program characterized by:

Citation Information

Patent Citations

  • Image analytics for elevator maintenance

    EP3409629A1