Method and mounting device for installation work in elevator hoistway

Abstract

This mounting device comprises: a main body suspended from a raising / lowering module installed in the elevator hoistway; at least two cameras arranged in the main body; an inertial measurement unit configured to measure the roll and pitch of the mounting device; and a pose calculation module for calculating the position and orientation of the mounting device. The pose calculation module is configured to: cause each of the cameras to capture an image containing at least one of two reference lines arranged inside the elevator hoistway; estimate the coordinates of the two reference lines in a device coordinate system from the image; and calculate, in an elevator hoistway coordinate system, five degrees of freedom (x, y, roll, pitch, and yaw) including the position (x, y) and yaw angle on the device on the basis of the coordinates of the two reference lines in the device coordinate system and the image.

Description

Method and mounting device for installation work in an elevator shaft

[0001] The present disclosure relates to a method and a mounting device for installation work in an elevator shaft.

[0002] It can be dangerous for humans to perform installation work inside an elevator shaft where an elevator car is moving. Therefore, methods are being studied to perform installation work inside the elevator shaft using automated robots. For example, wires (reference lines) are placed across the upper and lower parts of the elevator shaft to install the elevator car. Then, an automated robot installs the necessary components for the elevator car installation in the shaft based on the position of the wires. For the accurate installation of the elevator car, the automated robot needs to accurately identify the position of the wires. However, it is difficult to accurately identify the position of the wires due to vertical patterns formed on the side walls of the elevator shaft. Korean Published Patent No. 10-2018-0128905 discloses a technology for determining the position of an automated robot from a reference line inside the elevator shaft using a sensor mounted on a robot arm. However, according to Korean Published Patent No. 10-2018-0128905, since the robot arm moves to determine the position of the reference line, the center of gravity of the automated robot changes depending on the movement of the robot arm, making it difficult to accurately recognize the position of the robot and potentially reducing the precision of the operation.

[0003] The present disclosure relates to a method and a mounting device for installation work in an elevator shaft, and aims to improve the precision of the work by accurately recognizing the position and orientation of a device working within the shaft. In addition, to accurately recognize the position of a device working within the shaft, a method is provided for accurately recognizing a reference line that serves as a reference for position determination.

[0004] In one aspect of the present disclosure, a mounting device is provided for measuring the relative position and orientation of an installation operation in an elevator shaft. The mounting device comprises: a main body suspended from a lifting module installed in the elevator shaft; at least two cameras disposed on the main body; an inertial measuring device configured to measure the roll and pitch of the mounting device; and a pose calculation module for calculating the position and orientation of the mounting device. The pose calculation module is configured such that each of the cameras captures at least one of two reference lines disposed in the elevator shaft in an image, estimates the coordinates of the two reference lines in a device coordinate system from the image, and obtains five degrees of freedom (x, y, roll, pitch, yaw) including a position (x, y) and a yaw angle (yaw) on the device in a shaft coordinate system based on the coordinates of the two reference lines in the device coordinate system and the image.

[0005] In one embodiment, the mounting device may further include a distance sensor configured to measure a height (z) from the elevator shaft. The mounting device may further include a robot module that performs installation work, such as drilling or anchor insertion, within the elevator shaft using the five degrees of freedom and the height (z).

[0006] In one embodiment, at least some of the cameras may be positioned so that one image includes two reference lines placed in the elevator shaft.

[0007] In one embodiment, estimating the yaw angle may include performing the azimuth line intersection of at least two cameras arranged on one side.

[0008] In one embodiment, the yaw angle can be estimated based on the condition that the x-coordinates of the two reference lines in the elevator shaft coordinate system are the same, and the condition that the pose calculation module has already secured the distance between the two reference lines.

[0009] In one embodiment, the pose calculation module estimates the image display width from the actual thickness of the reference line and the distance from the camera, and based on this, can estimate the coordinates of the two reference lines in the device coordinate system.

[0010] In one embodiment, the mounting device further includes a light source, uses the light source to generate reflected light of the reference line, and uses the brightness of the reflected light to estimate the coordinates of the two reference lines in the device coordinate system.

[0011] In one embodiment, the at least two cameras include first to fourth cameras, and the first and third cameras may be configured to capture the first reference line among the two reference lines, and the second and fourth cameras may be configured to capture the second reference line among the two reference lines to obtain an image.

[0012] By accurately identifying and correcting the position and orientation of the elevator installation device, the stability of the installation device and the accuracy and precision of the operation can be ensured. By accurately positioning the elevator installation device, the stability of the operation within the elevator shaft can be improved.

[0013] Increased work accuracy and precision lead to higher work efficiency and energy savings. In other words, it offers ESG (Environment / Social / Governance) benefits.

[0014] FIG. 1 is a configuration diagram of a system according to one embodiment of the present disclosure.

[0015] FIG. 2 is an exemplary block diagram of a mounting device according to one embodiment of the present disclosure.

[0016] FIG. 3 is a flowchart of a baseline recognition method according to one embodiment of the present disclosure.

[0017] FIG. 4 is a flowchart of the operation execution according to one embodiment of the present disclosure.

[0018] FIGS. 5a to 5c are examples of image photographs of an elevator shaft according to one embodiment of the present disclosure.

[0019] FIGS. 6a to 6k are exemplary photographs of a reference line detection method according to one embodiment of the present disclosure being performed on an elevator shaft image.

[0020] Embodiments of the present disclosure are described below with reference to the attached drawings so that those skilled in the art can easily implement them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein.

[0021] In addition, to clearly explain the present disclosure in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification have been given similar reference numerals.

[0022] Throughout the specification, when a part is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0023] It should be understood that the technology described in this disclosure is not intended to be limited to specific embodiments and includes various modifications, equivalents, and / or alternatives to the embodiments of this disclosure.

[0024] As used in this disclosure, the expression “configured to” may be replaced, depending on the context, with, for example, “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of.” The term “configured to” may not necessarily mean only that which is “specifically designed to” in hardware. Instead, in some situations, the expression “device configured to” may mean that the device is “capable of” together with other devices or components.

[0025] The prior art described in this disclosure is incorporated herein by reference in its entirety, and it will be understood that a person skilled in the art may apply the contents described in the prior art to the parts briefly described in this disclosure.

[0026] Within the context of the present disclosure, the two terms “system” and “device” refer to the same concept and may be used interchangeably.

[0027] FIG. 1 is a configuration diagram of a system according to one embodiment of the present disclosure. Referring to FIG. 1, the system includes a lifting module (10), a mounting device (20), and a control module (30). The system may include a configuration in which the lifting module (10), the mounting device (20), and the control module (30) are interconnected to calculate the position and posture of the installation robot in the elevator shaft coordinate system (camera, IMU, ToF data fusion) and automatically perform drilling and anchoring operations using the calculated results. The mounting device (20) may include a main body (22), a robot module (24), a pose calculation module (25), and a work control module (26).

[0028] In one embodiment, the lifting module (10) moves the chassis up and down to a target height within the hoistway using a hoisting machine and a transfer system, and can provide the height (z reference and Δz) (distance from the top of the hoistway to the coordinate system of the mounted device (20)) of the coordinate system of the mounted device through a transfer amount sensor such as an encoder to the pose calculation module (24) or the work control module (26). For example, the z value is calculated based on the height of the elevator door opening of the lowest or highest floor and is used together with the signals of the ToF / distance sensor and the up and down transfer amount measuring device.

[0029] In one embodiment, the mounting device (20) is a work platform suspended from a lifting module and includes a camera for piano wire recognition, an IMU, and a distance sensor (e.g., ToF), and can simultaneously observe two reference lines (piano wires) installed parallel within the elevator shaft. The mounting device (20) can obtain reference lines through camera images, roll and pitch of the mounting device (20) through the IMU, and z information through ToF and an encoder. Based on this information, the pose calculation module (25) of the mounting device (20) estimates the relative position and orientation of the mounting device in the elevator shaft coordinate system and provides this to the work control module (26). The selected reference line (piano wire) detection result is used for setting anchor positions and generating robot work trajectories. The mounting device (20) includes actual construction tools such as drilling and anchor insertion, and processes rail brackets and holes for installing elevator doors at accurate positions so that they are aligned in a straight line.

[0030] In one embodiment, the mounting device (20) includes a main body (22), a robot module (24), a pose calculation module (25), and a work control module (26). The main body (22) is a work platform main body that moves up and down within the hoistway by means of a lifting module (10, rope / hoisting system) and provides a device coordinate system. The main body (22) may include a frame, a support (left / right expansion / contraction support), power / control wiring, an IMU / ToF, a control unit / mechanical unit, a camera, etc. The robot module (24) includes a work module that performs drilling / anchor insertion / processing within the hoistway. By converting the hoistway reference target point (hole / bracket position) provided by the main body (22) or the control module (30) into a robot reference trajectory, an approach→contact→processing→detachment sequence can be implemented.

[0031] In one embodiment, the pose calculation module (25) may function as a computational unit that includes the functions of image acquisition and image analysis. For example, the pose calculation module (25) acquires the pixel coordinates and orientations of two reference lines captured by a camera, estimates the coordinates of the two reference lines in the device coordinate system, and analytically calculates yaw and (x, y) using a parallel constraint of “x equals and y difference of two reference lines = gap D,” and then fuses the roll & pitch of the IMU and the z of the ToF / encoder to complete the 6 degrees of freedom based on the elevator shaft coordinate system. A detailed method for acquiring the coordinate system of the reference lines will be described later. The pose calculation module (25) may be integrated into the work control module (26) and embedded within the mounted device (20). The IMU is an inertial measurement device that measures acceleration and angular velocity using a 3-axis accelerometer + a 3-axis gyroscope (including a magnetometer if necessary) to estimate attitudes such as inclination (roll, pitch). ToF is a time-of-flight distance sensor / camera that can determine distance by emitting an IR / laser pulse and the time Δt for it to be reflected back. In this application, ToF can define / maintain an elevator shaft z=0 reference by obtaining the absolute distance to a reference plane (e.g., the bottom door floor) or Δz during inter-floor movement.

[0032] In one embodiment, the main body (22) may perform some functions of the pose calculation module (25), the work control module (26), or the control module (30). The pose calculation module (25), the work control module (26), or the control module (30) may perform functions by sharing them and may be physically separated or implemented as a single unit. In one embodiment, the work control module (26) controls the robot module using position and posture information provided by the pose calculation module (25), performs path planning and replanning along a target point (hole / bracket position, etc.), and controls the work sequence (approach → processing → exit).

[0033] In one embodiment, the control module (30) performs high-level work instructions and monitoring. That is, it is responsible for functions such as inputting work plans, setting quality standards, providing HMI, and aggregating and monitoring data, and manages the overall work flow by communicating with the work control module (26).

[0034] FIG. 2 is an exemplary block diagram of a mounting device according to one embodiment of the present disclosure. Referring to FIG. 2, the mounting device (200) is placed within an elevator shaft (100). The elevator shaft (100) is a space where elevator installation work is performed. Since building deformation occurs due to wind or other factors and the condition of the elevator shaft (100) is not ideal, in order to accurately place the anchor hole, it is necessary to obtain the coordinates of the elevator shaft (100) itself and the coordinates of the device (200) and use them to determine the exact location of the anchor hole. The elevator shaft coordinate system (110) is an absolute coordinate system that serves as a reference for the work. The reference of the elevator shaft coordinate system (110) can be set by the user. For example, it can be set by referring to actual measurement standards, such as the height of the lowest floor landing door opening. The reference line (120) may include, for example, a thin piano wire. For example, the piano wire may have a thickness of 0.5 mm to 1.6 mm. Reference lines (120) can be installed in two parallel lines within the elevator shaft at a constant interval D. Also, the thickness of the reference lines (120) is information known to the user and can be used later to obtain the exact coordinates of the reference lines (120). When the two wires are converted from the device coordinate system (210) to the elevator shaft coordinate system (110), the x values ​​must be the same and the difference in y values ​​must be D.

[0035] In one embodiment, the mounting device (200) includes a camera (220), an IMU, a transport system (not shown), etc. It is necessary to accurately know the x, y, z, roll, pitch, and yaw in the hoistway coordinate system (110) to perform automatic operations such as drilling. The mounting device (200) has a device coordinate system (210). The device coordinate system (210) is a coordinate system fixed at a point on the mounting device (200). By capturing an image with a camera and converting the device coordinate system (210) value of the wire estimated based on the image into the hoistway coordinate system and aligning it, the yaw and planar position (x, y) can be obtained. The camera (220) is a plurality of cameras placed on one side of the mounting device (200), and there may be at least two cameras. At least two cameras (220) can simultaneously capture a single reference line. The camera (220) can stably acquire the wire angle / coordinate even under conditions of obstruction, reflection, and tilt by securing multiple fields of view.

[0036] In one embodiment, the camera (220) includes first to fourth cameras (C1, C2, C3, C4), and the first and third cameras (C1, C3) can capture the first reference line (120a) among the two reference lines, and the second and fourth cameras (C2, C4) can capture the second reference line (120b) among the two reference lines to obtain an image. One camera can capture an image that includes one reference line. Alternatively, one camera can capture an image that includes two reference lines.

[0037] FIG. 3 is a flowchart of a baseline recognition method according to one embodiment of the present disclosure. The baseline recognition method is exemplary and may be performed by a pose calculation module (25) of a mounting device (200) shown in FIG. 2 or a control module (30) shown in FIG. 1. According to the method, a baseline (piano wire) crossing the side wall of an elevator shaft can be automatically detected from an image and its coordinates and angles can be calculated. The operation performed in the method according to the present disclosure may be performed by a pose calculation module (25) of a mounting device (200) or a control module (30) shown in FIG. 1.

[0038] In S305, at least two reference lines are placed across the upper and lower parts of the elevator shaft. The reference lines are wires installed vertically across the side walls of the shaft and are used as measurement references in subsequent steps. (The installation itself can be fixed or variable depending on the operating environment.) The thickness and spacing of the reference lines may be pre-stored in a mounted device or control module, or provided from an external source.

[0039] In S310, the mounted device uses cameras to capture images of the inside of the elevator shaft, including reference lines, to acquire images. The internal parameters of the cameras may be pre-corrected, and the cameras can be identified through filenames, metadata, etc. If necessary, images with lens distortion correction may be used. A single camera can capture two reference lines in a single frame. That is, each of the multiple cameras can capture two reference lines in a single image from different positions within a common time window.

[0040] In S315, both edges of each baseline are detected from the acquired image. Specifically, edge detection (e.g., Prewitt, Canny, etc.) can be performed after grayscale conversion and blurring for noise reduction so that the left and right boundaries of the baseline are represented as binary edges. The output of this step is two vertical edges corresponding to the baseline. In one embodiment, a directional filter that emphasizes the vertical component may be added.

[0041] In S320, a Hough transform is performed on the edge image. Accordingly, broken or jagged pixels can be integrated in the parameter space and aggregated into a single line candidate. For example, line detection is performed in a predetermined camera horizontal axis-based Hough angle window centered on the vertical direction, and a line detection score map is generated to represent the confidence of each line candidate. The angle window settings can be configured to include a tolerance for camera inclination (roll / tilt). In one embodiment, the range of the angle window may be [15°, 15°]. Accordingly, focus can be placed on baseline candidates. As a result of the Hough transform, an accumulation matrix H(ρ, θ) can be obtained. In the Hough transform, edge pixels are sent to the (ρ, θ) accumulation space, where ρ = xcosθ + ysinθ.

[0042] In S325, parallel line center emphasis correction is performed on both edges of the baseline using the Hough transform results. By searching the ρ-axis neighbor interval (interval configurable) in the accumulation matrix H, the score of the central location is amplified when left and right edges exist. More specifically, correction is performed to highlight the central straight line component located between them by utilizing the parallel pair pattern formed by the left and right edges of the baseline. Specifically, weights can be applied to increase the score of the central location when high values ​​are simultaneously observed in the left and right corresponding ranges (ρ-axis neighbor intervals) that are close to each other along the distance axis of the accumulation space. The weight profile can be designed to be largest at the center and decrease towards the outside, and the left and right corresponding ranges (ρ-axis neighbor intervals) can be determined based on the expected separation between two parallel edges (e.g., a range corresponding to the wire width in the image). As a result of this correction, a correction score map is obtained in which the central single-line candidate is emphasized. A correction score map with an emphasis on the central first-line candidate is obtained, which can be referred to as the correction accumulation matrix H_adj. For example, the score of the central ρ can be amplified by convolving a weighted kernel with respect to the ρ-axis or by using the correlation between left and right corresponding positions.

[0043] In one embodiment, the left-right corresponding range may be set to approximately 0.4 times or more and 0.8 times or less the expected display width of the wire in the image, and in another embodiment, it may be set to a fixed range of approximately 2 to 6 pixels. The weighting profile may be designed to be maximum at the center and monotonically decrease towards the outside (e.g., Gaussian or triangular (trapezoidal) shape). In addition, the left-right corresponding range may be automatically set based on the expected display width on the image calculated from the actual thickness of the wire and the distance from the camera. For example, based on the expected display width d_px, the corresponding range may be set to approximately (0.5 × d_px) ±20% to the left and right from the center, respectively.

[0044] In S330, Hough peaks are detected. For example, peaks are detected based on the result of parallel line center-weighted correction to obtain parameters of one or more candidate lines. Peaks are detected in the corrected score map (H_adj) to obtain parameters of one or more candidate lines (corresponding to position and direction). The number of peaks can typically be limited to one or two, and peaks that are close to each other can be combined into one by means of non-maximum suppression, etc.

[0045] In S335, a line is reconstructed in image space based on the parameters of the candidate line, and the pixel coordinates (top and bottom intersection points, etc.) and tilting angle of the reconstructed line are calculated. Reconstruction can be implemented using one or more of the following methods: a method using intersection with the screen boundary or a method restoring the longest line segment that coincides with the edge. Accordingly, the on-screen coordinates (e.g., top point, bottom point, center x-coordinate) and angle of the baseline are calculated. The coordinates and angle may be stored.

[0046] In one embodiment, detection accuracy can be improved by utilizing the reflected light characteristics of a baseline shown in an image. The baseline may include metal, and since metal has the property of reflecting light compared to the side walls of the elevator shaft (e.g., concrete), it may have higher luminance. Therefore, luminance scoring can be performed using the reflected light image of the baseline shown in the image. For example, among the parameters of one or more candidate lines from a captured image or a grayscale-converted image, the candidate with the highest luminance score can be selected as the baseline. For example, for each candidate line, the ambient light removal luminance score can be calculated by taking the maximum value of the center / left / right (±1 pixel, configurable) intensity profile in the normal direction of the line and subtracting the maximum value of the background (±5 pixels, configurable) profile.

[0047] In one embodiment, the score of the candidate having the highest score among the parameters of one or more candidate straight lines is compared with a predetermined standard, and based on the comparison result, said candidate may be selected as a baseline. For example, if there are multiple candidates, if the ratio of the corrected scores is greater than or equal to a preset value (e.g., 1.5), the candidate with the higher score may be selected as the baseline. In another embodiment, if the ratio of the corrected scores is less than or equal to a preset value (e.g., 1.5), the candidate with the higher luminance score may be selected as the baseline.

[0048] In one embodiment, in S335, a length filter can be used to select candidates having a length greater than a certain length. For example, peaks can be extracted from H_adj to restore line segments, but only line segments with a length greater than a preset rate can be kept as candidates.

[0049] In one embodiment, after S315, directional filtering that emphasizes the vertical component can be performed. Accordingly, the horizontal component can be removed and the baseline candidates can be narrowed further.

[0050] In one embodiment, after S325, baseline candidates can be further narrowed by removing information outside the region of interest through X-range masking. For example, if it falls outside the predefined range [x_min, x_max], H_adj can be masked and set to 0.

[0051] In one embodiment, a step of illuminating using a light source for luminance scoring may be further performed. This step may be performed by a mounting device (20).

[0052] According to the method of Fig. 3, the top point, bottom point, center x-coordinate (horizontal pixel coordinate), angle (tilting value), and image size of the baseline can be obtained from the image. These values ​​may be obtained from an image corrected for camera lens distortion. All of these values ​​are based on a coordinate system on the image.

[0053] Since the elevator shaft may differ from the design, working coordinates must be defined based on the actual location. As the alignment standard is the elevator shaft coordinate system, the hole positions of rail brackets and door anchors are defined relative to the elevator shaft (e.g., based on the actual height of the door opening). Because the design and actual conditions may differ, alignment with field standards is required. To achieve this, the positional orientation (x, y, z, roll (Φ), pitch (θ), yaw (ψ)) of the mounted device must be determined in the elevator shaft coordinate system. Roll (Φ) and pitch (θ) can be obtained from an IMU. Since the camera is fixed to the mounted device, the roll and pitch provided by the IMU can be prepared by correcting them relative to the camera. Z can be obtained through feed rate sensors such as ToF and / or encoders. Roll, pitch, and z can be expressed in the elevator shaft coordinate system. The camera angles of view, intrinsic parameters (e.g., focal length, principal point, skew, distortion coefficient), mounting direction offset (which direction the camera is facing relative to the x-axis of the mounted device coordinate system), camera position (camera coordinates in the mounted device coordinate system), and the height of the mounted device may be stored in advance.

[0054] In one embodiment, the azimuth angle (β) is calculated using the intrinsic parameters of the camera and pixel coordinates obtained according to the method of FIG. 3. The reference horizontal axis of the camera is set to 0°, and the horizontal direction angle corresponding to the center x-pixel of the detected reference line is defined as the azimuth angle (β).

[0055] At this time, a virtual straight line can be extended in the direction of the azimuth (β) obtained by adding the camera's installation offset. However, with only one camera, only the azimuth can be obtained, and the reference line coordinates (P1, P2) cannot be directly calculated. Therefore, at least two camera images of the same reference line captured at different locations are required. For example, if two cameras capture the same reference line, a straight line can be formed using the azimuth (including correction) obtained from each camera, and the intersection point of these straight lines can be calculated as the corresponding reference line coordinates (P1 or P2).

[0056] In another embodiment, P1 and P2 can be obtained through an intersection (triangulation) method in which two cameras each capture a reference line and the point where the two direction lines obtained from each camera intersect is defined as the wire coordinate. Since two cameras capture the reference line, there are two pixel coordinates obtained according to the method of FIG. 3 for each reference line. Using each pixel coordinate, two azimuth angles (β1, β2) are obtained for one reference line, and a straight line corresponding to each azimuth angle is generated to obtain the intersection point as P1 and P2. That is, the intersection point can be obtained using two azimuth angles per reference line.

[0057] In contrast, in the method according to one embodiment, since the reference lines are parallel and the distance between the reference lines is known, one intersection point (e.g., P1) can be obtained using two azimuths, and P2 can be obtained using one azimuth and the distance between the reference lines.

[0058] After determining P1 and P2, yaw(ψ) is estimated using them. When the coordinates of the reference lines in the device coordinate system are converted to the elevator shaft parallel coordinate system, the x-coordinates between the two reference lines must be identical, and the y-coordinates must differ by the distance between the reference lines. By using this as a condition for the equation, the unknown yaw value can be calculated.

[0059] Ideally, when the mounted device is precisely aligned with the hoistway, the line connecting P1 and P2 should point exactly up and down, that is, in the y-axis direction of the mounted device. In other words, the degree of misalignment between the line connecting P1 and P2 and the y-axis direction of the device is yaw(ψ). z can be measured using ToF and / or an encoder as described above. Accordingly, the position orientation (x, y, z, roll(Φ), pitch(θ), yaw(ψ)) of the mounted device can all be obtained.

[0060] In one embodiment, scaling can be performed so that the distance between P1 and P2 is equal to the distance between design baselines.

[0061] FIG. 4 is a flowchart of the operation execution according to one embodiment of the present disclosure.

[0062] Referring to FIG. 4, as described above, in S405, the current pose (x, y, z, Φ, θ, ψ) of the mounted device can be obtained using the values ​​calculated through S305 to S335.

[0063] In S410, in one embodiment, the work control module (26) can use the updated pose to convert the elevator reference target work point into coordinates that the robot can follow and generate a process trajectory. For example, the mounting device coordinate system can be converted to the elevator coordinate system. In one embodiment, the procedure for converting the mounting device coordinate system to the elevator coordinate system can be updated periodically.

[0064] In S415, according to one embodiment, the pose calculation module (25) updates the 6 degrees of freedom (x, y, z, Φ, θ, ψ) of the mounted device in real time. The operation control module can continuously monitor the deviation from the target pose and perform correction immediately if it exceeds a threshold value. For example, the 6 degrees of freedom (x, y, z, Φ, θ, ψ) are updated in real time throughout the operation, and the planar position error (Δx, Δy), height error (Δz), and pose error (ΔΦ, Δθ, Δψ) can be continuously monitored.

[0065] In S420, according to one embodiment, the work control module (26) can ensure traceability by recording data generated during the entire recognition and control process in chronological order for the reproducibility, performance verification, and failure analysis of the work. Recording can be performed on a non-transient storage medium of the mounted device (200) or on a storage transmitted to an external server, and can be indexed by a work unit (e.g., floor, section, process ID) and a device identifier.

[0066] First, frame key metadata can be recorded. For example, it may include the time of capture (local / UTC), camera ID and location, frame / filename, resolution, whether distortion correction was performed, and version identifiers (calibration revisions) of the internal and external camera parameters used. Environmental conditions (exposure / gain, auxiliary lighting on / off) are also recorded to be utilized for subsequent comparative analysis.

[0067] Second, the baseline (piano wire) detection results and selection criteria may be recorded. In one embodiment, the line detection scores before and after center-emphasis correction (e.g., Hough correction accumulated value H_adj, score ratio between candidates), the horizontal pixel coordinates (x) of the final selected candidate, the upper and lower intersection point coordinates, the slope of the baseline (vertical deviation angle), the candidate length, the number of candidates, whether non-maximum suppression and merging were performed, and whether X-range masking was applied may be stored. For luminance scoring, the measurement positions (offsets) of the center / left / right profiles and the background profile, the representative value of each profile (e.g., maximum value), and the final score (after background subtraction) may be included. Such selection criteria data is useful for reproducing the same results during subsequent reprocessing.

[0068] Third, the robot's estimated pose and control state can be recorded. For example, for each cycle, the estimated (x, y, z, Φ, θ, ψ), the error vector relative to the target pose, the tolerance (threshold) setting, and reliability indicators (e.g., sensor fusion weight, wire detection reliability, vibration RMS) may be included. Path following status (waypoint index, number of replans), tool status (drill / anchor motion flag, motion start / end times), and whether safety interlocks are satisfied may also be recorded.

[0069] Fourth, operational quality indicators can be recorded. For example, this may include the number of successes / retries per section, average / maximum position / attitude errors, cycle time, number of consecutive frames with no wire detected, and emergency stop cause codes. These indicators can be used for process optimization and to assess robustness against changes in field conditions.

[0070] Fifth, parameter snapshots can be recorded. Automatically configured parameters (e.g., expected display width in px, ρ-axis corresponding range (gapRange), merging threshold (Δρ, Δθ), minimum length threshold, weighted kernel shape and standard deviation, score fusion threshold ratio, etc.) and their calculation basis (actual wire thickness, camera-wire distance, focal length, etc.) can be saved at each point in time. This allows for offline reproduction processing to be performed with the same parameters for the same input.

[0071] Sixth, exception and alarm events can be recorded. For example, events such as a sudden drop in wire candidate confidence, detection failure due to occlusion / overexposure, ToF outlier detection, calibration inconsistency, and communication delay / packet loss, along with the corresponding control measures (slow scan, realignment, safe stop), can be logged together.

[0072] In one embodiment, the data may be stored redundantly on a frame-by-frame or process segment-by-process basis, and integrity verification information such as signatures and hashes may be added. The storage format may be implemented as CSV, binary, or a database, and traceability may be enhanced by storing raw images or regions of interest (ROI) together if necessary. Additionally, software build versions (algorithm revisions, model IDs) may be recorded to enable performance comparison before and after algorithm changes.

[0073] The records accumulated as described above can be utilized for various purposes, such as (i) automatic generation of field reports (alignment errors by floor / section, drilling location deviation, work time), (ii) tuning / performance improvement (threshold reset, automatic parameter range update), (iii) cause analysis (reproduction of input, parameter, and selection grounds for false detection / non-detection cases), and (iv) proof of safety compliance (logs of work performed within error limits). Accordingly, the data recording and traceability functions according to the present disclosure can serve as a foundation for maintaining and improving the reliability and quality of automated construction within elevator shafts over a long period.

[0074] FIGS. 5a to 5c are examples of image photographs of an elevator shaft according to one embodiment of the present disclosure. FIGS. 6a to 6k are exemplary photographs of an image photograph of an elevator shaft in which a reference line detection method according to one embodiment of the present disclosure is performed. FIGS. 5a to 6k are exemplary images for explaining the present disclosure, in which only one reference line is captured.

[0075] FIG. 5a is an original image of one side of the elevator shaft and a reference line captured using a camera. From the original image, it can be seen that the center of one side of the elevator shaft appears convex (distorted by the camera lens). FIG. 5b is an image in which the distortion of the original image has been corrected. Through distortion correction, one side of the elevator shaft is corrected to be flat. FIG. 5c is a grayscale image of the distortion-corrected image.

[0076] Figure 6a is an image in which edge detection was performed on a grayscaled image. Figure 6b is an enlarged image of a rectangular portion within the image of Figure 6a. Referring to Figure 6b, it is shown that both edges of the reference line have been detected. Additionally, it can be seen that noise (a pattern on one side of the elevator shaft, not the reference line) is present. Figure 6c is an image in which x-range masking was performed on the image of Figure 6b.

[0077] Figure 6d is an image obtained by applying the Hough transform to the image (Figure 6a) on which edge detection was performed. Referring to Figure 6e, the 20 locations with the highest Hough transform agreement, i.e., the locations with the largest accumulation matrix H, are marked with circles in the image obtained by applying the Hough transform. Figure 6f is an enlarged image of the rectangular portion within the image in Figure 6e. Referring to Figure 6f, the 20 locations with the largest H are marked with circles.

[0078] Figure 6g is an image in which parallel line center-focus correction is performed on both edges of the baseline using the Hough transform result. Referring to Figure 6g, it can be seen that the circles converge towards the center due to the parallel line center-focus correction. Here, the 20 circles with the largest correction accumulation matrix H_adj are marked as circles. Figure 6h is an enlarged image of the rectangular portion within the image of Figure 6g. Comparing Figure 6h and Figure 6f, it can be seen that the circles converge towards the center due to the parallel line center-focus correction.

[0079] Referring to Fig. 6i, two baseline candidates are displayed in the image after parallel line center emphasis correction. Referring to Fig. 6j, the baseline candidates are displayed in the image after edge detection.

[0080] Referring to Fig. 6k, this is an image showing the use of the reflected light characteristics of the baseline. Although there is only one actual baseline, two baseline candidates were found. Since the baseline containing metal will have more reflected light compared to the noise, this was indicated in the image.

[0081] The method according to the present disclosure can be implemented as processor-readable code on a processor-readable recording medium equipped with a server, system, equipment, computer, integrated control device, etc., used by a subject. A processor-readable recording medium includes all types of recording devices in which data that can be read by a processor is stored. Examples of processor-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc., and also include implementation in the form of a carrier wave, such as transmission over the Internet. Furthermore, the processor-readable recording medium may be distributed across networked computer systems, so that processor-readable code can be stored and executed in a distributed manner.

[0082] The device and method described above may be implemented as a hardware component, a software component, and / or a combination of a hardware component and a software component.

[0083] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

[0084] The embodiments described in this disclosure may also be implemented in a distributed computing environment in which tasks are performed by remote processing units connected via a communication network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

[0085] Although the embodiments have been described above with reference to the limited drawings, those skilled in the art can apply various technical modifications and variations based on the above. For example, suitable results may be achieved even if the described techniques are performed in a different order than described, and / or if the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0086] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below.

Claims

1. A mounting device for measuring the relative position and orientation of an installation work in an elevator shaft, A main body suspended from a lifting module installed in the above elevator shaft; At least two cameras disposed on the main body; An inertial measuring device configured to measure the roll and pitch of the above-mentioned mounted device; A pose calculation module that calculates the position and orientation of the mounted device, Includes, The pose calculation module above is, Each of the above cameras captures an image such that at least one of the two reference lines positioned in the elevator shaft is included in the image, and Estimate the coordinates of two reference lines in the device coordinate system from the above image, and Based on the coordinates of the two reference lines in the device coordinate system and the image, the five degrees of freedom (x, y, roll, pitch, yaw), including the position (x, y) and yaw angle (yaw) on the device, are configured to be obtained in the hoistway coordinate system. Mounted device.

2. In Paragraph 1, The mounting device further comprises a distance sensor configured to measure a height (z) from the elevator shaft.

3. In Paragraph 2, The mounting device further comprises a robot module that performs installation work, such as drilling and anchor insertion, within the elevator shaft using the 5 degrees of freedom and the height (z).

4. In Paragraph 1, A mounting device in which at least some of the above cameras are positioned so as to include two reference lines placed in the elevator shaft in one image.

5. In Paragraph 1, A mounting device comprising estimating the above position (x, y) and yaw angle by the intersection of the orientation lines of at least two cameras arranged on one side.

6. In Paragraph 1, A mounting device that estimates the above position (x, y) and yaw angle based on the condition that the x coordinates of the two reference lines in the elevator shaft coordinate system are the same, and the condition that the distance between the two reference lines is secured in advance by the pose calculation module.

7. In Paragraph 1, A mounted device, wherein the pose calculation module estimates the image display width from the actual thickness of the reference line and the distance from the camera, and based thereon, estimates the coordinates of the two reference lines in the device coordinate system.

8. In Paragraph 1, The above-mentioned mounting device further includes lighting, and A mounted device that uses the above lighting to generate reflected light of the above reference line and further uses the luminance of the above reflected light to estimate the coordinates of the two reference lines in the device coordinate system.

9. In Paragraph 1, The above at least two cameras include first to fourth cameras, and A mounting device configured such that the first and third cameras capture the first reference line among the two reference lines, and the second and fourth cameras capture the second reference line among the two reference lines to obtain an image.

Images