Construction site automatic supervision system

Abstract

According to an embodiment, an automatic construction site supervision system is provided, comprising: a shooting unit that captures an image of a construction site and generates image data; a first processing unit that specifies the position of a jack support on the image data; a second processing unit that calculates the two-dimensional coordinates of the reference point position of the jack support on the image data; a third processing unit that converts the two-dimensional coordinates of the reference point position into three-dimensional coordinates; a fourth processing unit that calculates the installation position of the jack support and the installation spacing of the jack support using the three-dimensional coordinates; and a verification unit that verifies the quality of the installation position and the installation spacing.

Description

Technology Field

[0001] One embodiment of the present invention relates to an automatic supervision system for construction sites. Background Technology

[0003] Jack supports refer to temporary structures consisting of circular steel pipe supports primarily used to reinforce slabs during heavy-duty work and demolition operations on the slab surface. When installing jack supports, they must be constructed in compliance with the structural review report specifying their installation locations and spacing; this is crucial for ensuring proper load distribution and structural stability.

[0004] Currently, to verify whether jack supports have been constructed in compliance with structural review documents, methods are utilized such as visual inspection or measuring installation locations and spacing using mechanical tools like tape measures.

[0005] However, this visual inspection method has the disadvantage that it is difficult to guarantee objectivity, and measurement methods using tools such as tape measures require a lot of time and manpower to measure all the jack supports installed on-site one by one. The problem to be solved

[0007] The technical problem that the present invention aims to solve is to provide an automatic construction site supervision system for automatically supervising the installation location and spacing of jack supports installed at a construction site using image information. means of solving the problem

[0009] According to an embodiment, an automatic construction site supervision system is provided, comprising: a shooting unit that captures an image of a construction site and generates image data; a first processing unit that specifies the position of a jack support on the image data; a second processing unit that calculates the two-dimensional coordinates of the reference point position of the jack support on the image data; a third processing unit that converts the two-dimensional coordinates of the reference point position into three-dimensional coordinates; a fourth processing unit that calculates the installation position of the jack support and the installation spacing of the jack support using the three-dimensional coordinates; and a verification unit that verifies the quality of the installation position and the installation spacing.

[0010] The first processing unit can determine the location of the pixels constituting the jack support by applying an object segmentation method to the image data.

[0011] The second processing unit above can calculate the center point of the lower part of the jack support where the jack support and the bottom surface come into contact as the reference point.

[0012] The second processing unit above can calculate the coordinates of the center point of the lower part of the jack support using the position of the pixel constituting the jack support.

[0013] The verification unit can verify the quality by comparing the installation location and the installation spacing with the specifications in the structural review document. Effects of the invention

[0015] The construction site automatic supervision system according to the embodiment can automatically supervise the installation location and spacing of jack supports installed at a construction site using video information.

[0016] In addition, objectivity and reliability of the inspection results can be ensured.

[0017] In addition, it can reduce the time and manpower required for supervision.

[0018] In addition, the safety of the building can be ensured by managing the installation of jack supports in compliance with relevant regulations. Brief explanation of the drawing

[0020] FIG. 1 is a block diagram of an automatic construction site supervision system according to an embodiment. FIG. 2 is a diagram for explaining the operation of a first processing unit according to an embodiment. FIG. 3 is a diagram illustrating the operation of a second processing unit according to an embodiment. FIG. 4 is a diagram illustrating the operation of a third processing unit according to an embodiment. FIG. 5 is a diagram illustrating the operation of the fourth processing unit according to an embodiment. FIG. 6 is a diagram illustrating the operation of a verification unit according to an embodiment. FIG. 7 is a flowchart of the operation sequence of an automatic construction site supervision system according to an embodiment. Specific details for implementing the invention

[0021] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

[0022] However, the technical concept of the present invention is not limited to some of the described embodiments but can be implemented in various different forms, and within the scope of the technical concept of the present invention, one or more of the components among the embodiments may be selectively combined or substituted.

[0023] In addition, terms used in the embodiments of the present invention (including technical and scientific terms) may be interpreted in a sense that is generally understood by those skilled in the art to which the present invention belongs, unless explicitly and specifically defined otherwise. Terms that are commonly used, such as terms defined in advance, may be interpreted in consideration of their meaning in the context of the relevant technology.

[0024] Furthermore, the terms used in the embodiments of the present invention are for the purpose of describing the embodiments and are not intended to limit the present invention.

[0025] In this specification, the singular form may include the plural form unless specifically stated otherwise in the text, and when described as "at least one of A and B and C (or more than one)," it may include one or more of all combinations that can be formed from A, B, and C.

[0026] In addition, terms such as first, second, A, B, (a), (b), etc. may be used to describe the components of the embodiments of the present invention.

[0027] These terms are intended merely to distinguish a component from other components and are not limited by the nature, order, sequence, etc., of the said component.

[0028] And, where it is stated that a component is 'connected', 'combined', or 'joined' to another component, this may include not only cases where the component is directly connected, combined, or joined to the other component, but also cases where it is 'connected', 'combined', or 'joined' due to another component located between the component and the other component.

[0029] Furthermore, when described as being formed or placed "above or below" each component, "above" or "below" includes not only cases where two components are in direct contact with each other, but also cases where one or more other components are formed or placed between the two components. Additionally, when expressed as "above or below," it may include the meaning of a downward direction as well as an upward direction relative to a single component.

[0030] Hereinafter, embodiments will be described in detail with reference to the attached drawings, provided that identical or corresponding components are given the same reference number regardless of the drawing symbols, and redundant descriptions thereof will be omitted.

[0032] FIG. 1 is a block diagram of an automatic construction site supervision system according to an embodiment.

[0033] Referring to FIG. 1, the construction site automatic supervision system (10) according to the embodiment may include a shooting unit (11), a first processing unit (12), a second processing unit (13), a third processing unit (14), a fourth processing unit (15), and a verification unit (16).

[0034] The camera unit (11) can capture images of the construction site and generate image data.

[0035] Multiple shooting units (11) may be installed within the construction site. For example, the shooting unit (11) may include at least one upper image sensor positioned above the construction site and at least one side image sensor positioned on the side of the construction site. Each of the upper image sensor and the side image sensor may be an IP camera capable of transmitting real-time video via wired or wireless communication. In the embodiment, the case where one image sensor is positioned on the side of the construction site will be described as an example.

[0036] The shooting unit (11) can generate image data by capturing an image including a plurality of jack supports. In the embodiment, the jack support may refer to a circular steel pipe support used as a temporary structure, mainly for reinforcing the slab during heavy lifting and dismantling operations on the upper part of the slab.

[0037] The shooting unit (11) can generate multiple image data using multiple images that are captured sequentially. For example, the shooting unit (11) can generate first image data by capturing a first image containing multiple jack supports, and can generate second image data by capturing a second image containing multiple jack supports. The first image and the second image may each be images captured sequentially in time, and one image data may mean a single frame. The shooting unit (11) can generate image data that is sequential in time using the first image and the second image that are captured sequentially.

[0038] The shooting unit (11) may be an image sensor that captures a subject using a COMS (complementary metal-oxide semiconductor) module or a CCD (charge coupled device) module. At this time, the input image frame is provided to the COMS module or CCD module within the shooting unit (11) through a lens, and the COMS module or CCD module converts the optical signal of the subject passing through the lens into an electrical signal (image data) and outputs it.

[0039] The shooting unit (11) may include a fisheye lens or a wide-angle lens with a wide field of view. Accordingly, it is possible for a single image sensor to capture the entire space inside the construction site.

[0040] Additionally, the shooting unit (11) may be a depth camera. The shooting unit (11) may be operated in any one of various depth recognition methods, and the image captured through the shooting unit (11) may include depth information. The shooting unit (11) may be, for example, a Kinect sensor. The Kinect sensor is a depth camera using a structured light projection method, and can obtain three-dimensional information of a scene by projecting a defined pattern image using a projector or laser and acquiring the image of the pattern projected through the camera. Such a Kinect sensor includes an infrared emitter that illuminates a pattern using an infrared laser, and an infrared camera that captures an infrared image, and an RGB camera that functions like a general webcam is positioned between the infrared emitter and the infrared camera. In addition, the Kinect sensor may further be configured with a microphone array and a pan-tilt unit that adjusts the angle of the camera.

[0041] The basic principle of the Kinect sensor is that when a laser pattern emitted from an infrared emitter is projected onto an object and reflected, the distance to the surface of the object is calculated using the position and size of the pattern at the reflection point. Based on this principle, the imaging unit (11) can irradiate a laser pattern into the space within the construction site and sense the laser pattern reflected from the object to generate image data containing depth information for each object.

[0042] In the embodiment, it will be explained that the image data generated by the shooting unit (11) is two-dimensional image data.

[0043] The first processing unit (12) can determine the location of the jack support on the image data. The first processing unit (12) can determine the location of the pixels constituting the jack support by applying an object segmentation method to the image data.

[0044] FIG. 2 is a drawing for explaining the operation of the first processing unit (12) according to an embodiment.

[0045] Referring to FIG. 2, the first processing unit (12) can divide image data into multiple zones. The first processing unit (12) can divide image data into multiple zones of equal size. At this time, the size of each zone can be determined according to the height, thickness, width, etc. of the jack support. For example, the first processing unit (12) can divide the image data by making the size of a single zone relatively smaller as the volume of the jack support decreases.

[0046] Alternatively, the first processing unit (12) may divide the image data into multiple areas of unequal size according to the distortion characteristics of the camera.

[0047] The first processing unit (12) can divide the area into smaller sections the further away from the camera when the image data is captured by a fisheye lens. In the case of image data captured by a fisheye lens, the further away from the focal point, the more distorted it appears. Therefore, the accuracy of abnormal object detection can be improved by finely dividing the area the further away from the camera.

[0048] The first processing unit (12) can divide the area into larger sections the further away from the camera when the image data is captured by a telephoto lens. In the case of image data captured by a telephoto lens, the perspective is compressed, and the concentration of objects is exaggerated, making them appear as if they are densely packed. Therefore, the accuracy of pixel detection can be improved by dividing the area into larger sections the further away from the camera.

[0049] Referring again to FIG. 1, the second processing unit (13) can calculate the two-dimensional coordinates of the reference point location of the jack support on the image data. The second processing unit (13) can calculate the center point of the lower part of the jack support where the jack support and the floor surface meet as the reference point. The second processing unit (13) can calculate the coordinates of the center point of the lower part of the jack support using the location of the pixels constituting the jack support.

[0050] The second processing unit (13) can identify the center point of the bottom part where the jack support meets the floor in the two-dimensional image data on a two-dimensional pixel coordinate system.

[0051] FIG. 3 is a diagram for explaining the operation of the second processing unit (13) according to an embodiment.

[0052] Referring to FIG. 3, the second processing unit (13) can calculate the coordinates of the center point (reference point) of the bottom part where the jack support contacts the floor. The second processing unit (13) calculates the height (H) of the jack support using the positions of the pixels constituting the jack support, and then calculates the pixels located within 1 / 20 of the height of the jack support from the bottom part of the jack support. Then, among the calculated pixels, it calculates two points that are the minimum (Xmin) and maximum (Xmax) on the horizontal axis (x), calculates the average between the horizontal axis (x) and vertical axis (y) values ​​respectively, and the calculated value can be calculated as the center point coordinates of the jack support.

[0053] Referring again to FIG. 1, the third processing unit (14) can convert the 2D coordinates of the reference point location into 3D world coordinates. Since the center point coordinates of the jack support calculated by the second processing unit (13) are values ​​calculated through 2D image data, they must be converted into 3D world coordinate values ​​(Object localization).

[0054] The third processing unit (14) can store internal parameters including the focal length of the shooting unit (11) and external parameter information such as the installation height of the image sensor and the pitch value with respect to the x-axis. Internal and external parameters can generally be obtained through calibration using a checkerboard or the like.

[0055] FIG. 4 is a diagram for explaining the operation of the third processing unit (14) according to an embodiment. Referring to FIG. 4, the third processing unit (14) calculates the rotation angle (θx) of the image sensor with respect to the x-axis using the vanishing point, and extracts the remaining necessary internal and external parameters based on the focal length (f) of the image sensor, the 2D image (pixel) coordinates of at least two points (principal points), and the 3D world coordinates.

[0056] The third processing unit (14) can convert the 2D coordinates of the reference point location into 3D coordinates according to the following mathematical formulas 1 to 6 using internal and external parameters of the image sensor that have been acquired and stored, or internal and external parameters of the image sensor extracted using the vanishing point.

[0057] While various operations can be performed to convert 2D coordinates to 3D coordinates, in the embodiment, the operation of normalizing the 2D coordinates can be performed. To this end, normalized coordinates can be generated by normalizing the focal length (f), which is a variable such as intrinsic and extrinsic parameters, to 1.

[0058] Subsequently, the third processing unit can convert the normalized 2D coordinates into coordinate values ​​of a 3D coordinate system. In addition, it can (re)convert the 3D coordinates into 3D world coordinates and, in this process, generate 3D world coordinates by reflecting 3D position information received from GNSS (Global Navigation Satellite System).

[0059] The third processing unit can first normalize the 2D coordinates calculated from the 2D image data in order to convert the 2D coordinates into 3D world coordinates. For example, the third processing unit can convert the pixel coordinates into normalized coordinates with f = 1 to eliminate the influence of the image sensor parameters. This can be expressed as shown in [Equation 1] below.

[0060]

[0061] In mathematical formula 1, x = column and y = row are two-dimensional coordinates, and cx, cy, fx, and fy are constants of the image sensor that may be obtained through camera calibration or are pre-stored values.

[0062] The third processing unit can perform a transformation operation of normalized 2D coordinates into a 3D coordinate system. The coordinates recognized by the third processing unit are represented in a 3D coordinate system with a focal length f = 1, and the 2D coordinates recognized through the normalization of the aforementioned 2D coordinates can be expressed in the 3D coordinate system as shown in [Equation 2] below.

[0063]

[0064] The third processing unit can convert 3D coordinates into 3D world coordinates. This can be expressed as shown in [Equation 3] below.

[0065]

[0066] In Equation 3, Pm represents the coordinates of the jack support in 3D world coordinates, and R is a rotation matrix that can be obtained through the aforementioned intrinsic and extrinsic parameters. This can be expressed as shown in [Equation 4] below.

[0067] In addition, in Equation 3, Pc is a value representing the coordinates of the jack support in a 3D coordinate system, and T is a 3-axis translation vector between the origin of the 3D world coordinate system and the origin of the 3D coordinate system, which can be obtained through GNSS.

[0068]

[0069] In mathematical formula 4, Rx is the rotation angle (roll) with respect to the x-axis, Ry is the rotation angle (roll) with respect to the y-axis, and Rz is the rotation angle (pitch) with respect to the z-axis.

[0070] At this time, if the 3D world coordinates of the image sensor center are denoted as Cm, the condition of [Equation 5] below holds.

[0071]

[0072] The third processing unit uses the formula in [Mathematical Equation 6] below, in the 3D world coordinate system and The point where the line connecting them meets the ground is the 3D world coordinate of the jack support It can be calculated as.

[0073]

[0074] In mathematical equation 6, since the ground in the 3D world coordinate system is defined as an arbitrary constant (k) (in reality, the height value of the ground can be obtained using a DEM), the value of k can be calculated using the Z value extracted from the construction site's DEM, and based on this, the final 3D world coordinates of the jack support ( You can obtain ).

[0075] Referring again to FIG. 1, the fourth processing unit (15) can calculate the installation location of the jack support and the installation spacing of the jack support using three-dimensional world coordinates. The fourth processing unit (15) can determine the installation location of the jack support using three-dimensional world coordinates and calculate the installation spacing of the jack support through distance calculation between three-dimensional world coordinates.

[0076] For example, if the 3D world coordinates of the first jack support are (x1, y1, z1) and the 3D world coordinates of the second jack support are (x2, y2, z2), the fourth processing unit (15) can calculate the installation spacing between the first jack support and the second jack support according to the following mathematical formula 7.

[0077]

[0078] FIG. 5 is a drawing for explaining the operation of the fourth processing unit (15) according to an embodiment. Referring to FIG. 5, the fourth processing unit (15) can display the calculated installation spacing (D1~D4) between jack supports as 2.05m, 2.16m, 3.54m, and 2.32m by overlapping them on the image data.

[0079] Referring again to FIG. 1, the verification unit (16) can verify the quality of the installation location and installation spacing. The verification unit (16) can verify the quality of the installation location and installation spacing by comparing them with the specifications in the structural review report. The verification unit (16) can verify the quality of the installation location by comparing the installation location of the jack support recorded in the structural review report with the installation location of the jack support calculated by the fourth processing unit (15) and comparing whether it is installed within the error range according to standard specifications. In addition, the verification unit (16) can verify the quality of the installation spacing by comparing the installation spacing of the jack support recorded in the structural review report with the installation spacing of the jack support calculated by the fourth processing unit (15) and comparing whether it is installed spaced apart within the error range according to standard specifications.

[0080] FIG. 6 is a diagram illustrating the operation of a verification unit (16) according to an embodiment, wherein the verification unit (16) can calculate the verification result as shown in FIG. 6 and output it as a graph. In FIG. 6, the Ground truth value is the position of the jack support in the structural review document, and Predicted is the installation position of the jack support calculated according to the embodiment. Referring to the graph in FIG. 6, it can be confirmed that the position of the jack support in the structural review document and the position of the jack support calculated according to the embodiment are located identically within a predetermined error range.

[0081] FIG. 7 is a flowchart of the operation sequence of an automatic supervision system for a construction site according to an embodiment. Referring to FIG. 7, first, the camera unit can capture images of the construction site to generate image data (S701).

[0082] Next, the first processing unit can determine the location of the jack support on the image data. The first processing unit can determine the location of the pixels constituting the jack support by applying an object segmentation method to the image data (S702).

[0083] Next, the second processing unit can calculate the 2D coordinates of the reference point location of the jack support on the image data. The center point of the bottom part of the jack support where the jack support contacts the floor surface can be calculated as the reference point. At this time, the second processing unit can calculate the coordinates of the center point of the bottom part of the jack support using the location of the pixels constituting the jack support (S703).

[0084] Next, the third processing unit can convert the 2D coordinates of the reference point location into 3D world coordinates. The third processing unit can convert the 2D coordinates of the reference point location into 3D world coordinates using the internal and external parameters of the image sensor that have been acquired and stored, or the internal and external parameters of the image sensor extracted using the vanishing point (S704).

[0085] Next, the fourth processing unit can calculate the installation position and installation spacing of the jack supports using three-dimensional world coordinates. The fourth processing unit can display the calculated spacing between the jack supports by overlapping it on the image data (S705).

[0086] Next, the verification unit can verify the quality of the installation location and installation spacing. The verification unit can verify the quality by comparing the installation location and installation spacing with the specifications in the structural review document. The verification unit can calculate the verification results and output them as a graph (S706).

[0087] The construction site automatic supervision system according to the embodiment is highly versatile as it can inspect the installation location and spacing of jack supports by utilizing easily obtainable image data. Furthermore, unlike conventional inspection methods using visual inspection or tape measures, the objectivity and reliability of the results can be ensured by performing automatic inspection using image data. In addition, since the installation location and spacing of all jack supports within a certain distance included in the image data can be inspected at once, inspections can be performed quickly with minimal manpower. Moreover, since construction sites contain various objects whose actual size can be determined through drawings, utilizing this makes it easier to extract internal and external parameters of the camera. Through this, the present invention can help ensure the safety of buildings by managing the installation of jack supports in compliance with relevant regulations.

[0089] The term "part" as used in this embodiment refers to a software or hardware component, such as a field-programmable gate array (FPGA) or an ASIC, and the "part" performs certain roles. However, the meaning of "part" is not limited to software or hardware. The "part" may be configured to reside in an addressable storage medium or configured to run one or more processors. Thus, as an example, the "part" includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and "parts" may be combined into a smaller number of components and "parts" or further separated into additional components and "parts." In addition, the components and '~parts' may be implemented to play one or more CPUs within the device or secure multimedia card.

[0090] Although the present invention has been described above with reference to preferred embodiments, those skilled in the art will understand that various modifications and changes can be made to the invention without departing from the spirit and scope of the invention as described in the following claims. Explanation of the symbols

[0092] 10: Automated Construction Site Supervision System 11: Filming Department 12: First processing unit 13: Second processing unit 14: Third processing unit 15: 4th Processing Unit 16: Verification Department

Claims

Claim 1 A construction site automatic supervision system comprising: a shooting unit that captures images of a construction site and generates image data; a first processing unit that specifies the location of pixels constituting a jack support on the image data; a second processing unit that, using the location of pixels constituting the jack support, calculates a reference point for the center point of the lower part of the jack support where the jack support contacts the floor surface, calculates pixels belonging to the lowest part of the jack support and a point located at a predetermined ratio of the height of the jack support from the lowest part, and calculates the 2D coordinates of the center point based on the calculated pixels; a third processing unit that converts the 2D coordinates of the reference point location into 3D world coordinates; a fourth processing unit that calculates the installation location of the jack support and the installation spacing of the jack support using the 3D world coordinates; and a verification unit that verifies the quality of the installation location and the installation spacing. Claim 2 In claim 1, the first processing unit applies an object segmentation method to the image data to specify the location of the pixels constituting the jack support in the construction site automatic supervision system. Claim 3 A construction site automatic supervision system according to claim 1, wherein the second processing unit calculates the lowest part of the jack support and pixels within 1 / 20 of the jack support height from the lowest part to calculate the 2D coordinates of the central point. Claim 4 In paragraph 3, the second processing unit calculates two points, one minimum and one maximum on the horizontal axis, among the calculated pixels, calculates the average between the horizontal axis and vertical axis values ​​respectively, and calculates the calculated value as the center point coordinate of the corresponding jack support, thereby establishing an automatic construction site supervision system. Claim 5 In claim 1, the verification unit is a construction site automatic supervision system that verifies the quality by comparing the installation location and the installation spacing with the specifications in the structural review document.

Images