Method and system for constructing indoor space information using drone equipped with near-infrared light

Abstract

The present invention relates to a method and a system for constructing indoor space information using a drone equipped with near-infrared light, wherein: a three-dimensional point cloud is generated by scanning the inside of a global navigation satellite system (GNSS) shielding structure to be photographed by a drone, and environmental information is generated by analyzing the inside of the structure. In addition, when the inside of the structure is photographed by the drone according to the generated environmental information, the image quality and resolution for constructing spatial information, the size of the drone to be inserted into the structure, the output of a light to be mounted on the drone, the position for attaching the light to the drone, and the photographing direction of a camera are determined, and a near-infrared lighting module is mounted onto the drone. After the drone equipped with the near-infrared lighting module photographs the inside of the structure while flying, spatial information of the inside of the structure is constructed by overlaying an image of the inside of the structure photographed by the drone onto a three-dimensional point cloud.

Description

Method and system for constructing indoor spatial information using a drone equipped with near-infrared lighting

[0001] The present invention relates to a method and system for constructing indoor spatial information using a drone, and more particularly, to a method and system for constructing indoor spatial information in a GNSS (Global Navigation Satellite System) shielded environment using a drone equipped with lighting in the 850 nm near-infrared (NIR) wavelength range.

[0002] Indoor inspections involve comprehensive investigations of indoor structures, risk assessments for collapse, and repairs. Currently, indoor inspections are conducted through visual observation and inspection. When experts conduct visual inspections, they photograph problematic areas on-site and analyze the footage after the inspection is complete.

[0003] Internal inspections of buildings, tunnels, sewer pipes, storage tanks, pipelines, and other structures typically involve deploying personnel to photograph and analyze problematic areas. However, this method of manually deploying personnel often leads to problems like injuries, deaths, and toxic gas leaks due to the need to enter dark areas.

[0004] Or, there are cases where the passage is narrow and personnel cannot enter with filming equipment to conduct an investigation.

[0005] It is necessary to investigate whether cracks have developed inside buildings, tunnels, sewer pipes, etc. due to long-term use, or whether there is a risk of collapse. Currently, the method used is to film while moving personnel with filming equipment, as described above. However, there are also places where filming is not possible due to issues such as space and terrain.

[0006] To address these traditional issues, robotic inspection techniques are now being used. When inspecting dark, narrow spaces like sewer pipes, not only lighting but also power is required to operate the robot. If the robot encounters an obstacle it cannot climb, it cannot move any further and must stop the inspection. Furthermore, high-power lighting is required to capture the interior properly, but this quickly drains power, preventing the entire pipe from being inspected.

[0007] Also, when using a drone, a drone equipped with a GNSS (Global Navigation Satellite System) that flies outdoors has a stable flight and does not consume much power, but a drone that flies indoors has a lot of work to do as the FC (Flight Controller) to fly normally in a dark environment, so it consumes a lot of power.

[0008] Conventional visible light lighting using LEDs has limitations: it generates a lot of heat, requires relatively high power, and makes accurate photographic surveys difficult in dusty indoor environments. Furthermore, when mounted on a drone, battery technology limitations significantly reduce mission execution time.

[0009] Since most underground structure detection environments are dark or low-light, a high-power light source is required to acquire high-resolution images that can illuminate the interior. However, with existing drones, detection is often impossible due to limitations in battery and lighting technology, as well as diffuse reflection and field of view obstruction when a large amount of dust is generated.

[0010] The present invention is intended to solve the problems of the above-mentioned prior art, and relates to a method for detecting and acquiring spatial information in a GNSS (Global Navigation Satellite System) shielded environment (indoors) using illumination in the 850 nm near-infrared (NIR) wavelength range, and provides a method and system for constructing indoor spatial information using a drone equipped with NIR illumination, which overcomes the limitations of LED illumination utilizing the existing visible light range when investigating artificial structures (factories, storage and stockpiling facilities, tunnels, sewer pipes, ships, etc.) and natural structures such as caves, which are difficult to detect inside due to their narrow structures or poor working environments, and enables photographic investigation, 3D modeling, SLAM (Simultaneous Localization And Mapping), and spatial information construction inside the structures.

[0011] A method for constructing indoor spatial information using a drone equipped with near-infrared lighting to achieve the above purpose comprises a first step of scanning the interior of a GNSS (Global Navigation Satellite System) shielding structure to be photographed by a drone to create a 3D point cloud and analyzing the interior of the structure to create environmental information, a second step of determining image quality and resolution for constructing spatial information when photographing with a drone according to the environmental information created in the first step, a third step of determining the size of a drone to be deployed in the structure according to the environmental information created in the first step, a fourth step of determining the output of lighting to be installed on the drone, the location to attach the lighting to the drone, and the shooting direction of a camera according to the environmental information created in the first step, a fifth step of mounting a near-infrared lighting module on the drone according to the content determined in the fourth step, a sixth step of flying the interior of the structure while photographing, and a seventh step of constructing spatial information on the interior of the structure by overlaying an image of the interior of the structure photographed by the drone onto the 3D point cloud created in the first step.

[0012] Specifically, step 1 is:

[0013] It consists of a step of obtaining a 3D point cloud containing location information by scanning the interior of a structure using a LiDAR sensor, and a step of generating environmental information by receiving inputs such as dust, illuminance, and the type and condition of the structure inside the structure.

[0014] Here, the state of the structure is composed of the size of the movement space inside the structure and the material of the movement space.

[0015] In the second stage, the image quality and resolution are primarily determined by considering the dust and illuminance inside the structure, and are then adjusted secondarily to reflect the size and material of the moving space, and then finally determined.

[0016] In addition, in the third stage, the size range of drones to be deployed in the structure is initially determined by considering the size of the movement space inside the structure, and the final decision is made based on the type and number of sensors required for dust and illuminance inside the structure among one or more drones initially determined.

[0017] In the fourth stage

[0018] Among the environmental information generated in the first step, the lighting output is increased or decreased depending on the level of illumination inside the structure, and the angle at which the lighting and camera face the wall of the moving space inside the structure is set differently depending on the material of the moving space.

[0019] And the near-infrared lighting module uses the 850nm wavelength range.

[0020]

[0021] Meanwhile, an indoor spatial information construction system using a drone equipped with near-infrared lighting to achieve the above purpose comprises: a point cloud generation unit that generates a 3D point cloud using data scanned using a LiDAR sensor of the interior of a GNSS (Global Navigation Satellite System) shielding structure to be photographed by a drone; an environmental information generation unit that generates environmental information by receiving dust, illuminance, and the type and condition of the structure inside the structure; an image quality determination unit that determines image quality and resolution for constructing spatial information when photographing with a drone according to the environmental information generated by the environmental information generation unit; a drone size determination unit that determines the size of a drone to be deployed to a structure according to the environmental information generated by the environmental information generation unit; a lighting camera determination unit that determines the output of lighting to be installed on a drone, the location of lighting to be attached to the drone, and the shooting direction of the camera according to the environmental information generated by the environmental information generation unit; and a lighting camera determination unit that overlays an image of the interior of a structure photographed by a drone equipped with a near-infrared lighting module according to the contents determined by the lighting camera determination unit onto the 3D point cloud generated by the point cloud generation unit to generate spatial information inside the structure. It consists of a spatial information construction department that is being built.

[0022] Here, the state of the structure is composed of the size of the movement space inside the structure and the material of the movement space.

[0023] Specifically, the image quality determination unit primarily determines the image quality and resolution by considering the dust and illuminance inside the structure, and secondarily adjusts the image quality and resolution by reflecting the size and material of the moving space.

[0024] Specifically, the drone size determination unit first determines the size range of drones to be deployed in the structure by considering the size of the movement space inside the structure, and then makes a final decision based on the type and number of sensors required for dust and illuminance inside the structure among one or more drones determined in the first stage.

[0025] Specifically, the lighting camera decision unit increases or decreases the lighting output according to the level of illumination inside the structure among the environmental information generated by the environmental information generation unit, and sets the angle at which the lighting and camera face the wall of the movement space differently according to the material of the movement space inside the structure.

[0026] And the near-infrared lighting module uses the 850nm wavelength range.

[0027] According to the method and system for constructing indoor spatial information using a drone equipped with near-infrared lighting according to the present invention, the image quality and resolution during drone shooting can be determined according to the internal environment of a GNSS (Global Navigation Satellite System) shielding structure to be photographed by the drone, thereby securing an optimal image suitable for the environment, and the optimal size of a drone to be deployed into the structure can be determined according to the internal environment of the structure, and by comprehensively determining the output of the lighting to be mounted on the drone, the location to attach the lighting, the shooting direction of the camera, etc., the drone equipped with a near-infrared lighting module having optimal conditions suitable for the internal environment of the structure can be used to photograph an image while flying inside the structure.

[0028] The present invention is an indoor exploration drone equipped with lighting in the 850nm near-infrared wavelength range that can fly in narrow and dangerous indoor spaces without spatial restrictions, and can take high-resolution photographs penetrating dust with low power, and obtain spatial information through this, which can be used to build indoor spatial information inside structures, conduct digital twin projects, and discover problems inside structures, so the usability of data taken by the drone is high.

[0029] Figure 1 is a photo showing the poor working environment of an underground structure (sewerage).

[0030] Figure 2 is a comparative analysis result of testing the performance of a visible light lighting using LEDs used in conventional drones and a near-infrared lighting module with a wavelength range of 850 nm mounted on the drone of the present invention in low-light, night, and dusty environments.

[0031] Figure 3 is a photograph showing the process of developing a near-infrared lighting module in the 850 nm wavelength range according to the present invention and conducting a flight test using a drone equipped with a near-infrared lighting module.

[0032] FIG. 4 is an example showing the steps of creating a 3D point cloud containing location information by scanning the inside of a structure using a LiDAR sensor according to the present invention, customizing a drone to fit the environment of an underground structure, verifying the operation by flying the customized drone through a narrow space of the underground structure, and finally constructing spatial information by overlaying an image of the inside of the underground structure taken by the drone using a near-infrared lighting module and a camera in the 850 nm wavelength range onto the 3D point cloud.

[0033] Figure 5 is a flowchart of a method for constructing indoor spatial information using a drone equipped with near-infrared lighting.

[0034] Figure 6 is an example of generating environmental information for a sewer pipe.

[0035] Figure 7 is a configuration diagram of an indoor spatial information construction system using a drone equipped with near-infrared lighting.

[0036] Figure 8 is a functional block diagram of the image quality determination unit, drone size determination unit, and lighting camera determination unit.

[0037] The advantages and features of the present invention and the method for achieving them will become clear with reference to the embodiments described in detail below together with the attached drawings.

[0038] However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms.

[0039] The embodiments in this specification are provided to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention.

[0040] And the present invention is defined only by the scope of the claims.

[0041] Accordingly, in some embodiments, well-known components, well-known operations, and well-known techniques are not specifically described to avoid obscuring the present invention.

[0042] Additionally, like reference numerals throughout the specification refer to like elements, and the terms used (referred to) herein are for the purpose of describing embodiments and are not intended to limit the present invention.

[0043] In this specification, the singular includes the plural unless specifically stated otherwise in the phrase, and the reference to an element or action as “including (or comprising)” does not exclude the presence or addition of one or more other elements or actions.

[0044] Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used in the sense commonly understood by a person of ordinary skill in the art to which the present invention belongs.

[0045] Also, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless they are defined otherwise.

[0046] Hereinafter, a preferred embodiment of the present invention will be described with reference to the attached drawings.

[0047] The present invention will be described in detail with reference to FIGS. 1 to 8.

[0048] [Method for Building Indoor Spatial Information Using a Drone Equipped with Near-Infrared Lighting]

[0049] Step 1 (S510)

[0050] By scanning the interior of a GNSS (Global Navigation Satellite System) shielding structure with a drone, a 3D point cloud is created, and the interior of the structure is analyzed to create environmental information.

[0051] Specifically, a worker enters the interior of a structure to be investigated by photographing it with a drone, carries a LiDAR sensor device, and scans the interior of the structure with the LiDAR sensor device while moving. The data scanned by the worker is entered into the system (1000).

[0052] The point cloud generation unit (100) obtains a 3D point cloud containing location information using data obtained by scanning the interior of a structure using a LiDAR sensor.

[0053] In the present invention, structures include both artificial structures such as factories, storage and stockpiling facilities, tunnels, sewer pipes, ships, and natural structures such as caves.

[0054] LiDAR sensors are devices that measure the positional coordinates of an object by emitting laser pulses and measuring the time it takes for the pulses to reflect and return. They capture and output precise 3D information about the real world over wide areas and long distances in the form of a 3D point cloud image. LiDAR sensors can precisely measure 3D positional information.

[0055] The environmental information generation unit (200) receives input from the worker on whether there is a lot / little / normal dust inside the structure, whether the illuminance (brightness) is bright / dark / normal, and the type of structure, such as a sewer pipe, cave, tunnel, or storage tank, and the condition of the structure, and generates environmental information.

[0056] The generated environmental information is recorded in the database (700) of the system (1000).

[0057] The condition of the structure includes the size of the movement space inside the structure and the material of the movement space.

[0058] For example, for sewer pipes, this includes the pipe's length, cross-sectional area, whether it's narrow or wide, whether the pipe wall is made of concrete or plastic, and the amount of water flowing underneath. Furthermore, the presence of harmful gases can be identified and entered.

[0059] Figure 6 illustrates an example of generating environmental information for a sewer pipe based on information identified and entered by a worker entering the structure. The environmental information is recorded in the database (700) of the system (1000).

[0060]

[0061] The environmental information of the structure is created by inputting the information into the system (1000) by a worker who enters the structure wearing a lidar sensor device.

[0062] The operator also checks whether GNSS signals are received inside the structure based on environmental analysis results. There are differences in drone control depending on whether GNSS signals are received inside the structure or not.

[0063] When GNSS reception is available, the drone can accurately measure its position and speed from satellites, allowing it to fly stably and reach its destination. However, when GNSS reception is unavailable, the drone cannot accurately determine its position and speed, which can lead to accumulated errors or the risk of collision with obstacles.

[0064] Therefore, in order to control a drone inside an underground sewer pipe where GNSS reception is impossible, it is necessary to attach a position correction sensor and a device that can protect the drone in case of collision with a wall or other surface.

[0065]

[0066] Step 2 (S520)

[0067] The image quality determination unit (300) determines the image quality and resolution for constructing spatial information during drone shooting based on the environmental information generated in the first step.

[0068] The image quality and resolution of drone footage are determined by reflecting the dust, illuminance, type and condition of the structure inside the structure.

[0069] Specifically, the image quality primary decision module (310) primarily determines the image quality and resolution by considering the dust and illuminance inside the structure, and the image quality secondary adjustment module (320) secondarily adjusts the image quality and resolution by reflecting the size and material of the moving space.

[0070] Image quality and resolution are two important factors that determine the clarity and detail of an image. Image quality refers to the color, brightness, contrast, and noise levels of an image, while resolution refers to the number of pixels contained in the image along its width and height.

[0071] Generally, higher image quality and resolution result in sharper and more detailed images. However, image quality and resolution are independent factors, so increasing just one does not necessarily result in a better image.

[0072] For example, even a high-resolution image will have poor color or contrast, or be noisy, resulting in poor image quality. Conversely, a low-resolution image will have high image quality if it has well-adjusted color and contrast, and minimal noise.

[0073] As shown in Figure 6, if the interior of a structure is dusty or dark, image quality can be increased or resolution reduced to reduce image noise. If the structure is complex or damaged, image quality can be decreased or resolution increased to preserve image details. This allows the drone to generate optimized images of the spatial information within the structure.

[0074] In dusty and dark environments, increasing both image quality and resolution will result in clearer images. However, higher image quality and resolution require the drone's camera, flight controller, and communication module to process and store more data, resulting in increased power consumption. Rapid battery drain shortens the drone's flight time, making it difficult to properly capture images inside structures.

[0075] The present invention overcomes these limitations and determines image quality and resolution through primary decision and secondary adjustment to enable optimal image acquisition and drone flight suitable for the internal situation of a structure.

[0076] Even when filming the interior of a structure by equipping a drone with near-infrared lighting utilizing the 850nm wavelength range, the quality and resolution of the image can be adjusted according to the situation.

[0077] As can be seen from the comparative analysis results in Figure 2, near-infrared lighting utilizing the 850 nm wavelength range has the advantage of being able to capture clear images even in dark places with a lot of dust.

[0078] Because near-infrared lighting has high penetration through particles like dust and fog, image quality can be improved or resolution reduced to reduce noise in cases of high dust or darkness in sewer pipes, as shown in Figure 6. This allows the drone to generate optimized images of spatial information inside structures using near-infrared lighting in the 850nm wavelength range.

[0079] For example, Figure 6 is as follows.

[0080] First Decision: Because the dust is thick and dark, image quality can be improved by reducing pixel size or increasing the number of pixels. Near-infrared lighting utilizing the 850nm wavelength range can penetrate the dust inside the underground sewer pipes and produce clear images. The resolution is also increased to ensure high image quality.

[0081] Secondary Adjustment: The sewer pipe is approximately 60 meters long and has a wide cross-sectional area of ​​approximately 2.5 meters. It's made of concrete. Because the drone must capture such a long and wide sewer pipe, the resolution determined in the first stage had to be lowered to reduce battery consumption.

[0082] The reference value for the secondary adjustment is stored in the database (700) of the system (1000), and this value can be changed.

[0083] For example, a standard value is set so that if it exceeds 50 m, it is judged as long, and if it is less than 50 m, it is judged as short. A standard value is set so that if the cross-sectional area exceeds 2 m in diameter, it is judged as wide, and if it is less than 2 m, it is judged as narrow.

[0084]

[0085] The video quality and resolution can be set as follows through the first decision and second adjustment.

[0086] The image quality primary decision module (310) reduces the pixel size by half or doubles the number of pixels compared to a general shooting environment with less dust and bright lighting in order to improve the image quality of an underground sewer pipe.

[0087] The pixel size and number of a typical shooting environment are stored in a database (700).

[0088] This allows the video quality to be maintained above 1080p even when the resolution is lowered from 1080p to 720p. The secondary video quality adjustment module (320) downgrades the resolution from 1080p to 720p.

[0089] According to the present invention, the resolution values ​​for image quality in drone filming range from low to high, for example, are as follows.

[0090] a. Low video quality range: 480p (720x480) or lower

[0091] The resolution in this range is 720x480 or less. The image size is small and the level of detail is poor. Enlarging the image dramatically reduces image quality.

[0092] This is the resolution range that can be used when the drone needs to fly for a long time and take pictures in a dust-free, bright environment and when expressing very detailed parts is not required.

[0093] B. Average video quality range: 720p (1280x720) ~ 1080p (1920x1080)

[0094] This range of resolutions ranges from 720p to 1080p. Images are large, yet offer a certain level of detail. Even when enlarged, the image quality does not deteriorate significantly.

[0095] It is mainly used to observe the internal condition of tunnels or sewer pipes. Since the tunnels or sewer pipes are long and have a wide cross-sectional area, the shooting range is wide and the flight is long, so in cases where the battery needs to be used for a long time, the resolution can be lowered to 720p. In cases where the length is not long and the cross-sectional area is not wide, the battery consumption does not need to be greatly concerned, so the resolution can be increased to 1080p.

[0096] High video quality range: 2K (2560x1440) ~ 4K (3840x2160)

[0097] This ranges from 2K to 4K resolution. The images are very large and offer exceptional detail. Even when enlarged, the image quality deteriorates little.

[0098] Storage and stockpiling facilities require close observation, as cracks within them can lead to major accidents. Drone footage of storage and stockpiling facilities requires high-quality images. This also applies to tunnels and sewer pipes.

[0099]

[0100] Referring to the example of Fig. 6 above, the case where the image quality is improved by reducing the pixel size or increasing the number of pixels because there is a lot of dust and it is dark, and the resolution value is lowered considering the length and size of the sewer pipe, was explained.

[0101] However, if the length of the sewer pipe is short, such as 15 m, or the diameter is narrow, such as 1 m, there will be no problem with the drone being unable to fly due to the battery draining quickly, so there is no problem with increasing the image quality and resolution.

[0102] Additionally, if the structure is a tunnel, dust will be minimal, illuminance will be moderate, and the image will likely be approximately 100 m long and 10 m wide in cross-section. In this case, considering the dust and illuminance inside the tunnel, image quality and resolution are initially set to moderate. Depending on the tunnel's length and cross-sectional area, image quality and resolution can be maintained at moderate or adjusted one level lower.

[0103] In this way, the image quality and resolution must be configured differently depending on the environment of the structure being detected by the drone.

[0104] The image quality and resolution information determined by the image quality determination unit (300) is output through a display so that the operator can check it.

[0105]

[0106] Step 3 (S530)

[0107] The drone size determination unit (400) determines the size of the drone to be deployed to the structure based on the environmental information generated in the first step.

[0108] Specifically, the drone size primary determination module (410) primarily determines the size range of the drone to be deployed into the structure by considering the size (cross-sectional area and length) of the movement space inside the structure.

[0109] Drone sizes are divided into small drones using propellers from 2 to 7 inches, medium drones using propellers from 7 to 10 inches, and large drones using propellers larger than 10 inches.

[0110] The first drone size determination module (410) determines a small drone of a size that can sufficiently fly inside the structure when the cross-sectional area inside the structure is narrow, such as about 1 m.

[0111] The drone size primary decision module (410) determines a drone of sufficient size to fly the length of the structure by considering the length of the structure, among small drones of a higher size and medium-sized drones, when the cross-sectional area inside the structure is approximately 1 to 3 m. This is because if the drone is small, it may not be able to fly the entire length of the structure due to battery capacity.

[0112] The drone size primary determination module (410) determines a drone of sufficient size to fly the length of the structure by considering the length of the structure and a medium-sized drone of a higher size if the internal cross-sectional area of ​​the structure is 3 m or more. This is because if the drone is small, it may not be able to fly the entire length of the structure due to battery capacity.

[0113] Drones travel along the walls of a structure to film. The larger the cross-sectional area of ​​the structure, the more frequent the drone must travel within it. Considering this, the drone's appropriate size is determined first.

[0114]

[0115] And the drone size final decision module (420) makes a final decision based on the type and number of sensors required for dust and illuminance inside the structure among one or more drones that have been primarily decided.

[0116] The type of sensor mounted on the drone varies depending on the degree of darkness and amount of dust inside the underground structure.

[0117] The interiors of underground structures are often dark and dusty, making drone flight difficult. Therefore, drones flying inside underground structures should be equipped with the following sensors.

[0118] Vision sensors: Use infrared cameras, thermal imaging cameras, etc. to recognize the surroundings even in dark environments.

[0119] Laser sensors: They can accurately measure objects over long distances, helping to determine the structure inside a structure.

[0120] However, when equipped with near-infrared lighting utilizing the 850nm wavelength range, an infrared camera, thermal imaging camera, or laser sensor may not be equipped, and only the sensors below may be equipped.

[0121] Ultrasonic sensor: Can quickly measure objects at short distances, helping to avoid collisions with obstacles.

[0122] Accelerometer: Detects the drone's movements to maintain stable flight.

[0123] Gyroscope: Detects the drone's rotation angle to maintain stable flight.

[0124]

[0125] Depending on the degree of darkness and amount of dust inside the underground structure, the type and number of drone sensors required are divided as follows.

[0126] In cases of extreme darkness and dust: infrared cameras, thermal imaging cameras, laser sensors, ultrasonic sensors, accelerometers, gyroscopes, etc.

[0127] When the darkness is not too bad and there is a lot of dust: infrared cameras, laser sensors, ultrasonic sensors, accelerometers, gyroscopes, etc.

[0128] When the darkness is not too bad and there is little dust: cameras, laser sensors, accelerometers, gyroscopes, etc.

[0129] However, when equipped with near-infrared lighting utilizing the 850nm wavelength range, the infrared camera, thermal imaging camera, and laser sensor can be omitted, and only the remaining sensors can be installed.

[0130] The more sensors a drone must be equipped with, the more its weight increases. Since the flight controller has more work to do and consumes more power, the battery size increases, and the drone also becomes larger.

[0131] As shown in Figure 6, in areas with a lot of dust and darkness in the sewer pipe, near-infrared lighting utilizing the 850 nm wavelength range is installed, so infrared cameras, thermal imaging cameras, and laser sensors are excluded, and ultrasonic sensors, accelerometers, and gyroscopes are installed.

[0132] The sewer pipe in Fig. 6 has a cross-sectional area of ​​2.5 m and a length of 60 m.

[0133] In the above example, the drone size primary decision module (410) first decides on a drone of sufficient size to fly the corresponding length by considering the length of the structure among the medium-sized drones and some of the upper-sized small drones.

[0134] The drone size primary decision module (410) will initially determine that, if the length of the sewer pipe is approximately 20 to 40 m, some of the upper-sized small drones and all of the medium-sized drones will be large enough to fly over that length. However, since the length is approximately 60 m, the flight length is judged to be long, so some of the upper-sized small drones are excluded and only medium-sized drones are initially determined.

[0135] The length of 60m is an example, and the length is classified as long or short based on the standard value of approximately 50m described above.

[0136] If it is initially decided to use only a medium-sized drone, the final drone size decision module (420) makes the final decision based on the type and number of sensors required for dust and illuminance inside the structure.

[0137] Because the underground sewer pipe in Figure 6 is dark and dusty, it is equipped with near-infrared lighting utilizing the 850 nm wavelength range. Therefore, infrared cameras, thermal imaging cameras, and laser sensors are excluded, and ultrasonic sensors, accelerometers, and gyroscopes are installed as standard.

[0138] Among the initially selected medium-sized drones, the final selection will be one that can be equipped with an ultrasonic sensor, accelerometer, and gyroscope. For reference, medium-sized drones can be equipped with all of these: ultrasonic sensors, accelerometers, and gyroscopes.

[0139] As mentioned above, a medium-sized drone uses a propeller between 7 and 10 inches. Since a medium-sized drone using a propeller between 7 and 10 inches can be equipped with an ultrasonic sensor, an accelerometer, and a gyroscope, the drone size final decision module (420) selects one of the medium-sized drone models using a propeller between 7 and 10 inches stored in the database (700).

[0140] Information on the size of the drone to be deployed in the structure determined by the drone size determination unit (400) is output through a display so that the worker can check it.

[0141]

[0142] Step 4 (S540)

[0143] The lighting camera decision unit (500) determines the output of the lighting to be installed on the drone, the location to attach the lighting to the drone, and the shooting direction of the camera based on the environmental information generated in the first step.

[0144] The lighting camera decision unit (500) is configured to include a lighting output decision module (510), a lighting position decision module (520), and a camera direction decision module (530).

[0145] The lighting output determination module (510) increases or decreases the lighting output according to the level of illumination inside the structure among the environmental information generated in the first step.

[0146] Lighting output is typically measured in lumens. A drone light with an output of 800 to 1,000 lumens provides sufficient illumination even in dark environments. The present invention allows for adjustment within an output range of 300 to 1,000 lumens.

[0147] If the interior is dark and dusty, it is recommended to output lighting of at least 800 lumens.

[0148] Near-infrared lighting utilizing the 850nm wavelength range allows for the operation of high-power light sources with low power consumption, so there is no problem in increasing lighting output in dark places such as sewer pipes.

[0149] The lighting positioning module (520) determines whether to attach only the main lighting to the upper part of the structure or to attach auxiliary lighting to the lower part of the structure. As shown in Fig. 6, in cases where water flows through a sewer pipe, the drone must illuminate and photograph the water flowing below while in flight. In such an environment, the lighting positioning module (520) determines whether to attach lighting to the upper and lower parts of the structure, respectively.

[0150] The camera direction determination module (530) sets the angle at which the lighting and camera face the wall of the movement space differently depending on the material of the movement space inside the structure.

[0151] For straight passageways like sewer pipes, it's best to illuminate them at a slight 45-degree angle rather than directly at them, and to also aim the camera at a 45-degree angle. Ensure the camera is facing the same direction as the light.

[0152] Shooting at an angle is crucial to avoid white-out results from light reflection. This means aiming for diffuse reflection, not direct reflection. Rough and uneven surfaces on the walls of a structure are particularly prone to diffuse reflection.

[0153] The camera direction determination module (530) sets the angle at which the lighting and camera face the wall differently depending on whether the wall material inside the structure is a rough cave, flat metal, or rough concrete.

[0154] In the case of storage tanks, the walls are made of steel, so near-infrared light with a wavelength of 850 nm can reflect like a mirror and return to the camera. To prevent this, it's recommended to keep the angle between the light and the camera facing the wall less than 45 degrees, around 30 degrees, to prevent reflected light from returning to the camera.

[0155] Conversely, because cave walls have uneven surfaces, near-infrared light can be scattered in various directions by the curved surfaces and returned to the camera. This can result in dark shadows or distortion in the image, or the shape or damage of the subject cannot be clearly revealed. To prevent this, it is recommended to increase the angle between the light and camera toward the wall, typically 60 to 70 degrees, to ensure that the scattered light returns to the camera.

[0156] Therefore, when moving and filming with a near-infrared lighting module and camera mounted on a drone, facing the wall at an angle, the angle at which the lighting module and camera face the wall must vary depending on the characteristics of the wall. This allows the drone to generate optimized images of spatial information within the structure using near-infrared lighting in the 850nm wavelength range.

[0157] In this way, the lighting camera decision unit (500) of the present invention sets the optimal angle at which the lighting and camera face the wall of the moving space according to the material inside the structure.

[0158] The information on the output of the light to be mounted on the drone, the location to attach the light to the drone, and the shooting direction of the camera, determined by the lighting camera decision unit (500), is output so that the worker can check it through the display.

[0159]

[0160] Step 5 (S550)

[0161] In step 4, a near-infrared lighting module is installed on the drone according to the decision.

[0162] The drone has been selected, and the angle at which the lighting and camera will face the walls of the moving space has also been determined. Based on these details, the lighting module and camera are mounted on the selected drone.

[0163] Typically, a lighting module is mounted on top of the drone to facilitate capturing the sides and top of the moving space. However, since sewer pipes have flowing water underneath, and the drone must navigate while maintaining a constant eye on the water level, it's necessary to also capture images from below. Therefore, drones filming flowing sewer pipes should be equipped with cameras and lighting not only on top but also on the bottom.

[0164]

[0165] Step 6 (S560)

[0166] A drone equipped with a near-infrared lighting module flies inside the structure and takes pictures.

[0167] Drone filming experts control the drone while viewing the footage captured and transmitted by the drone's camera in real time.

[0168]

[0169] Step 7 (S570)

[0170] Spatial information about the interior of the structure is constructed by overlaying the images of the interior of the structure captured by the drone onto the 3D point cloud created in the first stage.

[0171] The method of constructing spatial information inside a structure is performed in the following manner.

[0172] A. The drone flies inside the structure and captures orthophotos. An orthophoto is an image in which the direction and height from which it was captured are accurately recorded.

[0173] B. Estimate the drone's position and direction to calculate the coordinates of the orthophoto.

[0174] A. Match the 3D point cloud and the orthoimage using the coordinates of the orthoimage.

[0175] A. A 3D spatial information model is created by synthesizing the matched orthophoto and 3D point cloud.

[0176]

[0177] A 3D spatial information model of the interior of the structure is created using the above method, and once the 3D spatial information model is prepared, a drone is deployed inside at regular intervals (e.g., every few months) and a drone photography expert controls it to take pictures of the interior of the structure, and the captured images are then aligned with the 3D point cloud to create a 3D spatial information model.

[0178] By comparing the 3D spatial information model created in this way with a previously created 3D spatial information model, problems (cracks, collapse, etc.) inside the structure can be identified and diagnosed.

[0179]

[0180] [Indoor spatial information construction system using drones equipped with near-infrared lighting]

[0181] A detailed description is provided of an indoor spatial information construction system (1000) using a drone equipped with near-infrared lighting. Any overlapping information with that described above has been omitted.

[0182] An indoor spatial information construction system (1000) using a drone equipped with near-infrared lighting is configured to include a point cloud generation unit (100), an environmental information generation unit (200), an image quality determination unit (300), a drone size determination unit (400), a lighting camera determination unit (500), a spatial information construction unit (600), and a database (700).

[0183] The image quality determination unit (300) includes an image quality primary determination module (310) and an image quality secondary adjustment module (320).

[0184] The drone size determination unit (400) includes a drone size primary determination module (410) and a drone size final determination module (420).

[0185] The lighting camera decision unit (500) includes a lighting output decision module (510), a lighting position decision module (520), and a camera direction decision module (530).

[0186] The point cloud generation unit (100) generates a 3D point cloud containing location information using data scanned using a LiDAR sensor inside a GNSS (Global Navigation Satellite System) shielding structure to be photographed by a drone.

[0187] The environmental information generation unit (200) receives dust, illuminance, type and condition of the structure inside the structure and generates environmental information.

[0188] At this time, the state of the structure is composed of the size of the movement space inside the structure and the material of the movement space.

[0189] Specifically, the environmental information generation unit (200) receives input from the worker on whether there is a lot / little / normal dust inside the structure, whether the illuminance (brightness) is bright / dark / normal, and the type of structure, such as a sewer pipe, cave, tunnel, or storage tank, and the condition of the structure, and generates environmental information.

[0190] The generated environmental information is recorded in the database (700) of the system (1000).

[0191]

[0192] The image quality determination unit (300) determines the image quality and resolution for constructing spatial information during drone shooting based on the environmental information generated by the environmental information generation unit (200).

[0193] The image quality and resolution of drone footage are determined by reflecting the dust, illuminance, type and condition of the structure inside the structure.

[0194] Specifically, the image quality primary decision module (310) primarily determines the image quality and resolution by considering the dust and illuminance inside the structure, and the image quality secondary adjustment module (320) secondarily adjusts the image quality and resolution by reflecting the size and material of the moving space.

[0195] The image quality and resolution information determined by the image quality determination unit (300) is output through a display so that the operator can check it.

[0196]

[0197] The drone size determination unit (400) determines the size of the drone to be deployed to the structure based on the environmental information generated by the environmental information generation unit (200).

[0198] Specifically, the drone size primary determination module (410) primarily determines the size range of the drone to be deployed into the structure by considering the size (cross-sectional area and length) of the movement space inside the structure.

[0199] And the drone size final decision module (420) makes a final decision based on the type and number of sensors required for dust and illuminance inside the structure among one or more drones that have been primarily decided.

[0200] Information on the size of the drone to be deployed in the structure determined by the drone size determination unit (400) is output through a display so that the worker can check it.

[0201]

[0202] The lighting camera decision unit (500) determines the output of the lighting to be installed on the drone, the location to attach the lighting to the drone, and the shooting direction of the camera based on the environmental information generated by the environmental information generation unit (200).

[0203] The lighting output determination module (510) increases or decreases the lighting output according to the level of illumination inside the structure among the environmental information generated in the first step.

[0204] The lighting positioning module (520) determines whether to attach only the main lighting to the upper part of the structure or to attach auxiliary lighting to the lower part of the structure. As shown in Fig. 6, in cases where water flows through a sewer pipe, the drone must illuminate and photograph the water flowing below while in flight. In such an environment, the lighting positioning module (520) determines whether to attach lighting to the upper and lower parts of the structure, respectively.

[0205] The camera direction determination module (530) sets the angle at which the lighting and camera face the wall of the movement space differently depending on the material of the movement space inside the structure.

[0206] The camera direction determination module (530) sets the angle at which the lighting and camera face the wall differently depending on whether the wall material inside the structure is a rough cave, flat metal, or rough concrete.

[0207] In this way, the lighting camera decision unit (500) of the present invention sets the optimal angle at which the lighting and camera face the wall of the moving space according to the material inside the structure.

[0208] The information on the output of the light to be mounted on the drone, the location to attach the light to the drone, and the shooting direction of the camera, determined by the lighting camera decision unit (500), is output so that the worker can check it through the display.

[0209]

[0210] The spatial information construction unit (600) constructs spatial information inside the structure by overlaying an image of the inside of the structure taken by a drone equipped with a near-infrared lighting module according to the content determined by the lighting camera determination unit (500) onto a 3D point cloud generated by the point cloud generation unit (100).

[0211]

[0212] In the present invention, the near-infrared lighting module uses a wavelength range of 850 nm.

[0213] When exploring underground structures, the majority of environments are dark or low-light, so a high-power light source that can illuminate the interior is required. However, due to limitations in battery and lighting technology and dust generation causing diffuse reflection and obstruction of the field of vision, exploration often becomes impossible.

[0214] Accordingly, a special lighting system for indoor structure detection drones was developed that can operate a high-power light source with low power consumption and minimize dust scattering.

[0215] The present invention is an innovative indoor safety inspection and spatial information construction technology that simultaneously solves the problems of low brightness (flight weight, battery consumption, LED technology limitations) of existing drone light sources and the problem of dust obstruction of vision by utilizing special lighting in the 850 nm wavelength range.

[0216]

[0217] The present invention relates to a method and system for constructing indoor spatial information using a drone equipped with near-infrared lighting, which can secure an optimal image suitable for the environment by determining the image quality and resolution when shooting with a drone according to the environment inside a GNSS (Global Navigation Satellite System) shielding structure to be photographed by a drone, determine the optimal size of drone to be deployed in a structure according to the internal environment of the structure, and comprehensively determine the output of the light to be mounted on the drone, the location to attach the light, the shooting direction of the camera, etc., thereby flying inside the structure using a drone equipped with a near-infrared lighting module having optimal conditions suitable for the environment inside the structure and shooting an image, thereby constructing spatial information inside a GNSS shielding structure.

[0218] Although the present invention has been described in detail with reference to preferred embodiments, it should be understood that the embodiments described above are exemplary in all respects and not restrictive, as those skilled in the art can implement the present invention in other specific forms without changing the technical idea or essential characteristics thereof.

[0219] In addition, the scope of the present invention is defined by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention.

[0220]

[0221] 100...point cloud generation section

[0222] 200...Environmental Information Generation Department

[0223] 300...Video quality decision unit

[0224] 310...Video quality primary decision module

[0225] 320...2nd video quality adjustment module

[0226] 400...Drone size decision part

[0227] 410... Drone size first decision module

[0228] 420...Final decision module for drone size

[0229] 500...Lighting Camera Decision Section

[0230] 510...Lighting output determination module

[0231] 520...Light positioning module

[0232] 530...Camera direction determination module

[0233] 600...Spatial Information Construction Department

[0234] 700... database

[0235] 1000... An indoor spatial information system using drones equipped with near-infrared lighting.

Claims

1. The first step is to scan the interior of a GNSS (Global Navigation Satellite System) shielding structure to be photographed by a drone to create a 3D point cloud, and analyze the interior of the structure to create environmental information; A second step of determining the image quality and resolution for constructing spatial information when shooting with a drone based on the environmental information generated in the first step; A third step of determining the size of the drone to be deployed into the structure based on the environmental information generated in the first step; A fourth step of determining the output of the light to be mounted on the drone, the location to attach the light to the drone, and the shooting direction of the camera based on the environmental information generated in the first step; A fifth step of mounting a near-infrared lighting module on the drone according to the content determined in the fourth step; Step 6: A drone equipped with the above near-infrared lighting module flies inside the structure and takes pictures; and A method for constructing indoor spatial information using a drone equipped with near-infrared lighting, characterized in that it includes a seventh step of constructing spatial information inside a structure by overlaying an image of the inside of the structure taken by the drone onto a three-dimensional point cloud created in the first step.

2. In paragraph 1, The above first step is, A step of scanning the interior of a structure using a LiDAR sensor to obtain a 3D point cloud containing location information; and A method for constructing indoor spatial information using a drone equipped with near-infrared lighting, characterized by including a step of generating environmental information by receiving input of dust, illuminance, type and condition of the structure inside the structure.

3. In paragraph 2, The condition of the above structure is: A method for constructing indoor spatial information using a drone equipped with near-infrared lighting, characterized in that the structure comprises a size of the internal movement space and a material of the movement space.

4. In paragraph 3, In the second step above A method for constructing indoor spatial information using a drone equipped with near-infrared lighting, characterized in that the above image quality and resolution are primarily determined by considering dust and illuminance inside a structure, and secondarily adjusted by reflecting the size and material of the moving space.

5. In paragraph 3, In the third step above A method for constructing indoor spatial information using a drone equipped with near-infrared lighting, characterized in that the size range of drones to be deployed in a structure is first determined by considering the size of the movement space inside the structure, and the final decision is made based on the type and number of sensors required for dust and illuminance inside the structure among one or more drones determined in the first decision.

6. In paragraph 1, In the above 4th step A method for constructing indoor spatial information using a drone equipped with near-infrared lighting, characterized in that the output of lighting is increased or decreased according to the degree of illumination inside the structure among the environmental information generated in the first stage, and the angle at which the lighting and camera face the wall of the moving space inside the structure is set differently according to the material of the moving space.

7. In paragraph 1, A method for constructing indoor spatial information using a drone equipped with near-infrared lighting, characterized in that the above near-infrared lighting module uses a wavelength range of 850 nm.

8. A point cloud generation unit that creates a 3D point cloud using data scanned using a LiDAR sensor inside a GNSS (Global Navigation Satellite System) shielding structure to be photographed by a drone; An environmental information generation unit that generates environmental information by receiving inputs of dust, illuminance, type and condition of the structure inside the structure; An image quality determination unit that determines image quality and resolution for constructing spatial information when shooting with a drone based on the environmental information generated by the environmental information generation unit; A drone size determination unit that determines the size of a drone to be deployed into a structure based on the environmental information generated by the environmental information generation unit; A lighting camera determination unit that determines the output of the lighting to be installed on the drone, the location to attach the lighting to the drone, and the shooting direction of the camera based on the environmental information generated by the environmental information generation unit; and An indoor spatial information construction system using a drone equipped with near-infrared lighting, characterized in that it includes a spatial information construction unit that constructs spatial information of the interior of a structure by overlaying an image of the interior of a structure taken by a drone equipped with a near-infrared lighting module according to the content determined by the above-mentioned lighting camera decision unit onto a 3D point cloud generated by the above-mentioned point cloud generation unit.

9. In paragraph 8, The condition of the above structure is: An indoor spatial information construction system using a drone equipped with near-infrared lighting, characterized by including the size of the internal movement space of the structure and the material of the movement space.

10. In paragraph 9, An indoor spatial information construction system using a drone equipped with near-infrared lighting, characterized in that the image quality determination unit primarily determines the image quality and resolution by considering dust and illuminance inside the structure, and secondarily adjusts the image quality and resolution by reflecting the size and material of the moving space.

11. In paragraph 9, An indoor spatial information construction system using a drone equipped with near-infrared lighting, characterized in that the drone size determination unit first determines the size range of drones to be deployed in a structure by considering the size of the movement space inside the structure, and finally determines the type and number of sensors required for dust and illuminance inside the structure among one or more drones determined in the first stage.

12. In paragraph 8, An indoor spatial information construction system using a drone equipped with near-infrared lighting, characterized in that the above lighting camera decision unit increases or decreases the lighting output according to the degree of illumination inside the structure among the environmental information generated by the environmental information generation unit, and sets the angle at which the lighting and camera face the wall of the moving space differently according to the material of the moving space inside the structure.

13. In paragraph 8, A method for constructing indoor spatial information using a drone equipped with near-infrared lighting, characterized in that the above near-infrared lighting module uses a wavelength range of 850 nm.

Images