Electronic device for performing slicing on robot equipped with 3D lidar module and method operation thereof

Abstract

According to various embodiments, an electronic device for performing slicing of a robot equipped with a 3D LiDAR module comprises: a movable robot having a column installed at its top; a 3D LiDAR module installed at one end of the column; and a processor; The processor controls the mobile robot to move in a place where an internal space of a building or a movable internal space is formed, and while the mobile robot is moving, acquires at least one lidar data acquired from the internal space of the building or a movable internal space and point cloud data extracted by the mobile robot at regular intervals through the 3D lidar module, and when the at least one lidar data acquired from each side data located in a vertical direction relative to the direction in which the mobile robot is moving in the internal space of the building or a movable internal space exceeds a preset first threshold value, it determines that the mobile robot has approached any area of ​​the internal space of the building and controls the mobile robot to move to the center of the internal space of the building or a movable internal space, and when the at least one lidar data detects an object in the internal space of the building or a movable internal space and the at least one lidar data exceeds a preset second threshold value, it determines that the object is a minimum distance value capable of detecting the shadow of the object and controls the analysis of the object, and when the mobile robot approaches a preset target point, it is set to stop the movement of the mobile robot.

Description

Technology Field

[0001] Various embodiments of the present invention relate to an electronic device for performing slicing of a robot equipped with a three-dimensional LiDAR module and a driving method thereof. Background Technology

[0002] Recently, mapping technologies are being developed to improve accuracy for various regions and existing buildings. This aims to provide services that allow first-time visitors to easily discover their destinations and find their way based on these mapped technologies.

[0003] To address this, while outdoor areas can be easily mapped using satellite and vehicle mapping services, indoor structures make such mapping impossible. Therefore, mobile robots are used to map indoor and underground penetration holes.

[0004] However, LiDAR modules are generally used for such mapping, but conventionally, 3D LiDAR modules have been used. However, even if objects or sides in the new passageway can be detected, it is difficult to avoid them, and furthermore, when there is shadowing on an object, it is difficult to clearly identify it, which makes mapping difficult.

[0005] Therefore, in order to overcome these problems, there is a need for a system capable of detecting objects or sides and avoiding them while performing clear identification of the objects to enable accurate mapping. Prior art literature

[0006] Korean Patent Publication No. 10-2096096 (March 26, 2020) The problem to be solved

[0007] Accordingly, the present embodiment can provide an electronic device and a driving method thereof that enables accurate three-dimensional mapping while allowing reconstruction to be easily performed, allows slicing into a two-dimensional map based on an accurate three-dimensional map to be easily performed, and controls a mobile robot to accurately detect objects without damage while moving. means of solving the problem

[0008] According to various embodiments, an electronic device for performing slicing of a robot equipped with a 3D LiDAR module comprises: a movable robot having a column installed at its top; a 3D LiDAR module installed at one end of the column; and a processor; The processor controls the mobile robot to move in a place where an internal space of a building or a movable internal space is formed, and while the mobile robot is moving, acquires at least one lidar data acquired from the internal space of the building or a movable internal space and point cloud data extracted by the mobile robot at regular intervals through the 3D lidar module, and when the at least one lidar data acquired from each side data located in a vertical direction relative to the direction in which the mobile robot is moving in the internal space of the building or a movable internal space exceeds a preset first threshold value, it determines that the mobile robot has approached any area of ​​the internal space of the building and controls the mobile robot to move to the center of the internal space of the building or a movable internal space, and when the at least one lidar data detects an object in the internal space of the building or a movable internal space and the at least one lidar data exceeds a preset second threshold value, it determines that the object is a minimum distance value capable of detecting the shadow of the object and controls the analysis of the object, and when the mobile robot approaches a preset target point, it is set to stop the movement of the mobile robot. Effects of the invention

[0009] According to the present embodiment, while a mobile robot moves within a building interior space or a movable interior space, it can easily output a 3D map and a sliced ​​2D map based on various data acquired from an attached 3D LiDAR module, calculate the distance between the mobile robot and an object or side to prevent the mobile robot from colliding with the object or side in advance, and calculate the distance between the object and the mobile robot to easily identify the shadow of the object. Brief explanation of the drawing

[0010] FIG. 1 illustrates a block diagram of an electronic device and network according to various embodiments of the present invention. FIG. 2 is a flowchart illustrating how an electronic device operates according to various embodiments. FIG. 3 is a first flowchart illustrating a method for an electronic device to measure the distance of a wall surface according to various embodiments. FIG. 4 is a second flowchart illustrating a method for an electronic device to measure the distance of a wall surface according to various embodiments. FIG. 5 is an example diagram for visually displaying a mobile robot measuring the distance of a wall surface according to various embodiments. FIG. 6 is a flowchart illustrating a method for an electronic device to measure an object according to various embodiments. FIG. 7 is a first exemplary diagram for visually displaying a mobile robot detecting an object according to various embodiments. FIG. 8 is a second exemplary diagram for visually illustrating a mobile robot detecting an object according to various embodiments. FIG. 9 is an example of 2D and 3D slice mappings generated by an electronic device based on LiDAR data according to various embodiments. Specific details for implementing the invention

[0011] Hereinafter, various embodiments of this document are described with reference to the accompanying drawings. The embodiments and the terms used therein are not intended to limit the technology described in this document to specific embodiments and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In relation to the description of the drawings, similar reference numerals may be used for similar components. A singular expression may include a plural expression unless the context clearly indicates otherwise. In this document, expressions such as "A or B" or "at least one of A or B" may include all possible combinations of items listed together. Expressions such as "first," "second," "first," or "second" may modify said components regardless of order or importance and are used only to distinguish one component from another and do not limit said components. When it is mentioned that a certain (e.g., 1st) component is "(functionally or telecommunicationally) connected" or "connected" to another (e.g., 2nd) component, said certain component may be directly connected to said other component or connected through another component (e.g., 3rd component).

[0012] In this document, "configured to" may be used interchangeably with, depending on the context, for example, hardware- or software-wise, "suitable for," "capable of," "modified to," "made to," "capable of," or "designed to." In some cases, the expression "device configured to" may mean that the device is "capable of" in conjunction with other devices or components. For example, the phrase "processor configured to perform A, B, and C" may mean a dedicated processor for performing the corresponding operations (e.g., an embedded processor), or a general-purpose processor capable of performing the corresponding operations by executing one or more software programs stored in a memory device (e.g., a CPU or application processor).

[0013] An electronic device according to various embodiments of the present document may include, for example, at least one of a smartphone, a tablet PC, a desktop PC, a laptop PC, a netbook computer, a workstation, and a server.

[0014] Referring to FIG. 1, an electronic device (101) within a network environment (100) in various embodiments is described. The electronic device (101) may include a bus (110), a processor (120), a memory (130), an input / output interface (140), a display (150), a communication interface (160), a mobile robot (170), and a three-dimensional LiDAR module (180). In some embodiments, the electronic device (101) may omit at least one of the components or additionally include other components. The bus (110) may include a circuit that connects the components (110-180) to each other and transmits communication (e.g., control messages or data) between the components. The processor (120) may include one or more of a central processing unit, an application processor, or a communication processor (CP). The processor (120) may, for example, perform operations or data processing regarding the control or communication of at least one other component of the electronic device (101).

[0015] The memory (130) may include volatile or non-volatile memory. The memory (130) may store instructions or data related to at least one other component of the electronic device (101), for example. According to one embodiment, the memory (130) may store software or a program (140).

[0016] The input / output interface (140) can, for example, transmit commands or data input from a patient or other external device to other component(s) of the electronic device (101), or output commands or data received from other component(s) of the electronic device (101) to the patient or other external device.

[0017] The display (150) may include, for example, a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting diode (OLED) display, a micro-electromechanical system (MEMS) display, or an electronic paper display. The display (150) may display various content (e.g., text, images, videos, icons, or symbols, etc.) to a patient, for example. The display (150) may include a touch screen and may receive touch, gesture, proximity, or hovering input using, for example, an electronic pen or a part of the patient's body. The communication interface (160) may establish communication between, for example, the electronic device (101) and an external device (e.g., a first external electronic device (102), a second external electronic device (104), or a server (108)). For example, the communication interface (160) can be connected to a network (162) via wireless or wired communication to communicate with an external device (e.g., a second external electronic device (104) or a server (108)).

[0018] Wireless communication may include cellular communication using at least one of, for example, LTE, LTE-A (LTE Advance), CDMA (code division multiple access), WCDMA (wideband CDMA), UMTS (universal mobile telecommunications system), WiBro (Wireless Broadband), or GSM (Global System for Mobile Communications). According to one embodiment, wireless communication may include at least one of, for example, WiFi (wireless fidelity), Bluetooth, Bluetooth Low Energy (BLE), Zigbee, NFC (near field communication), Magnetic Secure Transmission, Radio Frequency (RF), or Body Area Network (BAN). According to one embodiment, wireless communication may include GNSS. GNSS may be, for example, GPS (Global Positioning System), Glonass (Global Navigation Satellite System), Beidou Navigation Satellite System (hereinafter "Beidou"), or Galileo, the European global satellite-based navigation system. Hereinafter, in this document, "GPS" may be used interchangeably with "GNSS". Wired communication may include at least one of, for example, USB (universal serial bus), HDMI (high definition multimedia interface), RS-232 (recommended standard 232), power line communication, or POTS (plain old telephone service).The network (162) may include at least one of a telecommunications network, for example, a computer network (e.g., LAN or WAN), the Internet, or a telephone network.

[0019] Each of the first and second external electronic devices (102, 104, 106) may be the same or a different type of device as the electronic device (101). According to various embodiments, all or part of the operations performed on the electronic device (101) may be performed on one or more other electronic devices (e.g., electronic devices (102, 104, 106), or a server (108). According to one embodiment, when the electronic device (101) needs to perform a function or service automatically or upon request, the electronic device (101) may request at least some of the associated functions from another device (e.g., electronic devices (102, 104, 106), or a server (108)) instead of performing the function or service itself or additionally. The other electronic device (e.g., electronic devices (102, 104, 106), or a server (108)) may perform the requested function or additional functions and transmit the result to the electronic device (101). The electronic device (101) may provide the requested function or service by processing the received result as is or additionally. For this purpose, for example, cloud computing, distributed computing, or client-server computing technologies may be used.

[0020] According to one embodiment, the mobile robot (170) may have a movable robot body (171) and a pillar (173) installed on the top of the robot body (171). According to one embodiment, a three-dimensional LiDAR module (180) may be installed on the top pillar (173). Additionally, the mobile robot (170) is a type of robot capable of independently understanding and moving in an environment, using a sophisticated set of sensors, artificial intelligence, machine learning, and computing for path planning, and can interpret and navigate the environment without being constrained by wired power. Furthermore, since the mobile robot (180) is equipped with cameras and sensors, when it encounters unexpected obstacles such as dropped boxes or crowds of people while navigating the environment, it can use navigation techniques such as collision avoidance to slow down, stop, or find a new path around the object and then continue the work. The mobile robot (180) in this embodiment can move and stop within a building interior space or a movable interior space.

[0021] According to one embodiment, the 3D LiDAR module (180) may be composed of a mapping technology that measures the distance to a target surface using laser light. This allows the 3D LiDAR module (180) to generate a 3D map of everything with remarkable accuracy, ranging from the interior of a building to a wide terrain area such as a movable tunnel. Additionally, the 3D LiDAR module (180) may be composed of two main components: a transmitter and a receiver. First, the transmitter may emit up to hundreds of thousands of laser light pulses in the direction of the surface being scanned. Subsequently, the receiver may receive the signal when the pulses reach the surface, either by returning or backscattering. Additionally, a processor (120) coupled with the 3D LiDAR module (180) may calculate the time it takes for the pulses to travel from the sensor to the target surface and return; the longer the time is set, the further the surface is identified from the 3D LiDAR module (180), and this technology may be called time-of-flight laser scanning. In the case of a mobile or airborne LiDAR system, the 3D LiDAR module (180) has an IMU (an inertial measurement unit composed of an accelerometer, gyroscope, and other sensors) and a GPS tracker for obtaining the XYZ coordinates of the transceiver at each point in time when a laser pulse is transmitted, and in the case of a mobile system inside an indoor space, data can be acquired by applying a triangulation algorithm and a Voronoi diagram algorithm. In this embodiment, the 3D LiDAR module (180) may apply a triangulation algorithm and a Voronoi diagram algorithm. In addition, unlike a 2D LiDAR module, the 3D LiDAR module (180) can simultaneously output and receive reflected light in various directions, thus having a significant advantage.

[0023] FIG. 2 is a flowchart illustrating how an electronic device operates according to various embodiments.

[0024] FIG. 9 is an example of 2D and 3D slice mappings generated by an electronic device based on LiDAR data according to various embodiments.

[0026] In operation 201, the electronic device (101) (e.g., the processor (120) of FIG. 1) can drive the mobile robot (170) to move within a building interior space or a movable interior space formed therein. According to one embodiment, the electronic device (101) can control the mobile robot (170) to move automatically from a starting point to a destination point within a building interior space or a movable interior space (e.g., a cave, a hole penetrating underground), as shown in FIG. 9. Additionally, the electronic device (101) may include a deep learning model to enable the mobile robot (170) to proceed autonomously. Here, the deep learning model can be described in more detail in operation 203.

[0027] In operation 203, the electronic device (101) (e.g., the processor (120) of FIG. 1) can acquire at least one lidar data acquired from an internal space of a building or a movable internal space through a 3D lidar module (180) while the mobile robot (170) is moving, and point cloud data extracted by the mobile robot (170) at regular intervals. According to one embodiment, the electronic device (101) can acquire at least one lidar data through a 3D lidar module (180) as shown in FIG. 9, and can acquire point cloud data extracted at regular intervals through a sensor module (not shown). Additionally, the point cloud data may be data used as a basis for the Voronoi diagram algorithm described later. Additionally, the electronic device (101) can accurately extract a lidar map in the form of a three-dimensional image (see (a) in FIG. 9) by inputting at least one lidar data and point cloud data into a deep learning model as shown in FIG. 9, and can output a plan view in the form of a two-dimensional image (see (b) in FIG. 9) by applying a slicing mode.

[0028] According to another embodiment, the deep learning model can be trained based on multiple LiDAR data and point cloud data, multiple 3D LiDAR maps, and multiple 2D LiDAR planar maps. Specifically, the deep learning model may utilize at least one of AlexNet, LENET-5, NIN, VGGNet, ResNet, WideResNet, GoogleNet, FractaNet, DenseNet, FitNet, RitResNet, HighwayNet, MobileNet, and DeeplySupervisedNet as a CNN structure. More specifically, LeNet-5 is the most recent model among LeNet models, created in the 1990s at Yann LeCun's laboratory, and can be used to recognize zip codes or numbers. Furthermore, the key point is that the LeNet structure is not significantly different from current CNNs, and it uses convolution and subsampling, and can be connected via fully-connected feature maps that are flattened into a straight line. Next, AlexNet is the model that won ILSVRC 2012, and it can be considered to have revolutionized deep learning models at that time. This is because AlexNet, with its CNN structure, was able to significantly reduce the top 5 errors of the past. Subsequently, AlexNet marked the beginning of the application of CNN techniques in image neural networks. While it proceeds with this structure, a unique feature here is that instead of applying multiple filters at once, it splits the process to two sides, allowing for analysis using two GPUs. ZFNet is very similar in structure to AlexNet; in fact, it achieved improved performance with only minor modifications to the parameters used in AlexNet. This demonstrates that to effectively train a CNN, one can examine how the filters are learned in the intermediate layers.As another example, GoogLeNet is a model that won LSVRC 2014. Compared to AlexNet, it has a deeper depth and a thicker width, but it can be seen that the number of parameters has been significantly reduced. This is because the concept of the Inception Module was introduced in GoogLeNet, and the Inception Module was applied based on the concept that if filter operations are performed non-linearly instead of linearly, more information can be found. The structure of GoogLeNet consists of a layer similar to a network within the network structure, and by forming this into a Network In Network (NIN) structure, it can be non-linear. Additionally, the deep learning model applies the structure of a CNN, and examples of this are not limited. Furthermore, the deep learning model can additionally learn the types of multiple mobile robots (170) and the movement lines of the mobile robots (170) so that the mobile robot (170) can move automatically. This can be applied so that the mobile robot (170) can move accurately to the starting point and the target point while being able to drive automatically using a deep learning model in the electronic device (101).

[0029] In operation 205, an electronic device (101) (e.g., processor (120) of FIG. 1) can acquire at least one LiDAR data and point cloud data from each side located vertically relative to the direction in which the mobile robot (170) moves within the building interior space or the movable interior space. According to one embodiment, if the at least one LiDAR data acquired from each side located vertically relative to the direction in which the mobile robot (170) moves within the building interior space or the movable interior space exceeds a preset first threshold value, the electronic device (101) can determine that the mobile robot (170) has approached any area within the building interior space and control the mobile robot (170) to move to the center of the building interior space or the movable interior space. This allows the electronic device (101) to automatically control the mobile robot (170) so that it does not collide as much as possible when approaching multiple sides as illustrated in FIG. 5 by extracting the distances on both sides, checking the distances of adjacent sides, and determining whether there is a possibility of colliding with the internal structure of a wall or side part through a preset first threshold value, and to prevent this in advance. The operation of the electronic device (101) can be specifically explained based on FIG. 3 to FIG. 5 below.

[0031] FIG. 3 is a first flowchart illustrating a method for an electronic device to measure the distance of a wall surface according to various embodiments.

[0033] In operation 301, the electronic device (101) (e.g., the processor (120) of FIG. 1) can calculate each first wall distance value by applying a triangulation algorithm from at least one LiDAR data through a 3D LiDAR module (180) to each side or object located in a vertical direction relative to the direction in which the mobile robot (170) moves within the building interior space or movable interior space. According to one embodiment, the electronic device (101) can detect the sides or walls at both ends located in a vertical direction relative to the direction of movement within the building by driving the 3D LiDAR module (180) in real time while the mobile robot (170) is moving. At this time, the electronic device (101) can calculate each first wall distance value by applying a triangulation algorithm based on at least one LiDAR data to the direct distance value between the mobile robot (170) and the sides or walls at both ends. Here, triangulation may be a method in which, given a point to be surveyed and two reference points, the angle formed by the base and the other two sides of the triangle formed by the point and the two reference points is measured, the length of the side is measured, and then the coordinates, distance, and even height of the point to be surveyed are extracted through calculations using the sine law, etc. Afterwards, the electronic device (101) can store the extracted first wall distance value in memory (e.g., memory (130) of FIG. 1).

[0034] In operation 303, the electronic device (101) (e.g., the processor (120) of FIG. 1) can calculate a second wall distance value for each side or object located in a vertical direction relative to the direction in which the mobile robot moves within the building interior space or movable interior space by applying a Voronoi diagram algorithm from at least one LiDAR data. According to one embodiment, the electronic device (101) can detect the side or wall at both ends located in a vertical direction relative to the direction of movement within the building by driving a 3D LiDAR module (180) in real time while the mobile robot (170) is moving. At this time, the electronic device (101) can calculate a second wall distance value, which is half the value of the distance between the two sides or walls, by applying a Voronoi diagram algorithm based on at least one LiDAR data to the direct distance value between the mobile robot (170) and the side or wall at both ends. Here, the Voronoi diagram algorithm is composed of a diagram that divides a plane into a set of points closest to a specific point, and can be configured to divide the plane by connecting the two closest points among the points on the plane and drawing the perpendicular bisectors of the next lines. Additionally, the Voronoi diagram algorithm refers to a method of creating boundary lines by dividing the space closest to each point based on several points (obstacles) on the plane, and can be an algorithm that serves to divide the plane into several cells (Voronoi Cells) based on the locations of the obstacles. Furthermore, each Voronoi Cell represents the area closest to the obstacle, and since the area is created centered on the point where the obstacle is located, when there are two obstacles, the boundary line of the area (= Voronoi Edge) can be formed to almost coincide with the midpoint between the adjacent obstacles, and the mobile robot (170) in this embodiment can calculate a second wall distance value to calculate a safe path when planning a path.Afterward, the electronic device (101) can store the extracted second wall distance value in memory (e.g., memory (130) of FIG. 1).

[0035] In operation 305, the electronic device (101) (e.g., the processor (120) of FIG. 1) can determine whether the first wall distance value is greater than the second wall distance value. According to one embodiment, the electronic device (101) can determine whether the first wall distance value, calculated by triangulation of the distance between the adjacent side or object and the mobile robot (170), is greater than the second wall distance value, which is half the distance between the side or object. Subsequently, based on this, the electronic device (101) can determine whether the mobile robot (170) is close to one side or wall without moving to the center line, based on the distances of both sides.

[0036] In operation 307, the electronic device (101) (e.g., the processor (120) of FIG. 1) may determine that the distance between the mobile robot and the side is the second wall distance value when the first wall distance value is determined to be smaller than the second wall distance value. According to one embodiment, if the first wall distance value is 30 cm and the second wall distance value is 50 cm, the electronic device (101) may determine that the first wall distance value is smaller than the second wall distance value and determine that the distance between the mobile robot (170) and the wall is 50 cm. This allows the electronic device (101) to control the mobile robot (170) to move to the center line by recognizing that the distance to the wall or object is very close.

[0037] Meanwhile, in operation 305, if the electronic device (101) (e.g., the processor (120) of FIG. 1) determines that the first wall distance value is greater than the second wall distance value, in operation 309, the electronic device (101) (e.g., the processor (120) of FIG. 1) can determine that the first wall distance value is greater than twice the value of the second wall distance. According to one embodiment, if the first wall distance value is 60 cm and the second wall distance value is 50 cm, the electronic device (101) can determine that the first wall distance value is greater than the second wall distance value. Subsequently, the electronic device (101) can determine that the first wall distance value, calculated by triangulation of the distance between the side and the mobile robot (170), is greater than twice the value of the second wall distance value, which is the distance between the side or the object. Based on this, the electronic device (101) can determine whether the mobile robot (170) is moving to the other side or wall without moving to the center line based on the distance between the two sides.

[0038] In operation 311, the electronic device (101) (e.g., the processor (120) of FIG. 1) may determine that the distance between the mobile robot and the side is the first wall distance value when the first wall distance value is determined to be greater than the second wall distance value but less than twice the second wall distance value. According to one embodiment, if the first wall distance value is 40 cm and the second wall distance value is 30 cm and twice the second wall distance value is 60 cm, the electronic device (101) may determine that the first wall distance value is greater than the second wall distance value but less than twice the second wall distance value, and determine that the distance between the mobile robot (170) and the wall is 40 cm. This allows the electronic device (101) to determine that the mobile robot (170) can maintain a constant distance between objects in both directions and continuously perform the work of the mobile robot (170).

[0039] Meanwhile, in operation 309, if the electronic device (101) (e.g., the processor (120) of FIG. 1) determines that the first wall distance value is greater than twice the second wall distance value, in operation 313, the electronic device (101) (e.g., the processor (120) of FIG. 1) may determine that the distance between the mobile robot and the side is twice the second wall distance value. According to one embodiment, if the first wall distance value is 70 cm and the second wall distance value is 30 cm and twice the second wall distance value is 60 cm, the electronic device (101) may determine that the first wall distance value is greater than twice the second wall distance value and determine that the distance between the mobile robot (170) and the wall is 60 cm. This allows the electronic device (101) to control the mobile robot (170) to move to the center line by recognizing that the distance to the wall or object is very far, but the distance to the wall or object in another direction is close.

[0041] FIG. 4 is a second flowchart illustrating a method for an electronic device to measure the distance of a wall surface according to various embodiments.

[0042] FIG. 5 is an example diagram for visually displaying a mobile robot measuring the distance of a wall surface according to various embodiments.

[0044] In operation 401, the electronic device (101) (e.g., the processor (120) of FIG. 1) can calculate a first wall distance value for the side closest to the other side located in the vertical direction relative to the direction in which the mobile robot moves within the building interior space or movable interior space from at least one lidar data. According to one embodiment, the electronic device (101) can detect the side or wall at both ends located in the vertical direction relative to the direction of movement within the building by driving a 3D lidar module (180) in real time while the mobile robot (170) is moving. At this time, the electronic device (101) can calculate a third wall distance value by applying a value of the lidar data reflected from at least one lidar data to the side or object (B1) closest to the direct distance value between the mobile robot (170) and the side or wall at both ends. Additionally, the electronic device (101) may store a third wall distance value, which is a distance value from an adjacent side or object (B1), in a memory (e.g., memory (130) of FIG. 1). Additionally, the method for calculating the third wall distance value may be calculated in the same way as described in FIG. 6, which will be described later.

[0045] In operation 403, the electronic device (101) (e.g., the processor (120) of FIG. 1) can calculate a second wall distance value by applying a Voronoi diagram algorithm from at least one LiDAR data for each side (B1, B2) located vertically relative to the direction in which the mobile robot moves within the building interior space or the movable interior space. Since this operation is identical to operation 303, it may be omitted without separate explanation.

[0046] In operation 405, the electronic device (101) (e.g., the processor (120) of FIG. 1) can calculate a fourth wall distance value by subtracting a first wall distance value from a second wall distance value. According to one embodiment, the electronic device (101) can subtract a third wall distance value, which captures the adjacent side or object (B1), from the second wall distance value calculated by applying a Voronoi diagram algorithm as another method of determining whether the mobile robot (170) is clearly close to the adjacent side or object (B1), and can output the calculated fourth wall distance value. Afterward, the electronic device (101) can store the fourth wall distance value in memory (e.g., the memory (130) of FIG. 1).

[0047] In operation 407, the electronic device (101) (e.g., the processor (120) of FIG. 1) can determine whether the fourth wall distance value exceeds the first threshold value. According to one embodiment, as shown in FIG. 5, the electronic device (101) can output a fourth wall distance value by subtracting the third wall distance value from the second wall distance value, which is the distance between the adjacent side or object (B) and the mobile robot (170). Subsequently, the electronic device (101) can determine whether the extracted fourth wall distance value exceeds the first threshold value by comparing it with a preset first threshold value. According to another embodiment, the first threshold value may include a preset distance percentage value based on the central portion of the distance between both sides located vertically relative to the direction in which the mobile robot (170) moves within the building interior space or the movable interior space. Specifically, the first threshold value can be set to 30% on each side relative to the central part, and can be set to 40%, 30%, and 20% respectively based on the distance value (e.g., second wall distance value of 1M or less, between 1M and 5M, or 5M or more) relative to the central part. Afterward, the electronic device (101) can determine whether the fourth wall distance value exceeds the corresponding range based on the first threshold value set by the manager.

[0048] In operation 409, if the electronic device (101) (e.g., the processor (120) of FIG. 1) determines that the mobile robot is not approaching the side when it determines that the fourth wall distance value does not exceed the first threshold value. According to one embodiment, if the fourth wall distance value is 68 cm, the second wall distance value is 1 M, and the percentage of the first threshold value is 40%, the electronic device (101) can determine that the fourth wall distance value does not exceed the first threshold value and is approaching the central part, and can perform the work while maintaining the operation of the mobile robot (170). Meanwhile, the electronic device (101) can control the mobile robot (170) to move as close to the center line as possible by taking into account its own thickness and width.

[0049] Meanwhile, in operation 407, if the electronic device (101) (e.g., the processor (120) of FIG. 1) determines that the third wall distance value exceeds the first threshold value, in operation 411, the electronic device (101) (e.g., the processor (120) of FIG. 1) may determine that the mobile robot has approached any area of ​​the building interior space. According to one embodiment, if the fourth wall distance value is 1.1M, the second wall distance value is 2M, and the percentage of the first threshold value is 30%, the electronic device (101) may determine that the fourth wall distance value exceeds the first threshold value and determine that there is a possibility of colliding with an adjacent side or object (B1).

[0050] In operation 413, the electronic device (101) (e.g., the processor (120) of FIG. 1) can control the mobile robot to move to an intermediate area of ​​the building's interior space. According to one embodiment, the electronic device (101) can control the movement of the mobile robot (170) by checking the current direction of movement of the mobile robot (170) and changing the direction of movement of the mobile robot (170) to a center line. This allows the electronic device (101) to move toward an adjacent side or object (B1), thereby preventing the mobile robot (170) from colliding with it in advance.

[0053] In operation 207, an electronic device (101) (e.g., the processor (120) of FIG. 1) can acquire at least one LiDAR data and point cloud data for an object detected by a mobile robot in a building interior space or a movable interior space. According to one embodiment, the electronic device (101) can control the analysis of an object by determining a minimum distance value capable of detecting the object's shadow when at least one LiDAR data detects an object in a building interior space or a movable interior space and at least one LiDAR data exceeds a preset second threshold value. This allows the electronic device (101) to automatically control the mobile robot (170) to detect an object, as illustrated in FIG. 7 and FIG. 8, to extract the distance to the object, confirm that the object has a shadow, and control the mobile robot (170) to maintain a minimum distance so that the part containing the object's shadow structure can be detected through a preset second threshold value. The operation of such an electronic device (101) can be specifically explained based on Figures 6 to 8 below.

[0055] FIG. 6 is a flowchart illustrating a method for an electronic device to measure an object according to various embodiments.

[0056] FIG. 7 is a first exemplary diagram for visually displaying a mobile robot detecting an object according to various embodiments.

[0057] FIG. 8 is a second exemplary diagram for visually illustrating a mobile robot detecting an object according to various embodiments.

[0059] In operation 601, an electronic device (101) (e.g., the processor (120) of FIG. 1) can detect an object (P) in a building interior space or a movable interior space from at least one LiDAR data. According to one embodiment, the electronic device (101) can confirm the presence of an object (P) based on at least one LiDAR data obtained in various directions through a 3D LiDAR module (180), as shown in FIG. 7 and FIG. 8. At this time, the 3D LiDAR module (180) can emit light in the form of a laser, and the angle value between each laser can be set to 15 degrees as the most efficient angle and can be configured as a fixed value as a unique function. Additionally, as shown in FIG. 8, the electronic device (101) can calculate an angle between a specific LiDAR value from the point where the object (P) is in contact with the floor based on the angle value between the lasers.

[0060] In operation 603, the electronic device (101) (e.g., the processor (120) of FIG. 1) can calculate the distance value between the object (P) and the mobile robot (170) based on the following [Equation 1].

[0061] [Mathematical Formula 1]

[0062]

[0063] Here, is the angle between a specific LiDAR value and the point where the object (P) is in contact with the floor, and is the height value of the 3D LiDAR module (180) relative to the floor, and is the distance value between the object (P) and the mobile robot (170).

[0064] According to one embodiment, the electronic device (101) may have a height value of a three-dimensional LiDAR module (180) with respect to the floor that is automatically fixed, and the most efficient height is within an error range of 50 cm. It can be set as follows. This means that the value of the mobile robot (170) can be equal to the height value of the 3D LiDAR module (180) relative to the floor, and since the angle between the point where the object (P) is in contact with the floor and the specific LiDAR value is constant, the distance value between the object (P) and the mobile robot (170) can be easily calculated based on this.

[0065] In operation 605, the electronic device (101) (e.g., the processor (120) of FIG. 1) can determine whether the distance value between the object (P) and the mobile robot (170) has exceeded a preset second threshold value. According to one embodiment, the second threshold value may be set differently depending on the height of the object (P). Specifically, the second threshold value may be set to 50 cm when the height of the object (P) is 30 cm or less, 70 cm when it is between 30 cm and 1 M, and 90 cm when it is 1 M or more. Additionally, the second threshold value may be determined as a minimum distance value for accurately detecting the shadow (S) of the object (P), and 85.7 cm may be calculated as the most efficient value for this. Subsequently, the electronic device (101) can determine whether the distance value between the object (P) and the mobile robot (170) has exceeded the second threshold value set by the operator.

[0066] In operation 607, if the electronic device (101) (e.g., the processor (120) of FIG. 1) determines that the distance value between the object (P) and the mobile robot (170) does not exceed a preset second threshold value, the mobile robot (170) can be moved until the mobile robot (170) exceeds the second threshold value. According to one embodiment, as shown in FIG. 8, if the second threshold value is set to 50 cm and the distance value between the object (P) and the mobile robot (170) is detected to be 30 cm, the electronic device (101) can determine that the distance value between the object (P) and the mobile robot (170) does not exceed the second threshold value. Subsequently, the electronic device (101) can control and move the mobile robot (170) until the distance value between the object (P) and the mobile robot (170) exceeds the second threshold value, and can return to operation 603 to perform a repetitive operation.

[0067] Meanwhile, in operation 605, if the electronic device (101) (e.g., the processor (120) of FIG. 1) determines that the distance value between the object (P) and the mobile robot (170) exceeds a preset second threshold value, in operation 609, the electronic device (101) (e.g., the processor (120) of FIG. 1) can continuously acquire at least one LiDAR data for a certain period of time or longer. According to one embodiment, as shown in FIG. 8, if the second threshold value is set to 50 cm and the distance value between the object (P) and the mobile robot (170) is detected to be 60 cm, the electronic device (101) can determine that the distance value between the object (P) and the mobile robot (170) exceeds the second threshold value. Subsequently, the electronic device (101) can continuously acquire at least one data in real time to determine whether the object (P) is fixed. This is because it is possible to determine whether an object (P) is a fixed object or a dynamic object (e.g., an animal), reflect it on the map if it is a fixed object, and ignore it if it is a dynamic object.

[0068] In operation 611, the electronic device (101) (e.g., the processor (120) of FIG. 1) can determine whether the time at least one lidar data is detected has exceeded a preset third threshold. According to one embodiment, the third threshold may be set to 10 seconds while emitting a laser toward an object (P), set to 5 seconds while emitting a laser toward an object (P), set to 15 seconds while emitting a laser toward an object (P), or may be varied and set by an operator in other ways. Subsequently, the electronic device (101) can determine in real time whether the time at least one lidar module is detected has exceeded based on the third threshold set by the manager.

[0069] In operation 613, if the electronic device (101) (e.g., the processor (120) of FIG. 1) determines that the time at least one lidar data is detected does not exceed a preset third threshold, it determines that the object (P) is a dynamic object and can ignore the object (P) without calculating its distance. According to one embodiment, if the electronic device (101) determines that the time at least one lidar data is detected in real time is 4 seconds and the third threshold is set to 10 seconds, it determines that the time at least one lidar data is detected does not exceed a preset third threshold. This allows the electronic device to determine that the detected object (P) is a mobile animal and ignore it without mapping it to a 3D and 2D map.

[0070] Meanwhile, in operation 611, if the electronic device (101) (e.g., the processor (120) of FIG. 1) determines that the detected time exceeds a preset third threshold, in operation 615, the electronic device (101) (e.g., the processor (120) of FIG. 1) determines that the object is a fixed object and can map the object to exist inside the building. According to one embodiment, if the electronic device (101) determines that the time of at least one lidar data that detects the object (P) in real time is 15 seconds or longer, and the third threshold is set to 10 seconds, the electronic device (101) may determine that the time of at least one lidar data that detects the object (P) in real time exceeds a preset third threshold. This can be configured to determine that the detected object (P) is a fixed object and to map it to a 3D and 2D map to reflect it inside the building.

[0072] In operation 209, the electronic device (101) (e.g., the processor (120) of FIG. 1) can stop the movement of the mobile robot (170) when the mobile robot (170) approaches a pre-specified target point. According to one embodiment, the electronic device (101) can automatically stop the movement of the mobile robot (170) when the mobile robot (170) arrives at a target point pre-set by the manager, after going through operations 201 and 203 and the process driven in FIG. 3 through 8. This allows the electronic device (101) to recognize that the role of the mobile robot (170) has ended and can reduce the inconvenience of wasting separate energy.

[0073] In operation 211, an electronic device (101) (e.g., processor (120) of FIG. 1) can transmit at least one LiDAR data and point cloud data to a server of a control system (e.g., server (108) of FIG. 1) through a communication interface (e.g., communication interface of FIG. 1). According to one embodiment, the electronic device (101) can transmit at least one data and point cloud data acquired while the mobile robot (170) is moving to a management server (e.g., server (108) of FIG. 1), and can also transmit a 3D map and a 2D map generated through a deep learning model, as shown in FIG. 9. Subsequently, the server of the control system (e.g., server (108) of FIG. 1) can easily verify the movement path of an internal space of a building or a movable internal space based on the data received from the electronic device (101).

[0075] According to the present embodiment, while the mobile robot (170) moves within a building interior space or a movable interior space, it can easily generate a 3D map and a sliced ​​2D map based on various data obtained from an attached 3D LiDAR module (180), and by calculating the distance between the mobile robot (170) and an object or side, it can prevent the mobile robot (170) from colliding with the object or side in advance, and by calculating the distance between the object and the mobile robot (170), it can easily identify the shadow of the object.

[0077] According to various embodiments, an electronic device for performing slicing of a robot equipped with a 3D LiDAR module comprises: a movable robot having a column installed at its top; a 3D LiDAR module installed at one end of the column; and a processor; The processor controls the mobile robot to move in a place where an internal space of a building or a movable internal space is formed, and while the mobile robot is moving, acquires at least one lidar data acquired from the internal space of the building or a movable internal space and point cloud data extracted by the mobile robot at regular intervals through the 3D lidar module, and when the at least one lidar data acquired from each side data located in a vertical direction relative to the direction in which the mobile robot is moving in the internal space of the building or a movable internal space exceeds a preset first threshold value, it determines that the mobile robot has approached any area of ​​the internal space of the building and controls the mobile robot to move to the center of the internal space of the building or a movable internal space, and when the at least one lidar data detects an object in the internal space of the building or a movable internal space and the at least one lidar data exceeds a preset second threshold value, it determines that the object is a minimum distance value capable of detecting the shadow of the object and controls the analysis of the object, and when the mobile robot approaches a preset target point, it is set to stop the movement of the mobile robot.

[0078] According to various embodiments, the processor is configured to calculate a first wall distance value for each side or object located in a vertical direction relative to the direction in which the mobile robot moves within the building interior space or movable interior space by applying a triangulation algorithm from the at least one lidar data, and to calculate a second wall distance value which is half the distance between sides by applying a Voronoi diagram algorithm to each side or object located in a vertical direction relative to the direction in which the mobile robot moves within the building interior space or movable interior space.

[0079] According to various embodiments, the processor is configured to determine the distance between the mobile robot and the side as the second wall distance value when the first wall distance value is smaller than the second wall distance value when the first wall distance value is compared with the first wall distance value and the second wall distance value is compared with the first wall distance value and the second wall distance value is compared with the first wall distance value and the first wall distance value is greater than the second wall distance value but less than twice the value of the second wall distance value when the first wall distance value is compared with the first wall distance value and the second wall distance value is compared with the first wall distance value and the second wall distance value is compared with the first wall distance value and the second wall distance value is compared with the first wall distance value and the second wall distance value when the first wall distance value is greater than twice the value of the second wall distance value when the distance between the mobile robot and the side is determined to be twice the value of the second wall distance value.

[0080] According to various embodiments, the first threshold value includes a pre-set distance percentage value based on the central portion of the distance between both sides located in a vertical direction based on the direction in which the mobile robot moves in the building interior space or movable interior space, and the processor calculates the third wall distance value for the adjacent side among the distances between both sides located in a vertical direction based on the direction in which the mobile robot moves in the building interior space or movable interior space from the at least one LiDAR data, calculates the second wall distance value set as half the length value of the distance between both sides located in a vertical direction based on the direction in which the mobile robot moves in the building interior space or movable interior space, compares the fourth wall distance value obtained by subtracting the third wall distance value from the second wall distance value with the first threshold value, and is configured to determine that the mobile robot has approached any area of ​​the building interior space when the fourth wall distance value exceeds the first threshold value.

[0081] According to various embodiments, the processor detects the object from the at least one LiDAR data, calculates a distance value between the detected object and the mobile robot based on the following [Equation 1], and is configured to control the analysis of the object by determining a minimum distance value capable of detecting the shadow of the object when the distance value between the mobile robots exceeds the second threshold value.

[0082] [Mathematical Formula 1]

[0083]

[0084] Here, is an angle between a specific LiDAR value and the point where the object contacts the floor, and is the height value of the 3D LiDAR module relative to the floor, and is the distance value between the object and the mobile robot.

[0085] According to various embodiments, when the processor detects the object from the at least one lidar data, it continuously acquires the at least one lidar data for a certain period of time or longer, and if it determines that the object is detected from the at least one lidar data at or above a preset third threshold value, it determines that the object is a stationary object, and if it determines that the object is detected from the at least one lidar data at or below the third threshold value, it determines that the object is a dynamic object, and is configured not to calculate the distance for the object determined to be a dynamic object.

[0086] According to various embodiments, a communication interface is further included; and the processor is configured to transmit the at least one lidar data and the point cloud data to a server of a control system through the communication interface.

[0088] As used in this document, the terms “module” or “part” include a unit composed of hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. “Module” or “part” may be a component formed integrally or a minimum unit or part thereof that performs one or more functions. “Module” or “part” may be implemented mechanically or electronically and may include, for example, an application-specific integrated circuit (ASIC) chip, field-programmable gate arrays (FPGAs), or programmable logic device known or to be developed that performs certain operations, and may be executed by a processor (120). At least part of the device (e.g., modules or functions thereof) or method (e.g., operations) according to various embodiments may be implemented as instructions stored in a computer-readable storage medium (e.g., memory (130)) in the form of a program module. When the above instruction is executed by a processor (e.g., processor (120)), the processor may perform a function corresponding to the above instruction. Computer-readable recording media may include a hard disk, a floppy disk, a magnetic medium (e.g., magnetic tape), an optical recording medium (e.g., CD-ROM, DVD, magneto-optical medium (e.g., floptical disk), built-in memory, etc. Instructions may include code generated by a compiler or code that can be executed by an interpreter. A module or program module according to various embodiments may include at least one of the aforementioned components, some of which may be omitted, or additionally include other components. Operations performed by a module, program module, or other components according to various embodiments may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0089] Furthermore, the embodiments disclosed in this document are presented for the purpose of explaining and understanding the disclosed technical content and are not intended to limit the scope of this disclosure. Accordingly, the scope of this disclosure should be interpreted to include all modifications or various other embodiments based on the technical concept of this disclosure.

[0091] [Private Information]

[0092] This invention is a result of the support of the following research and development project.

[0093] - Project ID: GRRC TUKorea2023-B03

[0094] - Research Project Name: Gyeonggi Regional Cooperation Research Center Project (GRRC)

[0095] - Organizing Institution: Korea University of Engineering

[0096] - Research Management Agency: Gyeonggi Provincial Government

[0097] - Research Period: July 1, 2024 – June 30, 2025

Claims

Claim 1 An electronic device for performing slicing of a robot equipped with a 3D LiDAR module, comprising: a movable robot having a column installed on its upper end; a 3D LiDAR module installed at one end of the column; and a processor; The processor controls the mobile robot to move in a location where an interior space of a building or a movable interior space is formed, and while the mobile robot is moving, acquires at least one LiDAR data acquired from the interior space of the building or the movable interior space and point cloud data extracted by the mobile robot at regular intervals through the 3D LiDAR module, and if the at least one LiDAR data acquired from each side data located in a vertical direction relative to the direction of movement of the mobile robot in at least one of the interior space of the building or the movable interior space exceeds a preset first threshold value, it determines that the mobile robot has approached any area of ​​the interior space of the building and controls the mobile robot to move to the center of the interior space of the building or the movable interior space, and if the at least one LiDAR data detects an object in at least one of the interior space of the building or the movable interior space and the at least one LiDAR data exceeds a preset second threshold value, it determines that the object is at least a distance value capable of detecting the shadow of the object and controls the analysis of the object, and if the mobile robot approaches a preset target point, it controls the movement of the mobile robot to stop. A first wall distance value is calculated for each side or object located in a vertical direction relative to the direction in which the mobile robot moves in at least one of the building interior space or movable interior space, by applying a triangulation algorithm from the at least one LiDAR data, andAn electronic device configured to calculate a second wall distance value, which is half the distance between sides, by applying a Voronoi diagram algorithm from at least one LiDAR data to each side or object located vertically relative to the direction in which the mobile robot moves within the building interior space or movable interior space. Claim 2 delete Claim 3 An electronic device according to claim 1, wherein the processor is configured to determine the distance between the mobile robot and the side as the second wall distance value when the first wall distance value is smaller than the second wall distance value as a result of comparing the first wall distance value and the second wall distance value, and determine the distance between the mobile robot and the side as the first wall distance value when the first wall distance value is larger than the second wall distance value but smaller than twice the second wall distance value as a result of comparing the first wall distance value and the second wall distance value, and determine the distance between the mobile robot and the side as twice the second wall distance value when the first wall distance value is larger than twice the second wall distance value as a result of comparing the first wall distance value and the second wall distance value. Claim 4 An electronic device according to claim 3, wherein the first threshold value includes a pre-set distance percentage value based on the central portion of the distances on both sides located in a vertical direction based on the direction in which the mobile robot moves in at least one of the building interior space or movable interior space, and the processor calculates the third wall distance value for the adjacent side among the distances on both sides located in a vertical direction based on the direction in which the mobile robot moves in the building interior space or movable interior space from the at least one LiDAR data, calculates the second wall distance value set as half the length value of the distances on both sides located in a vertical direction based on the direction in which the mobile robot moves in the building interior space or movable interior space, compares the fourth wall distance value obtained by subtracting the third wall distance value from the second wall distance value with the first threshold value, and is configured to determine that the mobile robot has approached any area of ​​the building interior space when the fourth wall distance value exceeds the first threshold value. Claim 5 An electronic device according to claim 1, wherein the processor detects the object from the at least one LiDAR data, calculates a distance value between the detected object and the mobile robot based on the following [Equation 1], and, when the distance value between the mobile robots exceeds the second threshold value, determines the distance value as the minimum distance value capable of detecting the shadow of the object, and is configured to control the analysis of the object. [Equation 1] Here, is an angle between a specific LiDAR value and the point where the object contacts the floor, and is the height value of the 3D LiDAR module relative to the floor, and is the distance value between the object and the mobile robot. Claim 6 An electronic device according to claim 5, wherein the processor, when detecting the object from the at least one lidar data, continuously acquires the at least one lidar data for a certain period of time or longer, and when it is determined that the object is detected from the at least one lidar data at or above a preset third threshold value, determines that the object is a fixed object, and when it is determined that the object is detected from the at least one lidar data at or below the third threshold value, determines that the object is a dynamic object, and is configured not to calculate the distance for the object determined to be a dynamic object. Claim 7 An electronic device according to claim 1, further comprising a communication interface; wherein the processor is configured to transmit the at least one lidar data and the point cloud data to a server of a control system through the communication interface.

Images