3D RECONSTRUCTION SYSTEM AND METHOD FOR IT

Abstract

The present invention relates to a 3D reconstruction technology. A 3D reconstruction system according to one embodiment of the present invention may comprise: a position verification device for checking the position of a vehicle entering a workspace, a scanning device for scanning the interior and exterior of the vehicle entering the workspace while scanning vehicle components located within the workspace, a 3D depth information generation device for generating 3D depth information about the vehicle and vehicle components based on the scan images generated by the scanning device, and a control device for controlling the scanning device based on the information generated by the position verification device and the information generated by the scanning device.

Description

Technical field The present invention relates to 3D reconstruction technology and, in particular, a 3D reconstruction system and a method therefor, which can be applied to an automated vehicle assembly line for which robots and scanning devices are required. Background technology In computer imaging technology, 3D reconstruction is a process of determining the shape and / or appearance of real-world objects. 3D reconstruction can be based on data and / or images of objects captured by various types of sensors. Examples of 3D reconstruction technologies include lidar-based systems, structured light-based systems, stereoscope image-based systems, and the like. A lidar-based system uses the principle of time-of-flight measurement by shining pulsed light into the environment, which is reflected by objects and returns to the sensor's detector, but has disadvantages in that it provides only sparse depth information and does not provide sufficient data for measurements, and is bulky. A structured light-based system projects a structured light pattern onto the surface of a target and simultaneously tracks the light pattern with a camera to calculate the object's position. However, it has disadvantages such as relatively long data acquisition times and difficulty in applying it to large objects at great distances. A stereoscopic image-based system captures three-dimensional depth information from images taken by two cameras arranged in parallel on the same baseline, but has a problem in that it is sensitive to objects without texture and reflective objects, such as those commonly found in manufacturing factories. Conventional 3D reconstruction technology has limitations, as it is not suitable for use in automated vehicle assembly lines that require robots and scanning devices. Therefore, there is a need for 3D reconstruction technology that can capture 3D depth information more efficiently than conventional technologies and can be applied to automated vehicle assembly lines that require robots and scanning devices. This background technology comprises technical information that the inventor had available to derive the present invention or that he acquired during the derivation process of the present invention, and cannot necessarily be referred to as prior art disclosed to the public before the filing of the present invention. revelation Technical problem The embodiment disclosed in the present invention was developed in accordance with the above-mentioned need and addresses a technical problem: to provide a 3D reconstruction system and a method for doing so, which can be used in automated vehicle assembly lines that require robots and scanning devices. Furthermore, the embodiment has a technical problem: providing a 3D reconstruction system and a method for doing so that can perform both the scanning of vehicles and the scanning of vehicle components in a single workspace. Furthermore, the embodiment has a technical problem: providing a 3D reconstruction system and a method for doing so that can capture motion information relating to robot arms / end effectors, which is used to calculate the position of scan sensors of scanning devices in real time during scanning. Furthermore, a technical problem of the embodiment is to provide a 3D reconstruction system and a method for doing so that can capture position change information about mounting points of vehicles / vehicle components. The technical problems of the present invention are not limited to the aspects mentioned above, and other problems covered by the present invention can be clearly understood by those skilled in the art in which the present invention belongs from the following description. Technical solution As a technical means to solve the technical problems described above, a 3D reconstruction system and a procedure for it can be provided, which can be applied to automated vehicle assembly lines that require robots and scanning devices. A 3D reconstruction system according to an embodiment of the present invention may comprise a position checking device configured to check the position of a vehicle entering a workspace, a scanning device configured to scan the interior and exterior of the vehicle entering the workspace while scanning vehicle components located within the workspace, a 3D depth information generation device configured to generate 3D depth information about the vehicle and vehicle components based on scan images generated by the scanning device, and a control device configured to control the scanning device based on information generated by the position checking device and information generated by the scanning device. According to one embodiment, the position checking device may include a marker installed on the vehicle, configured to provide position information about itself, and a position information generation unit configured to generate position and orientation information about the vehicle based on the position information provided by the marker. According to one embodiment, the control device can compare the 3D depth information generated by the 3D depth information generation device with an original CAD model and calculate position changes of mounting points of the vehicle and vehicle components. According to one embodiment, the scanning device may comprise a first scanning device configured to scan the exterior of the vehicle located in a first scan area outside an entrance to the workspace, a second scanning device configured to scan the exterior of the vehicle located in a second scan area adjacent to the entrance of the workspace defined within the workspace, and a third scanning device configured to scan the interior of the vehicle located in a third scan area defined within the workspace, while scanning vehicle components located in a fourth scan area within the workspace. According to one embodiment, the first and second scanning devices can have robot arms equipped with scan sensors, and the third scanning device can have an articulated robot arm with a scan sensor. According to one embodiment, the scanning device can have a plurality of first scanning devices, and the plurality of first scanning devices can scan the exterior of the vehicle within the first scanning area from different positions. According to one embodiment, the scanning device can have a plurality of second scanning devices, and the plurality of second scanning devices can scan the exterior of the vehicle within the second scanning area from different positions. According to one embodiment, the first to third scanning devices can perform scanning operations individually or jointly according to the position of the vehicle. According to one embodiment, a conveying device can be installed within the workspace, and the third scanning device can be installed on the conveying device. According to one embodiment, the third scanning device can generate a scan path based on vehicle-related position and orientation information generated by the position checking device and scan the interior of the vehicle while moving along the scan path. According to one embodiment, the third scanning device can scan the interior of the vehicle while moving according to scan path information provided by the control device. According to one embodiment, if the control device determines, based on position information about the vehicle generated by the position checking device, that the vehicle has left a preset scan area, it can control the third scanning device to scan the vehicle components. According to one embodiment, the third scanning device can generate motion information via a robot arm and an end effector while scanning the interior of the vehicle. According to one embodiment, the third scanning device can generate motion information via a robot arm and an end effector while scanning the vehicle components. According to one embodiment, if the control device determines, based on motion information generated while the third scanning device scans the interior of the vehicle, that a robot arm is in a resting state, it can control the third scanning device to scan the vehicle components. A 3D reconstruction method according to an embodiment of the present invention may comprise checking the position of a vehicle entering a workspace using a position checking device, scanning the interior and exterior of the vehicle entering the workspace using a scanning device, scanning vehicle components located within the workspace using the scanning device, and generating 3D depth information about the vehicle and vehicle components using a 3D depth information generation device based on scan images generated by the scanning device. According to one embodiment, scanning the interior and exterior of the vehicle using the scanning device may include scanning the exterior of the vehicle located in a first scan area outside an entrance to the workspace using a first scanning device, scanning the exterior of the vehicle located in a second scan area adjacent to the entrance of the workspace within the workspace using a second scanning device, and scanning the interior of the vehicle located in a third scan area within the workspace using a third scanning device. According to one embodiment, scanning using the third scanning device can include generating motion information via an articulated robot arm and an end effector contained in the third scanning device while scanning the interior of the vehicle. According to one embodiment, scanning vehicle components using the scanning device can include scanning vehicle components located in a fourth scan area defined within the workspace using a third scanning device. According to one embodiment, the scanning of vehicle components using the scanning device can include generating motion information via an articulated robot arm and an end effector contained in the third scanning device during the scanning of the vehicle components. According to one embodiment, the testing using the position testing device can include a marker installed on the vehicle that provides position information about itself, and a position information generation unit that generates position and orientation information about the vehicle based on position information from the marker. According to one embodiment, the 3D reconstruction method may further include a control device that compares the 3D depth information generated by the 3D depth information generation device with an original CAD model and calculates position changes of mounting points of the vehicle and vehicle components. According to one embodiment, scanning using the first scanning device can be carried out using a plurality of first scanning devices that scan the exterior of the vehicle within the first scan area from different positions. According to one embodiment, scanning using the second scanning device can be carried out using a plurality of second scanning devices that scan the exterior of the vehicle within the second scanning area from different positions. According to one embodiment, scanning the interior and exterior of the vehicle using the scanning device can include the first to third scanning devices performing scan operations individually according to the position of the vehicle. According to one embodiment, scanning the interior and exterior of the vehicle using the scanning device can include the first to third scanning devices performing scan operations jointly according to the position of the vehicle. According to one embodiment, scanning the interior of the vehicle using the third scanning device may include the third scanning device generating a scan path based on vehicle-related position and orientation information generated by the position checking device, and scanning the interior of the vehicle while moving along the scan path. Specific details regarding the means for solving the above-mentioned problems, as described above, are provided in the following description and drawings, based on various examples of the present invention. Beneficial effects According to the present invention, by carrying out path planning for a logistics robot based on a movement area with direction dependency, interferences or crossings between logistics robots can be prevented. According to one embodiment of the present invention, a 3D reconstruction system and a method for it can be provided which can be applied to automated vehicle assembly lines that require robots and scanning devices. According to one embodiment, the time required for scanning vehicles and vehicle components can be reduced, since both scanning vehicles and scanning vehicle components is possible in a single workspace. According to one embodiment, the time required to calculate the positions of scan sensors of scanning devices can be reduced because motion information relating to robot arms / end effectors, which are used to calculate the positions of scan sensors of scanning devices, can be captured in real time during scanning. According to one embodiment, tolerance models for manufacturing defects of parts and processes can be created because position change information about mounting points of vehicles / vehicle components can be captured. The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art based on the following description. The content of the problems to be solved, the means of solving the problems and the effects mentioned above do not specify any essential features of the claims, so that the scope of the rights of the claims is not limited by the facts described in the disclosure. Description of the drawings The drawings below serve to illustrate the embodiments of the present invention and provide embodiments together with detailed descriptions. However, the technical features of the present embodiments are not limited to specific drawings, and the features disclosed in the individual drawings can be combined to form new embodiments.Fig. 1 is a block diagram showing a configuration of a 3D reconstruction system according to one embodiment of the present invention. Fig. 2 is a drawing schematically showing a workspace according to one embodiment of the present invention. Fig. 3 is a drawing showing an example in which a scanning device according to one embodiment of the present invention scans the exterior of a vehicle located in a first scan area. Fig. 4 is a drawing showing an example in which a scanning device according to one embodiment of the present invention scans the exterior and interior of a vehicle located across the first to third scan areas. Fig. 5 is a drawing showing an example in which a scanning device according to one embodiment of the present invention scans vehicle components located in a fourth scan area.Figure 6a is a drawing of a scan sensor contained in a scanning device according to an embodiment of the present invention, viewed from the front. Figure 6b is a drawing schematically showing the scan sensor, viewed from direction A of Figure 6a. Figure 6c is a drawing schematically showing the scan sensor, viewed from direction B of Figure 6a. Figure 7 is a drawing illustrating a 3D reconstruction method according to an embodiment of the present invention. Mode of Revelation The advantages and features of the present invention, as well as methods for its implementation, will become clear with reference to the embodiments described in detail below in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but is implemented in various forms, and the present embodiments serve only to complete the disclosure of the present invention and to fully inform those who possess average knowledge in the field of technology to which the present invention belongs about the scope of the invention. The present invention is defined only by the scope of the claims. The shapes, sizes, ratios, angles, numbers, and the like disclosed in the drawings describing embodiments of the present invention are exemplary, so that the present invention is not limited to the depicted circumstances. Throughout the description, the same reference numerals refer to the same components. In the description of the present invention, detailed descriptions of related known technologies are omitted if it is determined that they might unnecessarily obscure the core of the present invention. When terms such as "has," "has," "is made of," and the like are used in the present description, other parts may be added unless "only" is used. When a component is expressed in the singular, this also includes the case of multiple components unless a specific express description is given. When interpreting components, it is assumed that a fault area is present, even if there is no separate explicit description of the fault area. When describing temporal relationships, if temporal precedence relationships are described as "after", "subsequently", "next", "before" and the like, non-continuous cases may also be included, unless "immediately" or "directly" is used. The terms "first," "second," and the like are used to describe different components, but these components are not limited by these terms. These terms are only used to distinguish one component from another. Therefore, a first component mentioned below can also be a second component within the meaning of the present invention. In describing the components of the present invention, terms such as "first," "second," "A," "B," "(a)," "(b)," and the like may be used. These terms serve only to distinguish the component from other components, and the nature, order, sequence, or number of the respective components are not limited by the term. When it is described that a component is "connected," "coupled," or "linked" to another component, the component may be directly connected or linked to the other component; however, it should be understood that another component may be "interposed" between each component, and may be indirectly connected or linked, unless a specific express description is given. "At least one" should be understood to encompass all combinations of one or more of the associated components. For example, the meaning of "at least one of the first, the second, and the third component" can be understood to include combinations of all components of two or more of the first, the second, and the third component, as well as of the first, the second, or the third component. Each feature of the various embodiments of the present description can be partially or completely combined or combined with one another, and various locking mechanisms and drives are technically possible, and each embodiment can be implemented independently of one another or together in an associated relationship. The following are embodiments of the present invention with reference to the accompanying drawings and embodiments as follows. For the sake of clarity, the scale of the components shown in the drawings differs from the actual scale, so that it is not limited to the scale shown in the drawings. The following describes a method and a system for improving the image quality of a vehicle display device according to an embodiment of the present invention with reference to the accompanying drawings. Fig. 1 is a block diagram showing the configuration of a 3D reconstruction system 1 according to an embodiment of the present invention, and Fig. 2 is a drawing schematically showing a workspace according to an embodiment of the present invention. For the purpose of illustration, the 3D reconstruction system 1 according to an embodiment of the present invention is described below by means of an example in which it performs work on vehicles, but the 3D reconstruction system 1 according to an embodiment of the present invention is not only applicable to vehicles. The 3D reconstruction system 1 according to one embodiment can scan vehicles and vehicle components that are work targets (here: scan targets) and perform a 3D reconstruction based on the scan results in order to generate 3D depth information about vehicles. The 3D Reconstruction System 1 can scan the exterior (or appearance or bodywork) and interior of vehicles. Accordingly, the 3D Reconstruction System 1 can perform exterior vehicle scanning, interior vehicle scanning, and scanning of vehicle components, and generate 3D depth information about vehicles based on the exterior vehicle scan results, the interior vehicle scan results, and the component scan results. The 3D reconstruction system 1 can perform scanning when vehicles and vehicle components are arranged in predefined spaces (or areas). For example, the 3D reconstruction system 1 can track the position of vehicles and perform vehicle scanning based on their location. According to one embodiment, the 3D reconstruction system 1 can perform scanning when vehicles are adjacent to the outside of the entrance of a workspace AR, when vehicles enter and are located in the workspace through the entrance, when vehicles are located in preset areas (e.g., interior scan workspaces) within the workspace, and when vehicle component containers (e.g., pallets) are located in preset areas (or spaces). With reference to Fig. 1 and Fig. 2, the 3D reconstruction system 1 according to one embodiment can comprise a position checking device 100, a scanning device 200, a 3D depth information generation device 300, a control device 400 and a storage device 500, although the configuration of the 3D reconstruction system 1 is not limited thereto. The position check device 100 can check the position of a vehicle V, which is a scan target, and generate information about the position and orientation of the vehicle V. For example, the position check device 100 can generate information about the position and orientation of the vehicle V and transmit this information to the control device 400 and store it in the storage device 500. The 3D reconstruction system 1 according to one embodiment performs a scan of the interior of the vehicle V, and in order to provide space for scanning the interior of the vehicle V, the upper section of the vehicle V has an open area. The position checking device 100 can check the position and orientation of the vehicle V entering the workspace AR in real time. For example, the vehicle V can enter the workspace AR via a conveyor device. According to one embodiment, the vehicle V can enter the workspace AR in a state in which it is mounted on a movable body moved by a conveyor device. According to one embodiment, the position checking device 100 can have a marker 110 attached to the vehicle V and configured to provide position information about itself, and a position information generation unit 120 installed within the working space AR and configured to generate information about the position and orientation of the vehicle V based on position information from the marker 110. For example, the marker 110 can provide 3D coordinate information (X, Y, Z) as position information about itself, and the position information generation unit 120 can be installed in a fixed form within the AR workspace. For example, the position checking device 100 can have a plurality of markings 110 and a plurality of position information generation units 120 and can be implemented as an ultra-wideband (UWB) based system, an ultrasound based system, a laser triangulation based system and the like. According to one embodiment, the information generated by the position checking device 100 can be used to generate optimal paths for scanning and collision avoidance. The scanning device 200 can scan the vehicle V and vehicle components and generate (or output) scan results (or scan images). The scan results can be provided to the 3D depth information generation device 300 and stored in the storage device 500. According to one embodiment, the scanning device 200 can scan the exterior (or appearance or bodywork) and the interior of the vehicle V. The scanning device 200 according to one embodiment can perform scanning of the vehicle exterior, scanning of the vehicle interior and scanning of vehicle components, and can generate vehicle exterior scan results, vehicle interior scan results and vehicle component scan results. According to one embodiment, the scanning device 200 can comprise a first scanning device 210, a second scanning device 220 and a third scanning device 230. Fig. 3 is a drawing showing an example in which the scanning device 200 according to an embodiment of the present invention scans the exterior of a vehicle V located in a first scan area SR1. As shown in Fig. 3, the first scanning device 210 can scan the exterior of the vehicle V, which is located within a first scan area SR1 that is adjacent to the outside of the entrance of the work space AR. For example, the first scanning device 210 can be installed adjacent to the entrance of the workspace AR so that it faces the first scan area SR1. For example, the first scanning device 210 can be installed outside or inside the entrance of the workspace AR so that it faces the first scan area SR1. For example, the first scanning device 210 can be permanently installed at the entrance of the workspace AR in order to scan the vehicle V from a specific direction. Here, “permanently installed” can mean that the first scan area for the first scanning device 210 is not changed during scanning according to the movement of the vehicle V. For example, the first scan area SR1 for the first scanning device 210 may be changed before the scanning process according to a system reset, the size of the vehicle V and the like, but once the scanning process is carried out, the first scan area SR1 may not be changed again before the scanning process is completed. According to one embodiment, the first scanning device 210 may comprise a first-1 scanning device 211 and a first-2 scanning device 212, which are associated and installed at both upper edges of the entrance to the workspace AR; however, it is not limited thereto. For example, the first scanning device 210 can have three or more scanning devices that scan the exterior of the vehicle V within the first scanning area SR1 from different positions. By scanning the exterior of the vehicle V from different directions using a plurality of first scanning devices 211, 212, more accurate scan results can be produced than by scanning the exterior of the vehicle V using a single scanning device. For example, the first-1 scan device 211 and the first-2 scan device 212 can have scan sensors SS1 SS. For example, the first-1 scan device 211 and the first-2 scan device 212 can be in the form of robot arms on which scan sensors SS1 are installed. According to one embodiment, the first scanning device can perform 210 scanning operations in response to trigger signals from the control device 400. Fig. 4 is a drawing showing an example in which the scanning device 200 according to an embodiment of the present invention scans the exterior and interior of a vehicle V arranged over the first to third scan area SR1 to SR3. As shown in Fig. 4, the second scanning device 220 can scan the exterior of the vehicle V, which is located within a second scan area SR2 that is adjacent to the entrance within the workspace AR. If the vehicle V is positioned over the first scan area SR1 and the second scan area SR1, both the first and second scanning devices can perform 220 scans. For example, the second scanning device 220 can be installed adjacent to the entrance within the workspace AR such that it faces the second scan area SR2. According to one embodiment, the second scanning device 220 can also be installed outside the entrance of the workspace AR so that it faces the second scan area SR2. For example, the second scanning device 220 can be fixed in place to scan the vehicle V from a specific direction. Here, “permanently installed” can mean that the second scan area SR2 for the second scan device 220 is not changed during scanning according to the movement of the vehicle V. For example, the second scan area SR2 for the second scan device 220 can be changed before the scanning process according to a system reset, the size of the vehicle V and the like, but once the scanning process is carried out, the second scan area SR2 may no longer be changed before the scanning process is completed. According to one embodiment, the second scanning device 220 can include a second-1 scanning device 221 and a second-2 scanning device 222, which are associated and installed at both upper edges of the entrance to the workspace AR; however, it is not limited to this. For example, the second scanning device 220 can have three or more scanning devices that scan the exterior of the vehicle V within the second scanning area SR2 from different positions. By scanning the exterior of the vehicle V from different directions using a plurality of second scanning devices 221, 222, more accurate scan results can be produced than by scanning the exterior of the vehicle V using a single scanning device. For example, the second-1 scan device 221 and the second-2 scan device 222 can have SS2 scan sensors. For example, the second-1 scan device 221 and the second-2 scan device 222 can have robot arms on which SS2 scan sensors are installed. Thus, the scanning device 200 includes the first scanning device 210 and the second scanning device 220, which scan the exterior of the vehicle V from different directions for different areas SR1 and SR2, and can therefore scan the exterior of the vehicle V from different angles. Since it is thus possible to generate 3D depth information about the vehicle V based on images scanned from different angles, the completeness of the 3D depth information can be improved. According to one embodiment, the second scanning device can perform 220 scan operations in response to trigger signals from the control device 400. As shown in Fig. 4, the third scanning device 230 can scan the interior of the vehicle V, which is located within a preset third scan area SR3 within the workspace AR. Here, the third scan area SR3 can overlap with the second scan area SR2, and if the vehicle V is positioned over the second and third scan areas SR2 and SR3, the second scanning device 220 and the third scanning device 230 can perform scanning operations together for a predetermined time. According to one embodiment, the third scanning device 230 can be rotated in different directions and implemented in a movable form to scan the interior of the vehicle V in detail. For example, the third scanning device 230 can be installed on a conveyor device C, which is provided in the upper section within the work area AR, and can scan the interior of the vehicle V while moving according to the operation of the conveyor device C. For example, the third scanning device 230 and the conveyor device C can be operated according to the control by the control device 400. One or more third scanning devices 230 can be installed within the workspace AR, and in this embodiment, by way of example, two third scanning devices 231, 232 are installed within the workspace AR. According to one embodiment, the conveying devices C1, C2 can be installed at both upper edges within the working space AR parallel to the vehicle entry direction, and one of the third scanning devices 231, 232 can be installed on each of the two conveying devices C1, C2. Of course, the installation structure of the conveyor device C is not limited to this, and, for example, the conveyor device C can be installed in a direction that crosses the vehicle entry direction, or it can be installed in an X-shape. According to this arrangement, the third scanning device 230 can move in conjunction with the operation of the conveyor device C and scan the interior of the vehicle V in more detail. According to one embodiment, the third scanning device 230 can have an articulated robot arm equipped with a scan sensor SS3 SS. When the third scanning device 230, which has an articulated robot arm, moves along the conveyor device C, the scan sensor SS3 of the third scanning device 230 can be brought into a position where high-resolution scanning is possible. According to one embodiment, the third scanning device 230 can scan the interior of the vehicle V while moving along optimal paths generated for scanning and collision avoidance based on information about the position and orientation of the vehicle V. For example, the third scanning device 230 can generate scan paths based on information about the position and orientation of the vehicle V and scan the interior of the vehicle V based on the generated scan paths. For example, the third scanning device 230 can scan the interior of the vehicle V based on scan path information transmitted by the control device 400. According to one embodiment, the third scanning device 230 can generate information relating to the movement of robot arms and end effectors while scanning the interior of vehicles. Motion information about the robot arms and end effectors can be used to calculate the position of the scan sensor SS3 of the third scanning device 230. The third scanning device 230 can generate (or output) vehicle interior scan information including vehicle interior scan images and motion information, and the vehicle interior scan information can be provided to the 3D depth information generation device 300 and stored in the storage device 500. According to one embodiment, the third scanning device can scan 230 vehicle components located in a preset fourth scan area SR4 within the workspace AR. Fig. 5 is a drawing showing an example in which the scanning device 200 according to an embodiment of the present invention scans vehicle components VP that are located in a fourth scan area SR4. Referring to Fig. 5, a plurality of fourth scan areas SR4 can be present within the workspace AR and arranged at the edges within the workspace AR. For example, the fourth scan area SR4 can be positioned so that it does not interfere with the movement of the third scan device 230. For example, the fourth scan area SR4 can be located at a predetermined distance from the upper section of the conveyor device C. Alternatively, the fourth scan area SR4 can also be located on the floor within the work area AR. According to one embodiment, vehicle component containers (e.g., pallets) P, which contain vehicle components VP, can be arranged in the fourth scan area SR4. For example, vehicle component containers P can be arranged in the fourth scan area SR4 by separate transport systems. For example, if transport systems place vehicle component containers P in the fourth scan area SR4 or remove vehicle component containers P that are placed in the fourth scan area SR4, they can supply counting information to the control device 400. For example, when placing vehicle component containers P in the fourth scan area SR4, transport systems can increase the current count information by one (+1) and when removing vehicle component containers P from the fourth scan area SR4, decrease the current count information by one (-1) and then transmit the updated count information to the control device 400. The third scanning device 230 can generate information about the movement of robot arms and end effectors during the scanning of vehicle components VP. This movement information about the robot arms and end effectors can be used to calculate the position of the scan sensor SS3 of the third scanning device 230. The third scanning device 230 can generate (or output) vehicle component scan information, including vehicle component scan images and motion information, and the vehicle component scan information can be provided to the 3D depth information generation device 300 and stored in the storage device 500. The 3D depth information generation device 300 can generate 3D depth information based on scan results (or scan images) produced by the scan device 200. The 3D depth information generated by the 3D depth information generation device 300 can be provided to the control device 400 and stored in the storage device 500. According to one embodiment, the 3D depth information generation device can generate 300 3D depth information about the vehicle V based on vehicle exterior scan images and vehicle interior scan images, and generate 3D depth information about vehicle components VP based on vehicle component scan images. According to one embodiment, the 3D depth information generation device can generate 300 3D depth information pieces about the vehicle V, taking into account motion information about robot arms and end effectors in relation to vehicle interior scan images. According to one embodiment, the 3D depth information generation device can generate 300 3D depth information about vehicle components VP, taking into account motion information about robot arms and end effectors in relation to vehicle component scan images. The control device 400 can receive various pieces of information generated (or output) by components within the 3D reconstruction system 1 and can perform the control of components within the 3D reconstruction system 1. According to one embodiment, the control device 400 can output trigger signals to control the scanning device 200 based on vehicle V-related position information. For example, the control device 400 can output trigger signals to control the first and second scanning devices 210, 220 based on vehicle V-related position information. If the control device 400 determines, based on vehicle V-related position information, that the vehicle V is in the first scan area, it can output trigger signals to the first scan device 210. And if the control device 400 determines, based on vehicle V-related position information, that the vehicle V is in the second scan area, it can output trigger signals to the second scan device 220. According to one embodiment, the control device 400 can control the scanning device 200 based on vehicle V-related position and orientation information generated by the position checking device 100. The control device 400 can generate scan paths for controlling the scan device 200 based on vehicle V-related position and orientation information. For example, the control device 400 can generate scan paths for controlling the third scan device 230 based on the vehicle V-related position and orientation information. Furthermore, the control device 400 can control the conveyor device C while performing the control of the third scanning device 230 based on the scan paths. This means that the control device 400 can control the third scanning device 230 and the conveyor device C in such a way that the scan sensor SS of the third scanning device 230 scans the interior of the vehicle V in detail. According to one embodiment, the control device 400 can, when it determines, based on motion information from robot arms / end effectors, that the robot arm of the third scanning device 230 is in a rest state, when it determines, based on vehicle V-related position information, that the vehicle has left the scan areas, when it determines, based on counting information from transport systems, that new vehicle component containers P are located in the fourth scan area SR4, when requests from higher-level systems are received, and the like, in cases where preset conditions are met, control the third scanning device 230 to perform a scan of vehicle components VP. According to one embodiment, the control device can compare 400 3D depth information with original CAD models and calculate position changes of mounting points of the vehicle V and the vehicle components VP. For example, the control device can calculate 400 position changes of mounting points of the vehicle V / vehicle components VP using subtraction methods. For example, the control device can calculate 400 position changes of mounting points of the vehicle V / vehicle components VP based on six degrees of freedom (DOF) transformation coordinates and rotation (roll, pitch, yaw). The position change information of the mounting points in relation to the vehicle V / vehicle components VP, generated by the control device 400, can be forwarded to higher-level systems and stored in the storage device 500. The position change information about mounting points of vehicles / vehicle components can be used as manufacturing statistics and is useful for automated robot training. The control device 400 can control conveying devices that move the vehicle V into the work area AR or move the vehicle V out of the work area AR. Figures 6a to 6c are drawings showing scan sensors SS included in the scanning device 200 according to an embodiment of the present invention, wherein Figure 6a is a drawing of the scan sensor SS seen from the front, Figure 6b is a drawing showing the scan sensor SS schematically from direction A of Figure 6a, and Figure 6c is a schematic representation of the scan sensor SS from direction B of Figure 6a. Referring to Figs. 6a to 6c, the scan sensor SS according to an embodiment of the present invention can emit light and scan objects based on reflected light that returns after the emitted light has been reflected by objects. According to one embodiment, the scan sensor SS can be configured to include a light source 1100, a camera 1200, a first polarizer 1300, a second polarizer 1400 and a distance sensor 1500; but the configuration of the scan sensor SS is not limited thereto. For example, the light source 1100, the camera 1200, the first polarizer 1300, the second polarizer 1400 and the distance sensor 1500 can be arranged in a main body 1600. The light source 1100 is a component that emits light and can be, for example, an LED (light-emitting diode) that emits infrared or visible light, but is not limited to this. In the main body 1600, a plurality of light sources 1100 can be arranged, and if a plurality of light sources 1100 are arranged, the plurality of light sources 1100 can be set up and arranged in a direction X. The camera 1200 can be positioned adjacent to the side of the light source 1100 and can capture images of objects by receiving light (reflected light) that returns after light emitted by the light source 1100 has been reflected by objects. The camera 1200 forms a pair with the light source 1100 and can be arranged adjacent to the light source 1100, forming a pair. A plurality of cameras 1200 can be arranged in the main body 1600, and if a plurality of cameras 1200 is arranged, the plurality of cameras 1200 can be set up and arranged in exactly one direction X. The light source 1100 and the camera 1200, which form a pair, can be set up and arranged in a different direction Y perpendicular to a direction X. The first polarizer 1300 can be arranged at a predetermined distance from the front of the light source 1100 and can prevent glare from light emitted by the light source 1100. If a plurality of light sources 1100 are arranged in the main body 1600, a plurality of first polarizers 1300 are also arranged accordingly, and the plurality of arranged first polarizers 1300 can be set up and arranged in a direction X. The second polarizer 1400 can be positioned at a predetermined distance from the front of the camera 1200 and can filter natural light that is reflected from objects and enters the camera 1200. If a plurality of cameras 1200 are arranged in the main body 1600, then a plurality of second polarizers 1400 are also arranged accordingly, and the plurality of arranged second polarizers 1400 can be set up and arranged in exactly one direction X. The 1500 distance sensor can measure distances to objects and, for example, measure distances using TOF (time-of-flight) methods, which can also be used to measure distances to distant objects; however, it is not limited to this. For example, the distance sensor 1500 can be set up and arranged in a different direction Y with the camera 1200. Fig. 7 is a drawing to illustrate a 3D reconstruction method according to an embodiment of the present invention. The step-by-step processes shown in Fig. 7 can be carried out by the 3D reconstruction system described with reference to Figs. 1 to 6c. The 3D reconstruction method according to an embodiment of the present invention is described with reference to Fig. 1, Fig. 2, Fig. 3, Fig. 4, Fig. 5, Fig. 6 to Fig. 7, assuming that a vehicle V, which is a work target, moves to a work space AR. First, in step S700, when the vehicle V moves to the workspace AR, the position testing device 100 can check the position of the vehicle V in real time and generate position and orientation information in relation to the vehicle V in real time. The generation of vehicle V-related position and attitude information can be performed continuously until the vehicle V scan is complete. For example, the vehicle V-related position and attitude information can be transmitted to the control device 400 and stored in the storage device 500. If the vehicle V is in the first scan area SR1, the first scanning device 210 can scan the exterior of the vehicle V in step S710. In step S710, the first scanning device 210 can perform the scanning of the vehicle V in response to trigger signals from the control device 400. Scan images (first vehicle exterior scan images) generated by the first scanning device 210 can be provided to the 3D depth information generation device 300 and stored in the storage device 500. If the vehicle V is in the second scan area SR2, the second scanning device 220 can scan the exterior of the vehicle V in step S720. In step S720, the second scanning device 220 can perform the scanning of the vehicle V in response to trigger signals from the control device 400. The scan images generated by scanning the second scanning device 220 (second vehicle exterior scan images) can be fed to the 3D depth information generation device 300 and stored in the storage device 500. If the vehicle V is in the third scan area SR3, the third scanning device 230 can scan the interior of the vehicle V in step S730. In step S730, the third scanning device 230 can generate scan paths based on vehicle V-related position and orientation information generated by the position checking device 100 and scan the interior of the vehicle V according to the generated scan paths. In step S730, the third scanning device 230 can scan the interior of the vehicle V according to the scan path information generated by the control device 400 based on the vehicle V-related position and orientation information generated by the position check device 100. In step S730, the third scanning device 230 can scan the interior of the vehicle V while moving in accordance with the operation of the conveyor device C on which it is installed. The scan images (vehicle interior scan images) generated by scanning the third scanning device 230 can be provided to the 3D depth information generation device 300 and stored in the storage device 500. In step S730, the third scanning device 230 can generate information about the movement of robot arms and end effectors while scanning the interior of vehicle V. The motion information of the robot arms / end effectors generated by the third scanning device 230 can be fed to the 3D depth information generation device 300 and stored in the storage device 500. Subsequently, in step S740, the third scanning device 230 can perform the scanning of vehicle components VP that are located in the fourth scan area SR4. In step S740, the third scanning device 230 can perform the scanning of vehicle components VP in response to instructions from the control device 400. According to one embodiment, if the control device 400 determines, based on motion information from robot arms / end effectors, that the robot arm of the third scanning device 230 is in a rest state, it can output vehicle component scan command signals to the third scanning device 230. According to one embodiment, when the control device 400 determines, based on vehicle V-related position information, that the vehicle has left the scan areas (first to third scan area), it can output vehicle component scan command signals to the third scan device 230. According to one embodiment, if the control device 400 determines, based on counting information from transport systems that transport vehicle components VP into the workspace AR, that new vehicle components VP are located in the fourth scan area SR4, it can output vehicle component scan command signals to the third scan device 230. According to one embodiment, the control device 400 can output vehicle component scan command signals to the third scan device 230 upon requests from higher-level systems. Scan images (vehicle component scan images) generated by the third scanning device 230 can be provided to the 3D depth information generation device 300 and stored in the storage device 500. In step S740, the third scanning device can generate 230 pieces of information relating to the movement of robot arms and end effectors during the scanning of vehicle components VP. The motion information of robot arms / end effectors generated by the third scanning device 230 can be provided to the 3D depth information generation device 300 and stored in the storage device 500. When the scanning of the vehicle V and the vehicle components VP is completed according to steps S710 to S740, the 3D depth information generation device 300 can generate 3D depth information based on the scan images in step S750. In step S750, the 3D depth information generation device can generate 300 3D depth information about the vehicle V based on vehicle exterior scan images and vehicle interior scan images, and generate 3D depth information about vehicle components VP based on vehicle component scan images. In step S750, the 3D depth information generation device 300 can generate 3D depth information about the vehicle V, taking into account robot arm / end effector-related motion information, which relates to the vehicle interior scan images. In step S750, the 3D depth information generation device 300 can generate 3D depth information about vehicle components VP, taking into account robot arm / end effector-related motion information, which refers to vehicle component scan images. The 3D depth information generated by the 3D depth information generation device 300 can be provided to the control device 400 and stored in the storage device 500. Subsequently, in step S760, the control device 400 can compare the 3D depth information with the original CAD models and calculate position changes of the mounting points of the vehicle V and the vehicle components VP. In step S760, the control device can calculate 400 position changes of the vehicle V / vehicle component mounting points using subtraction methods. For example, the control device can calculate 400 position changes of the vehicle V / vehicle component mounting points based on six degrees of freedom (DOF) transformation coordinates and rotation (roll, pitch, yaw). The position change information of the mounting points in relation to the vehicle V / vehicle components VP, generated by the control device 400, can be forwarded to higher-level systems and stored in the storage device 500. According to such an embodiment of the present invention, the time required for scanning the vehicle V / vehicle components VP can be reduced, since both the scanning of the vehicle V and the scanning of the vehicle components VP can be carried out in a single workspace AR. Furthermore, since motion information relating to robot arms / end effectors, which are used to calculate the positions of scan sensors SS of the third scanning device 230, is acquired in real time during the scanning of the interior of the vehicle V and the vehicle components VP, the time required to calculate the positions of the scan sensors SS of the third scanning device 230 can be reduced. Furthermore, since position change information about the mounting points of the vehicle V / vehicle components VP can be obtained by comparing 3D depth information generated based on scan images with original CAD models, tolerance models for manufacturing defects of parts and processes can be created. The embodiments of the present invention have been described in more detail above with reference to the accompanying drawings, but the present invention is not necessarily limited to these embodiments and can be modified and implemented in various ways within its scope without departing from the technical concept of the present invention. Therefore, the embodiments disclosed in this description do not serve to limit the technical concept of the present invention, but rather to illustrate it, and the scope of the technical concept of the present invention is not limited by these embodiments. Therefore, the embodiments described above are to be understood in every respect as exemplary and not as limiting.The scope of protection of the present invention should be interpreted in light of the claims, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of the rights of the present invention. Description of the reference numbers 1 3D reconstruction system 100 Position checking device 110 Marking 120 Position information generation unit 200 Scanning device 210 First scanning device 220 Second scanning device 230 Third scanning device 300 3D depth information generation device 400 Control device 500 Memory

Claims

3D reconstruction system (1) comprising: a position checking device (100) configured to check the position of a vehicle (V) entering a workspace (AR), a scanning device (200) configured to scan the interior and exterior of the vehicle (V) entering the workspace (AR) while scanning vehicle components located within the workspace (AR), a 3D depth information generation device (300) configured to generate 3D depth information about the vehicle and vehicle components based on scan images generated by the scanning device (200), and a control device (400) configured to control the scanning device (200) based on information generated by the position checking device (100) and information generated by the scanning device (200). 3D reconstruction system (1) according to claim 1, wherein the position verification device (100) comprises: a marker (110) installed on the vehicle (V) which is configured to provide position information about itself, and a position information generation unit (120) which is configured to generate position and orientation information about the vehicle based on the position information provided by the marker (110). 3D reconstruction system (1) according to claim 1, wherein the control device (400) is configured to compare the 3D depth information generated by the 3D depth information generation device (300) with an original CAD model and calculates position changes of mounting points of the vehicle (V) and the vehicle components. 3D reconstruction system (1) according to claim 1, wherein the scanning device (200) comprises: a first scanning device (210) configured to scan the exterior of the vehicle (V) located in a first scan area defined outside an entrance of the workspace (AR), a second scanning device (220) configured to scan the exterior of the vehicle (V) located in a second scan area defined adjacent to the entrance of the workspace (AR) within the workspace (AR), and a third scanning device (230) configured to scan the interior of the vehicle (V) located in a third scan area within the workspace (AR), while scanning the vehicle components located in a fourth scan area within the workspace (AR). 3D reconstruction system (1) according to claim 4, wherein the first (210) and the second (220) scanning device have robot arms equipped with scan sensors and the third scanning device (230) has an articulated robot arm with a scan sensor. 3D reconstruction system (1) according to claim 4, wherein the scanning device (200) comprises a plurality of first scanning devices (210) and the plurality of first scanning devices (210) scan the exterior of the vehicle (V) within the first scan area from different positions. 3D reconstruction system (1) according to claim 4, wherein the scanning device (200) has a plurality of second scanning devices (220) and the plurality of second scanning devices (220) scan the exterior of the vehicle (V) within the second scan area from different positions. 3D reconstruction system (1) according to claim 4, wherein the first to third scanning devices (210, 220, 230) are configured to perform scanning operations individually or jointly according to the position of the vehicle (V). 3D reconstruction system (1) according to claim 4, wherein a conveying device is installed within the workspace (AR) and the third scanning device (230) is installed on the conveying device. 3D reconstruction system (1) according to claim 4, wherein the third scanning device (230) is configured to generate a scan path based on the vehicle-related position and orientation information generated by the position checking device (100) and to scan the interior of the vehicle (V) while moving along the scan path. 3D reconstruction system (1) according to claim 4, wherein the third scanning device (230) is configured to scan the interior of the vehicle (V) while moving according to the scan path information provided by the control device (400). 3D reconstruction system (1) according to claim 4, wherein the control device (400) is configured such that, when it determines, based on the position information about the vehicle (V) generated by the position check device (100), that the vehicle (V) has left a preset scan area, it controls the third scan device (230) to scan the vehicle components. 3D reconstruction system (1) according to claim 5, wherein the third scanning device (230) is configured to generate motion information via a robot arm and an end effector while scanning the interior of the vehicle (V). 3D reconstruction system (1) according to claim 5, wherein the third scanning device (230) is configured to generate motion information about a robot arm and an end effector while scanning the vehicle components. 3D reconstruction system (1) according to claim 12, wherein the control device (400) is configured such that, based on the motion information generated while the third scanning device (230) scans the interior of the vehicle (V), it determines that a robot arm is in a resting state, it controls the third scanning device (230) to scan the vehicle components. 3D reconstruction method comprising: checking the position of a vehicle (V) entering a workspace (AR) using a position checking device (100) (S700), scanning the interior and exterior of the vehicle (V) entering the workspace (AR) using a scanning device (200) (S710, S720, S730), scanning vehicle components located within the workspace (AR) using the scanning device (200) (S740), and generating 3D depth information about the vehicle (V) and the vehicle components using a 3D depth information generation device (300) based on scan images generated by the scanning device (200) (S750). 3D reconstruction method according to claim 16, wherein the scanning of the interior and exterior of the vehicle (V) using the scanning device (200) comprises: scanning the exterior of the vehicle (V) located in a first scan area defined outside an entrance of the workspace (AR) using a first scanning device (210) (S710); scanning the exterior of the vehicle (V) located in a second scan area defined adjacent to the entrance of the workspace (AR) within the workspace (AR) using a second scanning device (220) (S720); and scanning the interior of the vehicle (V) located in a third scan area defined within the workspace (AR) using a third scanning device (230) (S730). 3D reconstruction method according to claim 17, wherein the scanning using the third scanning device (230) comprises generating motion information about an articulated robot arm and an end effector contained in the third scanning device (230) during scanning the interior of the vehicle (V). 3D reconstruction method according to claim 16, wherein scanning the vehicle components using the scanning device (200) comprises scanning the vehicle components located in a fourth scan area defined within the workspace (AR) using a third scanning device (230). 3D reconstruction method according to claim 19, wherein the scanning of the vehicle components using the scanning device (200) comprises generating motion information via an articulated robot arm and an end effector contained in the third scanning device (230) during the scanning of the vehicle components. 3D reconstruction method according to claim 16, further comprising comparing the 3D depth information with an original CAD model and calculating position changes of mounting points of the vehicle (V) and the vehicle components.

Images