Vehicle step difference measurement method, apparatus, and computer program

Abstract

A step difference measurement method according to an embodiment of the present application comprises the steps of: acquiring a surface image of a vehicle including a boundary area of a plurality of panels; setting a plurality of reference points facing each other on the basis of the boundary area in the surface image and cropping a first area including all of the plurality of reference points; extracting, from the first area, first and second point clouds corresponding to the plurality of reference points, respectively, and defining a coordinate system by using same; redefining the defined coordinate system by extracting a point cloud of one area on the basis of the defined coordinate system and correcting one or more unit vectors constituting the coordinate system by using same; extracting a point cloud of one area on the basis of the redefined coordinate system and projecting same onto one plane; and measuring a step difference of the vehicle by using data projected onto the one plane. According to an embodiment of the present invention, a step difference may be accurately measured on the basis of an image captured in an environment in which the distance between a measurement sensor and a vehicle is long.

Description

Vehicle step measurement method, device and computer program

[0001] The present application relates to a vehicle step measurement method, device and computer program, and more particularly, the present application relates to a vehicle step measurement method, device and computer program for measuring the gap and flush between vehicle panels based on an acquired image.

[0002] The gap between panels that make up a vehicle serves as a criterion for assessing the product's completeness. Therefore, to ensure high-quality products, gap measurements are performed during the production process. Traditionally, this involved manually touching or inserting measuring tools into gaps in the vehicle. However, this method relied on the skill of the person performing the measurement, delaying the unmanned process.

[0003] Accordingly, one method for unmanned operation has been proposed, which utilizes a non-contact optical tool, a step measurement sensor, and a robot. Step measurement requires a sensor that measures the vehicle surface, but the sensors used in existing step measurement solutions perform measurements at a close distance, approximately 10 cm away from the target (vehicle) to be measured. Step measurement sensors typically measure two dimensions (depth and width), so they require an environment where the measurement surface and the sensor are set parallel. However, in actual industrial environments, it is not easy to position the target completely parallel to the sensor. When the target is placed on a conveyor belt and moves, it is difficult to maintain parallelism, so a robot is used to achieve parallelism.

[0004] However, using robots to measure the height of vehicles on moving conveyor belts requires securing sufficient space to accommodate the robot's operating range and size. Furthermore, implementing robotic systems requires extensive safety equipment (safety nets, safety mats, indicators, etc.) due to safety regulations, making them only feasible in workplaces with ample space. Therefore, the development of technology capable of measuring the height of moving products within confined spaces is essential.

[0005]

[0006] The problem to be solved by the present invention is to provide a step measurement method, device and system capable of accurately measuring a step based on an image captured in an environment where the distance between the measurement sensor and the vehicle is far.

[0007] In addition, a problem to be solved by the present invention is to provide a method for measuring the step of a vehicle moving on a conveyor belt using a fixed measuring device.

[0008] In addition, the task to be solved by the present invention is to provide a step measurement method, device and system having a minimum error range.

[0009] The problems to be solved by the present invention are not limited to the problems described above, and problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present invention pertains from this specification and the attached drawings.

[0010] A step measuring method according to one embodiment of the present invention may include the steps of: acquiring a surface image of the vehicle including a boundary region of a plurality of panels; setting a plurality of reference points opposing each other based on the boundary region in the surface image, and cropping a first region including all of the plurality of reference points; extracting first and second point clouds corresponding to each of the plurality of reference points in the first region, and defining a coordinate system using the same; extracting a point cloud of an area based on the defined coordinate system, and using the same to correct one or more unit vectors constituting the coordinate system, thereby redefining the defined coordinate system; extracting a point cloud of an area based on the redefined coordinate system, and projecting the point cloud onto a plane; and measuring a step of the vehicle using data projected onto the plane.

[0011] According to one embodiment of the present invention, a step measurement device includes: an image collection unit that acquires a surface image of the vehicle including boundary regions of a plurality of panels; a processor that measures a step of the vehicle using the surface image; wherein the processor may include: an image processing unit that sets a plurality of reference points opposing the boundary regions in the surface image and crops a first region that includes all of the plurality of reference points; a surface coordinate system estimation unit that extracts first and second point clouds corresponding to each of the plurality of reference points in the first region, defines a coordinate system using the first and second point clouds, extracts a point cloud of an area based on the defined coordinate system, and corrects one or more unit vectors constituting the coordinate system using the extracted point cloud to redefine the defined coordinate system; and a step calculation unit that projects a point cloud corresponding to the first area based on the redefined coordinate system onto a plane and measures a step of the vehicle using data projected onto the plane.

[0012] The solutions to the problems of the present invention are not limited to the solutions described above, and solutions that are not mentioned can be clearly understood by a person having ordinary skill in the art to which the present invention pertains from this specification and the attached drawings.

[0013] According to one embodiment of the present invention, a step difference can be accurately measured based on an image captured in an environment where the distance between the measurement sensor and the vehicle is far.

[0014] Additionally, according to one embodiment of the present invention, the step of a vehicle moving on a conveyor belt can be measured even with a fixed measuring device.

[0015] Additionally, according to one embodiment of the present invention, the error in step measurement can be minimized.

[0016] Fig. 1 is a block diagram of a step measurement device according to one embodiment of the present application.

[0017] Fig. 2 is a flowchart showing a step measurement method according to one embodiment of the present application.

[0018] Figure 3 is a flowchart detailing a preprocessing step according to one embodiment of the present application.

[0019] FIG. 4 is a drawing illustrating an example of a result of a preprocessing step according to one embodiment of the present application.

[0020] FIG. 5 is a drawing for explaining an area to be preprocessed according to one embodiment of the present application.

[0021] Figure 6 is a flowchart detailing the coordinate system redefinition step according to one embodiment of the present application.

[0022] Figure 7 is a flowchart illustrating a first correction step according to one embodiment of the present application.

[0023] Figure 8 is a flowchart illustrating a second correction step according to one embodiment of the present application.

[0024] Fig. 9 is a flowchart detailing a step measurement step according to one embodiment of the present application.

[0025] FIG. 10 is a drawing for explaining the definition of a unit vector according to one embodiment of the present application.

[0026] Fig. 11 is a drawing for explaining a step measurement method according to one embodiment of the present application.

[0027] FIG. 12 is a drawing for explaining a step measurement system according to one embodiment of the present application.

[0028] The above-described purposes, features, and advantages of the present application will become more apparent through the following detailed description, taken in conjunction with the accompanying drawings. However, as the present application is susceptible to various modifications and various embodiments, specific embodiments will be illustrated in the drawings and described in detail below.

[0029] Throughout the specification, identical reference numbers, in principle, indicate identical components. Furthermore, components with identical functions within the scope of the same concept shown in the drawings of each embodiment are described using the same reference numbers, and redundant descriptions thereof will be omitted.

[0030] If a detailed description of a known function or configuration related to this application is deemed to unnecessarily obscure the gist of this application, such detailed description will be omitted. Furthermore, numbers (e.g., "first," "second," etc.) used throughout the description of this specification are merely identifiers used to distinguish one component from another.

[0031] In addition, the suffixes "module" and "part" for components used in the following examples are given or used interchangeably only for the convenience of writing the specification, and do not have distinct meanings or roles in themselves.

[0032] In the examples below, singular expressions include plural expressions unless the context clearly indicates otherwise.

[0033] In the examples below, terms such as “include” or “have” mean that a feature or component described in the specification is present, and do not preclude the possibility that one or more other features or components may be added.

[0034] For convenience of explanation, the sizes of components in the drawings may be exaggerated or reduced. For example, the sizes and thicknesses of each component shown in the drawings are arbitrarily shown for convenience of explanation, and the present invention is not necessarily limited to what is shown.

[0035] In some embodiments, where implementations are otherwise feasible, the order of specific processes may differ from the order described. For example, two processes described in succession may be performed substantially simultaneously, or in a reverse order from the order described.

[0036] In the following examples, when components are said to be connected, this includes not only cases where the components are directly connected, but also cases where components are interposed between the components and are indirectly connected.

[0037] For example, when it is said in this specification that components, etc. are electrically connected, it includes not only cases where the components, etc. are directly electrically connected, but also cases where components, etc. are interposed in between and are indirectly electrically connected.

[0038] Meanwhile, in this specification, 'step' can be understood as a concept that includes both the gap and the height difference (flush) between vehicle panels, and when a distinction is necessary, it is described as the gap and the height difference (flush).

[0039]

[0040] Hereinafter, the step measurement method, device and system of the present application will be described with reference to FIGS. 1 to 11.

[0041] Fig. 1 is a block diagram of a step measurement device (100) according to one embodiment of the present application. The step measurement device (100) is a device for measuring a step (a gap and a flush) between a plurality of panels, and is a device that can measure a step by acquiring a three-dimensional image of a vehicle surface including a plurality of panels and a step area and then analyzing the image.

[0042] A step measurement device (100) according to one embodiment of the present application may include an image collection unit (110), a processor (130), and may further include a storage unit (150) for storing data, and a communication unit (170) for transmitting measurement results to a server, etc.

[0043]

[0044] The image collection unit (110) can acquire a surface image of a vehicle including a boundary area of ​​a plurality of panels. The image collection unit (110) is an image acquisition device called a 3D camera or a 3D vision sensor, and may encompass devices that acquire 3D images including depth information, such as a 3D structured light camera, a ToF (Time-of-Flight) camera, and a stereo vision camera. The image collection unit (110) of the present invention is not limited by an image acquisition method or generation method, as long as it is a 3D measuring device that can acquire a point cloud - a collection of data points belonging to a 3D space.

[0045]

[0046] The processor (130) measures the step using the surface image acquired from the image collection unit (110). More specifically, the processor (130) may include an image processing unit (131), a preprocessing unit (133), a surface coordinate system estimation unit (135), and a step calculation unit (137). The processor (130) may load and execute a program for the overall operation of the step measurement device (100) from the storage unit (150). The processor (130) may be implemented as an AP (Application Processor), a CPU (Central Processing Unit), an MCU (Microcontroller Unit), or a similar device according to hardware, software, or a combination thereof. In this case, the hardware may be provided in the form of an electronic circuit that processes an electrical signal to perform a control function, and the software may be provided in the form of a program or code that drives a hardware circuit.

[0047]

[0048] The image processing unit (131) can set a plurality of reference points facing each other based on a boundary area in a surface image, and crop a first area including all of the reference points. At this time, the first area may be a rectangular shape, such as the red square box (R100) illustrated in FIG. 4, but the shape of the first area is not necessarily limited to the present embodiment. The plurality of reference points may be set by a machine learning framework for detecting reference points or a user setting. It is sufficient that the plurality of reference points are two or more points located at opposite positions based on the boundary area, and they do not have to be exactly symmetrical around the boundary area. However, the machine learning framework for detecting reference points may be a framework trained to detect two points symmetrical around a boundary area corresponding to a step, or a framework trained to detect two points on a straight line perpendicular to the boundary area. In other words, it may be a framework trained with an image including two points on a straight line perpendicular to the boundary area and / or two points symmetrical around the boundary area.

[0049]

[0050] The preprocessing unit (133) can remove noise from the cropped first region. Hereinafter, the noise removal method of the preprocessing unit (133) will be described with reference to FIG. 4. First, the preprocessing unit (133) can Gaussian blur the first region cropped by the image processing unit (131), as in R310. Next, the preprocessing unit (133) can detect an outline region. For example, the preprocessing unit (133) can detect an outline using an outline detection algorithm such as the Canny Edge algorithm, as in R330. Next, a dilation filter can be applied to the image from which the outline is detected, as in R350, to set the outline as the outline region. In this case, the algorithms and filters used in the outline detection or outline region setting process of the present invention are not limited to the examples described above, and other methods that can produce similar effects can also be utilized.

[0051] The preprocessing unit (133) can distinguish the region using connected components and labeling after detecting the outline region, and as a result, can obtain an image such as R370. Next, the preprocessing unit (133) can remove the point cloud of the third region excluding the outline region and the plurality of second regions each including a plurality of reference points in the first region. For example, referring additionally to FIG. 5, when an elemental image of the first region is generated as in FIG. 5 by the processing of the preprocessing unit (133), the second region (a) including the reference point (A) and the second region (b) including the reference point (B) can be maintained in the elemental image. In addition, the outline region can also be maintained. The preprocessing unit (133) can remove the point cloud corresponding to the third region, and the result can be derived as R390. That is, the third region is the remaining part of the first region excluding the second region and the outline region, which has a similar position and shape to the boundary region, but is different from the boundary region and is therefore distinguished. The reason why the preprocessing unit (133) removes the point cloud corresponding to the third area is because the point cloud corresponding to the third area is likely to be data caused by an anomaly existing at the boundary during the automobile step measurement. The point cloud or depth image results collected by the 3D camera device tend to include noise and artifacts. Here, noise can be understood as an electrical interference signal caused by the camera sensor or equipment, and artifacts can be understood as an anomaly captured due to the optical phenomenon of the projection and imaging system and the decoding of depth data. The object that the preprocessing unit (133) of the present invention seeks to remove is an artifact, which may be caused by mutual reflection between surfaces and distortion of the contrast discontinuity of the surface.That is, the preprocessing unit (133) of the present invention can reduce step measurement errors in areas where there is a sharp change from a surface with high absorption to a reflective surface, such as a point where the color changes from black to white, by removing the point cloud corresponding to the third area.

[0052]

[0053] Referring back to FIG. 1, the surface coordinate system estimation unit (135) can extract first and second point clouds corresponding to each of a plurality of reference points in the first region and use them to define a coordinate system. If the preprocessing unit (133) removes noise from the first region and transmits it, the surface coordinate system estimation unit (135) can extract first and second point clouds from the first region from which noise has been removed and use them to define a coordinate system. The surface coordinate system estimation unit (135) can extract a point cloud of an area based on the defined coordinate system and use it to correct one or more unit vectors constituting the coordinate system to redefine the defined coordinate system.

[0054] For example, referring to FIG. 10, the surface coordinate system estimation unit (135) can extract data of a first point cloud corresponding to a reference point (A) and a second point cloud corresponding to a reference point (B), and define a coordinate system using the two point clouds. First, the surface coordinate system estimation unit (135) can define a vector generated by the first and second point clouds as a base orient vector, and calculate a normal unit vector from each point cloud. Next, the surface coordinate system estimation unit (135) can define a normal unit vector of a point specified by the user among the normal unit vectors (the reference point (B) in the example of FIG. 10) as a base normal unit vector. In addition, a unit vector orthogonal to the above-described base orient vector and the reference normal unit vector can be defined as an approach unit vector. The surface coordinate system estimation unit (135) can recalculate the orient unit vector using the normal unit vector and the approach unit vector in order to raise the coordinate system by one dimension - to make it a homogeneous coordinate system. More specifically, since the homogeneous coordinate system has the characteristic that each vector is orthogonal, the orient unit vector is recalculated by dividing the vector product of the normal unit vector and the approach unit vector by the magnitude (scalar value) of the orient vector. The surface coordinate system estimation unit (135) can define a coordinate system using the orient unit vector, normal unit vector, and approach unit vector recalculated in this way.

[0055] After defining the coordinate system, the surface coordinate system estimation unit (135) can redefine the coordinate system to estimate a coordinate system parallel to the vehicle surface. The coordinate system redefinition can be performed as follows.

[0056] The surface coordinate system estimation unit (135) can extract only the point cloud (third point cloud) within a certain area (certain range) based on the defined coordinate system by masking it. Here, the area can vary depending on the user settings, and the size of the area affects the subsequent calculation of the average value of the normal vector and the coordinate system correction using it. In other words, if the size of the area is set to be large, an average result of a larger area can be obtained, and if the size is set to be small, a local result of the corresponding location can be obtained.

[0057] Meanwhile, the surface coordinate system estimation unit (135) can correct the unit vector of the coordinate system using the third point cloud (first correction). To explain the first correction in more detail, the surface coordinate system estimation unit (135) can extract a normal vector corresponding to the third point cloud and calculate an average of the vectors. The surface coordinate system estimation unit (135) can define the calculated average normal vector as a normal vector of the defined coordinate system and recalculate the approach unit vector based on this. Next, the orienting unit vector can be recalculated using the average normal vector and the recalculated approach unit vector in order to express the existing coordinate system again as a homogeneous coordinate system. Then, the recalculated approach unit vector is corrected again using the recalculated orienting unit vector and the unit average normal vector. The surface coordinate system estimation unit (135) can redefine the coordinate system through this operation (first redefinition).

[0058] After redefining the coordinate system, the surface coordinate system estimation unit (135) performs the task of extracting a point cloud (fourth point cloud) of an area based on the redefined coordinate system again. The surface coordinate system estimation unit (135) can use the fourth point cloud to correct the unit vector of the first redefined coordinate system (second correction), and use this to perform a second redefinition to redefine the first redefined coordinate system again.

[0059] To explain the second correction in more detail, the surface coordinate system estimation unit (135) can project the fourth point cloud onto the orient-approach plane of the first redefined coordinate system and cluster the projected data. As a result, multiple clusters can be generated, and the surface coordinate system estimation unit (135) can calculate the misalignment angle between the generated multiple clusters using SVM (Support Vector Machine), etc. Next, the surface coordinate system estimation unit (135) can rotate the orienting unit vector and the approach unit vector in the opposite direction of the misalignment angle between the two clusters based on the axis of the normal vector. Next, the surface coordinate system estimation unit (135) can redefine the coordinate system using the rotated orienting unit vector, the approach unit vector, and the normal unit vector (secondary redefinition). Thereafter, the surface coordinate system estimation unit (135) can extract a point cloud of an area based on the finally redefined coordinate system and transmit it to the step calculation unit (137).

[0060]

[0061] The step calculation unit (137) projects a point cloud of an area extracted based on a redefined coordinate system onto a plane, and can measure the step of a vehicle using the data projected onto the plane. Here, the plane onto which the point cloud is projected may be a protrusive-normal plane of the final redefined coordinate system, and the area projected onto the plane may be an area in the shape of a rectangular box.

[0062] The step calculation unit (137) can divide the projected data into multiple clusters and calculate support vectors by applying a support vector machine (SVM) to the multiple clusters. Furthermore, the support vectors can be used to calculate the distances between the multiple clusters. The step calculation unit (137) can define the distances between clusters thus calculated as gaps.

[0063] Furthermore, the step calculation unit (137) can generate a straight line corresponding to one of multiple clusters, and can calculate the height difference (flush) using the distance between this straight line and data of other clusters. More specifically, the step calculation unit (137) can calculate the height difference (flush) by calculating all distances between the straight line fitted to one cluster and data of other clusters, and then calculating the average value of the distances.*

[0064] Referring to the example of Fig. 11, the step calculation unit (137) can generate a straight line fitting to cluster 1, where this straight line can be understood as a reference surface parallel to the vehicle surface. The step calculation unit (137) can determine the average value of the distance between the straight line corresponding to cluster 1 and the data constituting cluster 2 as a height difference.

[0065]

[0066] The storage unit (150) of the step measurement device (100) can store various types of information. Various types of data can be temporarily or semi-permanently stored in the storage unit (150). Examples of the storage unit (150) may include a hard disk drive (HDD), a solid state drive (SSD), a flash memory, a read-only memory (ROM), a random access memory (RAM), etc. The storage unit (150) may be provided in a form that is built into the step measurement device (100) or a detachable form. The storage unit (150) may store various types of data necessary for the operation of the step measurement device (100), including an operating system (OS) for driving the step measurement device (100) or a program for operating each component of the step measurement device (100).

[0067]

[0068] The communication unit (170) of the step measurement device (100) can communicate with any external device including a server. In addition, for example, when the image collection unit (110) is configured separately from the step measurement device (100) (i.e., when the components of the step measurement device (100) are configured with the processor (130), the storage unit (150), and the communication unit (170), the surface image of the vehicle including the boundary area can be received from the image collection unit (110). In another embodiment, when part of the step measurement function of the processor (130) is performed in a separate terminal, the surface image of the vehicle collected by the image collection unit (110) can be transmitted to an external device that performs the function of the processor (130).

[0069] In addition, the step measurement device (100) can transmit and receive various data by connecting to a network through a communication unit (170). The transmission and reception unit can largely include a wired type and a wireless type. Since the wired type and the wireless type each have their own advantages and disadvantages, the step measurement device (100) may be equipped with both a wired type and a wireless type at the same time depending on the case. Here, in the case of the wireless type, a communication method of the WLAN (Wireless Local Area Network) series such as Wi-Fi can be mainly used. Alternatively, in the case of the wireless type, a communication method of the cellular communication series such as LTE or 5G can be used. However, the wireless communication protocol is not limited to the examples described above, and any appropriate wireless type of communication method can be used. Representative examples of the wired type include LAN (Local Area Network) or USB (Universal Serial Bus) communication, and other methods are also possible.

[0070]

[0071] Hereinafter, a step measurement method according to an embodiment of the present application will be described with reference to FIGS. 2 to 9.

[0072] Fig. 2 is a flowchart illustrating a step measurement method according to one embodiment of the present application. Referring to Fig. 2, the device can acquire a surface image of the vehicle including the boundary area of ​​a plurality of panels (S100). The entity performing step S100 is a 3D image acquisition device, which may be performed within the same hardware (step measurement device) as the processor, which is the entity performing step measurement in subsequent steps 200 to 700, or may be a separate hardware.

[0073] Next, the step measurement device can set a plurality of opposing reference points based on a boundary area in the surface image, and crop a first area including all of the plurality of reference points (S200). Here, the first area can be cropped into a rectangular shape. Next, the device can extract first and second point clouds corresponding to each of the plurality of reference points in the first area, and define a coordinate system using the first and second point clouds (S400). Before step S400, the device can optionally preprocess the first area (S300). After defining the coordinate system, the device can extract a point cloud of an area based on the defined coordinate system, and use the extracted point cloud to correct one or more unit vectors constituting the coordinate system to redefine the defined coordinate system (S500). Next, the device can extract the point cloud of an area based on the redefined coordinate system and project it onto a plane (S600). Finally, the device can measure the step of the vehicle using the data projected onto the plane, thereby obtaining the final result of the step measurement (S700).

[0074]

[0075] Below, specific examples of each step will be examined with reference to FIGS. 3 to 9.

[0076]

[0077] Referring to FIG. 3, the preprocessing step (S300) of the first region will be examined in more detail. The device can perform image filtering on the first region (S310). The image filtering in step S310 filters out high-frequency components unnecessary for deriving the outline, and a filter such as Gaussian blur can be used, for example. Then, an outline region can be detected in the first region. Specifically, this step can be performed through a process of detecting an outline in the first region (S330) and applying a dilation filter to set the dilated outline as the outline region (S350). Next, the device can remove boundary noise by dividing the region using connected componentization and labeling (S370), and then removing a point cloud corresponding to a plurality of second regions each including a plurality of reference points and a third region excluding the outline region (S390). The results of each step are as shown in Fig. 4, and R310 is an example of the results of S310, R330 is an example of the results of S330, R350 is an example of the results of S350, R370 is an example of the results of S370, and R390 is an example of the results of S390.

[0078]

[0079] Next, referring to FIG. 10, an example of a step (S400) for defining a coordinate system, i.e., a surface coordinate system for step measurement, will be specifically described. As illustrated in FIG. 10, the device can extract data of a first point cloud corresponding to a reference point (A) and a second point cloud corresponding to a reference point (B), and define a coordinate system using the two point clouds. The device can define a vector generated by the first and second point clouds as a reference orientation vector, and calculate a normal unit vector from each point cloud. In addition, among the normal unit vectors, the normal unit vector of a point specified by the user (the reference point (B) in the example of FIG. 10) can be defined as the reference normal unit vector. In addition, a unit vector orthogonal to the reference orient vector and the reference normal unit vector is defined as an approach unit vector, and then the orientation unit vector can be recalculated using the normal unit vector and the approach unit vector in order to match the characteristics of the homogeneous coordinate system. And using the recalculated orient unit vector, normal unit vector, and approach unit vector, we can define a coordinate system.

[0080] After step S400, the device can extract a point cloud of an area based on a defined coordinate system and use it to correct one or more unit vectors constituting the coordinate system to redefine the defined coordinate system (S500).

[0081]

[0082] Referring to FIG. 6, the coordinate system redefinition step will be examined in more detail as follows. In step S500, the device extracts a third point cloud of an area based on the coordinate system (S510). The unit vector of the coordinate system can be corrected using the third point cloud (S520). Next, the device can redefine the coordinate system using the unit vector corrected in step S520 (S530). The device extracts a fourth point cloud of an area based on the first redefined coordinate system (S540), and can correct the unit vector of the first redefined coordinate system using the fourth point cloud (S550). Then, the device can redefine the coordinate system redefined in step S530 using the unit vector corrected in step S550 (S560).

[0083]

[0084] Referring to Fig. 7, the first correction step (S520) for correcting the unit vector of the coordinate system will be examined in more detail. In step S520, the device extracts a normal vector corresponding to the third point cloud and calculates an average (S521), and defines the calculated average normal vector as the normal vector of the coordinate system, thereby recalculating the approach unit vector based thereon (S523). The device can recalculate the orienting unit vector using the average normal vector and the recalculated approach unit vector (S525), and can correct the recalculated approach unit vector using the recalculated orienting unit vector and the unit average normal vector (S527).

[0085]

[0086] Referring to Fig. 8, the step (S550) of correcting the unit vector of the first redefined coordinate system using the fourth point cloud will be examined in detail.

[0087] In step S550, the device can project the fourth point cloud onto the orientation-approach plane of the primary redefined coordinate system (S551), cluster the projected data (S553), and calculate a misalignment angle between multiple clusters (S555). In addition, the device can rotate the approach unit vector and the orientation unit vector in the opposite direction of the misalignment angle with respect to the axis of the normal vector of the primary redefined coordinate system (S557).

[0088]

[0089] Referring to Fig. 9, the step of measuring the step (S700) will be examined in more detail.

[0090] The device can cluster projected data (S710). Then, the gap can be calculated by calculating the distance between clusters (S730). The height difference (flush) can be calculated using the distance between the line corresponding to one cluster and the data of the other cluster. For example, the gap can be calculated through the support vector after calculating the support vector of the two separated clusters, and the height difference can be determined as the height difference by fitting a straight line to one of the two clusters, calculating the distance between the fitted straight line and the data of the other cluster, and then calculating the average of the distances, as illustrated in FIG. 11.

[0091] Next, a step measurement system according to an embodiment of the present invention will be described with reference to FIG. 12. The step measurement system (10) according to an embodiment of the present invention may include a 3D camera (500) and a server (1000).

[0092] A 3D camera (500) can obtain a surface image of a vehicle including a boundary region of a plurality of panels, and transmit the surface image to a server, and the server (1000) can receive the surface image and measure a step of the vehicle. The server (1000) can include an image processing unit (1310) that sets a plurality of reference points facing each other based on the boundary region in the surface image and crops a first region including all of the plurality of reference points, a surface coordinate system estimation unit (1350) that extracts first and second point clouds corresponding to each of the plurality of reference points in the first region, defines a coordinate system using the same, extracts a point cloud of an area based on the defined coordinate system, and corrects one or more unit vectors constituting the coordinate system using the same to redefine the defined coordinate system, and a step calculation unit (1370) that projects a point cloud corresponding to the first area based on the redefined coordinate system onto a plane and measures a step of the vehicle using data projected onto the plane. Since each component operates identically or similarly to the step measurement device (100) described above in FIG. 1, more specific details of each component can be inferred by referring to the descriptions of FIGS. 1 to 5 and 11.

[0093]

[0094] A step measurement device (100) and a step measurement system (10) according to an embodiment of the present invention have a primary purpose of finding a plane and a coordinate system parallel to the surface by using a point cloud and a texture image acquired by measuring a vehicle surface, and provide a method for measuring a step based on this coordinate system. Therefore, through the operation of the above-described method and device, the step measurement device (100) or server (1000) can measure a step with high accuracy using only an image without contacting a measuring tool or introducing a robot to set a measuring plane parallel to the surface.

[0095]

[0096] According to one embodiment of the present invention, a step measurement method, device, and computer program can accurately measure a step based on an image captured from a distance without contacting or inserting a measuring tool at a close distance. Furthermore, the step of a vehicle moving on a conveyor belt can be measured using a fixed measuring device without introducing a robot that requires a high price and a large installation space. According to the present invention, the step can be measured based on a coordinate system parallel to the surface of the vehicle, thereby minimizing errors due to movement.

[0097]

[0098] The features, structures, effects, etc. described in the embodiments above are included in at least one embodiment of the present invention, and are not necessarily limited to just one embodiment. Furthermore, the features, structures, effects, etc. exemplified in each embodiment can be combined or modified in other embodiments by those skilled in the art to which the embodiments pertain. Therefore, the contents related to such combinations and modifications should be construed as falling within the scope of the present invention.

[0099] In addition, although the above description focuses on the embodiments, these are merely examples and do not limit the present invention. Those skilled in the art to which the present invention pertains will appreciate that various modifications and applications not exemplified above are possible without departing from the essential characteristics of the present embodiments. In other words, each component specifically shown in the embodiments can be modified and implemented. In addition, differences related to such modifications and applications should be interpreted as being included within the scope of the present invention defined in the appended claims.

Claims

1. In a method for measuring the step of a vehicle, A step of acquiring a surface image of the vehicle including a boundary area of ​​a plurality of panels; A step of setting a plurality of reference points facing each other based on the boundary area in the surface image, and cropping a first area including all of the plurality of reference points; A step of extracting first and second point clouds corresponding to each of the plurality of reference points in the first region and defining a coordinate system using the same; A step of extracting a point cloud of an area based on a defined coordinate system and using the point cloud to correct one or more unit vectors constituting the coordinate system to redefine the defined coordinate system; A step of extracting a point cloud of an area based on a redefined coordinate system and projecting it onto a plane; A step measurement method comprising a step of measuring a step of the vehicle using data projected onto the above plane.

2. In paragraph 1, Further comprising a preprocessing step of removing noise from the cropped first region, The steps for defining the above coordinate system are: A step of extracting first and second point clouds corresponding to each of the plurality of reference points in the first area from which the noise has been removed, and defining a coordinate system using the same; A step measurement method including:

3. In paragraph 1, The steps to redefine the above coordinate system are: A step of extracting a third point cloud of an area based on the above coordinate system; A first correction step of correcting the unit vector of the coordinate system using the third point cloud; A first redefinition step of redefining the coordinate system using the unit vector corrected in the first correction step; A step of extracting a fourth point cloud of an area based on the first redefined coordinate system; A second correction step of correcting the unit vector of the first redefined coordinate system using the fourth point cloud; A second redefinition step of redefining the first redefined coordinate system using the unit vector corrected in the second correction step; A step measurement method including:

4. In paragraph 1, The step of measuring the step of the above vehicle is: A step of clustering the above projected data; A step of calculating the distance between clusters to derive the gap; A step of calculating a height difference (flush) by using the distance between a straight line corresponding to one cluster among the above clusters and data of another cluster; A step measurement method including:

5. In paragraph 2, A step measurement method characterized in that the first region has a square shape.

6. In paragraph 2, The above preprocessing step is; A step of detecting a contour area in the first area; A step of removing a point cloud corresponding to a plurality of second regions each including the plurality of reference points in the first region and a third region excluding the outline region; A step measurement method including:

7. In paragraph 6, A step measurement method further comprising a step of removing noise by image filtering the first area before detecting the outline area.

8. In paragraph 6, The above outline area detection step is, A step of detecting an outline in the first region and applying a dilation filter; A step of setting the expanded outline as the outline area; A step measurement method including:

9. In paragraph 1, The above projection step is, A step measurement method comprising a step of projecting a point cloud of an area extracted based on the above-described redefined coordinate system onto an approach-normal plane of the above-described redefined coordinate system.

10. In paragraph 1, The above multiple reference points are A step measurement method set by a machine learning framework or user settings that detects a reference point.

11. In paragraph 3, The above first correction step is, A step of extracting a normal vector corresponding to the third point cloud and calculating an average; A step of recalculating the approach unit vector using the produced average normal vector; A step of recalculating an orienting unit vector using a unit average normal vector using the above average normal vector and a recalculated approach unit vector; A step of correcting the recalculated approach unit vector using the recalculated orient unit vector and the unit mean normal vector; A step measurement method including:

12. In paragraph 3, The second correction step is: A step of projecting the fourth point cloud onto the orient-approach plane of the first redefined coordinate system; A step of clustering the projected data and calculating the misalignment angle between multiple clusters; A step of rotating the approach unit vector and the orient unit vector in the opposite direction of the twisted angle based on the axis of the normal vector of the first redefined coordinate system; A step measurement method including:

13. A computer-readable recording medium recording a program for executing a method according to any one of claims 1 to 12.

14. In a device for measuring the step of a vehicle, An image collection unit for acquiring a surface image of the vehicle including a boundary area of ​​a plurality of panels; A processor for measuring the step of the vehicle using the surface image; The above processor, An image processing unit that sets a plurality of reference points facing each other based on the boundary area in the surface image and crops a first area including all of the plurality of reference points; A surface coordinate system estimation unit that extracts first and second point clouds corresponding to each of the plurality of reference points in the first area, defines a coordinate system using the first and second point clouds, extracts a point cloud of an area based on the defined coordinate system, and redefines the defined coordinate system by correcting one or more unit vectors constituting the coordinate system using the first and second point clouds; A step calculation unit that projects a point cloud corresponding to the first area onto a plane based on a redefined coordinate system and measures the step of the vehicle using the data projected onto the plane; A step measuring device including:

15. In paragraph 14, Further comprising a preprocessing unit for removing noise from the cropped first region, The above first area is, A step measurement device, which is an area from which noise has been removed in the above preprocessing unit. In a system for measuring a step of a vehicle including a 16.3D camera and a server, A 3D camera that acquires a surface image of the vehicle including a boundary area of ​​a plurality of panels and transmits the surface image to a server; A server for receiving the surface image and measuring the step of the vehicle; The above server, An image processing unit that sets a plurality of reference points facing each other based on the boundary area in the surface image and crops a first area including all of the plurality of reference points; A surface coordinate system estimation unit that extracts first and second point clouds corresponding to each of the plurality of reference points in the first area, defines a coordinate system using the first and second point clouds, extracts a point cloud of an area based on the defined coordinate system, and redefines the defined coordinate system by correcting one or more unit vectors constituting the coordinate system using the first and second point clouds; A step calculation unit that projects a point cloud corresponding to the first area onto a plane based on a redefined coordinate system and measures the step of the vehicle using the data projected onto the plane; A step measurement system including:

Images