Vehicle level difference measuring method, device, and computer program

Abstract

A bump measurement method according to one embodiment of the present application includes the steps of: acquiring a surface image of the vehicle including a boundary region of a plurality of panels; setting a plurality of reference points in the surface image that face each other based on the boundary region, and cropping a first region that includes all of the reference points; extracting first and second point clouds corresponding to the plurality of reference points from the first region and defining a coordinate system using the extracted point clouds; extracting a point cloud of a region based on the defined coordinate system and correcting one or more unit vectors that constitute the coordinate system using the extracted point cloud to redefine the defined coordinate system; extracting the point cloud of a region based on the redefined coordinate system and projecting it onto a plane; and measuring a bump on the vehicle using the data projected onto the plane. According to one embodiment of the present invention, a bump can be accurately measured based on an image captured in an environment where a measurement sensor and the vehicle are far apart.

Description

[Technical Field]

[0001] The present application relates to a vehicle step measurement method, device, and computer program, and more particularly, to a vehicle step measurement method, device, and computer program that measure gaps and flushes between vehicle panels based on acquired images. [Background technology]

[0002] The gap between panels that make up a vehicle is a criterion for judging the product's completeness. Therefore, to produce high-quality products, gaps must be measured during the production process. Traditionally, this has been done by a person directly touching or inserting a measuring tool into the gaps in the vehicle, but this has had the problem of being affected by the level of skill of the person taking the measurements, delaying the unmanned process.

[0003] For this reason, one method proposed for unmanned operation is to use a non-contact optical step measurement sensor and a robot. Step measurement requires a sensor that measures the surface of the vehicle, but the sensors used in existing step measurement solutions perform measurements at a short distance of approximately 10 cm from the object (vehicle) to be measured. Step measurement sensors generally measure in two dimensions (depth and width), so they require an environment where the measurement surface and the sensor are set up parallel. However, in actual industrial environments, it is not easy to position the object to be measured perfectly parallel to the sensor. When the object to be measured is placed on a moving conveyor belt, it is not possible to maintain a parallel state, so a robot is used to achieve this.

[0004] However, in order to use a robot to measure the height difference of vehicles positioned on a moving conveyor belt, it is necessary to secure space that takes into account the robot's operating range and size, and since safety regulations require large-scale safety equipment (safety nets, safety mats, indicators, etc.) to introduce a robot system, it can only be introduced in workplaces with ample space. Therefore, there is a need to develop technology that can measure the height difference of moving products within a limited space. Summary of the Invention [Problem to be solved by the invention]

[0005] One problem that the present invention aims to solve is to provide a step measurement method, device, and system that can accurately measure steps based on images taken in an environment where the distance between the measurement sensor and the vehicle is long.

[0006] Another problem that the present invention aims to solve is to provide a method that can measure the step of a vehicle moving on a conveyor belt using a fixed measuring device.

[0007] Another problem to be solved by the present invention is to provide a step measurement method, device, and system that have a minimum error range.

[0008] The problems to be solved by the present invention are not limited to those described above, and problems not mentioned will be clearly understood by those skilled in the art to which the present invention pertains from this specification and the accompanying drawings. [Means for solving the problem]

[0009] A bump measurement method according to one embodiment of the present invention may include the steps of: acquiring a surface image of the vehicle including a boundary region of a plurality of panels; setting a plurality of reference points in the surface image that face each other based on the boundary region, and cropping a first region that includes all of the plurality of reference points; extracting first and second point clouds corresponding to each of the plurality of reference points from the first region and defining a coordinate system using the extracted point clouds; extracting a point cloud of a region based on the defined coordinate system and correcting one or more unit vectors that constitute the coordinate system using the extracted point cloud to redefine the defined coordinate system; extracting the point cloud of a region based on the redefined coordinate system and projecting it onto a plane; and measuring a bump on the vehicle using the data projected onto the plane.

[0010] A bump measurement device according to one embodiment of the present invention includes an image collection unit that acquires a surface image of the vehicle, including a boundary region of a plurality of panels; a processor that measures bumps on the vehicle using the surface image; the processor may include an image processing unit that sets a plurality of reference points in the surface image that face each other based on the boundary region, and crops a first region that includes all of the plurality of reference points; a surface coordinate system estimation unit that extracts first and second point clouds corresponding to each of the plurality of reference points from the first region, defines a coordinate system using the extracted point clouds, extracts a point cloud for a region based on the defined coordinate system, and corrects one or more unit vectors that constitute the coordinate system using the extracted point clouds to redefine the defined coordinate system; and a bump calculation unit that projects the point cloud corresponding to the first region onto a plane based on the redefined coordinate system, and measures bumps on the vehicle using the data projected onto the plane.

[0011] The means for solving the problems of the present invention are not limited to the above-mentioned means, and any unmentioned means will be clearly understood by a person having ordinary skill in the art to which the present invention pertains from this specification and the accompanying drawings. [Effects of the Invention]

[0012] According to an embodiment of the present invention, it is possible to accurately measure a step based on an image captured in an environment where the distance between the measurement sensor and the vehicle is large.

[0013] Furthermore, according to one embodiment of the present invention, even a fixed measuring device can measure the step of a vehicle moving on a conveyor belt.

[0014] Furthermore, according to an embodiment of the present invention, it is possible to minimize errors in step measurement. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a block diagram of a step measuring device according to an embodiment of the present application; [Figure 2] 1 is a flowchart illustrating a step measurement method according to an embodiment of the present application. [Figure 3] 1 is a flow chart embodying a pre-processing step according to an embodiment of the present application. [Figure 4] 1 is a diagram illustrating an example of a result of a pre-processing step according to an embodiment of the present application. [Figure 5] 1 is a diagram for explaining a region to be preprocessed according to an embodiment of the present application; [Figure 6] 1 is a flowchart embodying a coordinate system redefinition step according to an embodiment of the present application; [Figure 7] 1 is a flowchart embodying a first correction stage according to an embodiment of the present application. [Figure 8] 4 is a flowchart embodying a second correction stage according to an embodiment of the present application. [Figure 9] 1 is a flow chart embodying a step measurement step according to an embodiment of the present application; [Figure 10] 1 is a diagram for explaining the definition of a unit vector according to an embodiment of the present application; [Figure 11] 1 is a diagram for explaining a step measurement method according to an embodiment of the present application; [Figure 12]1 is a diagram for explaining a step measurement system according to an embodiment of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0016] The above-mentioned objects, features, and advantages of the present application will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. However, since the present application can be modified in various ways and can have various embodiments, the following will describe in detail specific embodiments by way of example in the drawings.

[0017] The same reference numerals will generally refer to the same elements throughout the specification. Furthermore, elements having the same functions within the same concept shown in the drawings of each embodiment will be described using the same reference numerals, and redundant description thereof will be omitted.

[0018] If it is determined that a detailed description of a known function or configuration related to this application may unnecessarily obscure the gist of this application, the detailed description will be omitted. In addition, numbers (e.g., 1, 2, etc.) used in the description of this specification are merely identification symbols for distinguishing one component from another.

[0019] Furthermore, the suffixes "module" and "section" used in the following embodiments for components are given or used interchangeably solely for the sake of ease of writing the specification, and do not have any distinct meanings or roles in themselves.

[0020] In the following embodiments, the singular expression includes the plural expression unless the context clearly indicates otherwise.

[0021] In the following embodiments, terms such as "include" or "have" mean that the features or components described in the specification are present, and do not preclude the possibility that one or more other features or components may be added.

[0022] In the drawings, the size of elements may be exaggerated or reduced for the sake of convenience of explanation. For example, the size and thickness of each element shown in the drawings are arbitrarily shown for the sake of convenience of explanation, and the present invention is not necessarily limited to those shown in the drawings.

[0023] If an embodiment can be implemented differently, the order of certain processes may be performed differently than described. For example, two processes described in succession may be performed substantially simultaneously or may be performed in the reverse order from that described.

[0024] In the following embodiments, when elements are said to be connected, this includes not only the case where the elements are directly connected, but also the case where the elements are indirectly connected with another element interposed between them.

[0025] For example, when it is stated herein that components are electrically connected, this includes not only cases where the components are directly electrically connected, but also cases where the components are indirectly electrically connected through an intervening component.

[0026] Meanwhile, in this specification, "step" can be understood as a concept that includes both gaps and flushes between vehicle panels, and when distinction is necessary, it is described as gaps and flushes.

[0027] The step measuring method, device, and system of the present application will be described below with reference to FIGS.

[0028] 1 is a block diagram of a step measurement device 100 according to an embodiment of the present application. The step measurement device 100 is a device for measuring steps (differences in gap and height) between multiple panels, and is capable of measuring steps by acquiring a 3D image of the surface of a vehicle including multiple panels and step areas and analyzing the image.

[0029] The step measurement device 100 according to one embodiment of the present application may include an image collection unit 110 and a processor 130, and may further include a memory unit 150 for storing data and a communication unit 170 for transmitting measurement results to a server or the like.

[0030] The image collecting unit 110 can acquire a surface image of a vehicle including boundary regions of multiple panels. The image collecting unit 110 is an image acquiring device called a 3D camera or a 3D vision sensor, which may encompass devices that acquire 3D images including depth information, such as a 3D structure light camera, a ToF (Time-of-Flight) camera, or a stereo vision camera. The image collecting unit 110 of the present invention is not limited by an image acquisition method or generation method as long as it is a 3D measurement device that can acquire a point cloud (a collection of data points belonging to a 3D space).

[0031] The processor 130 measures the step height using the surface image acquired by the image collecting unit 110. More specifically, the processor 130 may include an image processing unit 131, a preprocessing unit 133, a surface coordinate system estimating unit 135, and a step height calculating unit 137. The processor 130 may load and execute programs for the overall operation of the step height measuring device 100 from the storage unit 150. The processor 130 may be embodied as hardware, software, or a combination thereof, such as an application processor (AP), central processing unit (CPU), microcontroller unit (MCU), or similar device. In this case, the hardware may be provided in the form of an electronic circuit that processes electrical signals and performs control functions, and the software may be provided in the form of a program or code that drives the hardware circuit.

[0032] The image processor 131 may set a plurality of reference points in the surface image that are opposite to each other with respect to the boundary region, and crop a first region that includes all of the reference points. In this case, the first region may have a rectangular shape, such as the red rectangular box R100 shown in FIG. 4, but the shape of the first region is not necessarily limited to this embodiment. The plurality of reference points may be set by a machine learning framework for detecting the reference points or by user setting. The plurality of reference points may be two or more points that are located at opposite positions with respect to the boundary region, and do not need to be precisely symmetrical about the boundary region. However, the machine learning framework for detecting the reference points may be a framework trained to detect two points that are symmetrical about the boundary region corresponding to a step, or two points on a line perpendicular to the boundary region. That is, the framework may be trained with an image that includes two points on a line perpendicular to the boundary region and / or two points that are symmetrical about the boundary region.

[0033] The pre-processing unit 133 may remove noise from the cropped first region. A noise removal method performed by the pre-processing unit 133 will be described below with reference to FIG. 4. First, the pre-processing unit 133 may perform Gaussian blur processing on the first region cropped by the image processing unit 131 as in R310. Next, the pre-processing unit 133 may detect a contour region. For example, the pre-processing unit 133 may detect a contour using a contour detection algorithm such as the Canny Edge algorithm as in R330. Next, the pre-processing unit 133 may set the contour as a contour region by applying a dilation filter to the image in which the contour is detected as in R350. In this case, the algorithms and filters used in the contour detection and contour region setting processes of the present invention are not limited to the above examples, and any number of different methods that can achieve similar effects may be used.

[0034] After detecting the contour region, the pre-processing unit 133 may segment the region using connected components and labeling, thereby obtaining an image such as R370. Next, the pre-processing unit 133 may remove a point cloud of a third region, which is obtained by excluding a plurality of second regions each including a plurality of reference points and the contour region from the first region. For example, referring to FIG. 5 additionally, when the elementized image of the first region is generated as shown in FIG. 5 through the processing of the pre-processing unit 133, a second region (a) including reference point A and a second region (b) including reference point B may be maintained as elementized images. The contour region may also be maintained. The pre-processing unit 133 may remove a point cloud corresponding to the third region, resulting in an image such as R390. That is, the third region is the remaining portion of the first region excluding the second region and the contour region. While the third region has a similar position and shape to the boundary region, it is distinct from the boundary region because it differs from the boundary region. The pre-processing unit 133 removes the point cloud corresponding to the third region because it is highly likely to be data resulting from abnormal phenomena present at the boundary during bump measurement for a vehicle. Point clouds or depth image results collected by a 3D camera device tend to contain noise and artifacts. Here, noise can be understood to refer to electrical interference signals from the camera sensor or equipment, and artifacts can be understood to refer to optical phenomena in the projection and imaging system and abnormal phenomena captured by decoding the depth data. The pre-processing unit 133 of the present invention aims to remove artifacts, which can be caused by inter-reflection between surfaces and distortion due to discontinuities in surface contrast. That is, by removing the point cloud corresponding to the third region, the pre-processing unit 133 of the present invention can reduce bump measurement errors in areas where there is an abrupt transition from a highly absorptive surface to a reflective surface, such as a transition from black to white.

[0035] 1 again, the surface coordinate system estimation unit 135 may extract first and second point clouds corresponding to a plurality of reference points from the first region and define a coordinate system using the extracted point clouds. If the pre-processing unit 133 removes noise from the first region before transmitting, the surface coordinate system estimation unit 135 may extract the first and second point clouds from the noise-removed first region and define a coordinate system using the extracted point clouds. The surface coordinate system estimation unit 135 may extract a point cloud for one region based on the defined coordinate system and redefine the defined coordinate system by correcting one or more unit vectors constituting the coordinate system using the extracted point clouds.

[0036] For example, referring to FIG. 10, the surface coordinate system estimation unit 135 may extract data of a first point cloud corresponding to reference point A and a second point cloud corresponding to reference point B and define a coordinate system using both point clouds. First, the surface coordinate system estimation unit 135 may define a vector generated by the first and second point clouds as a base orient vector and calculate a normal unit vector from each point cloud. Next, the surface coordinate system estimation unit 135 may define the normal unit vector of a point specified by the user (reference point B in the example of FIG. 10) among the normal unit vectors as a base normal unit vector. Then, a unit vector orthogonal to the base orient vector and the base normal unit vector may be defined as an approach unit vector. The surface coordinate system estimation unit 135 may recalculate the orient unit vector using the normal unit vector and the approach unit vector to express the coordinate system one-dimensionally—to make it a homogeneous coordinate system. More specifically, since a homogeneous coordinate system has the property that each vector is orthogonal, the orient unit vector is recalculated by dividing the vector product of the normal unit vector and the approach unit vector by the magnitude (scalar value) of the orient vector. The surface coordinate system estimation unit 135 can define a coordinate system using the recalculated orient unit vector, normal unit vector, and approach unit vector.

[0037] After the coordinate system is defined, the surface coordinate system estimation unit 135 can redefine the coordinate system to estimate a coordinate system parallel to the surface of the vehicle. The coordinate system redefinition can be performed as follows.

[0038] The surface coordinate system estimation unit 135 can mask and extract only the point cloud (third point cloud) within one area (certain range) based on the defined coordinate system. Here, the one area can be changed depending on the user's setting, and the size of the area affects the subsequent calculation of the average normal vector and the coordinate system correction using the calculated average normal vector. In other words, if the size of the area is set large, an average result for a larger area can be obtained, and if the size is set small, a local result for the corresponding position can be obtained.

[0039] Meanwhile, the surface coordinate system estimation unit 135 may correct the unit vector of the coordinate system using the third point cloud (first correction). More specifically, the first correction may be performed by extracting normal vectors corresponding to the third point cloud and calculating the average of the vectors. The surface coordinate system estimation unit 135 may define the calculated average normal vector as the normal vector of the defined coordinate system and recalculate the approach unit vector based on the calculated average normal vector. Next, to re-express the existing coordinate system in a homogeneous coordinate system, the surface coordinate system estimation unit 135 may re-calculate the orient unit vector using the average normal vector and the recalculated approach unit vector. The recalculated approach unit vector may then be corrected again using the recalculated orient unit vector and the unit average normal vector. The surface coordinate system estimation unit 135 may redefine the coordinate system through this process (first redefinition).

[0040] After redefining the coordinate system, the surface coordinate system estimation unit 135 again performs an operation of extracting a point cloud (fourth point cloud) of one area based on the redefined coordinate system. The surface coordinate system estimation unit 135 corrects the unit vector of the first redefined coordinate system using the fourth point cloud (second correction), and can then perform a second redefinition to redefine the first redefined coordinate system using the fourth point cloud.

[0041] To explain the second correction in more detail, the surface coordinate system estimation unit 135 may project the fourth point cloud onto the orient-approach plane of the first redefined coordinate system and cluster the projected data. As a result, multiple clusters may be generated, and the surface coordinate system estimation unit 135 may calculate the twist angle between the generated multiple clusters using a support vector machine (SVM) or the like. Next, the surface coordinate system estimation unit 135 may rotate the orient unit vector and the approach unit vector in the opposite direction of the twist angle between the two clusters, based on the axis of the normal vector. Next, the surface coordinate system estimation unit 135 may redefine the coordinate system using the rotated orient unit vector, approach unit vector, and normal unit vector (secondary redefinition). Thereafter, the surface coordinate system estimation unit 135 may finally extract a point cloud of one region based on the redefined coordinate system and transmit the extracted point cloud to the step calculation unit 137.

[0042] The step calculation unit 137 may project the point cloud of the extracted area onto a plane based on the redefined coordinate system and measure the step of the vehicle using the data projected onto the plane. Here, the plane onto which the point cloud is projected may be the approach-normal plane of the final redefined coordinate system, and the area projected onto the plane may be a rectangular box-shaped area.

[0043] The step calculation unit 137 can separate the projected data into multiple clusters and calculate support vectors by applying SVM (Support Vector Machine) to the multiple clusters. Then, the step calculation unit 137 can calculate the distance between multiple clusters using the support vectors. The step calculation unit 137 can define the calculated inter-cluster distance as a gap.

[0044] Furthermore, the level difference calculation unit 137 can generate a line corresponding to one of the multiple clusters and calculate the height difference (flush) using the distance between this line and data of other clusters. More specifically, the level difference calculation unit 137 can calculate all the distances between the line fitted to one cluster and data of other clusters, and then calculate the height difference (flush) by averaging the distances.

[0045] 11, the step calculation unit 137 can generate a straight line fitted to cluster 1, and at this time, this straight line can be understood as a reference surface parallel to the surface of the vehicle. The step calculation unit 137 can calculate the average value of the distance between the straight line corresponding to cluster 1 and the data constituting cluster 2, and determine this as the height difference.

[0046] The memory unit 150 of the step measurement device 100 can store various types of information. Various types of data can be stored temporarily or semi-permanently in the memory unit 150. Examples of the memory unit 150 include a hard disk drive (HDD), a solid state drive (SSD), a flash memory, a read-only memory (ROM), and a random access memory (RAM). The memory unit 150 can be provided in a form that is built into the step measurement device 100 or in a removable form. The memory unit 150 can store various types of data necessary for the operation of the step measurement device 100, including an operating system (OS) for driving the step measurement device 100 and programs for operating each component of the step measurement device 100.

[0047] The communication unit 170 of the step measurement device 100 can communicate with any external device, including a server. Also, for example, if the image collection unit 110 is configured separately from the step measurement device 100 (i.e., if the components of the step measurement device 100 are the processor 130, the storage unit 150, and the communication unit 170), the image collection unit 110 can receive a surface image of the vehicle including the boundary area from the image collection unit 110. In another embodiment, if part of the step measurement function of the processor 130 is performed by a separate terminal, the surface image of the vehicle collected by the image collection unit 110 can be transmitted to an external device that performs the function of the processor 130.

[0048] In addition, the step measurement device 100 can connect to a network via the communication unit 170 to transmit and receive various data. Transmitting and receiving units can be broadly classified into wired and wireless types. Since wired and wireless types each have advantages and disadvantages, the step measurement device 100 may be provided with both wired and wireless types in some cases. Here, in the case of a wireless type, a communication method based on a WLAN (Wireless Local Area Network) such as Wi-Fi can be used. Alternatively, in the case of a wireless type, a communication method based on cellular communication, for example, LTE or 5G, can be used. However, the wireless communication protocol is not limited to the examples given above, and any appropriate wireless communication method can be used. In the case of a wired type, typical examples include a local area network (LAN) and universal serial bus (USB) communication, but other methods are also possible.

[0049] A step measuring method according to an embodiment of the present application will be described below with reference to FIGS.

[0050] 2 is a flowchart illustrating a step height measuring method according to an embodiment of the present application. Referring to FIG. 2, an apparatus can acquire a surface image of the vehicle including boundary regions of a plurality of panels (S100). The subject performing step S100 is a 3D image capturing device, which may be performed within the same hardware (step height measuring device) as a processor, which is the subject performing step height measurement in subsequent steps 200 to 700, or may be separate hardware.

[0051] The bump measurement device may then set a plurality of reference points in the surface image, each of which is located on a boundary region, and crop a first region that includes all of the reference points (S200). Here, the first region may be cropped into a rectangular shape. Next, the device may extract first and second point clouds corresponding to the plurality of reference points from the first region and define a coordinate system using the extracted point clouds (S400). Prior to step S400, the device may optionally preprocess the first region (S300). After defining the coordinate system, the device may extract a point cloud for a region based on the defined coordinate system and redefine the defined coordinate system by correcting one or more unit vectors that make up the coordinate system using the extracted point cloud (S500). Next, the device may extract the point cloud for a region based on the redefined coordinate system and project it onto a plane (S600). Finally, the device may obtain a final bump measurement result by measuring the bump on the vehicle using the data projected onto the plane (S700).

[0052] Specific embodiments of each step will be discussed in detail below with reference to FIGS.

[0053] The first region preprocessing step (S300) will be described in more detail with reference to FIG. 3. The apparatus may perform image filtering on the first region (S310). The image filtering in step S310 filters out high-frequency components that are not necessary for contour detection. For example, a filter such as Gaussian blur may be used. Then, a contour region may be detected in the first region. Specifically, this step may involve detecting a contour in the first region (S330) and applying a dilation filter to set the dilated contour as the contour region (S350). Next, the apparatus may segment the region using connected component classification and labeling (S370). Then, the apparatus may remove boundary noise by removing point clouds corresponding to a third region excluding a plurality of second regions and a plurality of contour regions, each including a plurality of reference points (S390). The results for each stage are shown in FIG. 4, where R310 is an embodiment of the results for S310, R330 is an embodiment of the results for S330, R350 is an embodiment of the results for S350, R370 is an embodiment of the results for S370, and R390 is an embodiment of the results for S390.

[0054] Next, an example of the step of defining a coordinate system, i.e., a surface coordinate system for measuring step height (S400), will be described in detail with reference to FIG. 10. As shown in FIG. 10, the apparatus can extract data of a first point cloud corresponding to reference point A and a second point cloud corresponding to reference point B and define a coordinate system using both point clouds. The apparatus can define a vector generated by the first and second point clouds as a reference orientation vector and calculate a normal unit vector from each point cloud. Among the normal unit vectors, the user can define the normal unit vector of a specific point (reference point B in the example of FIG. 10) as a reference normal unit vector. After defining a unit vector perpendicular to the reference orientation vector and the reference normal unit vector as an approach unit vector, the orient unit vector can be recalculated using the normal unit vector and approach unit vector to match the homogeneous coordinate system characteristics. The coordinate system can then be defined using the recalculated orient unit vector, normal unit vector, and approach unit vector.

[0055] After step S400, the device extracts a point cloud of an area based on the defined coordinate system, and can use this to redefine the defined coordinate system by correcting one or more unit vectors that make up the coordinate system (S500).

[0056] The coordinate system redefinition step will be described in more detail with reference to FIG. 6. In step S500, the apparatus extracts a third point cloud of one region based on the coordinate system (S510) and may correct the unit vectors of the coordinate system using the third point cloud (S520). Next, the apparatus redefines the coordinate system using the unit vectors corrected in step S520 (S530). The apparatus extracts a fourth point cloud of one region based on the first redefined coordinate system (S540) and may correct the unit vectors of the first redefined coordinate system using the fourth point cloud (S550). Then, the apparatus redefines the coordinate system redefined in step S530 using the unit vectors corrected in step S550 (S560).

[0057] The first correction step (S520) of correcting the unit vector of the coordinate system will be described in more detail with reference to Figure 7. In step S520, the device extracts normal vectors corresponding to the third point cloud and calculates their average (S521), and defines the calculated average normal vector as the normal vector of the coordinate system, thereby recalculating the approach unit vector (S523). The device recalculates the orient unit vector using the average normal vector and the recalculated approach unit vector (S525), and corrects the recalculated approach unit vector using the recalculated orient unit vector and the unit average normal vector (S527).

[0058] Referring to FIG. 8, the step of correcting the unit vector of the first redefined coordinate system using the fourth point cloud (S550) will be explained in detail.

[0059] In step S550, the apparatus may project the fourth point cloud onto the orient-approach plane of the primary redefined coordinate system (S551), cluster the projected data (S553), calculate a twist angle between multiple clusters (S555), and rotate the approach unit vector and the orient unit vector in the opposite direction of the twist angle with respect to the axis of the normal vector of the primary redefined coordinate system (S557).

[0060] Referring to FIG. 9, the step of measuring the step (S700) will be explained in more detail.

[0061] The device may cluster the projected data (S710). Then, the distance between the clusters may be calculated to calculate the gap (S730). The height difference (flush) may be calculated using the distance between a line corresponding to one of the clusters and the data of the other cluster. For example, the gap may be calculated using the support vectors after calculating the support vectors of the two separated clusters. The height difference may be determined by fitting a line to one of the two clusters, calculating the distance between the fitted line and the data of the other cluster, and then averaging these values, as shown in FIG. 11.

[0062] Next, a step measurement system according to an embodiment of the present invention will be described with reference to Fig. 12. The step measurement system 10 according to an embodiment of the present invention may include a 3D camera 500 and a server 1000.

[0063] The 3D camera 500 can acquire a surface image of the vehicle, including the boundary regions of the panels, and transmit the surface image to the server. The server 1000 can receive the surface image and measure bumps on the vehicle. The server 1000 can include an image processor 1310 that sets a plurality of reference points facing each other based on the boundary region in the surface image and crops a first region including all of the reference points; a surface coordinate system estimator 1350 that extracts first and second point clouds corresponding to the plurality of reference points from the first region, defines a coordinate system using the extracted point clouds, extracts a point cloud for a region based on the defined coordinate system, and corrects one or more unit vectors constituting the coordinate system using the extracted point clouds to redefine the defined coordinate system; and a bump calculator 1370 that projects the point cloud corresponding to the first region onto a plane based on the redefined coordinate system and measures bumps on the vehicle using the data projected onto the plane. Each component operates in the same or similar manner as the bump measurement device 100 described above with reference to FIG. 1, and more specific details of each component can be inferred by referring to the descriptions of FIGS. 1 through 5 and 11.

[0064] The step measurement device 100 and step measurement system 10 according to one embodiment of the present invention provide a method for measuring steps based on a coordinate system, with the objective of finding a plane and coordinate system parallel to the surface using a point cloud and texture image acquired by measuring the surface of a vehicle. Therefore, through the operation of the above-described method and device, the step measurement device 100 or server 1000 can measure steps with high accuracy using only images, without the need for contact with a measurement tool or the introduction of a robot to set a measurement plane parallel to the surface.

[0065] According to a step measurement method, device, and computer program according to an embodiment of the present invention, step heights can be accurately measured based on images captured at a long distance, without the need to contact or insert a measuring tool at close range. Furthermore, step heights of vehicles moving on a conveyor belt can be measured using a fixed measuring device, without the need to introduce a robot, which requires high costs and a large installation space. According to the present invention, step heights can be measured based on a coordinate system parallel to the surface of the vehicle, which has the advantage of minimizing errors due to movement.

[0066] The features, structures, effects, etc. described in the above embodiments are included in at least one embodiment of the present invention and are not necessarily limited to only one embodiment. Furthermore, the features, structures, effects, etc. exemplified in each embodiment can be combined or modified in other embodiments by a person skilled in the art to which the embodiment belongs. Therefore, content related to such combinations and modifications should be interpreted as being included in the scope of the present invention.

[0067] Furthermore, although the above description has focused on the embodiments, these are merely examples and do not limit the present invention. Those skilled in the art will recognize that various modifications and applications not exemplified above are possible within the scope of the essential characteristics of the present invention. In other words, each component specifically illustrated in the embodiments can be modified and implemented. Differences related to such modifications and applications should be construed as being included within the scope of the present invention as defined by the appended claims.

Claims

1. A method for measuring a step of a vehicle, acquiring a surface image of the vehicle including boundary regions of a plurality of panels; setting a plurality of reference points in the surface image, the reference points being opposite to each other with respect to the boundary region, and cropping a first region including all of the plurality of reference points; extracting first and second point clouds corresponding to the plurality of reference points from the first region, respectively, and defining a coordinate system using the extracted point clouds; extracting a point cloud of an area based on the defined coordinate system, and correcting one or more unit vectors constituting the coordinate system using the extracted point cloud to redefine the defined coordinate system; A step of extracting a point cloud of an area based on the redefined coordinate system and projecting it onto a plane; a step of measuring a step of the vehicle using the data projected onto the one plane.

2. further comprising a pre-processing step of removing noise from the cropped first region; The step of defining the coordinate system comprises:

2. The step measuring method of claim 1, further comprising: extracting first and second point clouds corresponding to the plurality of reference points, respectively, from the first region from which the noise has been removed, and defining a coordinate system using the extracted point clouds.

3. The step of redefining the coordinate system includes: extracting a third point cloud of an area based on the coordinate system; a first correction step of correcting a unit vector of the coordinate system using the third point cloud; a first redefinition step of redefining the coordinate system using the unit vector corrected in the first correction step; extracting a fourth point cloud of the one region based on the first redefined coordinate system; a second correction step of correcting unit vectors of the first redefined coordinate system using the fourth point cloud; 2. The step measuring method according to claim 1, further comprising: a second redefining step of redefining the coordinate system redefined primarily using the unit vector corrected in the second correcting step.

4. The step of measuring the step of the vehicle includes: clustering the projected data; Calculating the distance between the clusters to calculate the gap; 2. The method of claim 1, further comprising: calculating a height flush using a distance between a line corresponding to one of the clusters and data of another cluster.

5. The method of claim 2, wherein the first area has a rectangular shape.

6. The pre-treatment step comprises: detecting a contour region in the first region; 3. The step measuring method according to claim 2, further comprising: removing point clouds corresponding to a third region excluding the contour region and a plurality of second regions each including the plurality of reference points from the first region.

7. The method of claim 6 , further comprising the step of removing noise by image filtering the first region before detecting the contour region.

8. The step of detecting the contour region includes: detecting contours in the first region and applying a dilation filter; 7. The step measuring method according to claim 6, further comprising the step of: setting the expanded contour line as the contour region.

9. The projecting step includes:

2. The step measurement method according to claim 1, further comprising a step of projecting a point cloud of a region extracted with reference to the redefined coordinate system onto an approach-normal plane of the redefined coordinate system.

10. The plurality of reference points The step measurement method according to claim 1 , wherein the reference point is set by a machine learning framework or user setting.

11. The first correction step comprises: extracting and averaging normal vectors corresponding to the third point cloud; recalculating the approach unit vector using the calculated average normal vector; recalculating a unit mean normal vector using the mean normal vector and an orient unit vector using the recalculated approach unit vector; 4. The step measurement method according to claim 3, further comprising the step of: correcting the recalculated approach unit vector using the recalculated orient unit vector and the unit mean normal vector.

12. The second correction step includes: projecting the fourth point cloud onto an orient-approach plane of the primary redefined coordinate system; A step of clustering the projected data and calculating twist angles between a plurality of clusters; 4. The step measurement method according to claim 3, further comprising the step of rotating an approach unit vector and an orient unit vector in a direction opposite to the twist angle with respect to an axis of a normal vector of the linearly redefined coordinate system.

13. A computer-readable recording medium on which a program for executing the method according to any one of claims 1 to 12 is recorded.

14. In a device for measuring a step in a vehicle, an image acquisition unit that acquires a surface image of the vehicle including boundary regions of a plurality of panels; a processor for measuring a step in the vehicle using the surface image; The processor: an image processing unit that sets a plurality of reference points in the surface image, the reference points being opposite to each other with respect to the boundary region, and crops a first region that includes all of the plurality of reference points; a surface coordinate system estimation unit that extracts first and second point clouds corresponding to the plurality of reference points from the first region, defines a coordinate system using the extracted point clouds, extracts a point cloud for one region based on the defined coordinate system, and corrects one or more unit vectors constituting the coordinate system using the extracted point clouds to redefine the defined coordinate system; a step calculation unit that projects a point cloud corresponding to the first region onto a plane based on a redefined coordinate system, and measures a step on the vehicle using the data projected onto the plane.

15. a pre-processing unit that removes noise from the cropped first region; The first region is The step measuring device according to claim 14 , wherein the area is one area from which noise has been removed in the pre-processing unit.

16. A system for measuring a step on a vehicle including a 3D camera and a server, a 3D camera that captures a surface image of the vehicle including a boundary region of a plurality of panels and transmits the surface image to a server; a server that receives the surface image and measures the level difference of the vehicle; The server an image processing unit that sets a plurality of reference points in the surface image, the reference points being opposite to each other with respect to the boundary region, and crops a first region that includes all of the plurality of reference points; a surface coordinate system estimation unit that extracts first and second point clouds corresponding to the plurality of reference points from the first region, defines a coordinate system using the extracted point clouds, extracts a point cloud for one region based on the defined coordinate system, and corrects one or more unit vectors constituting the coordinate system using the extracted point clouds to redefine the defined coordinate system; a step calculation unit that projects a point cloud corresponding to the first area onto a plane based on a redefined coordinate system, and measures a step on the vehicle using the data projected onto the plane.

Images