Inspection system and method for a vehicle underbody

Abstract

The present invention relates to a vehicle underbody inspection system and method, the inspection system including a vehicle recognition unit for acquiring a vehicle ID by recognizing a vehicle entering an inspection process, a vision system for photographing a vehicle underbody by a plurality of cameras disposed vertically and at an angle below a vehicle movement direction (Y-axis) and along a width direction (X-axis) of the vehicle, and an inspection server for detecting assembly defects of components by performing at least one of a first visual inspection by matching object images of each component based on a rule-based algorithm or a secondary deep learning inspection by a deep learning engine, by acquiring a vehicle underbody image photographed by operating the vision system with setting information appropriate to a vehicle type and specifications according to the vehicle ID.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit of Korean Patent Application No. 10-2020-0074743, filed on June 19, 2020, with the Korean Intellectual Property Office, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This invention relates to a vehicle undercarriage inspection system and method, and more specifically, to an inspection system and method for automatically inspecting the undercarriage of vehicles assembled in a factory. Background Technology

[0004] Typically, vehicles are assembled in a factory through multiple production lines, and the assembly status of the vehicles is inspected at the completion stage.

[0005] For example, when an assembled vehicle enters the undercarriage inspection process, the operator, positioned inside a pit, visually inspects the assembly status of the undercarriage components by looking upwards. Additionally, after the inspection, the operator manually prepares an inspection report.

[0006] However, traditional methods of inspecting the underside of vehicles require continuous and repetitive upward looking movements in pits set up under the vehicle, which can lead to musculoskeletal problems in the operator's neck and shoulders.

[0007] Furthermore, there is the problem of increased fatigue due to the operator's upward-looking movements, and the occurrence of human errors due to traditional inspection methods may increase quality degradation and customer dissatisfaction due to the after-processing and distribution of defective vehicles.

[0008] Furthermore, when problems occur in the future, relying on the inspection report prepared by the operator will make it difficult to determine the exact cause, except for the operator who directly inspects the vehicle.

[0009] The information disclosed in the background section is intended only to enhance the understanding of the background technology of this application, and therefore the information it may contain does not constitute prior art that is already known to those skilled in the art in this country. Summary of the Invention

[0010] An exemplary embodiment of the present invention provides a vehicle undercarriage inspection system and method, which automatically detects assembly defects of components by acquiring vehicle undercarriage images using a vision system with various shooting angles and analyzing the images based on deep learning.

[0011] According to an exemplary embodiment of the present invention, a vehicle undercarriage inspection system in an in-line process includes: a vehicle identification unit for acquiring a vehicle ID by identifying a vehicle entering the inspection process; a vision system for capturing images of the vehicle undercarriage using multiple cameras positioned below the vehicle's direction of movement (Y-axis) and vertically and at an angle along the vehicle's width direction (X-axis); and an inspection server for detecting assembly defects of components by acquiring vehicle undercarriage images and performing at least one of a first visual inspection that matches object images of each component using a rule-based algorithm or a second deep learning inspection using a deep learning engine, wherein the vehicle undercarriage images are captured by operating the vision system with settings appropriate to the vehicle type and specifications based on the vehicle ID.

[0012] The vision system may include: a plurality of vertical cameras, which are equally spaced along the width direction on the upper surface of the base and capture images of the horizontal assembly components under the vehicle; a tilting camera, which is angled on both sides of the base to obtain images of the wheel housing part inside the tire; and a camera controller, which drives the plurality of vertical cameras and the tilting camera to capture images of the entire area under the vehicle (e.g., the underside of a moving vehicle) according to operating instructions from the inspection server, and sends the captured images of the vehicle underside to the inspection server.

[0013] The multiple cameras can be used as area scan camera types for correcting the inspection cycle time and inspection location for each frame.

[0014] The multiple cameras can be used as a global shutter type to capture images of the underside of a vehicle (e.g., the underside of a moving vehicle).

[0015] The vision system may include LED flat panel lights disposed on the upper surface of the base and LED ring lights disposed on the mounting surface of each tilting camera, and each light is filtered for diffuse reflection by a polarizing filter.

[0016] The vision system can adjust the tilt angle (θ) of each tilt camera through a tilt camera mounting unit including at least one servo motor, and change the setting position in the up / down and left / right directions.

[0017] The vision system may further include a vertical lift for vertically changing the position of a base on which multiple cameras are mounted.

[0018] The vision system can be installed to move back and forth according to the equipment environment via a linear motion (LM) guided type moving device mounted on the lower part.

[0019] The inspection server may include: a communication unit, which includes at least one wired / wireless communication device for communicating with the vehicle identification unit and the vision system; an image processing unit, which distinguishes and stores vehicle undercarriage images captured at the same time point for each ID of the vertical and tilt cameras; a database (DB), which stores various programs and data for vehicle undercarriage inspection, and stores data in DB form by matching the inspection result data of each vehicle ID with the corresponding image; and a controller, which analyzes the vehicle undercarriage images to detect the assembly status and defects of components, but for vehicles with normal first visual inspection results, the second deep learning inspection is omitted, and the second deep learning inspection is only performed on vehicles with poor first visual inspection results.

[0020] The image processing unit can generate a single vehicle undercarriage image by matching images captured by multiple camera IDs, and adjust noise and brightness by preprocessing the vehicle undercarriage image and removing the background to extract the region of interest (ROI) of each component.

[0021] The controller can perform ROI correction based on the vehicle's position on the vehicle's undercarriage image, identify the object image of each component (i.e., the entire image) in the vehicle's undercarriage image, and perform ROI position correction on a per-object-image basis.

[0022] The controller can convert the object image into a grayscale image, compare the regional feature values ​​of each angle through template matching, extract the matching score based on the comparison, and determine that it is defective if the matching score is less than a predetermined reference value.

[0023] The controller can divide the ROI into multiple regions defined according to the characteristics of the component, use labels to distinguish the divided regions, and compare the label ratio of each region divided by the label with a reference ratio to determine whether the component has assembly defects.

[0024] The controller can compare the label ratios in the regions divided by the labels to determine whether the component has an assembly defect based on the change in ratio when any label ratio is omitted.

[0025] Deep learning engines can use convolutional neural networks (CNNs) to analyze images that are deemed defective by the first visual inspection, and learn after labeling normal and defective components.

[0026] The deep learning engine can determine whether a component is defective based on whether the component exists in the analyzed image, and identify images with backgrounds similar to normal components as defective.

[0027] On the other hand, according to one aspect of the present invention, a method for inspecting the undercarriage of a vehicle is provided, which is a method for inspecting the undercarriage of a vehicle entering the inspection process in an in-line process, the method comprising: a) obtaining a vehicle ID by means of a barcode or wireless communication antenna of the vehicle entering the inspection process; b) taking pictures of the undercarriage of the vehicle by driving a vision system positioned below the vehicle's direction of movement (Y-axis), wherein multiple cameras are arranged vertically and at an angle along the vehicle's width direction (X-axis); c) performing a first visual inspection of the object image of each component by means of a rule-based algorithm after acquiring the undercarriage image; d) omitting a secondary deep learning inspection for vehicles whose first visual inspection result is normal, and performing a secondary deep learning inspection by means of a deep learning engine only for vehicles whose first visual inspection result indicates defects, to determine whether a component is defective.

[0028] Performing a first visual inspection may include: performing region of interest (ROI) correction on the vehicle undercarriage image based on the vehicle's position; performing object image recognition on each component (i.e., the entire image) in the vehicle undercarriage image; and performing ROI position correction on an object image basis; converting the object image to a grayscale image; comparing the region feature values ​​at each angle using template matching; extracting a matching score based on the comparison; and determining a defect if the matching score is less than a predetermined reference value; dividing the ROI into multiple regions defined according to the characteristics of the component; distinguishing the divided regions by labels; and comparing the label ratio of each region divided by the labels with a reference ratio to determine if the component has an assembly defect; and comparing the label ratios in the regions divided by the labels with each other, and determining if the component has an assembly defect based on the change in ratio when one label ratio is omitted.

[0029] Performing a deep learning inspection may include: using a convolutional neural network (CNN) to analyze images only for those that are defective according to the first visual inspection; determining whether a component is defective based on the presence of the component in the analyzed image; and identifying images with backgrounds similar to normal components as defective.

[0030] After step d), the process may further include: displaying the result identified as the final defect on the screen, sending an alert to the operator, and storing the inspection result data, including the vehicle ID and the location of the defect, component item, inspection time, and object image, in the DB.

[0031] According to an exemplary embodiment of the present invention, the effect is that inspecting the underside of a vehicle using a vision system with various shooting angles can prevent musculoskeletal disorders and human errors caused by conventional operator visual inspection.

[0032] In addition, by using precise settings and deep learning data based on vision systems suitable for various vehicle types / specifications, and by determining whether defects have occurred based on objective and quantitative data, the reliability of inspections can be improved and the cost of on-site claims can be reduced.

[0033] Furthermore, by storing the image-based vehicle undercarriage inspection results in a database, it is possible to predict assembly problems at that time. By using this as a basis for correcting the causes and processes of assembly defects that occur frequently, it is expected that assembly quality and production volume in the factory can be improved. Attached Figure Description

[0034] Figure 1 and Figure 2 This is a schematic diagram illustrating the configuration of a vehicle undercarriage inspection system viewed from the side (X-axis) and front (Y-axis) according to an exemplary embodiment of the present invention.

[0035] Figure 3 A schematic diagram illustrating the configuration of a vision system according to an exemplary embodiment of the present invention.

[0036] Figure 4 A schematic diagram illustrating a list of inspection components for the undercarriage of a vehicle according to an exemplary embodiment of the present invention.

[0037] Figure 5 A flowchart illustrating a method for inspecting the underside of a vehicle according to an exemplary embodiment of the present invention is provided.

[0038] Figure 6 The diagram illustrates in detail the first and second inspection processes of a method for inspecting the underside of a vehicle according to an exemplary embodiment of the present invention.

[0039] Figure 7 This is a schematic diagram illustrating the operation mechanism of the first and second inspection processes according to an exemplary embodiment of the present invention.

[0040] Figure 8 This is a schematic diagram illustrating a second deep learning inspection and analysis method according to an exemplary embodiment of the present invention.

[0041] Figure 9 This is a schematic diagram illustrating the inspection result data NG according to an exemplary embodiment of the present invention.

[0042] Figure 10 and Figure 11 This is a schematic diagram illustrating the configuration of a vehicle undercarriage inspection system according to another exemplary embodiment of the present invention. Detailed Implementation

[0043] It should be understood that the term "vehicle" or "of a vehicle" or other similar terms as used herein generally includes motor vehicles, such as passenger cars including sport utility vehicles (SUVs), buses, trucks, and various commercial vehicles, boats including various vessels and ships, aircraft, etc., and includes hybrid vehicles, electric vehicles, plug-in hybrid electric vehicles, hydrogen-powered vehicles, and other alternative fuel vehicles (e.g., vehicles powered by non-fossil fuels). As mentioned herein, a hybrid vehicle is a vehicle with two or more power sources, such as both gasoline and electric power.

[0044] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should be further understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of the stated feature, value, step, operation, element, and / or component, but do not exclude the presence or addition of one or more other features, values, steps, operations, elements, components, and / or groups thereof. As described herein, the term “and / or” includes any and all combinations of one or more of the associated enumerations. Throughout the specification, unless expressly stated to the contrary, the term “comprising” and variations such as “including” or “comprising of” should be understood to imply the inclusion of the stated element but not exclude any other element. Furthermore, the terms “unit,” “device,” “section,” and “module” described in the specification mean a unit for performing at least one function and operation, and can be implemented by hardware components or software components and combinations thereof.

[0045] Furthermore, the control logic of this application can be implemented as a non-transient computer-readable medium on a computer-readable medium, containing executable program instructions that are executed by a processor, controller, etc. Examples of computer-readable media include, but are not limited to, ROM, RAM, optical disc (CD)-ROM, magnetic tape, floppy disk, flash drive, smart card, and optical data storage devices. The computer-readable medium can also be distributed across a network-connected computer system, allowing it to be stored and executed in a distributed manner, for example, via a telematics server or a controller area network (CAN).

[0046] In the detailed description below, only certain exemplary embodiments of the invention are shown and described simply by way of illustration. Those skilled in the art will recognize that various modifications can be made to the described embodiments without departing from the spirit or scope of the invention. Accordingly, the drawings and description should be considered illustrative rather than restrictive in nature. Throughout the description, the same reference numerals denote the same elements.

[0047] Throughout this specification, terms such as first, second, "A", "B", "(a)", "(b)" are used only to describe the individual components and should not be construed as limiting those components. These terms are used only to distinguish constituent components from other constituent components, and the nature or order of the constituent components is not limited by these terms.

[0048] In this specification, it should be understood that when a component is referred to as "connected" or "attached" to another component, the component may be directly connected or attached to the other component, or it may be connected or attached to the other component through other components inserted therein. In this specification, it should be understood that when a component is referred to as "directly connected or directly attached" to another component, the component may be connected or attached to the other component without any other components inserted therein.

[0049] A vehicle undercarriage inspection system and method according to an exemplary embodiment of the present invention will now be described with reference to the accompanying drawings.

[0050] Figure 1 and Figure 2 This is a schematic diagram illustrating the configuration of a vehicle undercarriage inspection system viewed from the side (X-axis) and front (Y-axis) according to an exemplary embodiment of the present invention.

[0051] refer to Figure 1 and Figure 2 The vehicle undercarriage inspection system according to an exemplary embodiment of the present invention includes a vehicle identification unit 10, a vision system 20, and an inspection server 30.

[0052] Vehicle identification unit 10 identifies vehicles that have entered the vehicle inspection process and are mounted on a hanger (e.g., a conveyor) that serves as an in-line process moving device.

[0053] The vehicle identification unit 10 can be implemented using at least one of a barcode scanner 11 and a wireless communication antenna 12.

[0054] Barcode scanner 11 identifies the barcode affixed to the vehicle and sends the vehicle's ID (e.g., vehicle identification number (VIN)) to inspection server 30. Barcode scanner 11 can be an automatic scanner fixed to the barcode's attachment location or a manual scanner used by an operator.

[0055] The wireless communication antenna 12 may be a directional antenna, and it identifies vehicle entry based on the access of the wireless OBD installed on the vehicle entering the vehicle inspection process, thereby sending the vehicle ID (e.g., VIN) received wirelessly to the inspection server 30. Additionally, the vehicle identification unit 10 may also include a vehicle inspection sensor (not shown) that checks for vehicle entry and inspects hangers without vehicle attachments using ultrasonic or laser signals.

[0056] The vision system 20 is positioned at the lower part of the vehicle's direction of movement (Y-axis) and captures images of the vehicle's underside using multiple cameras 22a and 22b positioned at different angles according to the vehicle's width direction (X-axis).

[0057] For cameras 22a and 22b, area scan type cameras can be used to correct the inspection cycle time and inspection position for each frame. Additionally, cameras 22a and 22b can use a global shutter method with high frame rate and high resolution to capture the underside of fast-moving vehicles.

[0058] Figure 3 A schematic diagram illustrating the configuration of a vision system according to an exemplary embodiment of the present invention.

[0059] refer to Figure 3 According to an exemplary embodiment of the present invention, the vision system 20 has four vertical cameras 22a equally spaced along the width direction (X-axis) on the upper surface of the base 21, and two tilting cameras 22b tilted along a predetermined direction (e.g., a direction tilted upwards at a predetermined angle from the bottom of the base) on the side of the base 21. Here, the four vertical cameras 22a acquire images of the horizontally assembled components of the vehicle's undercarriage, and the two tilting cameras 22b respectively capture images of the wheel-receiving portion inside the tire at an angle.

[0060] Additionally, the vision system 20 includes an LED flat panel light 23a disposed on the upper surface of the base 21 and an LED ring light 23b disposed on the mounting surface of the tilting camera 22b. The LED flat panel light 23a and the LED ring light 23b can suppress diffuse reflection in the direct light method by increasing the light size relative to the inspected target, and minimize diffuse reflection by using a polarizing filter.

[0061] The vision system 20 uses four vertical cameras 22a and two tilting cameras 22b to capture the entire area under the vehicle. At this point, the vision system 20 ensures a field of view of approximately 400mm or more compared to the actual vehicle to accommodate various variations such as vehicle type, specifications, component distribution, and hanger positioning.

[0062] The vision system 20 can adjust the tilt angle (θ) of the tilt camera 22b and adjust the up / down and left / right setting positions via the tilt camera mounting unit 24, which includes at least one servo motor.

[0063] In addition, the vision system 20 can vertically adjust the position of the base 21 on which multiple cameras 22a and 22b are mounted by the vertical lift 25.

[0064] The camera controller 26 controls the overall operation of the vision system 20 and includes at least one microcontroller unit (MCU), hardware, software, memory, and communication devices within the housing for this purpose.

[0065] The camera controller 26 drives the vision system 20 to capture the entire area under the vehicle according to the operation instructions of the inspection server 30, and sends the captured images of the vehicle's underside to the inspection server 30.

[0066] By further including an LM (linear motion) guided forward and backward movement device 27 mounted below the camera controller 26, the vision system 20 can be mounted to be movable along the vehicle's direction of movement (Y-axis direction) according to the equipment environment.

[0067] The vision system 20 is not limited to the number of cameras and lights mentioned above, but can be changed according to the inspection conditions / environment of the vehicle.

[0068] refer to Figure 2 According to an exemplary embodiment of the present invention, the inspection server 30 includes a communication unit 31, an image processing unit 32, a database (DB) 33, and a controller 34.

[0069] The communication unit 31 includes at least one wired / wireless communication device for communicating with the vehicle identification unit 10 and the vision system 20.

[0070] When a vehicle is detected entering, the controller 34 sends an operation command to the vision system 20 via the communication unit 31 and receives the vehicle undercarriage image captured by the vision system 20.

[0071] The image processing unit 32 divides the vehicle undercarriage images captured at the same time point by each ID of the vertical camera 22a and the tilt camera 22b of the vision system 20, and stores them in the database 33.

[0072] The image processing unit 32 generates a single vehicle undercarriage image by matching images captured by each ID of a plurality of vertical cameras 22a and tilt cameras 22b.

[0073] The image processing unit 32 adjusts the noise and brightness of the image by preprocessing the vehicle undercarriage image, and removes the background to extract the region of interest (ROI) of each component.

[0074] Figure 4 A schematic diagram illustrating a list of inspection components for the undercarriage of a vehicle according to an exemplary embodiment of the present invention.

[0075] refer to Figure 4 The vehicle's undercarriage includes various components such as underbody panels, drivetrain components, anti-roll bars, muffler fasteners, and tie rods, which serve as key inspection components. These components can vary depending on the vehicle type and specifications. Inspection types include whether undercarriage components are installed, whether they are properly joined, whether they are of different specifications, component spacing, damage, and leaks.

[0076] According to an exemplary embodiment of the present invention, DB 33 stores various procedures and data for vehicle undercarriage inspection, and generates a database by matching the inspection result data of each vehicle ID with the corresponding image. That is, DB 33 ensures quality management by storing the location, type, time, and process of defects in the vehicle's undercarriage components based on inspection result data, and can be used as the basis for component defect causes and corresponding component defect handling and correction.

[0077] The controller 34 controls the overall operation of the vehicle undercarriage inspection system, which automatically detects the assembly status of components and the occurrence of component defects by analyzing images of the vehicle undercarriage acquired using the vision system 20 according to an exemplary embodiment of the present invention.

[0078] For this purpose, the controller 34 can be implemented as at least one processor operated by a predetermined program, which can be programmed to perform each step in the vehicle undercarriage inspection method according to an exemplary embodiment of the present invention.

[0079] The following diagram describes the method for inspecting the underside of a vehicle, but it specifically illustrates the characteristics of the controller 34.

[0080] Figure 5 A flowchart illustrating a method for inspecting the underside of a vehicle according to an exemplary embodiment of the present invention is provided.

[0081] Figure 6 The diagram illustrates in detail the first and second inspection processes of a method for inspecting the underside of a vehicle according to an exemplary embodiment of the present invention.

[0082] Figure 7 This is a schematic diagram illustrating the operation mechanism of the first and second inspection processes according to an exemplary embodiment of the present invention.

[0083] First, refer to Figure 5 According to an exemplary embodiment of the present invention, the controller 34 of the inspection server 30 receives the vehicle ID from the vehicle identification unit 10 to identify the entry of the vehicle (S1).

[0084] Based on the setting information corresponding to the vehicle ID, the controller 34 operates the vision system 20 to capture images of the vehicle's underside (S2). At this time, the controller 34 can query the vehicle type and specifications matching the vehicle ID from the production information system (MES) or database 33, and issue operating instructions to set the vision system 20 using the camera and its corresponding light settings.

[0085] The controller 34 receives vehicle underside images captured by the vision system 20 (S3). In this case, the controller 34 can perform preprocessing of the vehicle underside images via the image processing unit 32 and extract the ROI of each component.

[0086] As a first step, the controller 34 performs a first visual inspection analysis, which analyzes the image acquired by the image processing unit 32 using a rule-based algorithm (S5).

[0087] refer to Figure 6 and Figure 7 Describe in detail the first visual inspection and analysis process.

[0088] The controller 34 performs overall ROI correction based on the vehicle's position on the image of the vehicle's undercarriage (S41). At this time, the controller 34 can perform ROI correction based on the vehicle's position by utilizing the entire image frame by frame.

[0089] When assembling a vehicle, since the paint on the connecting components such as bolts may peel off, or production or assembly deviations may occur for each component, the controller 34 can extract the object image of each component from the entire image and perform ROI position correction on a per-object-image basis (S42).

[0090] The controller 34 converts the object image into a grayscale image and compares the feature values ​​of a predetermined region at each angle with the template of a predetermined template by matching the converted grayscale object image with the template (S43). That is, the controller 34 can compare the features of a predetermined region at each angle of the object image (e.g., a predetermined region every 20 degrees) with the features of a predetermined region at each angle of the template. At this time, the controller 34 can extract a matching score based on the above comparison. If the matching score is less than a predetermined reference value, it is determined that the matching is defective.

[0091] The controller 34 can perform histogram matching by using a pre-generated histogram of the orientation of edge pixels in each segmented region of a reference object image as a template. This histogram matching method is suitable for recognizing objects or vehicle components whose internal patterns are not complex but whose external contour information of the object image is unique. However, components such as bolts have polygonal (square, hexagonal, etc.) heads that rotate during engagement, which may affect the detection rate in the case of template matching. Therefore, the controller 34 can compare the matching rate with the template by calculating the average number of pixels in a predetermined region at each angle of the object image.

[0092] The controller 34 divides the ROI into multiple regions defined according to the characteristics of the components, and uses labels to distinguish the divided regions (S44). For example, a component such as a bottom cover has paint markings on its parts for assembly work. Within a specific ROI of the bottom cover, the paint markings can be divided into body labels (label 1), paint marking labels (label 2), and bolt color labels (label 3). Therefore, the ROI can be divided into regions according to each label.

[0093] The controller 34 compares the label ratio of each area divided by the label with each predetermined reference ratio (S45). For example, body labels, paint marking labels, and bolt color labels can exist at predetermined ratios in the areas divided by the label. In the areas divided by the label, there can be 50% body labels, 30% paint marking labels, and 20% bolt color labels. Therefore, the controller 34 can compare each ratio of the body labels (label 1 = 50%), paint marking labels (label 2 = 30%), and bolt color labels (label 3 = 20%) with each reference ratio in the areas divided by the label. By comparing these ratios, the presence of a defect can be determined by the deviation between the label ratio and the reference ratio.

[0094] In addition, the controller 34 compares the ratios of each label in the area divided by the label with each other (S46). For example, by comparing the ratio of the vehicle body label in the area divided by the label (label 1 = 50%) with the ratio of the bolt color label (label 3 = 20%), it is possible to determine whether a defect exists based on the ratio that changes when a joint component is missing.

[0095] Additionally, the controller 34 can compare the vector components (magnitude and orientation) of pixels in a predetermined region at each angle with a reference value using a histogram of gradient (HOG) algorithm to identify those smaller than the predetermined value as defects (S47).

[0096] Refer again Figure 5 If, after performing the first visual inspection analysis as described above, it is determined to be OK without any defects (S5; Yes), then the controller 34 displays the vehicle undercarriage inspection result (OK) on the screen (S6). Then, the controller 34 converts the inspection result data and stores it in the database (S10). At this point, if the vehicle is determined to be normal in the first visual inspection analysis, the controller 34 ends the inspection without performing the secondary deep learning inspection described later.

[0097] This is because, due to the time interval during which the first visual inspection result is output within a few seconds after the vehicle passes through the vision system 20, performing a second deep learning inspection on all vehicles requires a large computational load and a significant amount of time.

[0098] On the other hand, when a vehicle enters the repair process due to poor assembly, it is usually removed from the hanger and the operator has to lift it again for a re-inspection to check the undercarriage of the vehicle, which is tedious and labor-intensive.

[0099] Therefore, according to an exemplary embodiment of the present invention, the controller 34 does not immediately send the vehicle identified as defective in the first inspection to the repair process, and uses secondary deep learning inspection analysis for verification.

[0100] In other words, if the result of the first visual inspection analysis is determined to be defective (NG) (S5; No), the controller 34 performs a second deep learning inspection analysis before sending the vehicle to the repair process (S7).

[0101] Here, for reference Figures 6 to 8 Describe in detail the secondary deep learning inspection and analysis process.

[0102] Figure 8 This is a schematic diagram illustrating a second deep learning inspection and analysis method according to an exemplary embodiment of the present invention.

[0103] refer to Figure 6 and Figure 8 The controller 34 sends the image identified as defective to the deep learning engine 34a (S71), and operates the deep learning engine 34a, which is suitable for the corresponding component, by using the image as input information (S72).

[0104] The deep learning engine 34a separates the input image into separate learning images that are 1.5 times the size of the ROI (Region of Interest). At this point, the deep learning engine 34a analyzes the image using convolutional neural networks (CNNs), but convolution operations can be performed by applying a 3×3 filter to the input image.

[0105] Deep learning engine 34a learns after labeling normal components and defective components. In this case, because the number of defective images is limited compared to normal images (determined by whether or not a component is present), deep learning engine 34a can learn images with backgrounds similar to the defects.

[0106] This deep learning engine 34a utilizes a technique that distinguishes between normal and defective components based on the presence / absence of defects without detecting defective components. That is, it only determines whether additional defects exist in images identified as defective by a rule-based algorithm based on a first visual inspection analysis.

[0107] Refer again Figure 5 If the result of the above-mentioned secondary deep learning inspection and analysis is determined to be normal (OK) (S8; Yes), the controller 34 will display the final vehicle undercarriage inspection result (OK) determined to be normal on the screen (S6). Then, the controller 34 stores the inspection result data in DB (S10).

[0108] On the other hand, if the result of the secondary deep learning inspection analysis determines that there is a defect, the controller 34 can display the vehicle undercarriage inspection result (NG) determined to be the final defect on the screen and can warn the operator about the vehicle undercarriage inspection result (S9). Then, the controller 34 stores the inspection result data in the DB (S10).

[0109] Figure 9 This is a schematic diagram illustrating the inspection result data NG according to an exemplary embodiment of the present invention.

[0110] refer to Figure 9 According to an exemplary embodiment of the present invention, the controller 34 graphically displays the vehicle ID and the location of the defects on the vehicle undercarriage based on the vehicle undercarriage inspection results, and generates data of the corresponding parts, locations, component items, inspection times and images that have been saved.

[0111] The controller 34 accumulates the generated inspection result data in DB 33 and stores it in DB to ensure quality management based on statistical data, and can be used in the future as a basis for assembling defect causes and defect assembly processes for correcting defects that occur more frequently.

[0112] As described above, according to an exemplary embodiment of the present invention, the following effects are achieved: inspecting the underside of a vehicle using a vision system with various shooting angles can avoid musculoskeletal disorders and human errors caused by conventional operator visual inspection.

[0113] In addition, by using precise settings and deep learning data based on vision systems suitable for various vehicle types / specifications, and by determining whether defects have occurred based on objective and quantitative data, the reliability of inspections can be improved and the cost of on-site claims can be reduced.

[0114] Furthermore, by storing the results of image-based undercarriage inspections of vehicles in a database, it is possible to predict assembly problems at that time. By using this data as a basis for correcting the causes and processes of assembly defects that frequently occur, it is expected that the assembly quality and production volume of the factory can be improved.

[0115] Although exemplary embodiments have been described above, the invention is not limited thereto, and various other modifications are possible.

[0116] For example, in the above exemplary embodiments of the present invention, a fixed vision system 20 has been described as being mounted on a gantry to photograph the underside of a moving vehicle; however, the present invention is not limited thereto, and the vision system 20 may take pictures while the vehicle is in motion.

[0117] Figure 10 and Figure 11 This is a schematic diagram illustrating the configuration of a vehicle undercarriage inspection system according to another exemplary embodiment of the present invention.

[0118] First, refer to Figure 10 In the above exemplary embodiments, the process of mounting the vehicle on the hanger and moving it is mainly described. However, depending on various processing methods, the inspection can be carried out while the vehicle is parked at a predetermined inspection position.

[0119] Therefore, as the vision system 20 moves along the Y-axis in the space under the stationary vehicle via a linear motion (LM) guided front / rear movement device 27 mounted on the bottom, the vision system 20 can capture images of the entire area under the vehicle.

[0120] Additionally, refer to Figure 11 The vision system 20 is mounted on the end effector of the multi-joint robot and can take pictures of the underside of the vehicle while it moves through the kinematic posture control of the robot, or the front / rear movement device 27 can be set on the bottom of the robot so as to take pictures while it moves.

[0121] According to the present invention, there is an advantage that the vehicle undercarriage inspection system is capable of being adapted to various environmental conditions in the production line process.

[0122] The embodiments of the present invention described above are not implemented solely by methods and apparatus, but can be implemented by a program for performing functions corresponding to the exemplary embodiments of the present invention, or by a recording medium on which the program is recorded. These embodiments can be implemented by those skilled in the art through the description of the exemplary embodiments above.

[0123] While the invention has been described in conjunction with what are now considered exemplary embodiments, it should be understood that the invention is not limited to the disclosed embodiments. Rather, the invention is intended to cover various modifications and equivalents included within the spirit and scope of the appended claims.

Claims

1. A system for inspecting a vehicle underbody of a production line process, comprising: a vehicle recognition unit for acquiring a vehicle ID by recognizing a vehicle entering an inspection process; a vision system for photographing a vehicle underbody by a plurality of cameras disposed vertically and at an angle in a vehicle width direction below a vehicle moving direction, wherein the vehicle moving direction is a Y-axis direction and the vehicle width direction is an X-axis direction; and an inspection server for performing a first visual inspection by matching an object image of each component based on a rule-based algorithm by acquiring a vehicle underbody image photographed by operating the vision system with setting information suitable for a vehicle type and specifications according to the vehicle ID, omitting a secondary deep learning inspection for a vehicle showing a normal result of the first visual inspection, and performing a secondary deep learning inspection only for a vehicle showing a defective result of the first visual inspection to determine whether the component is defective through a deep learning engine, thereby detecting an assembly defect of the component; wherein the inspection server is configured to divide a region of interest into a plurality of regions defined according to characteristics of the component during the first visual inspection, distinguish the divided regions with tags, and compare a tag ratio of each region distinguished by the tags with a reference ratio to determine whether the component has an assembly defect.

2. The inspection system of a vehicle underbody of a production line process according to claim 1, wherein, The vision system comprises: a plurality of vertical cameras disposed at equal intervals in a width direction on an upper surface of a base and photographing a horizontally assembled part of the vehicle underbody; an inclined camera disposed at an angle on both sides of the base to acquire an image of a wheel housing portion inside a tire; and a camera controller photographing an entire area of the vehicle underbody by driving the plurality of vertical cameras and the inclined camera according to an operation instruction of the inspection server and transmitting the photographed vehicle underbody image to the inspection server.

3. The inspection system of vehicle underbodies of a production line process according to claim 1, wherein, The plurality of cameras apply an area scan camera type to correct an inspection period time and an inspection position of each frame.

4. The inspection system of vehicle underbodies of a production line process of claim 1, wherein, The plurality of cameras apply a global shutter type to photograph the vehicle underbody.

5. The inspection system of vehicle underbodies of a production line process of claim 2, wherein, The vision system includes an LED flat light disposed on the upper surface of the base and an LED ring light disposed on a mounting surface of each inclined camera, and each light filters diffuse reflection through a polarizing filter.

6. The inspection system of vehicle underbodies of a production line process of claim 2, wherein, The vision system adjusts an inclination angle of each inclined camera through an inclined camera mounting unit including at least one servo motor and changes a setting position in an up / down direction and a left / right direction.

7. An inspection system of a vehicle underbody of a production line process according to claim 6, wherein, The vision system further includes a vertical lifter for vertically changing a position of the base on which the plurality of cameras are disposed.

8. The vehicle underbody inspection system of claim 2, wherein, The vision system is installed to reciprocally move according to a device environment through a front and rear moving device of a linear motion guide type mounted at a lower portion.

9. The vehicle underbody inspection system of claim 2, wherein, The inspection server comprises: a communication unit including at least one wired / wireless communication device for communicating with the vehicle recognition unit and the vision system; an image processing unit distinguishing and storing a vehicle underbody image photographed at the same time point for each ID of the vertical cameras and the inclined cameras; a database storing various programs and data for vehicle underbody inspection and storing data in the form of a database by matching inspection result data for each vehicle ID with a corresponding image; and a controller analyzing a vehicle underbody image to detect an assembly state and defects of components.

10. An inspection system of a vehicle underbody of a production line process according to claim 9, wherein, The image processing unit generates a single vehicle underbody image by matching images captured by a plurality of camera IDs, adjusts noise and brightness by pre-processing work on the vehicle underbody image, and removes a background to extract a region of interest for each component.

11. An inspection system of a vehicle underbody of a production line process according to claim 10, wherein, The controller performs region of interest correction according to a vehicle position on the vehicle underbody image, recognizes an object image for each component in the vehicle underbody image, and performs position correction of the region of interest in units of the object image.

12. An inspection system of a vehicle underbody of a production line process according to claim 11, wherein, The controller converts the object image into a gray scale image, compares region feature values for each angle by template matching, extracts a matching score according to the comparison, and determines that there is a defect if the matching score is less than a predetermined reference value.

13. An inspection system of a vehicle underbody of a production line process according to claim 12, wherein, The controller compares label ratios in regions divided by labels to each other, thereby determining whether components have assembly defects according to a ratio that changes when any one label ratio is omitted.

14. The vehicle underbody inspection system of claim 10, wherein, The deep learning engine analyzes images using a convolutional neural network only for images that are defective as a first visual inspection result, and learns after labeling normal components and defective components.

15. An inspection system of a vehicle underbody of a production line process according to claim 14, wherein, The deep learning engine determines whether a component is defective according to whether the component exists in an analyzed image, and determines an image of a background similar to a normal component as defective. 16.An inspection method of a vehicle underbody, comprising: a) obtaining a vehicle ID through a barcode or a wireless communication antenna of a vehicle entering an inspection process; b) photographing a vehicle underbody by driving a vision system disposed below a vehicle movement direction, wherein a plurality of cameras are disposed vertically and at an angle along a vehicle width direction, the vehicle movement direction is a Y-axis direction, and the vehicle width direction is an X-axis direction; c) performing a first visual inspection of an object image for each component through a rule-based algorithm by acquiring a vehicle underbody image; d) omitting a secondary deep learning inspection for a vehicle that is normal as a first visual inspection result, and performing a secondary deep learning inspection through a deep learning engine only for a vehicle that is shown as defective in the first visual inspection to determine whether components are defective; wherein performing the first visual inspection includes dividing a region of interest into a plurality of regions defined according to characteristics of components, distinguishing the divided regions by labels, and comparing a label ratio of each region divided by the labels with a reference ratio to determine whether components have assembly defects.

17. The inspection method of claim 16, wherein, Performing the first visual inspection includes: performing region of interest correction according to a vehicle position on the vehicle underbody image, recognizing an object image for each component in the vehicle underbody image, and performing position correction of the region of interest in units of the object image; The object image is converted into a gray scale image, the area characteristic values of each angle are compared by template matching, and a matching score is extracted according to the comparison, and if the matching score is less than a predetermined reference value, it is determined to be defective; Each label ratio in the area divided by the label is compared with each other, and whether the assembly has an assembly defect is determined according to the ratio when one label ratio is omitted.

18. The inspection method of claim 16, wherein, The deep learning inspection includes: An image is analyzed using a convolutional neural network only for an image having a defective result in the first visual inspection; It is determined whether the component is defective according to whether the component exists in the analyzed image, and an image having a background similar to a normal component is determined to be defective.

19. The inspection method of claim 16, further comprising: After step d), a result determined as a final defect is displayed on a screen, and inspection result data including a vehicle ID and a defect occurrence position, a component item, an inspection time, and an object image are stored in a database.

Images