Automatic visual inspection equipment for workpieces with complex curved surfaces

Abstract

To provide an automatic vision test device for a workpiece with a complex curved surface.SOLUTION: A camera and a light source are provided in a test unit supporting frame. One camera and a plurality of light sources arranged in the camera form one test unit. The distance between the light sources and an object is at least five times as large as the field of vision of the camera. When an imaging work is performed using the same test unit, the light sources are sequentially emit light. The camera takes an image of the object every time the light sources emit light one time. One imaging by the camera requires one light source alone to emit light. The regions with the highest illuminance of the object of each light source in the same test unit do not overlap with each other.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to the technical field of visual inspection, and more particularly to an automatic visual inspection device for workpieces having complex curved surfaces. [Background technology]

[0002] Visual inspection technology, which uses image sensors instead of human eyes to measure and evaluate objects, is widely used in industry. A typical machine vision system consists of three main components: lighting, lenses, and cameras. Because there are no universal lighting devices for machine vision inspection, optimal imaging is achieved by selecting the appropriate lighting device for each application. Furthermore, different lighting methods can reveal different types of defects during defect inspection. On production lines, it is not uncommon for a single lighting method to fail to reveal all types of defects, resulting in a high rate of oversight. Parts with complex curved surfaces, in particular, are not suitable for a single lighting method. Examples include a "U"-shaped rail or a hood with multiple stamped slopes.

[0003] For example, steelworks produce standard lengths of rail, usually 25m or 100m. These standard lengths are transported to a rail welding base, where they are welded together to form a 500m-long seamless rail. The rail is then transported to the railway line for installation. Specialized equipment is used to weld each 500m-long rail into a seamless rail, determining the desired position. This eliminates the need for mechanical connections and ensures smooth train operation. To form a welded rail base, two rails are welded together to form a weld tumor, which protrudes above the rail surface. The weld tumor is then polished to a smooth transition, meeting industry standards for welds and welds around the rail, ensuring smooth and safe train operation.

[0004] Existing rail surface quality inspection methods rely on manual inspection or the use of a micrometer to measure the rail surface depth at two or more points, while mechanized and automated inspection programs do not. Manual inspections cannot achieve the required inspection rate for pits and cracks on the surface of the rail base material, and cannot inspect the grinding traces and grinding quality of rail welds, making it impossible to accurately measure the processing costs of rail welds.

[0005] Taking food casings as an example, commonly used 2D visual inspection equipment uses strip light source lighting, which cannot fully detect defects such as dents, scratches, and holes on oblique surfaces. Coupled with the fact that food casings are made of pressed stainless steel, which causes severe light interference, the defect inspection rate and accuracy are low. Summary of the Invention

[0006] According to a first aspect of the present invention, there is provided an automatic visual inspection device capable of inspecting the surface quality of a workpiece having a complex curved surface.

[0007] Surface quality inspection equipment, which mainly consists of a camera and a surface or ring light source, is commonly used on industrial production lines. The light source is constantly turned on and the camera shutter is controlled to capture a two-dimensional image or a series of images, which are then used to inspect the surface for defects. This type of visual inspection equipment has the problem that the light source position cannot be changed and the lighting method is single. For workpieces with complex curved surfaces, it is not possible to excite light from multiple angles, making it impossible to see all types of defects on the workpiece surface, resulting in a high rate of false inspections.

[0008] Furthermore, some visual inspection devices are primarily composed of a camera and laser line, and the camera shines a laser line beam onto the workpiece to take a picture, then converts the beam into a point cloud of 3D information to inspect the surface quality.The problem with this type of inspection device is that the laser line spacing is large, usually in the millimeter range, so a sparse point cloud is obtained, making it impossible to obtain dense information about the workpiece surface; it is suitable for quantitative measurement of specific areas, and is not easily adapted to inspecting surface defects (scratches, dents, protruding pucks, dents, etc.).

[0009] SUMMARY OF THE INVENTION An object of the present invention is to provide a visual inspection apparatus that can present a wide variety of surface defects using a general-purpose illumination method and acquire high-density three-dimensional surface information of an object.

[0010] An automatic visual inspection device for workpieces with complex curved surfaces, comprising: at least one defect inspection unit, the defect inspection unit including a camera, a light source, and an inspection unit support frame, the camera and the light source being mounted on the inspection unit support frame; The light sources in the same inspection unit are designed not to overlap with each other in the area of ​​strongest illumination on the object under inspection, where the area of ​​strongest illumination on the object refers to the circular envelope area of ​​the light source with the highest reflectance on the object.

[0011] Preferably, the light sources are controlled by a controller having preset lighting rules for the light sources, each having its own light source code, and each camera having its own camera code, and the lighting rules include the correspondence between the camera codes and the light source codes and the lighting order of the light sources. For example, camera number 1 corresponds to light source numbers 01, 04, and 06, and light source numbers 01, 04, and 06 are individually lit in sequence.

[0012] Preferably, the inspection unit support frame is circular, the center of the inspection unit support frame is located within the mounting area of ​​the camera, the surface of the inspection unit support frame is used as a reference plane, and each light source in each group has an equal projection distance from the camera on the reference plane, where the equal projection distance is not an absolute value in a mathematical sense but is determined to be equal within an allowable deviation range taking into account various factors in engineering applications.

[0013] Preferably, the inspection unit frame is polygonal, and the inspection unit frame consists of a rectangular tube member and a triangular connecting member assembled by fastening members, the rectangular tube member forms the skeleton of the inspection unit frame, the triangular connecting member is arranged between the connecting portions of the two rectangular tube members, one mounting edge of the triangular connecting member contacts one of the rectangular tube members and is fixed by the fastening member, and the second mounting edge of the triangular connecting member contacts the other rectangular tube member and is fixed by the fastening member, and the included angle formed by the connection of the two rectangular tube members is the same as the included angle formed by the two mounting edges of the triangular connecting member.

[0014] Furthermore, a relay is fixedly attached to the rectangular tube member, each light source corresponds to a relay, and an electric wire is connected between the relay and the light source, one end of the electric wire is fixed to the connection terminal of the relay, the other end of the electric wire is fixed to the connection terminal of the light source, and the electric wire starts from the output of the relay and first extends outward away from the rectangular tube member and then extends inward toward the rectangular tube member, thereby ensuring a stable electrical connection between the light source and the relay.

[0015] Furthermore, signal input wires are provided at the inputs of the relays, and each signal input wire is led from the input of the relay first to the front and then to the rear, with the input of the relay at the front and the output of the relay at the rear, thereby ensuring a stable connection between the relay and the signal input leads.

[0016] Preferably, each light source is mounted on its light source support frame, the light source support frame including a peg and a pendant, the peg is fixed to the inspection unit support frame, the light source is fixed to the pendant, the pendant is parallel to the light source support frame, the pendant and the peg are rotatably connected, and a resistor exists between the pendant and the peg.

[0017] Furthermore, the rotation range between the pendant and the peg is ±10°, which allows the same group of light sources to be projected on the reference plane at the same distance from the camera.

[0018] Furthermore, the peg has a mounting portion extending in a direction away from the inspection unit support frame, the peg has a beam portion and a connecting portion extending in a direction toward the inspection unit support frame, the light source is fixed to the beam portion, the connecting portion overlaps the mounting portion, and a light source rotation axis is located between the connecting portion and the mounting portion, and the light source rotation axis is fixed to the connecting portion or the light source rotation axis is fixed to the mounting portion. By rotating the light source rotation axis, the irradiation range of the light source on the measured object can be adjusted.

[0019] Furthermore, the light source rotation shaft is fixed to the connector and connected to the output shaft of the light source drive motor. The presence of the light source rotation shaft allows the light source position to be adjusted to accommodate objects of different sizes. Using an encoder or angle measuring device, the relationship between the object size and the light source rotation angle can be obtained, facilitating quick calibration of the light source position when changing objects.

[0020] Preferably, with the camera at the center and any one plane perpendicular to the camera axis as the reference plane, the distance between the light sources and the camera is equal for light sources in the same group, and random for light sources in different groups.

[0021] Preferably, the polar coordinate angles of the light source are randomly set around the camera, so that the light source can be irradiated onto the object as discretely as possible, and images of the workpiece to be measured can be obtained using multiple different light irradiating methods, revealing a greater variety of surface defects.

[0022] Preferably, the ratio of the distance between any two adjacent light sources to the shooting distance from the camera to the subject is 0.1 to 0.2. By setting it in this way, the image captured by the camera will be clear and the areas illuminated by the light sources on the subject will not overlap.

[0023] The object to be inspected is a railway rail, and the solution using the visual inspection device described above is applied. The quality requirements for the rail surface include the clean removal of welds and weld pegs around the welds, a smooth polished surface, a round and smooth rail foot (the curved section at the transition from the rail waist to the rail bottom), no lateral polishing, and no loss of rail material. "Lateral" means along the length of the rail.

[0024] An automated visual inspection system for rail surface quality, comprising at least one set of inspection systems, the inspection systems targeting rails that are located on a production line and have completed a weld removal process, the inspection systems being set up after the first rail weld removal process, or there are two sets of inspection systems, the first set of inspection systems being set up after rail misalignment inspection and before the first rail weld removal process, and the second inspection device being installed after the first rail weld removal process.

[0025] Furthermore, the rail inspection device has a frame on which the automatic inspection device is mounted, the frame has side panels and a top panel, the side panels and the top panel enclose the frame in a darkroom, one side panel has an entrance through which the rail can enter, and the opposite side panel has an exit through which the rail can exit.

[0026] Preferably, the side panels may be provided with doors or beds, or the side panels may be removably attached to the frame.

[0027] Preferably, there are four groups of inspection devices, each corresponding to one steel rail inspection area, and the inspection devices surround the inspection area, with the steel rail entering the inspection area along its axis. The distance between two opposing inspection devices is 1050 mm or more, making the inspection device suitable for steel rail inspection in long-rail production lines.

[0028] Preferably, the four sets of inspection devices include an upper surface inspection mechanism aligned with the upper surface of the rail, a lower surface inspection mechanism aligned with the lower surface of the rail, a first rail lower surface inspection mechanism aligned with the rail lower surface, and a second rail lower surface inspection mechanism aligned with another rail lower surface, and the two rail lower surface inspection mechanisms include an upper inspection unit, an intermediate inspection unit, and a lower inspection unit, each inspection unit having a camera, a light source, and a light source control device, the camera axis of the upper inspection unit being inclined downward at an angle of 27° to 34° relative to the horizontal, the camera axis of the intermediate inspection unit being parallel to the horizontal, and the camera axis of the lower inspection unit being inclined upward at an angle of 41° to 49° relative to the horizontal.

[0029] Preferably, any set of inspection devices on the rail waist surface includes a back plate, and the upper inspection unit, intermediate inspection unit and lower inspection unit are each attached to the back plate via their respective position adjustment mechanisms, and the back plate includes a main plate, upper wings and lower wings, the main plate is parallel to the height direction of the rail to be inspected, the upper wings intersect the main plate, the upper part of the upper wings is closer to the rail to be inspected than the lower part of the upper wings, and the lower wings intersect the main plate, and the upper part of the lower wings is farther away from the rail to be inspected than the lower part of the lower wings.

[0030] Preferably, the camera of the upper inspection unit is positioned near the area where the upper blade and the main board intersect, and a portion of the light source of the upper inspection unit is set in the area covered by the upper blade, and another portion of the light source of the upper inspection unit is set in the area covered by the main board.

[0031] Preferably, the camera and its light source of the intermediate detection unit are set in the area covered by the main board.

[0032] Preferably, the camera of the lower inspection unit and its light source are positioned in the area covered by the lower vanes.

[0033] In this way, any set of rail waist surface inspection equipment can achieve full-surface inspection of the surface located below the rail upper part, the side of the rail waist, the small circular surface at the transition between the surface below the rail upper part and the side of the rail waist, the large circular surface at the transition from the rail waist to the rail lower part, and the vertical surfaces adjacent to the rail waist and the rail lower part.

[0034] Furthermore, in the manufacture of long rails, the height of the welding rod (weld seam) at the bottom of the rail must be 0.5 mm or more after removal of the welding tumor, which necessitates quantitative measurement of the rail after grinding, and there are also demands for rail surface inspection.

[0035] Based on the rail surface quality requirements, the inspection indicators for rail surface defects after welding tumor removal are specified as the presence or absence of lateral grinding marks and the presence or absence of a step between the grinding surface and the rail base material, and a rail surface defect inspection method is proposed using the above-mentioned inspection device or inspection system.

[0036] A method for inspecting the surface quality of steel rails uses the above-mentioned inspection device to inspect surface defects on steel rails. In each photographing task, the same inspection unit sequentially lights up a light source, and a camera takes images of the object being measured. The camera has only one light source lit for each photograph. Each photographing task obtains multiple images of the object being measured, and each image of the object being measured corresponds to a light source. When the object is illuminated, the image contains both object information and light information. The spatial information and inter-frame information of the object image are combined to reconstruct a normal map of the object's surface. Positions in the normal map that satisfy a gradient change feature threshold are identified and marked as defects. This allows for inspection of steps between the ground surface and the rail base material. By combining the spatial information and inter-frame information of the object to obtain a normal map of the object's surface, it is possible to eliminate interference from false defects caused by highlights and chromatic aberrations on the ground surface, and retain only true processing defects in the normal map.

[0037] This method uses the solution provided by the above-mentioned automatic surface quality inspection system for complex curved workpieces, and performs full surface quality inspection of rails using the above-mentioned automatic rail inspection system.

[0038] Preferably, lines are extracted from the normal diagram, and it is determined whether the direction of the lines is parallel or perpendicular to the length direction of the rail. If a line perpendicular to the length direction of the rail appears, it can be marked as a defect. For long rails, the length direction of the rail is fixed, so the length direction of the rail can be used as the basis for judgment. This makes it possible to check for polishing marks in the lateral direction of the rail.

[0039] Preferably, before obtaining the normal map of the measurement object, the image of the measurement object is first subjected to threshold division, and image regions smaller than a set threshold are removed, and only the remaining regions are subjected to inter-frame information extraction and spatial information extraction.

[0040] Preferably, after inspecting the rail surface for defects, a line laser is used to illuminate the rail surface with a linear light bar, and a camera is used to obtain the rail and light bar in the rail-light bar image. Quantitative measurements are then performed using the laser. The light bar is then divided into discrete points along the X direction from the rail-light bar image, and the pixel coordinates of the light bar center are extracted. The minimum vertical coordinate of the light bar is then found. The normal vector (n) of any pixel point on the light bar is used to extract the pixel coordinate of the light bar center. x ,n y ) to find the pixel coordinates of the center of the light bar (x0+tn x ,y0+tn y ) where

number

[0041] Preferably, before calculating the light bar center coordinate, the light bar boundary of the rail light bar image is roughly extracted using a threshold method, and the light bar width w is calculated from the roughly extracted light bar boundary to generate a Gaussian convolution kernel of the corresponding width. Finally, this Gaussian convolution kernel is used for the inner region of the light bar to accurately extract the light bar center using the above formula calculation to obtain the light bar center coordinate; The pixel coordinate of the center of the i-th light bar is (x i ,y i), the horizontal pixel coordinate of the center point of the light bar is x i Then, the point with the smallest vertical coordinate of the center of the light bar is determined as the feature point C, and a differential calculation and Gaussian filter are performed on each center of the light bar, and the first derivative k i ,

number

number

[0042] Specifically, the specific implementation process of the light bar center coordinate extraction method is as follows: Step 1: Set a threshold value th and input the rail / light bar image I to be processed. Step 2: Preprocess the rail / light bar image I. Step 3: Scan all pixels in the column sequentially to find the coordinates of the first pixel in the column that is greater than th and the last pixel that is greater than th. Record these as (B1, col) and (B2, col), respectively, and use these as the coordinates of the light bar boundary obtained by rough extraction. Step 4: Using the light bar boundaries obtained by rough extraction, calculate the light bar width for each column. The light bar width for column i is w i = |B1-B2|, the maximum light bar width is w max =max{w i Note that |N≧i≧1}, where N is the number of columns in the image. Step 5: Arithmetic array M = {10, 20, , w max} to generate element m in the array. i Using this, length and width are calculated

number

number

[0043] Specifically, the specific algorithm steps of the weld seam location algorithm are as follows: Step 1: Extract the light bar center from the light image of the line structure, and define the i-th light bar center as P i and the image coordinates are written as (x i ,y i ) Step 2: Light bar center P extracted in Step 1 i The above is repeated, and the point with the smallest ordinate is set as the feature point C. Step 3: For the centroid of the light bar extracted in Step 1, calculate the first derivative at each point using equation (4-2), and filter using a one-dimensional Gaussian filter. The filter size is set to 51, μ is set to 0, and σ is set to 17. Step 4: Using the first-order differential obtained by Gaussian filtering in Step 3, calculate the second-order differential at each center point. Step 5: Search for the minimum value of the second-order differential on the left and right sides of the characteristic point C, and determine the minimum value of the second-order differential on the left and right sides of the characteristic point C as the characteristic points of the rail weld boundary.

[0044] Preferably, a systematic correction is made when calculating the three-dimensional coordinates of the light bar center. A horizontal auxiliary measurement standard C is introduced, and the distance m from the auxiliary measurement standard C to the calibration standard B is i First, determine the distance n from the measurement point to B. i The distance m from C to B at the corresponding point i The displacement can be obtained by subtracting from H1 to obtain Hi, calculating H1 and H2 at selected measurement points located on both sides of the weld, and calculating |H1-H2|.

[0045] The smoke collector cover is the workpiece to be inspected, and the above-mentioned visual inspection device is applied to the production line of the smoke collector cover for oil smoke machines.

[0046] An automated visual inspection station for use in a fume hood production line, the visual inspection station being set up after a fume hood stamping station, the stamping station and the visual inspection station performing material transport by a loading and unloading robot, the visual inspection station including an inspection chamber and an inspection system located within the inspection chamber, the inspection chamber having a supply opening, the inspection chamber having a material tray, the material tray being located within an area covered by the supply opening when viewed from the front, the inspection system having a side located above the material tray and the supply opening, a plurality of side shades surrounding the inspection system, the inspection chamber having a top shade above the inspection chamber, the top shade and side shades surrounding a darkroom, and the inspection system including the inspection device as described above.

[0047] Furthermore, the material carriage includes a carriage and a lifting component, the lifting component includes a fixed part and a moving part, the moving part is fixed to the carriage. Both sides of the carriage are provided with minimum stops, and the distance between the two minimum stops is smaller than the width of the workpiece to be measured. The width of the carriage is smaller than the distance between the two minimum stops.

[0048] The system further includes a robot arm that retrieves the workpiece to be measured from the press line at a first position and whose conveying end rotates 180° to reach a second position, and a conveying end that enters the conveying opening when the conveying end is located at the second position. When the movable part of the lifting kit is at the lowest position, the distance from the ceiling surface of the lowest stopper to the ceiling surface of the carriage is greater than the height of the feed end.

[0049] Furthermore, when the movable part of the lifting kit is at the highest position, the distance between the carriage ceiling surface and the light source of the inspection device is five times the field of view of the camera of the inspection device.

[0050] Furthermore, the workpiece to be measured has a first inspection area including a top surface, a bottom surface, and a connecting surface, which are four inclined surfaces with the top and bottom surfaces parallel to each other, and a second inspection area. The second inspection area is a folded edge perpendicular to the top surface. The first inspection device inspects the first inspection area, and the second inspection device inspects the second inspection area.

[0051] In a second aspect of the present invention, there is provided a data processing system for reconstructing three-dimensional surface information of a workpiece to be measured from a plurality of images illuminated from different angles.

[0052] A data processing system includes a data processor, a graphics processor, and a data memory. The data processor reads the input image from the data memory into the graphics processor, which identifies the preset image network. The image includes the input image and is then subjected to a maximum pooling layer and a method resolution unit to maximize the effective features of the input image. The feature extractor then passes it through an initial fusion module, which then fuseds the input image information with the initial information fusion module's output. The feature extractor then extracts inter-frame information and a spatial information extractor. The inter-frame information extractor and spatial information extractor use 3D convolution, with inter-frame information occupying one dimension and spatial information occupying two dimensions. The input of the inter-frame information extractor is the output of the initial fusion module, and the input of the spatial information extractor is the output of the inter-frame information extractor. The output of the spatial information extractor serves as the input of the maximum pooling layer.

[0053] Furthermore, the normal-directed solution unit first performs a convolution operation on the output of the max pooling layer, and then performs L2 regularization, and the result of the regularized output is a normal-directed graph.

[0054] Furthermore, the final convolution operation to the solution unit uses three channels.

[0055] Additionally, other convolutional operations in the image recognition network use 128 channels.

[0056] Furthermore, the step of fusing spatial information and inter-frame information of the image of the measurement object includes:

number

[0057] Each image has three channels: R, G, and B, and the light source information map at each capture is characterized by three coordinate values: X, Y, and Z, with each coordinate direction value stored in one channel. By running the three-channel information of each image and the three-channel information of the corresponding light source information map, these six channels of information on a single 6x1x1 convolution core, this image and the corresponding light source information are fused, preventing the one-to-one correspondence from being disturbed in subsequent operations.

[0058] Furthermore, interframe information extraction is performed using an Interframe Information Extractor (IRFE), which consists of a convolution kernel of size 5x1x1, a Leaky Relu activation function, and a dropout layer.

number

[0059] Furthermore, the step of extracting inter-frame information using the IRFE includes setting an initial input of the IRFE to F0;

number

[0060] Furthermore, spatial information extraction is performed using a convolution kernel of size 1 × 3 × 3 and a Leaky Relu activation function (IAFE). i (·)=σ(Con 1×3×3 i This is done using an IAFE (spatial information extractor) consisting of (·)), where · represents the input to the spatial information extractor, 1×3×3 i (·) represents the i-th convolution of the input, and the convolution kernel of size 1×3×3 is σ(Con 1×3×3 i (·)) is Con 1×3×3 i Activation for (·) is shown.

[0061] Furthermore, the initial input to IAFE is F IRF K Then,

number

[0062] Furthermore, F IAF L as input to max pooling, F MAX =MP(F IAF L ),(9) MP stands for max-pooling operation, and F MAX denotes the output of max pooling. This step extracts the most important information between different channels and also fixes the input channels for later operations.

[0063] Furthermore, F MAX The normal vector is calculated using the input.

number

[0064] The inspection device of the present invention is for acquiring images and can be used as an independent acquisition device. The data processing system of the present invention can operate if an image is input, and the input image may be acquired by the inspection device of the present invention or by another route. [Effects of the Invention]

[0065] The beneficial effects of the present invention are as follows: 1. The camera and three or more light sources can form multiple groups of different reflection paths and light paths, forming three or more directions of the light source irradiated object. Each group of different locations of high-reflection areas and light paths can be used to depict the shapes of defective and normal products from multiple angles through the integration of deep learning, thereby obtaining complete regional data that can effectively solve the problem that a single perspective may not reflect the true appearance of the product.

[0066] 2. After optimizing the relative positions of the light source and camera, a standardized light source-camera module can be produced, and the layout of the light source-camera module can be adjusted for different shapes and sizes of objects to be measured. Multiple surface defect types can be displayed using a universal lighting method, solving the problem of different products requiring different lighting methods for different inspection methods and different defect types.

[0067] 3. Image fusion of multiple optical paths captures complete area data, achieving excellent shine resistance.

[0068] 4. When used in rail inspection, it can replace the manual visual inspection and micrometer measurement required on current long-rail production lines, and fully automate the qualitative defect inspection of rail surface quality and quantitative inspection of processing allowance, achieving high inspection efficiency and high inspection accuracy.Qualitative defect inspection can automatically inspect milling marks and grinding marks on rail welds, automatically identify cross-grinding patterns and alarms, and automatically identify the removal, angle, and step of dirty welds and alarms.

[0069] When used for fume hood inspection, it can automate the production of the entire fume hood assembly line and identify the location of defects; multiple consecutive defects of the same type in the same location can indicate a malfunction in the press mold or process.

[0070] The data processing device of the present invention combines the optical information, inter-frame information, and spatial information of an image, removes interference such as highlights, and retains the surface shape information of the workpiece, thereby better presenting the surface condition of the workpiece and improving the defect inspection rate. [Brief explanation of the drawings]

[0071] [Figure 1] FIG. 10 is a schematic diagram of an inspection device in which the inspection unit bracket is circular. [Figure 2] FIG. 10 is a schematic diagram of an inspection device in which the inspection unit support columns are rectangular. [Figure 3] This is the wiring diagram of Figure 2. [Figure 4] FIG. 1 shows an inspection of a rail using an inspection unit consisting of multiple cameras and light sources. [Figure 5] FIG. 5 is a layout diagram of the inspection device of FIG. [Figure 6] FIG. 6 is a layout diagram of the upper inspection section, middle inspection section, and lower inspection section of FIG. 5. [Figure 7] FIG. 2 is a schematic perspective view of a camera and a light source. [Figure 8] FIG. 1 is a layout diagram of a camera and a light source. [Figure 9] FIG. 10 is a schematic diagram of a visual inspection station when inspecting a smoke collection cover. [Figure 10] FIG. 10 is a schematic diagram showing the removal of the light-shielding plate in FIG. 9. [Figure 11] 1 is a schematic diagram of a network for image fusion using deep learning. [Figure 12] FIG. 1 is a schematic diagram of a process for fusing images using deep learning. [Figure 13] This is a comparison of rendering image results using the MERL dataset. DETAILED DESCRIPTION OF THE INVENTION

[0072] The present invention will be described in detail below with reference to the drawings. The present invention provides an automatic rail inspection device for inspecting the surface quality of rails. The surface quality of rails after grinding is inspected, for example, quantitatively and qualitatively.

[0073] As shown in FIG. 1, in some embodiments, an automated visual inspection device for complex curved workpieces has at least one defect inspection unit, which includes a camera 1, a light source 2, and an inspection unit support frame 3. The camera 1 and light source 2 are mounted on the inspection unit support frame 3. The distance between the light source 2 and the object is at least five times the field of view of the camera, and the light sources are alternately turned on when the same inspection device is performing a photographing operation. Each light source in the same inspection unit is designed to not overlap with each other in the area of ​​the object where the illumination intensity of the light source is strongest. The area on the object where the light source is most strongly illuminated refers to the circular envelope area of ​​the light source with the highest reflectivity on the object.

[0074] As shown in FIG. 1 , some embodiments provide an automated inspection device including a camera 1, a light source 2, and an inspection unit support frame 3, where the camera 1 and the light source 2 are mounted on the inspection unit support frame 3. The camera 1 and multiple light sources 2 configured for the camera 1 form an inspection unit, and the light points of the light source 2 in the same inspection unit cover independent areas on the object. In a single shooting task of the same inspection unit, the light sources 2 are sequentially turned on, and the camera 1 photographs the object after the light source 2 is turned on. Only one light source 2 is turned on for each shooting by the camera 1. The positions of the light source 2 and the camera 1 are relatively fixed for each shooting. Multiple object images are acquired for each shooting task, and each object image corresponds to an image containing both object information and light information when the light source 2 illuminates the object. By fusing the spatial information and inter-frame information of the object images, a three-dimensional image representing the surface texture of the object can be obtained.

[0075] In some embodiments, the light sources are controlled by a controller having predefined lighting rules for light sources, each with a unique light source code, and cameras, each with a unique camera code. The lighting rules include the correspondence between camera codes and light source codes, the order in which the light sources are turned on, etc. The camera codes correspond to the light source codes; for example, camera 1 corresponds to light source 01, light source 04, and light source 06. Light source 01, light source 04, and light source 06 are turned on individually in this order.

[0076] 1, in some embodiments, the light sources 2 are fixed to the inspection unit support frame 3 by light source brackets 21, one light source is provided on each light source bracket 21, the light source bracket 21 includes a hanging ear 211 and a pendant 212, the hanging ear 211 is fixed to the inspection unit support frame 3, the light source is fixed to the pendant 212, the pendant 212 and the hanging ear 211 are rotatably connected, and the pendant 212 and the hanging ear 211 have the following damping: The rotation range between the pendant 212 and the peg 211 is ±10°. This allows the light sources of the same group to be projected equidistantly from the camera on the reference plane.

[0077] 1, the inspection unit support frame 3 is circular, and the center of the circle of the inspection unit support frame 3 is located within the mounting area of ​​the camera 1. Using the surface of the inspection unit support frame 3 as a reference plane, the light sources 2 of each light source group are projected onto the reference plane at equal distances from the camera 1. The equal projection distances referred to here are not absolute values ​​in a mathematical sense, but are equal within an allowable range of deviation, taking into consideration various factors in engineering applications.

[0078] In another embodiment, as shown in FIGS. 2 and 3, the inspection unit support frame is polygonal and the inspection unit support frame 3 is a rectangular parallelepiped. The inspection unit support frame 3 is formed by assembling rectangular pipe members 31 and triangular connecting members 32 with fasteners, and the rectangular pipe members 31 form the skeleton of the inspection unit support frame 3. A triangular connector 32 is provided between the joints of the two rectangular pipe members 31, and one mounting edge 321 of the triangular connector 32 abuts against one rectangular pipe member 31 and is fixed by a fastener. A second mounting edge 322 of the triangular connector 32 abuts against the other rectangular pipe member 31 and is fixed by a fastener. The angle formed by the connection of the two rectangular pipe members 31 is equal to the angle between the two mounting edges 321, 322 of the triangular connector 32.

[0079] 3, there is a relay 33 fixedly attached to the rectangular tube member, each light source 1 corresponds to a relay 33, and there is an electric wire 34 between the relay 33 and the light source, one end of the electric wire 34 is fixed to a terminal of the relay 33, and the other end of the electric wire 34 is fixed to a terminal of the light source 2, and the electric wire 34 starts from the output end of the relay 33 and first extends in an outer circumferential direction away from the rectangular tube member 31, then in an inner circumferential direction of the rectangular tube member 31. This ensures a stable electrical connection between the light source and the relay.

[0080] As shown in Figure 3, the input of the relay 33 is provided with a signal input lead 35, and each signal input lead 35 is led from the input of the relay 33, extending first forward and then backward. The input end of the relay 33 is used as the front, and the output end of the relay 33 is used as the rear. This ensures a stable connection between the relay 33 and the signal input lead 35. The signal input lead 35 is connected to an external power supply or signal source.

[0081] 1 and 2, in some embodiments, the peg 211 has an attachment portion extending away from the inspection unit support frame 3, the pendant 212 has a beam and a connection portion extending toward the inspection unit support frame 3, and the light source is fixed to the beam. The connection portion overlaps the attachment portion, and a light source pivot 213 is located between the connection portion and the attachment portion, and the light source pivot 213 is fixed to the connection portion or the light source pivot 213 is fixed to the attachment portion. By rotating the beam, the illumination range of the light source on the object can be adjusted.

[0082] 1, in some embodiments, the light source rotation shaft 213 is fixed to the connector, and the light source rotation shaft 213 is connected to the output shaft of the light source drive motor. The presence of the light source rotation shaft 213 allows the position of the light source to be adjusted, making it possible to adapt to objects of different sizes. Using an encoder or angle measuring device, the relationship between the size of the object and the rotation angle of the light source can be determined, allowing for quick calibration of the light source position when the object is changed.

[0083] 7 and 8, with camera 1 at the center and any plane perpendicular to the axis of camera 1 as the reference plane, light sources 2 are grouped according to the distance from camera 1, with light sources in the same group being at the same distance from camera 1 and light sources in different groups being at random distances from camera 1. As shown in the figures, two light sources 2-1 are grouped together and three light sources 2-2 are grouped together.

[0084] As shown in Figures 7 and 8, the ratio of the distance between any two adjacent light sources to the shooting distance from camera 1 to the subject is 0.1 to 0.2, and with this setting, camera 1 captures a clear image and the illumination areas of the light sources on the subject do not overlap.

[0085] For the production of long rails, the requirements for rail surface quality include: the welds around the weld pegs must be clean, the grinding surface must be smooth, the rail bottom foot (the transition from the rail bottom surface to the rail waist at curved sections) must be round and smooth, and there must be no cross grinding, and the rail base material must not be damaged.

[0086] When the above-mentioned automatic inspection device is applied to rail surface quality inspection, an automatic rail inspection system is formed. In some embodiments, the automatic rail inspection system is as shown in Figure 4, the inspection system includes at least one set of the above-mentioned inspection device, the system targets rails located on a production line, and the inspection system is set up after the completion of initial milling of the weld nub or the completion of initial grinding of the weld nub, or after the first rail weld removal process, or there are two sets of inspection systems, the first set of inspection system is set up after the rail misalignment inspection, and the second inspection device is set up after the first rail weld removal process.

[0087] As shown in FIG. 4 , in some embodiments, the rail inspection system has a frame 4 on which the automatic inspection device is mounted, the frame 4 having side panels and a top panel, the side panels and the top panel enclosing the interior of the frame 4 in a darkroom, and one of the side panels having an entrance through which the rails can enter and an exit on the opposite side panel through which the rails can exit.

[0088] In some embodiments, the side panels have doors or beds thereon, or the side panels are removably attached to the frame 4.

[0089] In long rail manufacturing, rail surface inspection requirements also include full cross-sectional inspection of rail welds.

[0090] 5 and 6, in some embodiments, there is a group of four inspection devices corresponding to the rail inspection area, the inspection devices surround the inspection area, and the rail enters the inspection area along the axial direction of the prism. The distance between two opposing inspection devices is set to 1050 mm or more, making it possible to inspect rails on long railway lines.

[0091] As shown in Figures 5 and 6, in some embodiments, the four sets of inspection mechanisms include an upper inspection mechanism 41 aligned on the upper surface of the rail, a lower inspection mechanism 42 aligned on the lower surface of the rail, a first rail waist inspection mechanism 43 aligned on one rail waist surface, and a second rail waist inspection mechanism 44 aligned on the other rail waist surface. Each of the two rail waist inspection mechanisms 43, 44 has an upper inspection section 4S, a middle inspection section 4Z, and a lower inspection section 4X. Each inspection unit has a camera 1, a light source 2, and a light source controller. The camera 1 axis of the upper inspection unit 4S is angled downward at an angle of 27° to 34° relative to the horizontal, the camera 1 axis of the middle inspection unit 4Z is parallel to the horizontal, and the camera 1 axis of the lower inspection unit 4X is angled upward at an angle of 41° to 49° relative to the horizontal.

[0092] As shown in Fig. 6, in some embodiments, an arbitrary set of inspection devices for a rail waist surface includes a back panel to which an upper inspection unit 4S, a middle inspection unit 4Z, and a lower inspection unit 4X are respectively attached by their respective position adjustment mechanisms, and the back panel includes a main panel 4Z1, an upper wing panel 4Z2, and a lower wing panel 4Z3, and the main panel 4Z1 is arranged parallel to the height direction of the rail M to be inspected. The upper wing panel 4Z2 intersects with the main panel 4Z1, and the upper part of the upper wing panel 4Z2 is located closer to the rail M to be inspected than the lower part of the upper wing panel 4Z2. The lower wing panel 4Z3 intersects with the main panel 4Z1, and the upper part of the lower wing panel 4Z3 is located farther from the rail M to be inspected than the lower part of the lower wing panel 4Z3.

[0093] As shown in FIG. 6, in some embodiments, the camera 1 of the upper inspection unit 4S is located near the area where the upper blade 4Z2 and the main board 4Z1 intersect, and part of the light source of the upper inspection unit 4S is set in the area covered by the upper blade 4Z2, and another part of the light source of the upper inspection unit 4S is set in the area covered by the main board 4Z1.

[0094] As shown in FIG. 6, in some embodiments, the camera 1 and its light source of the middle inspection unit 4Z are set in the area covered by the main board 4Z1.

[0095] As shown in FIG. 6, in some embodiments, the camera 1 and its light source of the lower inspection unit 4X are set in the area covered by the lower vane 4Z3.

[0096] In this way, any set of rail waist surface inspection equipment can achieve full-surface inspection of the surface located below the rail upper part, the side of the rail waist, the small circular surface at the transition between the surface below the rail upper part and the side of the rail waist, the large circular surface at the transition from the rail waist to the rail lower part, and the vertical surfaces adjacent to the rail waist and the rail lower part.

[0097] Furthermore, in the manufacture of long rails, rail surface inspection requires that the height of the weld bar (weld seam) at the bottom of the rail after grinding be 0.5 mm or more, and it is also true that there is a need to quantitatively measure the rail after grinding.

[0098] In some embodiments, the inspection device has a line laser 5, the axis of the line laser 5 and the axis of the camera 1 are oblique, the light source is turned on, the line laser 5 does not emit a laser, and the light source is not turned on when the line laser 5 emits a laser line.

[0099] Based on rail surface quality requirements, the presence or absence of lateral grinding marks and the presence or absence of a step between the ground surface and the rail base material are specified as inspection indicators for rail surface defects after grinding, and a method for inspecting rail surface defects using the above-mentioned inspection device or inspection system is proposed.

[0100] As shown in Figures 4 to 6, some embodiments of a method for inspecting surface defects on steel rails using a camera 1 and a group of light sources include a camera 1 and multiple light sources configured as an inspection unit. The same inspection unit is used for each shooting task, and the light sources are turned on in sequence. The light sources are turned on once, and then the camera 1 captures an image of the object to be measured. The camera 1 only has one light source illuminating it for each shooting. Multiple object images are acquired for each shooting task, and each object image corresponds to an image containing both object information and light information when the light source illuminates the object. The spatial information and inter-frame information of the object are combined to obtain a normal map of the object's surface. Positions on the normal map that satisfy a gradient change threshold are identified and marked as defects. This allows for inspection of steps between the ground surface and the rail base material. By combining the spatial information and inter-frame information of the object image to obtain a normal map of the object's surface, it is possible to eliminate interference from false defects caused by highlights and chromatic aberrations on the ground surface and retain only true processing defects in the normal map. The structure of the inspection unit utilizes the structure of the inspection device described above. The normal map is obtained using the above-mentioned data processing system.

[0101] In some embodiments, lines are extracted from the normal diagram, and it is determined whether the direction of the lines is parallel or perpendicular to the length of the rail, and if a line perpendicular to the length of the rail appears, the product is deemed defective. For long rails, the length direction of the rail is fixed. Therefore, the length direction of the rail can be used as the basis for judgment. By doing so, grinding marks in the horizontal direction of the rail can be checked.

[0102] In some embodiments, before obtaining the object's normal map, the image of the object is thresholded to remove image regions smaller than a set threshold, and only the remaining regions are subjected to inter-frame information extraction and spatial information extraction.

[0103] The visual inspection device was applied to a food recovery line for cooking hoods, with fume hoods being the inspected products.

[0104] As shown in Figures 9 and 10, in the automated visual inspection station of the fume hood production line, the visual inspection is installed after the fume hood stamping station, and the stamping station and the visual inspection station are transported by a loading and unloading robot. The visual inspection station is composed of an inspection room 6 and an inspection device installed in the inspection room. The inspection room is provided with a material inlet 61, and a material tray 62 is arranged in the area covered by the material inlet 61 when viewed from the front. The inspection device is located above the material tray 62, and above the material supply inlet 61 there are side shades 63, which surround the inspection device therein, and above the inspection room there is a top shade 64, and the top shade 64 and side shades 63 surround the darkroom. The inspection system is composed of inspection devices such as those described above.

[0105] The material tray 62 comprises a material tray 621 and a lifting assembly 622. The lifting assembly 622 comprises a fixed part and a moving part, and the moving part is fixed to the material tray 62. The lifting assembly is, for example, an electric actuator, a cylinder, or the like.

[0106] In some embodiments, as shown in Figure 9, the lowest limit 65 is provided next to the pallet 621. The lowest limit may be one or two symmetrical limits, and the distance between the two lowest limits is less than the width of the workpiece to be measured. The width of the pallet is less than the distance between the two limit members of the lowest level.

[0107] In some embodiments of the fume hood testing, the robot includes a robot arm and a feed end, where the feed end picks up a part to be tested from the stamping line at a first position, the feed end rotates 180 degrees to a second position, and the robot arm enters the feed opening with the feed end at the second position. When the movable part of the lifting assembly is at its lowest position, the distance from the top surface of the lowest member to the top surface of the pallet is greater than the height of the feed end.

[0108] When the movable portion of the lifting assembly is in its highest position, the distance between the top surface of the pallet and the light source of the inspection device is at least five times the field of view of the camera of the inspection device.

[0109] In some embodiments of the fume hood inspection, the inspected part comprises a first inspection area and a second inspection area, the first inspection area comprises a top surface, a bottom surface, and a joint surface, the joint surface is four chamfered surfaces, the top surface and the bottom surface are parallel, and the second inspection area is a folded portion perpendicular to the top surface. The first inspection device uses the first inspection area as an inspection target, and the second inspection device uses the second inspection area as an inspection target.

[0110] The present invention provides a data processing system for reconstructing three-dimensional surface information of an object from a number of images illuminated from different angles.

[0111] An image is read and input into a graphics processor, which is pre-programmed with an image recognition network as shown in Figures 11 and 12. The image recognition network is composed of an image input, a feature extractor that acquires effective features of the input image, a max pooling layer, and a normalized solution. The image is input into an initial fusion module that matches the current image with its lighting information, and the fused lighting information output from the initial fusion module is input into a feature extractor that includes an inter-frame information extractor and a spatial information extractor. The input of the inter-frame information extractor is the output of the initial fusion module, the input of the spatial information extractor is the output of the inter-frame information extractor, and the output of the spatial information extractor is used as the input of the max pooling layer.

[0112] In some embodiments, the normal solver unit performs a convolution operation on the output of the max pooling layer, followed by L2 regularization, with the regularized output producing a normal map. In some embodiments, the final convolution operation of the normal solver unit uses 3 channels. In some embodiments, 128 channels are used for other convolution operations in the image recognition network.

[0113] As shown in Figures 11 and 12, in some embodiments, spatial information and inter-frame information in an image of a measurement object are extracted, and both the spatial information and the inter-frame information are expressed as three-dimensional convolution, where the three-dimensional convolution includes two spatial dimensions and one inter-frame dimension, the value of the one-dimensional inter-frame dimension of the three-dimensional convolution of the spatial information is a set value, the two-dimensional spatial dimension is the spatial information value of the image, and the value of the three-dimensional convolution of the inter-frame information, and in the three-dimensional convolution of the spatial information, the two-dimensional spatial dimension is the set value, and the one-dimensional inter-frame dimension is the inter-frame information value.

[0114] In some embodiments, the step of fusing spatial information and inter-frame information of the measured object image comprises:

number

[0115] Each image has three channels: R, G, and B, and the light source information map at each capture is characterized by three coordinate values: X, Y, and Z, with each coordinate direction value stored in one channel. By running the three-channel information of each image and the three-channel information of the corresponding light source information map, these six channels of information on a single 6x1x1 convolution core, this image and the corresponding light source information are fused, preventing the one-to-one correspondence from being disturbed in subsequent operations.

[0116] In some embodiments, interframe information extraction is performed using an Interframe Information Extractor (IRFE), which consists of a convolution kernel of size 5x1x1, a Leaky Relu activation function, and a dropout layer.

number

[0117] In some embodiments, the step of extracting inter-frame information using an IRFE sets the initial input of the IRFE to F0.

number

[0118] In some embodiments, spatial information extraction is performed using a convolution kernel of size 1×3×3, a Leaky Relu activation function (IAFE), i (·)=σ(Con 1×3×3 i This is done using an IAFE (spatial information extractor) consisting of (·)), where · represents the input to the spatial information extractor, 1×3×3 i (·) represents the i-th convolution of the input, and the convolution kernel of size 1×3×3 is σ(Con 1×3×3 i (·)) is Con 1×3×3 i Activation for (·) is shown.

[0119] Furthermore, the initial input to IAFE is F IRF K Then,

number

[0120] In some embodiments, F IAF L as input to max pooling, F MAX =MP(F IAF L ),(9) MP denotes the max-pooling operation, and F_MAX denotes the output of max-pooling. This step extracts the most important information between different channels and also fixes the input channels for later operations.

[0121] Furthermore, F MAX The normal vector is calculated using the input.

number

[0122] As shown in Figure 11, the present invention discloses a 3D convolution-based 3D model for non-Lambertian surface photometry, which includes an information fusion layer, inter-frame information extraction, spatial information extraction, a max pooling layer, and a regression layer, and the regression layer is a normal solver unit.

[0123] In the information fusion layer, each image and the corresponding light source information are fused, and in subsequent processing, images and light sources are matched one-to-one. Inter-frame information extraction is used to extract inter-frame information and obtain the information between input image frames necessary for our normal map estimation. Spatial information extraction extracts structural information from a single image and restores a correct map. Max pooling layers are used for dimensionality reduction, removing redundant information, compressing features, simplifying network complexity, reducing computational effort, and reducing memory consumption.

[0124] The information fusion layer consists of a 3D convolutional layer C1, followed by a LeakyReLU activation function, with an output dropout ratio of 0.2.

[0125] It can be seen that the inter-frame information extraction consists of three layers: a three-dimensional convolutional layer C2, a three-dimensional convolutional layer C3, and a three-dimensional convolutional layer C4, each of which is followed by a LeakyReLU activation function, with a dropout rate of 0.2.

[0126] Spatial information extraction consists of three layers: a 3D convolutional layer C5, a 3D convolutional layer C6, and a 3D convolutional layer C7, each followed by a Leaky ReLU activation function.

[0127] The regression layer consists of three layers: a 2D convolutional layer C8, a 2D convolutional layer C9, and a 2D convolutional layer C10, each followed by a Leaky ReLU activation function, and the 2D convolutional layer C10 is followed by an L2 regularization function.

[0128] The method for recovering the normal map of an object for quantitative measurement is implemented by a deep learning network including an information fusion layer, inter-frame information extraction, spatial information extraction, a max pooling layer, and a regression layer, as shown in Figure 3 .

[0129] The inter-frame information extraction consists of three convolutional layers: a three-dimensional convolutional layer C2, a three-dimensional convolutional layer C3, and a three-dimensional convolutional layer C4. Each three-dimensional convolutional layer is followed by a LeakyReLU activation function, which outputs a dropout ratio of 0.2. The three-dimensional convolutional layer C2, the three-dimensional convolutional layer C3, and the three-dimensional convolutional layer C4 contain M 1*1 feature maps.

[0130] The spatial information extraction consists of three layers: a three-dimensional convolutional layer C5, a three-dimensional convolutional layer C6, and a three-dimensional convolutional layer C7, each followed by a Leaky ReLU activation function. The three-dimensional convolutional layer C5, the three-dimensional convolutional layer C6, and the three-dimensional convolutional layer C7 contain 1N*N feature maps. In image processing, inter-frame information extraction processing is performed first, followed by spatial information extraction processing. Here, in Example 1, M=1, N=3; in Example 2, M=3, N=3; in Example 3, M=5, N=3; and in Example 4, M=7, N=3.

[0131] Specific examples 5 to 7 will be described below. As shown in Figure 11, the 3D convolution-based non-Lambertian photometric stereo model includes an information fusion layer, inter-frame information extraction, spatial information extraction, max pooling layer, and regression layer.

[0132] Inter-frame information extraction includes three-dimensional convolutional layers: a three-dimensional convolutional layer C2, a three-dimensional convolutional layer C3, and a three-dimensional convolutional layer C4. Each three-dimensional convolutional layer is followed by a LeakyReLU activation function, and the output value dropout ratio of each LeakyReLU activation function is 0.2. The three-dimensional convolutional layer C2, the three-dimensional convolutional layer C3, and the three-dimensional convolutional layer C4 each include M 1*1 feature maps.

[0133] For spatial information extraction, there are three layers: 3D convolutional layer C5, 3D convolutional layer C6, and 3D convolutional layer C7, each of which is followed by a LeakyReLU activation function. 3D convolutional layer C5, 3D convolutional layer C6, and 3D convolutional layer C7 contain 1N*N feature maps. In image processing, spatial information extraction is performed first, followed by inter-frame information extraction. Here, in Example 5, M=5, N=1; in Example 6, M=5, N=3; and in Example 7, M=5, N=5.

[0134] The analysis results of the convolution data in Examples 1 to 8 were measured for images of different shapes, and the data shown in Table 1 was obtained. [Table 1]

[0135] In Table 1, the first row is the sample name, the first column is the convolution kernel size, and the median is the average angle error. Table 1 shows the effect that differences in convolution kernel size have on the network structure. N is the convolution kernel size of the inter-frame extractor, M is the convolution kernel size of the spatial extractor, and based on the average angle error values ​​of each example, the final selection was made to set the convolution kernel size of the inter-frame extractor at 5 and the convolution kernel size of the spatial extractor at 3.

[0136] A model was constructed according to the selected convolution layer data with an inter-frame extractor convolution kernel of size 5 and a spatial extractor convolution kernel of size 3, and this model was compared with other models and tested on the DiLiGenT dataset for images of different shapes, and the results are shown in Table 2. [Table 2]

[0137] The first row of Table 2 shows the sample name, the first column shows the method name, and the value in the middle shows the average angular error. This table shows a cross-sectional comparison between the proposed model and current state-of-the-art methods, with CNN-PS18, PS-FCN18, TM18, and DPSN17 all being deep learning-based methods, and the remaining methods all being based on traditional mathematical analysis. The average angular error values ​​show that the proposed model offers significant improvements over both traditional methods and deep learning-based methods.

[0138] A model was constructed according to the selected convolution layer data with an inter-frame extractor convolution kernel of size 5 and a spatial extractor convolution kernel of size 3, and this model was compared with other models of different sparse-input photometric stereo methods on the DiLiGenT dataset for the test data shown in Table 3. [Table 3]

[0139] In Table 3, the first row represents the input image, the first column represents the model name, and the middle value represents the average angular error. This is a cross-sectional comparison of our model with current state-of-the-art photometric stereo methods for sparse inputs. JU-19, CH-18, and SI-18 are all deep learning-based methods. JU-19 and CH-18 have specific structures for the sparse input problem, while the other methods are all conventional methods. The rest are conventional methods. Table 3 shows that our model does not have a complex structure for sparse inputs, and by simply using an inter-frame space extractor to improve information utilization, it can achieve better results for 16-image and 10-image inputs.

[0140] We selected a convolution kernel size of 5 for the inter-frame extractor and 3 for the spatial extractor, and constructed a model based on the convolution layer data. Figure 13 shows a comparison of images processed by this model photometric stereo method and images processed by the PS-FCN model stereo method based on the rendered images of the MERL dataset. As can be seen from Figure 13, this model has relatively high image processing stability and good accuracy compared to PS-FCN.

[0141] From the above, it has become clear that the present invention has good accuracy in normal vector recovery, the spatial information provides good information correction for high-illumination points and shadow areas, and adding spatial information can improve the robustness of the algorithm for abnormal areas, and the model of the present invention has high normal recovery accuracy while maintaining a high calculation speed, making it promising for industrial application.

[0142] The embodiments of the present invention can be used alone as technical solutions or can be combined with each other to form complex solutions. It will be understood that the embodiments described in the present invention are some preferred examples of embodiments and features, and that anyone skilled in the art can make some changes and modifications under the essence of the description of the present invention, which are also considered to fall within the scope of the present invention and the limits of the independent and dependent claims.

Claims

1. An automated visual inspection device for a workpiece having a complex curved surface, comprising: at least one defect inspection unit, the defect inspection unit including a camera, a light source, and an inspection unit support frame, the camera and the light source being mounted on the inspection unit support frame; One camera and multiple light sources arranged on the camera constitute one inspection unit, the distance between the light source and the object is five times or more the field of view of the camera, when photographing work is performed by the same inspection unit, the light sources are turned on sequentially, the camera photographs an image of the object each time the light source is turned on, only one light source is turned on in one photographing by the camera, the areas of the object with the strongest illumination of each light source in the same inspection unit do not overlap with each other, A visual inspection device, wherein each light source is provided on a corresponding light source support frame, the light source support frame includes a peg and a pendant, the peg is fixed to the inspection unit support frame, a light source is fixed to the pendant, the pendant is parallel to the light source support frame, the pendant and the peg are rotatably connected, and a resistance exists between the pendant and the peg.

2. 2. The visual inspection device according to claim 1, wherein the inspection unit support frame is circular, the center of the inspection unit support frame is located within the mounting area of ​​the camera, and the surface of the inspection unit support frame is used as a reference plane, and the projection distances of each light source in each group of light sources and the camera on the reference plane are equal.

3. the inspection unit support frame is polygonal, and the inspection unit support frame is assembled by a rectangular pipe member and a triangular connecting member via a fastening member, and the rectangular pipe member forms a skeleton of the inspection unit support frame; a triangular connecting member is provided between the connecting portions of the two rectangular pipe members, a first mounting edge of the triangular connecting member abuts against one rectangular pipe member and is fixed by a fastening member, and a second mounting edge of the triangular connecting member abuts against the other rectangular pipe member and is fixed by a fastening member, and the included angle formed by the two rectangular pipe members after connection is the same as the included angle between the two mounting edges of the triangular connecting member; 2. The visual inspection device according to claim 1.

4. The visual inspection device of claim 3, wherein a relay is fixed to the rectangular tube member, each light source corresponds to one relay, and an electric wire is provided between the relay and the light source, one end of the electric wire is fixed to the connection terminal of the relay, and the other end of the electric wire is fixed to the connection terminal of the light source, and the electric wire starts from the output end of the relay and first extends outward away from the rectangular tube member, and then extends inward toward the rectangular tube member, thereby ensuring a stable electrical connection between the light source and the relay.

5. 5. The visual inspection device of claim 4, wherein signal input wires are provided at the input ends of the relay, and each signal input wire is drawn out from the input end of the relay, first extending in a forward direction and then extending in a rearward direction, wherein the input end of the relay is the forward direction and the output end of the relay is the rearward direction.

6. 2. The visual inspection device of claim 1, wherein the range of rotation between the pendant and the peg is ±10 degrees.

7. the peg has a mounting portion extending away from the inspection unit support frame, the peg has a beam portion and a connecting portion extending toward the inspection unit support frame, the light source is fixed to the beam portion, The visual inspection device according to claim 6, characterized in that the connecting portion and the mounting portion partially overlap, a light source rotation axis is provided between the connecting portion and the mounting portion, and the light source rotation axis is fixed to the connecting portion or the light source rotation axis is fixed to the mounting portion.

8. 8. The visual inspection device according to claim 7, wherein the light source rotation shaft is fixed to the connecting portion, and the light source rotation shaft is connected to an output shaft of a light source drive motor.

9. 2. The visual inspection device according to claim 1, wherein the camera is set as the center and any one plane perpendicular to the camera axis is set as the reference plane, the light sources are grouped according to the distance between the light source and the camera, the distance between the light source and the camera is equal for light sources in the same group, and the distance between the light source and the camera for light sources in different groups is random.

10. The visual inspection device according to claim 9, characterized in that the camera is the center, the polar coordinates of the light source are random, and the ratio of the distance between any two adjacent light sources to the shooting distance from the camera to the subject is 0.1 to 0.

2.

11. 2. The visual inspection device of claim 1, wherein the light sources are controlled by a controller, and a lighting rule for the light sources is preset in the controller, each light source has a unique light source code, and the camera has a unique camera code, and the lighting rule includes a correspondence between the camera codes and the light source codes and a lighting sequence for the light sources.

Images