A method and system for image recognition of surface defects in automotive body panels

Abstract

This invention discloses a method and system for image recognition of surface defects in automotive body panels, relating to the field of defect detection technology. The method includes: extracting the main body outline, area, and edge sharpness based on multi-view images of the same automotive body panel, viewpoint calibration parameter sets, and a three-dimensional reference model of the body panel; selecting complete reference images of the body panel and matching local image blocks at the same location according to the coordinate relationship of the standard curved surface of the body panel to generate a reference image set for the body panel; determining the coating color distribution and reflection areas based on the reference image set, correcting the reflection areas to continuous coating texture, and generating a reflection correction texture map; and extracting multi-layer alignment features through the reflection correction texture map, continuous bright reflection areas, and the original bright texture residual map. This invention improves the accuracy of defect localization through edge posterior bi-branch reconstruction, body panel detection partitioning, and verification using a normal texture image block memory set.

Description

Technical Field

[0001] This invention relates to the field of defect detection technology, and in particular to a method and system for image recognition of surface defects in automotive body panels. Background Technology

[0002] As automobile manufacturing processes become more automated, flexible, and intelligent, the surface quality inspection of automotive body panels is gradually shifting from manual visual inspection to image recognition methods based on machine vision. Current methods for identifying surface defects in automotive body panels typically involve acquiring images of the panels at fixed workstations or from multiple viewpoints using industrial cameras. These images are then combined with color space conversion, edge extraction, texture analysis, depth feature extraction, and target detection methods to identify scratches, dents, particles, indentations, and coating anomalies. For body panels with complex curvatures and coating reflective properties, such as doors, fenders, hoods, and side panels, visual inspection methods usually require combining illumination control, image enhancement, feature alignment, and candidate region selection to obtain texture and boundary features suitable for defect assessment.

[0003] Existing methods for identifying automotive body panel defects still suffer from insufficient stability when dealing with highly reflective coatings, variations in surface brightness, and normal structural boundaries. On one hand, light strip reflections, surface glare, and localized overexposure can easily obscure shallow scratches, shallow dents, and paint particles, causing true defects to appear as discontinuous textures or weak edges in the image. On the other hand, hole edges, folded structures, reinforcing rib projections, and normal stamping lines can easily produce similar grayscale abrupt changes to true defect boundaries, making it difficult for ordinary edge detection or target detection methods to reliably distinguish between normal structural textures and surface defect textures. Furthermore, localized reflections and changes in shooting angle within a single image can also cause false defect responses, affecting the reliability of the surface defect identification results for body panels. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides an image recognition method for surface defects in automotive body panels to solve the problem of easy confusion between weak texture defects and normal structural boundaries in highly reflective curved body panels.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a method for image recognition of surface defects in automotive body panels, comprising, Based on multi-view images of the same automotive body panel, view calibration parameter groups, and 3D benchmark model of the body panel, the main body outline, area and edge sharpness are extracted. Complete body panel benchmark images are selected and local image block groups at the same position are matched according to the coordinate relationship of the standard curved surface of the body panel to generate a body panel benchmark image set. The coating color distribution and reflection area are determined based on the reference image set of the cover, and the reflection area is corrected to a continuous coating texture to generate a reflection correction texture map. Multi-layer alignment features are extracted from the reflection correction texture map, continuous bright reflection region and original bright texture residual map, and the directional texture distributed along the standard surface coordinate relationship of the cover is enhanced by combining the anomaly difference map obtained by reverse distillation to generate an anomaly difference directional feature map. The abnormal difference direction feature map is fed into the edge posterior dual-branch reconstruction. The suspected defects are located by using local texture destruction, abnormal boundary trend and cover detection partition to generate a candidate set of structural verification defects. A normal texture image block memory set is constructed based on the candidate set of structural verification defects, and texture verification at the same location is performed in combination with the reference image set of the cover part to generate the surface defect recognition result of the automotive cover part.

[0007] As a preferred embodiment of the image recognition method for surface defects of automotive body panels according to the present invention, the local image block group at the same position is generated by establishing the coordinate relationship of the standard curved surface of the body panel based on the outer contour and vertex position of the main body of the complete body panel reference image, the view calibration parameter group and the three-dimensional reference model of the body panel, mapping the main body area of ​​the other multi-view images to the corresponding curved surface position of the three-dimensional reference model of the body panel, and cropping the corresponding local image block according to the reference image position.

[0008] As a preferred embodiment of the automotive body panel surface defect image recognition method of the present invention, the specific steps for generating the body panel reference image set are as follows: Receive multiple images of the same automotive body panel formed at continuous work stations and adjacent viewpoints, read the camera intrinsic parameters, camera extrinsic parameters, lens distortion parameters and installation pose corresponding to each image, perform multi-color space transformation and body panel main area localization on each image, extract the main body outer contour, vertex position, area and edge sharpness, and generate body panel main evaluation set. By eliminating images to be identified that have incomplete main body regions, edge shadows, or areas whose area does not meet the range of the complete cover area through the evaluation set of the main body of the cover, the images to be identified with the complete outer contour of the main body and the highest edge clarity are retained to generate a complete cover reference image. The complete cover reference image and the local image block group at the same position are matched according to the coordinate relationship of the standard curved surface of the cover, and the visibility state, occlusion state and reprojection error state at the corresponding positions are recorded to generate the cover reference image set.

[0009] As a preferred embodiment of the automotive body panel surface defect image recognition method of the present invention, the specific steps for generating the reflection correction texture map are as follows: The coating hue distribution, coating saturation distribution, and coating brightness distribution are extracted from the complete cover reference image in the cover reference image set to generate the coating color distribution, and the normal coating brightness distribution is determined based on the coating color distribution; Based on the normal coating brightness distribution, the continuous high-brightness reflection area and the normal coating texture area are distinguished. The brightness residual, edge residual and local texture residual of the continuous high-brightness reflection area are saved to generate the original high-brightness texture residual map. The brightness of the continuous high-brightness reflection area is reduced, and the coating texture is compensated along the adjacent normal coating texture area to generate the reflection correction texture map.

[0010] As a preferred embodiment of the automotive body panel surface defect image recognition method of the present invention, the abnormal difference map is obtained by reverse distillation, and the specific steps are as follows: The teacher encoder was trained and fixed using calibration images of qualified parts of the same cover model, and normal stamping structure lines, hole edges and folded contours were used as normal structure areas for training. The teacher encoder is used to process the reflection correction texture map, extract the normal surface code corresponding to normal coating, normal curved surface and normal stamping structure, and generate teacher surface features; A single-class bottleneck embedding is used to compress the normal coating texture, normal surface orientation and normal stamping structure information in the teacher's surface features to generate normal surface embedding features. The single-class bottleneck embedding is formed by sequentially connecting a normal surface feature compression layer, a low-dimensional normal embedding layer and a normal surface feature pushback layer. The student decoder is used to back-push back the embedded features of the normal surface to recover the normal surface feature map that is consistent with the position of the main body area of ​​the complete cover, and generate back-push surface features. The student decoder is trained using feature reconstruction loss and structural partition consistency loss. By comparing the feature differences between the teacher surface features and the back surface features at the same cover location, areas that deviate from normal coatings, normal curved surfaces, and normal stamping structures are extracted to generate an anomaly difference map.

[0011] As a preferred embodiment of the automotive body panel surface defect image recognition method of the present invention, the specific steps for generating the abnormal difference direction feature map are as follows: Shallow texture features and deep semantic features are extracted by reflection-corrected texture map, and weak texture residuals that coincide with shallow edge details in the original bright texture residual map are superimposed. The outer contour and vertex position of the main body of the complete cover reference map are mapped to the same cover position according to the standard surface coordinate relationship of the cover, generating multi-layer alignment features. The abnormal texture locations in the multi-layer alignment features are marked by continuous regions in the abnormal difference map that are higher than the abnormal difference threshold of the corresponding partition, and the shallow edge details and weak texture residuals corresponding to the abnormal texture locations are preserved to generate multi-layer alignment features with abnormal indications. The system uses multi-layer alignment features with anomaly indicators to extract horizontal and vertical extended textures, oblique extended textures, and arc extended textures extending along the principal curvature direction of the cover surface. The system also marks the location of abnormal textures within the horizontal, vertical, oblique, and arc extended textures to generate directional enhanced texture features. By connecting texture fragments corresponding to discontinuous scratches, thin tear and edge indentation, diagonal scratches, curved indentation and local arc-shaped defects through directional enhancement texture features, an abnormal difference directional feature map is generated.

[0012] As a preferred embodiment of the automotive body panel surface defect image recognition method of the present invention, the specific steps for generating the structural verification defect candidate set are as follows: The local texture destruction branch is input using anomaly difference direction feature map, original bright texture residual map and local texture slice set to extract local texture destruction features; Using the abnormal difference direction feature map, normal structural boundary and cover detection partition input boundary posterior branch, the edge posterior features are extracted, the local texture slices on the surface of the cover are enhanced in detail, and the real defect boundary, normal stamping structure line, hole edge and fold contour are distinguished to generate local texture damage results and boundary trend abnormal results; Based on the comparison of local texture destruction results, boundary trend abnormal results and abnormal difference maps at the same standard surface position, regions that simultaneously possess texture destruction, boundary abnormality and abnormal difference response are retained by candidate region cascade recombination, and the retained regions are cascaded small-scale localized to generate suspected defect image blocks. Based on the three-dimensional reference model of the cover part, the structural boundary calibration results in the calibration image of the qualified part, and the process inspection location file, the hole edges, folded structures, stiffener projections, and normal assembly boundaries in the complete cover part reference drawing are determined. The cover part is divided into inspection zones, and suspected defect image blocks are verified using the cover part inspection zones. Suspected defect image blocks located in the structural exclusion area of ​​the cover part inspection zone are removed, and a candidate set of structural verification defects is generated.

[0013] As a preferred embodiment of the automotive body panel surface defect image recognition method of the present invention, the specific steps for constructing the normal texture image block memory set are as follows: Load the offline normal texture library pre-built from the calibration images of qualified parts of the same cover part model, use the candidate set of structural verification defects to determine the candidate areas in the complete cover part reference image, take the candidate areas, hole edges, folded structures and reinforcing rib projection areas as exclusion areas, and retain the coating areas outside the exclusion areas in the complete cover part reference image to generate normal coating sampling areas. Normal coating image blocks with the same color distribution as the coating are extracted from the normal coating sampling area. The texture direction, edge trend and brightness distribution characteristics of the normal coating image blocks are preserved and arranged according to the standard surface coordinate relationship of the complete cover reference map. The offline normal texture library and normal coating image blocks are used together to generate a normal texture image block memory set.

[0014] As a preferred embodiment of the automotive body panel surface defect image recognition method of the present invention, the specific steps for generating the automotive body panel surface defect recognition result are as follows: Extract local image patch features of each candidate region in the candidate set of structural verification defects, and perform nearest neighbor search with the normal texture image patch memory set. Delete candidate regions whose nearest neighbor search distance is lower than the upper limit of the normal texture search distance and are located in continuous bright reflection areas, do not have original bright texture residuals and do not have corresponding partition abnormal difference responses, and generate verification candidate regions. Based on the local image patch group at the same position in the reference image set of the cover, the verification candidate region is mapped to the visible corresponding position of adjacent view and continuous work position according to the coordinate relationship of the standard curved surface of the cover. The invisible region, the occluded region and the edge distortion region are excluded. The direction, boundary trend and texture destruction features of the corresponding position are extracted to generate the texture verification result at the same position. Based on the same-position texture verification results, candidate regions with consistent standard surface position, consistent boundary trend, consistent texture damage characteristics, and nearest neighbor retrieval distance not lower than the upper limit of normal texture retrieval distance are retained. When the brightness intensity of defects is inconsistent under different viewpoints, retention judgment is made based on standard surface position, boundary trend, texture damage characteristics, and abnormal difference response to generate the surface defect recognition results of automotive body panels.

[0015] Secondly, the present invention provides an image recognition system for surface defects in automotive body panels, comprising, The image processing module extracts the main body outline, area and edge sharpness based on multi-view images of the same automotive body panel, view calibration parameter group and three-dimensional reference model of the body panel, filters the complete body panel reference image and matches local image block groups at the same position according to the coordinate relationship of the standard curved surface of the body panel to generate a body panel reference image set. The texture correction module determines the coating color distribution and reflection area based on the reference image set of the cover, corrects the reflection area into a continuous coating texture, and generates a reflection correction texture map. The difference extraction module extracts multi-layer alignment features by using the reflection correction texture map, continuous bright reflection area and original bright texture residual map, and combines the abnormal difference map obtained by reverse distillation to enhance the directional texture distributed along the coordinate relationship of the standard surface of the cover, and generates an abnormal difference directional feature map. The region verification module sends the abnormal difference direction feature map into the edge posterior dual-branch reconstruction, and uses local texture destruction, abnormal boundary trend and cover detection partition to jointly locate suspected defects and generate a candidate set of structural verification defects. The verification output module constructs a normal texture image block memory set based on the structural verification defect candidate set, and performs same-position texture verification in combination with the cover reference image set to generate the surface defect identification result of the automotive cover.

[0016] The beneficial effects of this invention are as follows: By mapping two-dimensional image blocks to the coordinate relationship of the standard curved surface of the cover part through the viewpoint calibration parameter group and the three-dimensional reference model of the cover part, it is ensured that the same position verification of multiple views and continuous work stations is established on the physical position of the actual curved surface of the cover part; by linking the reflection correction texture map, the continuous bright reflection area and the original bright texture residual map, the shallow defects in the bright area are not repaired by the brightness fall-off process; by generating an anomaly difference map through the teacher encoder trained with normal samples, single-class bottleneck embedding and student decoder, and constraining the direction texture enhancement with the partition anomaly difference threshold, the discontinuous scratches, oblique scratches, curve indentations and local arc defects can be continuously expressed in the main direction of the curved surface; by participating in the edge posterior bi-branch reconstruction through the detection partition of the cover part, the true defect boundary is distinguished from the hole edge, folded structure, reinforcing rib projection and normal assembly boundary; by jointly constructing a normal texture image block memory set through the offline normal texture library and the online normal texture library after pollution screening, and combining the multi-view visibility and reprojection error threshold for same position verification, the stability and reliability of the identification of surface defects of high reflectivity automotive cover parts are improved. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a method for image recognition of surface defects in automotive body panels.

[0019] Figure 2 This is a schematic diagram of an image recognition system for surface defects in automotive body panels.

[0020] Figure 3 This is a schematic diagram of the generation of candidate sets for edge posterior bibranch recombination and structural verification defects.

[0021] Figure 4 This diagram illustrates the construction of a normal texture image block memory set and a texture kernel at the same location. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0025] Reference Figures 1-4 As one embodiment of the present invention, this embodiment provides a method for image recognition of surface defects in automotive body panels, comprising the following steps: S1. Extract the main body outline, area and edge sharpness from multi-view images of the same automotive body panel, filter the complete body panel reference image and match local image block groups at the same location to generate a body panel reference image set.

[0026] S1.1. When the same automotive body panel passes through the continuous workstation shooting area and the adjacent viewing angle shooting area, the photoelectric sensor triggers the fixedly installed industrial camera to collect the surface image of the body panel. The industrial camera transmits the surface image of the body panel to the image processing equipment through the image acquisition card. The image processing equipment receives multiple images to be identified formed by the same automotive body panel under the continuous workstation and adjacent viewing angle. It saves the imaging time, workstation number and viewing angle number of each image to be identified in correspondence with the image pixel matrix, and simultaneously reads the camera intrinsic parameters, camera extrinsic parameters, lens distortion parameters and the installation pose of the camera relative to the transport reference coordinate system for the corresponding workstation and viewing angle to form a viewing angle calibration parameter group. It also converts each image to be identified into a red-green-blue image, a hue-saturation-brightness image and a brightness-chroma image, respectively, to obtain a multi-color space image group.

[0027] Among them, the viewpoint calibration parameter group is obtained by the camera calibration board of the same automotive body panel inspection production line, the repeated positioning calibration of qualified body panels, and the workstation installation pose calibration; the image processing equipment also receives the three-dimensional reference model of the body panel corresponding to the same body panel model. The three-dimensional reference model of the body panel includes the outer surface curved surface of the body panel, hole edges, folded structure, reinforcing rib projection, normal assembly boundary, and corresponding standard curved surface coordinates, which are used to map the two-dimensional image position to the physical position on the real curved surface of the body panel.

[0028] The red, green, and blue images preserve surface texture; the hue, saturation, and brightness images distinguish coating colors and highlight reflections; and the brightness and chromaticity images separate brightness variations from chromaticity variations.

[0029] The edge of the luminance channel is extracted using the luminance and chromaticity images in the multi-color space image group. The background color interference is eliminated using the hue and saturation luminance images. Continuous closed regions that coincide with the edge of the luminance channel are retained in the red, green and blue images. The continuous closed regions are determined as the main body region of the cover. If there are multiple continuous closed regions in an image to be identified, the continuous closed region with the largest area is retained as the main body region of the cover, thus obtaining the main body region of the cover corresponding to each image to be identified.

[0030] Extract the main outline of the cover body area, and use the minimum circumscribed quadrilateral to determine the positions of the four vertices of the main outline. Calculate the area of ​​the main body area of ​​the cover body according to the positions of the four vertices, as expressed by: ; in, Indicates the first The area of ​​the main body region of the covered object in the image to be identified is expressed in pixels squared. Indicates the first The outer contour of the subject in the image to be identified The position of each vertex; when hour, This indicates the position of the first vertex.

[0031] S1.2. Edge sharpness is calculated using the luminance channels within the main body area of ​​the cover. First, a Laplacian operation is performed on the luminance channels within the main body area of ​​the cover to obtain an edge response map. Then, the variance of the edge response map is calculated as the edge sharpness. The Laplacian variance is derived from classic image sharpness evaluation methods. The Laplacian operation highlights edges with abrupt changes in grayscale, and the variance reflects the degree of dispersion of the edge response. The expression is: ; in, Indicates the first The edge sharpness of the image to be identified; Indicates the first The number of pixels in the main area of ​​the overlay in the image to be identified; Indicates the first The main area of ​​the overlay in the image to be identified The Laplacian response value of each pixel; Indicates the first The average value of all Laplacian response values ​​in the main body region of the overlay in the image to be identified.

[0032] The main body outline, vertex position, area, and edge sharpness of each image to be identified are arranged to generate a body panel evaluation set. Images with unclosed main body outlines, outlines truncated by image boundaries, edge sharpness lower than the median edge sharpness of images in the same batch, or area areas that do not meet the complete body panel area range are removed from the evaluation set. The remaining images are then selected for the complete body panel reference image screening. The complete body panel area range is determined by the qualified calibration images of the same automotive body panel. Qualified calibration images are images of qualified body panels acquired by continuous-station industrial cameras and adjacent-view industrial cameras under the same installation position, focal length, and lighting conditions, with at least 10 qualified calibration images. First, the area of ​​the main body region of each qualified calibration image is calculated using the above area calculation method. Then, the median of the area of ​​all main body regions of the body panels is taken as the qualified calibration area. The lower limit of the area range of the complete coverage component is taken as The upper limit of the area range of the complete coverage component is taken as That is, the area range of the complete cover is The above values ​​are based on the area statistics of the calibration images of qualified parts, and are determined by the fluctuation of the shooting position and the changes in imaging from adjacent viewpoints of the fixed-position industrial camera under the same cover part model. Values ​​lower than these values ​​are not considered acceptable. The image to be identified was determined to have a missing main body region, which is higher than... The image to be identified was determined to have a subject region that was adhered to by the background.

[0033] S1.3. Select the image with the most complete outer contour and the clearest edge from the remaining images to be identified, and generate a complete cover reference image; if the number of remaining images to be identified is zero, then re-receive multiple images of the same automotive cover formed at continuous work stations and adjacent viewpoints, and do not generate a cover reference image set.

[0034] The coordinate relationship of the standard surface of the cover is established by using the outer contour and vertex positions of the main body of the complete cover reference image, the view calibration parameter set, and the three-dimensional reference model of the cover. The four vertex positions of the complete cover reference image are mapped to the four corner points of the standard rectangular area. Perspective transformation is used to determine the initial coordinate correspondence between the complete cover reference image and the standard rectangular area. Then, the pixel positions in the complete cover reference image are distorted according to the camera intrinsic parameters, camera extrinsic parameters, and lens distortion parameters. The distorted pixel rays are then projected onto the outer surface of the three-dimensional reference model of the cover to obtain the standard surface coordinates corresponding to the pixel positions.

[0035] The main body area of ​​the cover in the remaining multi-view images is mapped to the corresponding surface position of the three-dimensional reference model of the cover through the coordinate relationship of the standard curved surface of the cover. The corresponding local image block is cropped according to the reference image position in the complete cover reference image. The corresponding local image block retains the station number, view number, view calibration parameters, standard curved surface coordinates and pixel reprojection position of the original image to be identified, and generates a group of local image blocks at the same position.

[0036] When generating local image block groups at the same location, the image processing device uses the standard surface coordinates on the three-dimensional reference model of the cover as the basis for judging the same physical location, rather than using a single two-dimensional perspective coordinate as the final basis. For areas in adjacent viewpoints that cannot be reliably projected due to surface occlusion, hole occlusion, edge occlusion, excessive angle between the line of sight and the surface normal, image edge distortion, or truncation of the main outline, they are marked as invisible areas, occluded areas, or edge distortion areas, and are excluded from the effective verification area of ​​the local image block group at the same location.

[0037] The same position judgment is jointly limited by the surface position error threshold and the pixel reprojection error threshold. The surface position error threshold and the pixel reprojection error threshold are determined by the statistical analysis of the repeated mapping error of the qualified part calibration image under continuous work station and adjacent viewpoint. When the standard surface coordinate error and pixel reprojection error corresponding to two local image blocks do not exceed the corresponding threshold, they are determined to belong to the same physical position of the cover. If either threshold is exceeded, the same position texture verification is not performed.

[0038] The complete cover reference image and the local image block group at the same position are matched according to the coordinate relationship of the standard curved surface of the cover, so that each reference image position in the complete cover reference image has a corresponding local image block from continuous work positions and adjacent viewpoints, and the visibility state, occlusion state and reprojection error state of the corresponding position are recorded to generate the cover reference image set.

[0039] S2. Determine the coating color distribution and reflection area based on the reference image set of the cover, correct the reflection area to a continuous coating texture, and generate a reflection correction texture map.

[0040] S2.1. Obtain the complete cover reference image and the complete cover main body area from the cover reference image set, retain the red, green and blue images, hue saturation and brightness images and brightness and chromaticity images in the complete cover main body area, and retain the surface position index of the complete cover main body area under the standard surface coordinate relationship of the cover.

[0041] In the hue, saturation, and brightness images, the hue channel uses an 8-bit image range of 0 to 179, while the saturation and brightness channels use an 8-bit image range of 0 to 255. The hue, saturation, and brightness distributions are statistically analyzed according to the main area of ​​the complete cover to generate the coating color distribution. The coating color distribution is derived from the main area of ​​the cover within the reference image of the complete cover, without using external color samples, to avoid coating color deviations caused by different workstation lighting.

[0042] The coating color distribution is used to filter the pixels in the main area of ​​the complete cover, and the pixels whose hue falls into the main distribution of coating hue and whose saturation falls into the main distribution of coating saturation are retained to form a coating pixel set.

[0043] The brightness values ​​of the coating pixel set are arranged from smallest to largest. The brightness values ​​between the 5th percentile and the 95th percentile are taken as normal coating brightness samples to generate normal coating brightness distribution. The 5th percentile to the 95th percentile are derived from the brightness statistics of the coating pixel set in the reference image of the complete cover. Brightness values ​​below the 5th percentile correspond to local occlusion and dark noise, while brightness values ​​above the 95th percentile correspond to light strip reflection, curved surface reflection and local overexposure.

[0044] S2.2. The normal coating brightness distribution is used to distinguish between continuous high-brightness reflection areas and normal coating texture areas. When the pixel brightness in the main body area of ​​the complete cover is higher than the 95th percentile of the normal coating brightness distribution and the pixel saturation is lower than the 25th percentile of the coating saturation distribution, the pixel is marked as a high-brightness reflection pixel. Connectivity analysis is performed on adjacent high-brightness reflection pixels, and a 3×3 structuring element is used to perform a closing operation to connect the broken edges to obtain the continuous high-brightness reflection area. The coating pixels in the main body area of ​​the complete cover that do not fall into the continuous high-brightness reflection area and whose brightness is between the 5th and 95th percentiles of the normal coating brightness distribution constitute the normal coating texture area.

[0045] Meanwhile, the brightness residual, edge residual, and local texture residual of the continuous high-brightness reflection area in the original complete cover reference map are saved separately to generate the original high-brightness texture residual map. The original high-brightness texture residual map does not participate in the brightness fallback replacement, but serves as constraint information for subsequent abnormal difference maps and texture verification at the same location, in order to prevent shallow scratches, shallow dents, and paint particles from being directly erased during the brightness fallback process.

[0046] When reducing the brightness of a continuous high-brightness reflection area, first search for adjacent normal coating texture areas along the outer contour of the continuous high-brightness reflection area, and then extract the 25th percentile, median, 75th percentile, brightness gradient direction, and coating texture residual of the adjacent normal coating texture areas.

[0047] It should be noted that the median brightness range of adjacent normal coating texture areas is the closed interval formed by the 25th percentile to the 75th percentile of the pixel brightness within the adjacent normal coating texture areas. The 25th percentile and the 75th percentile of the brightness are derived from the ascending statistics of the 8-bit brightness channel pixel values ​​within the adjacent normal coating texture areas, and the values ​​range from 0 to 255. The 25th percentile to the 75th percentile are selected as the median brightness range because this range belongs to the middle half of the brightness distribution of the normal coating texture area, which can eliminate the interference of residual dark noise and residual bright reflection in the adjacent normal coating texture areas on the brightness drop.

[0048] The process proceeds layer by layer from the edge of the continuous high-brightness reflection area inwards. The brightness of each pixel is adjusted to the median brightness range of the adjacent normal coating texture area. The median brightness of the adjacent normal coating texture area is used as the target value for brightness reduction of each pixel. Then, the coating texture residual is compensated according to the brightness gradient direction of the adjacent normal coating texture area to obtain the brightness reduction texture area. The brightness reduction texture area retains the original hue distribution and saturation distribution, and only changes the brightness component and texture residual of the continuous high-brightness reflection area to avoid the coating color being incorrectly changed.

[0049] The brightness reduction texture area is replaced with the continuous bright reflection area in the complete overlay reference map, while retaining the weak texture residual information in the normal coating texture area, the true edge of the overlay, and the original bright texture residual map. If there is no continuous bright reflection area in the main body area of ​​the complete overlay, the main body area of ​​the complete overlay in the complete overlay reference map is directly used as the reflection correction texture map. If the continuous bright reflection area coincides with the true edge of the overlay, brightness reduction and texture compensation are only performed inside the continuous bright reflection area without changing the position of the true edge of the overlay, and the reflection correction texture map is finally generated.

[0050] Among them, the reflection-corrected texture map is used to reduce the interference of bright reflections on subsequent feature extraction, and is not used as a sole criterion for defect judgment; the continuous bright reflection regions and the original bright texture residual map follow the reflection-corrected texture. Figure 1 This information is then passed to subsequent steps, enabling reverse distillation, directional texture enhancement, and co-position texture verification to simultaneously reference both the corrected continuous coating texture and the original weak texture residue before correction.

[0051] S3. Extract multi-layer alignment features from the reflection-corrected texture map, continuous bright reflection regions, and original bright texture residual map, and enhance the surface main direction texture by combining the anomaly difference map obtained by reverse distillation to generate an anomaly difference direction feature map.

[0052] S3.1. Extract shallow texture features and deep semantic features from the main body area of ​​the complete cover in the reflection-corrected texture map; the shallow texture features consist of edge response, local grayscale changes and fine-grained texture in the reflection-corrected texture map, and are superimposed with weak texture residuals that coincide with shallow edge details in the original bright texture residual map; the deep semantic features consist of normal coating area, normal curved surface area and normal stamping structure area in the main body area of ​​the complete cover; according to the outer contour and vertex position of the main body in the reference map of the complete cover and the coordinate relationship of the standard curved surface of the cover, map the shallow texture features and deep semantic features to the same standard curved surface position of the cover to generate multi-layer alignment features.

[0053] The deep semantic features from the multi-layer alignment features are fed into the teacher encoder. The teacher encoder extracts the normal surface codes corresponding to normal coatings, normal curved surfaces, and normal stamping structures to generate teacher surface features. The teacher encoder is a convolutional encoder trained on the surface image of the normal cover part. The surface image of the normal cover part comes from the qualified part calibration image of the same automotive cover part. The qualified part calibration image does not contain scratches, pits, particles, and edge indentations. The teacher surface features record the normal coating texture, normal curved surface orientation, and normal stamping structure information within the main body area of ​​the complete cover part.

[0054] The training samples for the teacher encoder are grouped into zones based on cover color, surface curvature, hole edges, folded edges, reinforcing rib projections, and normal assembly boundaries. This ensures that normal samples cover different coating colors, different surface areas, and different normal structural areas. After training, the teacher encoder parameters are fixed so that it outputs only surface feature representations corresponding to the normal cover surface distribution on the cover to be identified. Normal stamping structure lines, hole edges, and folded edges are included in the training samples as normal structural regions and are marked as structural prior regions in subsequent cover detection zones to avoid misclassifying normal stamping structures as abnormalities.

[0055] S3.2. A single-class bottleneck embedding is used to compress the normal coating texture, normal surface orientation, and normal stamping structure information in the teacher surface features to generate normal surface embedding features. The single-class bottleneck embedding retains the common features of the normal cover surface at the same cover position and weakens the local disturbances caused by suspected defect textures. The normal surface embedding features are back-pushed through the student decoder to restore the normal surface feature map that is consistent with the position of the main body area of ​​the complete cover, and generate back-pushed surface features. The student decoder only learns the back-pushing relationship from the normal surface embedding features to the normal surface feature map, and the back-pushed surface features serve as a normal reference for the teacher surface features.

[0056] The single-class bottleneck embedding is formed by sequentially connecting a normal surface feature compression layer, a low-dimensional normal embedding layer, and a normal surface feature back-pull layer. It is trained using only the normal coating region, normal curved surface region, and normal structure region in the qualified part calibration image. The student decoder uses the teacher's surface features as the supervision target and is trained using feature reconstruction loss and structural partition consistency loss. The feature reconstruction loss is used to constrain the back-pull surface features to be close to the normal surface features, and the structural partition consistency loss is used to constrain the normal stamping structure, hole edge, and folded edge contour to maintain the normal structural expression in the back-pull result.

[0057] To prevent small-scale defects from being directly reconstructed as normal textures by single-class bottleneck embedding, the student decoder does not use candidate defect regions of the overlay to be detected as normal training samples during training, and does not directly input shallow edge details and original highlight texture residuals into single-class bottleneck embedding; shallow edge details and original highlight texture residuals are only used as constraint information in the subsequent abnormal texture location marking and orientation texture enhancement stages.

[0058] By comparing the feature differences between the teacher surface features and the back surface features at the same cover location, areas that deviate from normal coatings, normal curved surfaces, and normal stamping structures are extracted to generate an anomaly difference map.

[0059] Feature differences are calculated using Euclidean distance in mathematics. The feature vector of the teacher's surface at the same cover location has the same dimension as the feature vector of the back-projected surface. Euclidean distance is used to represent the degree of deviation between two feature vectors of the same dimension, and its expression is: ; in, Indicates the position of the cover part in the standard surface coordinate relationship. Abnormal difference values ​​at; Indicates position Teacher's surface characteristics Each feature component; Indicates position The surface feature of the back push Each feature component; This represents the common feature dimension of teacher surface features and backward surface features.

[0060] It should be noted that the common feature dimension Instead of calculating the number of pixels in the image to be recognized, the area of ​​the main body of the overlay, or the number of candidate regions on an ad hoc basis, the feature output layer is pre-determined based on the number of channels in the corresponding feature output layers of the teacher encoder and student decoder during their construction. In this embodiment, the last convolutional feature layer of the teacher encoder is selected as the teacher surface feature output layer, which is set to have 256 feature channels; the final reconstruction layer of the student decoder is used as the pushback surface feature output layer, which is also set to have 256 feature channels. After the student decoder outputs the pushback surface features, the spatial dimensions and position indices of the pushback surface features are adjusted to match those of the teacher surface features according to the coordinate relationship of the standard curved surface of the overlay, so that the same position... Both the teacher surface features and the back-pull surface features contain 256 feature components arranged in the same channel order; therefore, the common feature dimension in this embodiment is... Set the number to 256. When using other feature channel configurations, set the teacher surface feature output layer and the backpropagation surface feature output layer to have the same number of feature channels, and determine this same number of feature channels as the common feature dimension. This ensures that each teacher's surface feature component has a corresponding backpropagation surface feature component with the same position and channel order.

[0061] The abnormal difference values ​​at each location within the main body area of ​​the complete cover are arranged according to the coordinate relationship of the standard curved surface of the cover to obtain an abnormal difference map.

[0062] The abnormal difference threshold of the abnormal difference map is determined by the verification sample of normal cover parts in the calibration image of qualified parts. Specifically, the upper limit of normal abnormal difference values ​​is calculated according to the coating color partition, surface curvature partition and normal structure partition, and the upper limit is used as the abnormal difference threshold of the corresponding partition. In the cover parts to be tested, only when the abnormal difference value of the same standard curved surface position exceeds the abnormal difference threshold of the corresponding partition will it enter the abnormal texture position marking.

[0063] S3.3. Anomaly difference maps are used to mark the locations of abnormal textures in the multi-layer alignment features, and the shallow edge details corresponding to the abnormal texture locations are preserved to generate multi-layer alignment features with anomaly indicators. The abnormal texture locations are determined by continuous regions in the anomaly difference map that are higher than the corresponding partition anomaly difference threshold. When the continuous region coincides with the shallow edge details or the weak texture residual information in the original bright texture residual map, it is retained as an abnormal texture location. When the continuous region does not coincide with the shallow edge details or the original bright texture residual, it is treated as a normal surface brightness fluctuation. The multi-layer alignment features with anomaly indicators preserve both the abnormal locations in the anomaly difference map and the edge details in the shallow texture features.

[0064] A multi-layer alignment feature extraction method with anomaly indication is used to extract directional textures distributed along the coordinate relationship of the standard surface of the cover. The directional textures include horizontal, vertical, and diagonal textures, as well as arc-shaped textures extending along the principal curvature directions of the cover surface. Each directional texture is determined by combining the Sobel gradient direction with the principal surface direction in the 3D reference model of the cover. The Sobel gradient is derived from the classic image gradient calculation method, and the direction of texture change is represented by the horizontal and vertical gradients, expressed as follows: ; in, Indicates the position of the cover part in the standard surface coordinate relationship. Texture direction at the location; Indicates position The horizontal gradient at that location; Indicates position The vertical gradient at that location.

[0065] When the texture direction is close to the horizontal direction, it is classified as a horizontally extended texture; when the texture direction is close to the vertical direction, it is classified as a vertically extended texture; when the texture direction is between the horizontally extended texture and the vertically extended texture, it is classified as a diagonally extended texture; when the texture direction changes continuously along the principal curvature direction of the cover surface, it is classified as an arc-shaped extended texture, thus obtaining the directional texture region.

[0066] S3.4. Within the directional texture region, anomaly difference maps are used to pinpoint the locations of anomalous textures. Continuous texture responses are enhanced along horizontally extending, vertically extending, obliquely extending, and arc-extending textures, respectively, generating directional enhanced texture features. For discontinuous anomalous texture locations on the same horizontally extending texture, adjacent texture segments are connected according to the direction of the horizontal extending texture. For discontinuous anomalous texture locations on the same vertically extending texture, adjacent texture segments are connected according to the direction of the vertical extending texture. For discontinuous anomalous texture locations on the same obliquely extending texture, adjacent texture segments are connected according to the direction of the oblique extending texture. For discontinuous anomalous texture locations on the same arc-extending texture, adjacent texture segments are connected according to the direction of the principal curvature of the cover surface. The connected texture segments retain the anomalous locations and shallow edge details in the anomaly difference map, resulting in directional enhanced texture features.

[0067] By connecting the texture fragments corresponding to discontinuous scratches, thin tear and edge indentation, as well as oblique scratches, curved indentation and local arc defects through directional enhancement texture features, and removing normal coating texture fragments that do not have abnormal difference map markers, an abnormal difference directional feature map is generated.

[0068] Among them, the abnormal difference map plays a reverse constraint role on directional texture enhancement: directional texture fragments that do not exceed the abnormal difference threshold of the corresponding partition and do not coincide with the original bright texture residual do not participate in texture connection; directional texture fragments located in the continuous bright reflection area but simultaneously have the original bright texture residual and abnormal difference response are retained for subsequent edge posterior bi-branch reconstruction.

[0069] S4. The abnormal difference direction feature map is fed into the edge posterior dual-branch reconstruction. The suspected defects are located by using local texture destruction, abnormal boundary trend and cover detection partition. A candidate set of structural verification defects is generated.

[0070] S4.1. Under the coordinate relationship of the standard curved surface of the cover, the local texture slices of the cover surface are divided according to the fixed-size sliding window; the side length of the fixed-size sliding window is determined according to the side length of the circumscribed rectangle of the smallest resolvable texture segment in the abnormal difference direction feature map, and the movement step of the fixed-size sliding window is half of the side length of the fixed-size sliding window, so that the overlapping area is retained between adjacent local texture slices; each local texture slice retains the abnormal texture position, horizontal extension texture, vertical extension texture, diagonal extension texture, arc extension texture, continuous bright reflection area marker, original bright texture residual and shallow edge details, to obtain a local texture slice set.

[0071] The edge posterior bi-branch reconstruction includes a local texture destruction branch and a boundary posterior branch. The input of the local texture destruction branch is a set of local texture slices, anomaly difference maps, and original highlight texture residual maps, and the output is the local texture destruction result. The input of the boundary posterior branch is anomaly difference direction feature maps, normal structural boundaries in the complete cover reference map, and cover detection partitions, and the output is boundary trend anomaly results. The fusion method of the two branches is candidate region cascade reconstruction, that is, only the positions that simultaneously satisfy the constraints of local texture destruction, boundary trend anomalies, anomaly difference responses, and cover detection partitions are retained as suspected defect regions.

[0072] Local texture destruction features are extracted using a set of local texture slices. The grayscale continuity, texture direction continuity, and abnormal difference response continuity within each local texture slice are checked pixel by pixel. When there are regions in a local texture slice where grayscale abrupt changes, texture direction breaks, and abnormal difference responses overlap, the overlapping regions are marked as local texture destruction regions. Adjacent broken texture fragments are extended along the main direction of the local texture destruction region, and texture fragments that overlap with shallow edge details or original bright texture residuals are preserved to generate local texture destruction results.

[0073] S4.2. Extract posterior edge features using anomaly difference direction feature map, search for continuous edge segments along the boundary trend of the cover surface, and match the continuous edge segments with the normal stamping structure lines, hole edges, and folded edge contours in the complete cover reference map; when the continuous edge segment is located at the boundary position of the normal stamping structure line, hole edge, and folded edge contour, the continuous edge segment is marked as a normal structure edge; when the continuous edge segment deviates from the normal structure edge and coincides with the anomaly difference response, the continuous edge segment is marked as a true defect boundary, generating anomaly boundary trend results.

[0074] Among them, the posterior basis of the edge posterior features is the prior structural position of the hole edge, folded structure, stiffener projection and normal assembly boundary under the standard surface coordinate relationship provided by the three-dimensional reference model of the cover part; when the edge response is consistent with the prior structural position, it is preferentially treated as a normal structural edge; when the edge response deviates from the prior structural position and coincides with the abnormal difference response and local texture destruction result, it is treated as a real defect boundary.

[0075] Based on the comparison of local texture damage results, abnormal boundary trend results, and abnormal difference maps at the same cover location; when a local texture damage area, a real defect boundary, and an abnormal difference response exist simultaneously at the same cover location, the corresponding location is retained as a suspected defect area; when only a local texture damage area, only a real defect boundary, or only an abnormal difference response exist at the same cover location, the corresponding location is excluded as a normal coating texture disturbance, a normal structural edge, or a residual light area; after completing the comparison of all cover locations, suspected defect areas are generated.

[0076] S4.3. Using suspected defect regions for cascaded small-scale localization, firstly, the first layer of local image blocks is cropped using the bounding rectangle of the suspected defect region. Then, within the first layer of local image blocks, the second layer of local image blocks is cropped along the texture segment with the most concentrated abnormal difference response. The second layer of local image blocks retains the texture destruction center, boundary trend center, and abnormal difference response center of the suspected defect region, generating a suspected defect image block. If the area of ​​the suspected defect region is smaller than the minimum cropping area of ​​the first layer of local image blocks, the suspected defect image block is cropped after expanding to the minimum cropping area with the center of the suspected defect region as the center.

[0077] The inspection zones for the cover are divided based on the hole edges, folded structures, reinforcing rib projections, and normal assembly boundaries in the complete cover reference drawing. The hole edges correspond to the closed hole contours in the complete cover reference drawing, the folded structures correspond to the continuous broken line edges extending along the outer contour of the cover in the complete cover reference drawing, the reinforcing rib projections correspond to the regularly extending raised edges within the main body area of ​​the complete cover, and the normal assembly boundaries correspond to the boundary lines corresponding to the mounting structures within the main body area of ​​the complete cover. The inspection zones for the cover mark the areas where the normal structural boundaries are located as structural exclusion areas, and mark the cover coating surfaces outside the structural exclusion areas as valid inspection areas.

[0078] The structural information of hole edges, folded structures, reinforcing rib projections, and normal assembly boundaries comes from the three-dimensional reference model of the cover part, the structural boundary calibration results in the calibration image of qualified parts, and the process inspection location file of the corresponding cover part model. The image processing equipment maps the above structural information to the complete cover part reference image through the view calibration parameter group, and makes the position correspondence with the suspected defect image block through the standard curved surface coordinate relationship.

[0079] S4.4. Use the cover detection partition to verify suspected defective image blocks, and check the position of the texture destruction center, boundary trend center, and abnormal difference response center of each suspected defective image block in the cover detection partition; when the texture destruction center, boundary trend center, and abnormal difference response center are all located in the effective detection area, the corresponding suspected defective image block is retained; when any center is located in the structure exclusion area, the corresponding suspected defective image block is removed; if the suspected defective image block crosses the effective detection area and the structure exclusion area, only the image block part located in the effective detection area that simultaneously has texture destruction, boundary abnormality, and abnormal difference response is retained to generate a candidate set of structural verification defects.

[0080] Among them, the cover detection partition is not only eliminated in the candidate output stage, but also participates in the boundary trend anomaly judgment as a structural prior of the boundary posterior branch; the edge response located in the structural exclusion area and consistent with the normal structural boundary does not enter the structural verification defect candidate set, and the area located in the effective detection area and simultaneously meets the abnormal difference response, texture destruction and boundary trend anomaly is entered into the structural verification defect candidate set.

[0081] S5. Construct a normal texture image block memory set based on the structural verification defect candidate set, and perform same-position texture verification in combination with the cover reference image set to generate the surface defect identification result of the automotive cover.

[0082] S5.1. Using the candidate set of structural verification defects, candidate regions are determined in the coordinate relationship of the standard curved surface of the complete cover part in the reference image of the complete cover part. The candidate regions, hole edges, folded structures, and stiffener projection areas are marked as exclusion regions. The coating regions outside the exclusion regions that fall within the coating color distribution are retained in the reference image of the complete cover part to generate normal coating sampling regions. If the normal coating sampling region in the reference image of the complete cover part is occupied by the candidate region, the coating region at the same position of the cover part that does not fall within the exclusion region is selected from the local image block group at the same position in the reference image set of the cover part and added to the normal coating sampling region.

[0083] Figure 4 The exclusion items shown correspond specifically to candidate regions, hole edges, folded structures, and reinforcing rib projection areas. The image processing device generates candidate region exclusion masks based on the structural verification defect candidate set, and generates hole edge exclusion masks, folded structure exclusion masks, and reinforcing rib projection exclusion masks based on the 3D reference model of the cover and the structural boundary calibration results. The four types of exclusion masks are then merged in the coordinate relationship of the cover's standard curved surface. Image regions located within any exclusion mask are not considered normal coating sampling regions; only regions that do not fall within any of the four exclusion masks and whose hue and saturation distributions match the coating color distribution are considered normal coating sampling regions.

[0084] The normal texture image block memory set includes an offline normal texture library and an online normal texture library. The offline normal texture library is pre-constructed from calibration images of qualified parts of the same cover model and is classified and saved according to coating color, surface curvature, structural partitions and shooting angle. The online normal texture library is only generated from normal coating sampling areas in the current cover to be tested, after being jointly screened by multi-view consistency, low abnormal differences, discontinuous high brightness reflection, non-structural boundaries and non-candidate regions.

[0085] Figure 4 The offline library shown corresponds to the offline normal texture library, which is built before the cover to be inspected enters the inspection station. It is used to provide a stable normal texture reference for the same cover model under different coating colors, different surface curvatures, different structural partitions and different shooting angles. Figure 4 The online library shown corresponds to the online normal texture library, which is generated only during the image processing of the current overlay to be detected. It is used to supplement the normal texture changes of the current overlay to be detected caused by local lighting, the current shooting angle, and the current surface reflection state. Both the offline and online normal texture libraries use normal coating image blocks as the basic storage unit. Each storage unit stores the source library mark, overlay model, coating color category, surface curvature category, structural partition, shooting angle, standard surface coordinates, texture direction, edge trend, and brightness distribution characteristics of the normal coating image block.

[0086] Online normal coating sampling areas must simultaneously meet the following conditions: the corresponding standard curved surface position is visible in adjacent viewpoints or continuous workstations; the corresponding position does not fall within a continuous high-brightness reflection area; the abnormal difference value at the corresponding position does not exceed the abnormal difference threshold of the corresponding partition; the corresponding position does not belong to the edge of a hole, a folded structure, a reinforcing rib projection, or a normal assembly boundary; and the texture direction and brightness distribution at the corresponding position are consistent within the local image block group at the same position. Areas that do not simultaneously meet the above conditions will not be added to the online normal texture library.

[0087] According to the cropping size of the suspected defect image blocks in the structural verification defect candidate set, normal coating image blocks are cropped in the normal coating sampling area; the cropping positions of the normal coating image blocks are arranged according to the coordinate relationship of the standard curved surface of the cover, and the hue distribution and saturation distribution of the normal coating image blocks fall into the coating color distribution of S2; texture direction, edge trend and brightness distribution features are extracted from the normal coating image blocks. The texture direction is obtained by using the Sobel gradient direction and the main direction of the surface in S3, the edge trend is obtained by using the edge posterior features in S4, and the brightness distribution features are obtained by using the 8-bit brightness channel statistics in S2, forming a normal coating image block feature set.

[0088] S5.2. Arrange the normal coating image block feature set according to the standard surface coordinate relationship of the complete cover reference image to generate a normal texture image block memory set; each normal coating image block feature in the normal texture image block memory set retains the standard surface coordinates, texture direction, edge trend and brightness distribution features of the cover, and serves as a normal coating reference for nearest neighbor retrieval of the candidate set of structural verification defects.

[0089] Figure 4 middle, This represents the total number of normal texture classification groups actually formed after classifying according to the combination of coating color, surface curvature, structural partitioning, and shooting angle. The image processing device reads the coating color category, surface curvature category, structural partitioning, and shooting angle of each normal coating image block in both the offline and online normal texture libraries. Normal coating image blocks with identical classification information are grouped into the same normal texture classification group. The normal texture classification groups containing valid normal coating image blocks are then sequentially numbered. The number of normal texture classification groups after numbering is the total number of normal texture classification groups. Figure 4 In .

[0090] Figure 4 middle, This represents the number of normal coating image patch features that are pre-reserved for each normal texture classification group, i.e., the sample size of a single normal texture classification group. In this embodiment, the pre-reserved sample size for each normal texture classification group is 100. Take 100. Each normal texture classification group is first loaded with normal coating image patch features that match the classification information in the offline normal texture library, and then supplemented with samples from the online normal texture library that have passed the contamination screening. When the number of valid samples in a normal texture classification group exceeds 100, the samples are uniformly retained according to the standard surface coordinate distribution in the corresponding surface area of ​​the cover, so that the 100 retained samples cover the different surface positions corresponding to the classification group. When the number of valid samples is less than 100, the actual number of valid samples is saved, and the storage positions not occupied by valid samples do not participate in the subsequent nearest neighbor search.

[0091] Figure 4 The first row represents the first normal texture classification group, with numbers 1 to... The first row represents the features of each normal coating image patch in the first normal texture classification group; the second row represents the second normal texture classification group, where the numbers continue sequentially from the first normal texture classification group; the remaining classification groups are numbered sequentially in the same way; the last row represents the... Normal texture classification group, last number express Each normal texture classification group has a preset capacity of [number]. The total storage capacity corresponding to the time.

[0092] Figure 4 The feature information stored in the image corresponds specifically to the standard surface coordinates, texture direction, edge trend, and brightness distribution characteristics of each normal coating image block.

[0093] When generating the normal texture image block memory set, the offline normal texture library is first loaded as the basic reference, and then the online normal texture library that has passed the pollution screening is used as a supplementary reference. If the number of samples in the online normal texture library is lower than the minimum number of samples in the corresponding overlay detection partition, or if the nearest neighbor distance distribution in the online normal texture library is abnormally concentrated around the candidate region, it is determined that the online normal texture library has a pollution risk, and the corresponding online samples are not added to the normal texture image block memory set.

[0094] The upper limit of the normal texture retrieval distance is determined by using a normal texture image patch memory set. Specifically, within the normal texture image patch memory set, each normal coating image patch feature is searched for nearest neighbors with the other normal coating image patch features. The nearest neighbor search uses Euclidean distance to calculate the distance between feature vectors of the same dimension. The nearest neighbor distances corresponding to all normal coating image patch features are arranged from smallest to largest, and the 95th percentile is taken as the upper limit of the normal texture retrieval distance. The 95th percentile comes from the distance statistics within the normal texture image patch memory set, which can retain the main distribution range of normal coating texture and exclude discrete texture interference at the edges of a small number of normal coating sampling areas.

[0095] When the upper limit of the nearest neighbor distance between the offline normal texture library and the online normal texture library in the same detection partition is inconsistent, the upper limit of the normal texture retrieval distance determined by the offline normal texture library is used as the main threshold, and the online normal texture library is only used to supplement the local brightness and texture direction changes of the same object to be detected.

[0096] S5.3. Extract the local image patch features of each candidate region in the candidate set of structural verification defects, and perform nearest neighbor search on the local image patch features and the normal texture image patch memory set to obtain the nearest neighbor search distance for each candidate region; when the nearest neighbor search distance of the candidate region is lower than the upper limit of the normal texture search distance, and the candidate region is located in the continuous bright reflection region obtained in S2 and does not have the weak texture residual in the original bright texture residual map, does not have the corresponding partition abnormal difference response, and has not passed the multi-view same position verification, the candidate region is judged as normal texture perturbation and deleted; the remaining candidate region after deleting normal texture perturbation is used as the verification candidate region.

[0097] Based on the local image patch group at the same position in the cover reference image set, the verification candidate region is mapped to the corresponding position of adjacent view and continuous station according to the coordinate relationship of the standard curved surface of the cover. The texture direction, edge trend and texture destruction features are extracted at the corresponding position. When the verification candidate region falls into the corresponding position of the local image patch group at the center position of the complete cover reference image, and the corresponding position has the same type of texture direction, same trend edge and the same type of texture destruction features, the texture verification result at the same position is generated.

[0098] For cases where the brightness of defects varies due to changes in the direction of surface reflection under different viewing angles, it is not required that the brightness intensity of each viewing angle be completely consistent. Instead, the verification criteria are based on the consistency of the standard surface position, the boundary trend, the texture damage type, and the abnormal difference response. For viewing angles marked as invisible areas, occluded areas, or edge distortion areas, these viewing angles are not used as negative verification criteria. Only viewing angles that are visible and whose reprojection error meets the threshold and continuous workstations are used for verification.

[0099] Based on the same-position texture verification results, candidate regions with consistent position, consistent boundary trends, consistent texture destruction features, and nearest neighbor retrieval distance not lower than the upper limit of normal texture retrieval distance are retained, while candidate regions that fail the same-position texture verification are deleted. The retained candidate regions are converted back to the image position in the complete cover reference map according to the coordinate relationship of the cover standard surface, and the surface defect recognition results of automotive cover are generated.

[0100] This embodiment also provides an image recognition system for surface defects in automotive body panels, including: The image processing module extracts the main body outline, area and edge sharpness based on multi-view images of the same automotive body panel, view calibration parameter group and three-dimensional reference model of the body panel, filters the complete body panel reference image and matches local image block groups at the same position according to the coordinate relationship of the standard curved surface of the body panel to generate a body panel reference image set. The texture correction module determines the coating color distribution and reflection area based on the reference image set of the cover, corrects the reflection area into a continuous coating texture, and generates a reflection correction texture map. The difference extraction module extracts multi-layer alignment features by using the reflection correction texture map, continuous bright reflection area and original bright texture residual map, and combines the abnormal difference map obtained by reverse distillation to enhance the directional texture distributed along the coordinate relationship of the standard surface of the cover, and generates an abnormal difference directional feature map. The region verification module sends the abnormal difference direction feature map into the edge posterior dual-branch reconstruction, and uses local texture destruction, abnormal boundary trend and cover detection partition to jointly locate suspected defects and generate a candidate set of structural verification defects. The verification output module constructs a normal texture image block memory set based on the structural verification defect candidate set, and performs same-position texture verification in combination with the cover reference image set to generate the surface defect identification result of the automotive cover.

[0101] In summary, this invention maps two-dimensional image blocks to the standard curved surface coordinates of the cover part through a set of viewpoint calibration parameters and a three-dimensional reference model of the cover part, ensuring that the same-position verification of multiple views and continuous workstations is based on the actual physical position of the curved surface of the cover part. Through the linkage of reflection correction texture map, continuous bright reflection area, and original bright texture residual map, shallow defects in the bright area are prevented from being repaired by the brightness fall-off process. Anomaly difference maps are generated by a teacher encoder trained with normal samples, single-class bottleneck embedding, and student decoder, and directional texture enhancement is constrained by a partitioned anomaly difference threshold, enabling discontinuous scratches, oblique scratches, curve indentations, and local arc-shaped defects to be continuously expressed in the main direction of the curved surface. The cover part detection partition participates in edge posterior bi-branch reconstruction, distinguishing between real defect boundaries and hole edges, folded structures, reinforcing rib projections, and normal assembly boundaries. A normal texture image block memory set is constructed by combining an offline normal texture library with an online normal texture library that has been filtered for contamination, and the same-position verification is performed by combining multi-view visibility and reprojection error thresholds, improving the stability and reliability of defect identification on the curved surface of high-reflectivity automotive cover parts.

[0102] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for image recognition of surface defects in automotive body panels, characterized in that: include, Based on multi-view images of the same automotive body panel, view calibration parameter groups, and 3D benchmark model of the body panel, the main body outline, area and edge sharpness are extracted. Complete body panel benchmark images are selected and local image block groups at the same position are matched according to the coordinate relationship of the standard curved surface of the body panel to generate a body panel benchmark image set. The coating color distribution and reflection area are determined based on the reference image set of the cover, and the reflection area is corrected into a continuous coating texture to generate a reflection correction texture map. Multi-layer alignment features are extracted from the reflection correction texture map, continuous bright reflection region and original bright texture residual map, and the directional texture distributed along the standard surface coordinate relationship of the cover is enhanced by combining the anomaly difference map obtained by reverse distillation to generate an anomaly difference directional feature map. The abnormal difference direction feature map is fed into the edge posterior dual-branch reconstruction. The suspected defects are located by using local texture destruction, abnormal boundary trend and cover detection partition to generate a candidate set of structural verification defects. A normal texture image block memory set is constructed based on the candidate set of structural verification defects, and texture verification at the same location is performed in combination with the reference image set of the cover part to generate the surface defect recognition result of the automotive cover part.

2. The method for image recognition of surface defects in automotive body panels as described in claim 1, characterized in that: The local image block group at the same position is generated by establishing the coordinate relationship of the standard curved surface of the cover part based on the outer contour and vertex position of the main body of the complete cover part reference image, the view calibration parameter group, and the three-dimensional reference model of the cover part. The main body area of ​​the other multi-view images is mapped to the corresponding curved surface position of the three-dimensional reference model of the cover part, and the corresponding local image block is cropped according to the reference image position.

3. The method for image recognition of surface defects in automotive body panels as described in claim 2, characterized in that: The specific steps for generating the reference image set for the overlay are as follows: Receive multiple images of the same automotive body panel formed at continuous work stations and adjacent viewpoints, read the camera intrinsic parameters, camera extrinsic parameters, lens distortion parameters and installation pose corresponding to each image, perform multi-color space transformation and body panel main area localization on each image, extract the main body outer contour, vertex position, area and edge sharpness, and generate body panel main evaluation set. By eliminating images to be identified that have incomplete main body regions, edge shadows, or areas whose area does not meet the range of the complete cover area through the evaluation set of the main body of the cover, the images to be identified with the complete outer contour of the main body and the highest edge clarity are retained to generate a complete cover reference image. The complete cover reference image and the local image block group at the same position are matched according to the coordinate relationship of the standard curved surface of the cover, and the visibility state, occlusion state and reprojection error state at the corresponding positions are recorded to generate the cover reference image set.

4. The method for image recognition of surface defects in automotive body panels as described in claim 3, characterized in that: The specific steps for generating the reflection-corrected texture map are as follows: The coating hue distribution, coating saturation distribution, and coating brightness distribution are extracted from the complete cover reference image in the cover reference image set to generate the coating color distribution, and the normal coating brightness distribution is determined based on the coating color distribution; Based on the normal coating brightness distribution, the continuous high-brightness reflection area and the normal coating texture area are distinguished. The brightness residual, edge residual and local texture residual of the continuous high-brightness reflection area are saved to generate the original high-brightness texture residual map. The brightness of the continuous high-brightness reflection area is reduced, and the coating texture is compensated along the adjacent normal coating texture area to generate the reflection correction texture map.

5. The method for image recognition of surface defects in automotive body panels as described in claim 4, characterized in that: The abnormal difference map was obtained through reverse distillation, and the specific steps are as follows: The teacher encoder was trained and fixed using calibration images of qualified parts of the same cover model, and normal stamping structure lines, hole edges and folded contours were used as normal structure areas for training. The teacher encoder is used to process the reflection correction texture map, extract the normal surface code corresponding to normal coating, normal curved surface and normal stamping structure, and generate teacher surface features; A single-class bottleneck embedding is used to compress the normal coating texture, normal surface orientation and normal stamping structure information in the teacher's surface features to generate normal surface embedding features. The single-class bottleneck embedding is formed by sequentially connecting a normal surface feature compression layer, a low-dimensional normal embedding layer and a normal surface feature pushback layer. The student decoder is used to back-push back the embedded features of the normal surface to recover the normal surface feature map that is consistent with the position of the main body area of ​​the complete cover, and generate back-push surface features. The student decoder is trained using feature reconstruction loss and structural partition consistency loss. By comparing the feature differences between the teacher surface features and the back surface features at the same cover location, areas that deviate from normal coatings, normal curved surfaces, and normal stamping structures are extracted to generate an anomaly difference map.

6. The method for image recognition of surface defects in automotive body panels as described in claim 5, characterized in that: The specific steps for generating the abnormal difference direction feature map are as follows: Shallow texture features and deep semantic features are extracted by reflection-corrected texture map, and weak texture residuals that coincide with shallow edge details in the original bright texture residual map are superimposed. The outer contour and vertex position of the main body of the complete cover reference map are mapped to the same cover position according to the standard surface coordinate relationship of the cover, generating multi-layer alignment features. The abnormal texture locations in the multi-layer alignment features are marked by continuous regions in the abnormal difference map that are higher than the abnormal difference threshold of the corresponding partition, and the shallow edge details and weak texture residuals corresponding to the abnormal texture locations are preserved to generate multi-layer alignment features with abnormal indications. The system uses multi-layer alignment features with anomaly indicators to extract horizontal and vertical extended textures, oblique extended textures, and arc extended textures extending along the principal curvature direction of the cover surface. The system also marks the location of abnormal textures within the horizontal, vertical, oblique, and arc extended textures to generate directional enhanced texture features. By connecting texture fragments corresponding to discontinuous scratches, thin tear and edge indentation, diagonal scratches, curved indentation and local arc-shaped defects through directional enhancement texture features, an abnormal difference directional feature map is generated.

7. The method for image recognition of surface defects in automotive body panels as described in claim 6, characterized in that: The specific steps for generating the candidate set of structural verification defects are as follows: The local texture destruction branch is input using anomaly difference direction feature map, original bright texture residual map and local texture slice set to extract local texture destruction features; Using the abnormal difference direction feature map, normal structural boundary and cover detection partition input boundary posterior branch, the edge posterior features are extracted, the local texture slices on the surface of the cover are enhanced in detail, and the real defect boundary, normal stamping structure line, hole edge and fold contour are distinguished to generate local texture damage results and boundary trend abnormal results; Based on the comparison of local texture destruction results, boundary trend abnormal results and abnormal difference maps at the same standard surface position, regions that simultaneously possess texture destruction, boundary abnormality and abnormal difference response are retained by candidate region cascade recombination, and the retained regions are cascaded small-scale localized to generate suspected defect image blocks. Based on the three-dimensional reference model of the cover part, the structural boundary calibration results in the calibration image of the qualified part, and the process inspection location file, the hole edges, folded structures, stiffener projections, and normal assembly boundaries in the complete cover part reference drawing are determined. The cover part is divided into inspection zones, and suspected defect image blocks are verified using the cover part inspection zones. Suspected defect image blocks located in the structural exclusion area of ​​the cover part inspection zone are removed, and a candidate set of structural verification defects is generated.

8. The method for image recognition of surface defects in automotive body panels as described in claim 7, characterized in that: The specific steps for constructing the normal texture image block memory set are as follows: Load the offline normal texture library pre-built from the calibration images of qualified parts of the same cover part model, use the candidate set of structural verification defects to determine the candidate areas in the complete cover part reference image, take the candidate areas, hole edges, folded structures and reinforcing rib projection areas as exclusion areas, and retain the coating areas outside the exclusion areas in the complete cover part reference image to generate normal coating sampling areas. Normal coating image blocks with the same color distribution as the coating are extracted from the normal coating sampling area. The texture direction, edge trend and brightness distribution characteristics of the normal coating image blocks are preserved and arranged according to the standard surface coordinate relationship of the complete cover reference map. The offline normal texture library and normal coating image blocks are used together to generate a normal texture image block memory set.

9. The method for image recognition of surface defects in automotive body panels as described in claim 8, characterized in that: The specific steps for generating the surface defect identification results of automotive body panels are as follows: Extract local image patch features of each candidate region in the candidate set of structural verification defects, and perform nearest neighbor search with the normal texture image patch memory set. Delete candidate regions whose nearest neighbor search distance is lower than the upper limit of the normal texture search distance and are located in continuous bright reflection areas, do not have original bright texture residuals and do not have corresponding partition abnormal difference responses, and generate verification candidate regions. Based on the local image patch group at the same position in the reference image set of the cover, the verification candidate region is mapped to the visible corresponding position of adjacent view and continuous work position according to the coordinate relationship of the standard curved surface of the cover. The invisible region, the occluded region and the edge distortion region are excluded. The direction, boundary trend and texture destruction features of the corresponding position are extracted to generate the texture verification result at the same position. Based on the same-position texture verification results, candidate regions with consistent standard surface position, consistent boundary trend, consistent texture damage characteristics, and nearest neighbor retrieval distance not lower than the upper limit of normal texture retrieval distance are retained. When the brightness intensity of defects is inconsistent under different viewpoints, retention judgment is made based on standard surface position, boundary trend, texture damage characteristics, and abnormal difference response to generate the surface defect recognition results of automotive body panels.

10. A surface defect image recognition system for automotive body panels, based on the surface defect image recognition method for automotive body panels according to any one of claims 1 to 9, characterized in that: include, The image processing module extracts the main body outline, area and edge sharpness based on multi-view images of the same automotive body panel, view calibration parameter group and three-dimensional reference model of the body panel, filters the complete body panel reference image and matches local image block groups at the same position according to the coordinate relationship of the standard curved surface of the body panel to generate a body panel reference image set. The texture correction module determines the coating color distribution and reflection area based on the reference image set of the cover, corrects the reflection area into a continuous coating texture, and generates a reflection correction texture map. The difference extraction module extracts multi-layer alignment features by using the reflection correction texture map, continuous bright reflection area and original bright texture residual map, and combines the abnormal difference map obtained by reverse distillation to enhance the directional texture distributed along the coordinate relationship of the standard surface of the cover, and generates an abnormal difference directional feature map. The region verification module sends the abnormal difference direction feature map into the edge posterior dual-branch reconstruction, and uses local texture destruction, abnormal boundary trend and cover detection partition to jointly locate suspected defects and generate a candidate set of structural verification defects. The verification output module constructs a normal texture image block memory set based on the structural verification defect candidate set, and performs same-position texture verification in combination with the cover reference image set to generate the surface defect identification result of the automotive cover.

Images