Vehicle paint surface scratch detection device fused with multi-modal imaging

Abstract

The invention discloses a vehicle paint surface scratch detection device fused with multi-modal imaging, and relates to the technical field of vehicle scratch detection, and the device is characterized in that multiple paths of annular light sources are configured, time-sharing exposure and continuous acquisition are carried out, and a multi-modal image is determined; setting a detection constraint condition; constructing a scratch detection module; identifying a multi-modal image, executing morphological processing and self-adaptive binarization processing of a grayscale image, performing multi-modal superposition fusion and scratch feature identification, and integrating and outputting a scratch detection list; and carrying out homologous induction marking, and carrying out visualization on a terminal display interface. The technical problems that existing vehicle paint surface scratch detection is limited to single-mode image acquisition and processing, tiny scratches and deep scratches on a vehicle paint surface are difficult to comprehensively capture, the detection efficiency is low, and the accuracy of a detection result is poor are solved, and the technical effect of improving the vehicle paint surface scratch detection efficiency and accuracy is achieved.

Description

Technical Field

[0001] The present application relates to the technical field related to automobile scratch detection, and in particular to a vehicle paint scratch detection device integrating multi-modal imaging. Background Art

[0002] With the booming development of the automobile industry and the increasing demand of consumers for the appearance quality of vehicles, vehicle paint scratch detection has become an indispensable part of automobile manufacturing, repair and maintenance. In recent years, with the rapid development of computer vision, image processing and artificial intelligence technology, scratch detection technology based on machine vision has gradually emerged. However, traditional scratch detection methods use single-mode imaging technology for image acquisition and processing. For example, using only visible light images, it is difficult to fully capture the complex features of the vehicle paint surface, especially for tiny scratches, hidden deep scratches or scratches that are difficult to identify under different lighting conditions. The detection effect is not ideal, it is difficult to guarantee the accuracy and consistency of the detection results, and the scratch detection efficiency is low.

[0003] Therefore, the current technologies related to vehicle paint scratch detection are limited to single-modality image acquisition and processing, which makes it difficult to fully capture tiny scratches and deeper scratches on the vehicle paint, leading to technical problems such as low detection efficiency and poor accuracy of detection results. Summary of the invention

[0004] The present application provides a vehicle paint scratch detection device that integrates multi-modal imaging, thereby solving the technical problems of existing vehicle paint scratch detection, namely, that the image acquisition and processing is limited to a single modality, making it difficult to fully capture tiny scratches and deeper scratches on the vehicle paint, which in turn leads to low detection efficiency and poor accuracy of detection results. The application achieves the technical effect of improving the efficiency and accuracy of vehicle paint scratch detection.

[0005] The present application provides a vehicle paint scratch detection device that integrates multi-modal imaging, and the device includes: a multi-modal image determination module, which is used to configure a multi-channel ring light source, perform time-sharing exposure and continuous acquisition, and determine a multi-modal image, wherein the multi-modal image has a light source modal number based on the same target area; a detection constraint condition setting module, which is used to set the detection constraint condition, wherein the detection constraint condition includes an interference constraint condition for the natural texture of the surface and an optical amplification condition based on the detection requirement; a scratch detection module construction module, which is used to construct a scratch detection module, wherein the scratch detection module includes a pre-processing unit based on the detection constraint condition and morphological operation, and a fusion detection unit connected to the back; a grayscale image processing module, which is used to identify the multi-modal image, perform morphological processing and adaptive binarization processing of the grayscale image based on the pre-processing unit, transfer the grayscale image to the fusion detection unit for multi-modal superposition fusion and scratch feature recognition, and integrate and output a scratch detection list; a homology induction marking module, which is used to perform homology induction marking on the scratch detection list and visualize it on a terminal display interface.

[0006] In a possible implementation, the multimodal image determination module also performs the following processing: setting the light source angle and the light source type, and combining to determine a multi-path ring light source; setting the light source exposure order and the neighborhood light source exposure interval, and determining the acquisition rules, wherein the neighborhood light source exposure interval is determined based on the exposure lag effect; and performing time-sharing exposure and continuous acquisition based on the multi-path ring light source and the acquisition rules.

[0007] In a possible implementation, the grayscale image processing module also performs the following processing: acquiring a multimodal grayscale image; performing morphological processing on the multimodal grayscale image to determine a primary processed image; traversing the primary processed image, performing binarization processing and optical magnification processing based on the detection constraint conditions, and determining a preprocessed image.

[0008] In a possible implementation, the grayscale image processing module also performs the following processing: determining the structural element characteristics and the morphological processing scale based on detection and recognition requirements; setting an operation processing mode, wherein the operation processing mode is determined based on a combination of a closing operation and an opening operation; and performing morphological processing on the multimodal grayscale image based on the structural element characteristics, the morphological processing scale and the operation processing mode.

[0009] In a possible implementation, the grayscale image processing module also performs the following processing: performing a binarization process on the image background and texture to determine a primary processing result; based on the interference constraint condition, binarizing the natural texture and the non-natural texture to determine a secondary processing result; traversing the secondary processing result, performing level marking on the non-natural texture, and determining a binary image.

[0010] In a possible implementation, the grayscale image processing module further performs the following processing: based on the optical magnification condition, performing boundary segmentation on the binary image to determine a segmented image, wherein the magnification method includes geometric magnification and contrast enhancement; traversing the segmented image to perform image optical magnification processing.

[0011] In a possible implementation, the grayscale image processing module further performs the following processing: introducing visual calculation Gestalt and setting verification levels; and performing scratch trajectory verification on the scratch detection sheet based on the verification levels.

[0012] In a possible implementation, the homology summary marking module further performs the following processing: performing a first summary marking on the scratch detection sheet based on homology of scratch type; and performing a second summary marking on the scratch detection sheet based on homology of repair mode.

[0013] The vehicle paint scratch detection device proposed in the present application is configured with a multi-channel annular light source, performs time-sharing exposure and continuous acquisition, and determines a multimodal image, wherein the multimodal image has a light source modal number based on the same target area; sets detection constraints, wherein the detection constraints include interference constraints for natural surface textures and optical amplification conditions based on detection requirements; constructs a scratch detection module, wherein the scratch detection module includes a preprocessing unit based on the detection constraints and morphological operations, and a fusion detection unit connected to the back; identifies the multimodal image, performs morphological processing and adaptive binarization processing of the grayscale image based on the preprocessing unit, and transfers it to the fusion detection unit for multimodal superposition fusion and scratch feature recognition, and integrates and outputs a scratch detection list; performs homology induction marking on the scratch detection list, and visualizes it on the terminal display interface. The present invention solves the technical problem that the existing vehicle paint scratch detection is limited to a single-mode image acquisition and processing, which makes it difficult to fully capture the tiny scratches and deeper scratches on the vehicle paint, thus leading to low detection efficiency and poor accuracy of detection results. The technical effect of improving the efficiency and accuracy of vehicle paint scratch detection is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solution of the embodiment of the present disclosure, the accompanying drawings of the embodiment of the present disclosure will be briefly introduced below. A flow chart is used in the present application to illustrate the operations performed by the device according to the embodiment of the present application. It should be understood that the previous or following operations are not necessarily performed precisely in order. On the contrary, various steps can be processed in reverse order or simultaneously as needed. At the same time, other operations can also be added to these processes, or one or more operations can be removed from these processes.

[0015] Figure 1A schematic diagram of the structure of a vehicle paint scratch detection device integrating multimodal imaging provided in an embodiment of the present application; Figure 2 A schematic diagram of the execution flow of the grayscale image processing module in the vehicle paint scratch detection device integrating multimodal imaging provided in an embodiment of the present application.

[0016] Explanation of the reference numerals: multimodal image determination module 10 , detection constraint condition setting module 20 , scratch detection module construction module 30 , grayscale image processing module 40 , homology induction labeling module 50 . DETAILED DESCRIPTION

[0017] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below.

[0018] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0019] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict, and the terms "first\second" involved are merely to distinguish similar objects and do not represent a specific ordering of objects. The terms "including" and "having" and any variations are intended to cover non-exclusive inclusions, for example, a process, device, product, or server that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or components that are not clearly listed or inherent to these processes, products, or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by technicians in the technical field of this application. The terms used herein are for the purpose of describing the embodiments of the present application only.

[0020] The present application embodiment provides a vehicle paint scratch detection device integrating multi-modal imaging, such as Figure 1 As shown, the vehicle paint scratch detection device integrating multimodal imaging includes: a multimodal image determination module 10, a detection constraint condition setting module 20, a scratch detection module construction module 30, a grayscale image processing module 40, and a homology induction labeling module 50.

[0021] The multimodal image determination module 10 is used to configure a multi-channel ring light source, perform time-sharing exposure and continuous acquisition, and determine a multimodal image, wherein the multimodal image has a light source modality number based on the same target area.

[0022] Preferably, multiple ring light sources are configured to acquire multimodal images, wherein the multiple light sources may include LED lights of different wavelengths or spectral bands, infrared light sources, ultraviolet light sources, etc., and the number is configured according to the actual situation of the vehicle. The design of the ring light source can ensure that the light is evenly irradiated to the target area, reduce the influence of shadows and reflections, and thus improve the quality of the multimodal image; multiple ring light sources are used for time-sharing exposure and continuous acquisition. Specifically, since the images generated by different light sources may have different characteristics and information, each light source is activated in turn through time-sharing exposure and the corresponding vehicle detection image is acquired, ensuring that only one light source is activated at the same time point to avoid interference between different light sources, and on the basis of time-sharing exposure, the images generated by each light source are continuously acquired. to form a multimodal image for subsequent vehicle paint scratch detection and analysis. The multimodal image contains multiple vehicle paint scratch detection images generated based on different light sources. Each modal image reflects the characteristics and information of the target area under different light sources. In order to distinguish and identify these multimodal images, each modal image is assigned a light source modal number based on the same target area. This light source modal number is unique and corresponds to the light source used when acquiring the image. Through this light source modal number, each modal image, as well as the relationship and difference between them, can be easily tracked and identified. By comparing and analyzing images of different modalities, the characteristics and positions of scratches can be more effectively identified, which helps to improve the accuracy and reliability of scratch detection.

[0023] The specific configuration of the multimodal image determination module 10 also includes setting the light source angle and the light source type, and combining to determine a multi-path ring light source; setting the light source exposure order and the neighborhood light source exposure interval, and determining the acquisition rules, wherein the neighborhood light source exposure interval is determined based on the exposure lag effect; and performing time-sharing exposure and continuous acquisition based on the multi-path ring light source and the acquisition rules.

[0024] Preferably, the angle of the light source refers to the irradiation direction of the light source relative to the target area (such as the paint surface of the vehicle). By adjusting the angle of the light source, the distribution and reflection of the light on the target area can be changed, thereby affecting the quality and characteristics of the image. The type of light source refers to the type of light source used, such as LED, halogen lamp, fluorescent lamp, etc. Different types of light sources have different spectral characteristics, brightness and stability, which will affect the color, contrast and clarity of the image. Specifically, light sources of different angles and types are combined to form a multi-channel ring light source. These light sources can be evenly distributed around the target area or configured at specific angles and distances to obtain more comprehensive image information. During the image acquisition process, the correct lighting can be used to separate it from the background. For example, for planar raised defects, low-angle lighting can be used to illuminate the edges of the raised parts; for smooth defects or curved surface defects, diffuse light can be used; at the same time, the use of multiple light sources for image acquisition and fusion can ensure the accuracy and completeness of scratch defects.

[0025] Preferably, the light source exposure order and the neighboring light source exposure interval are set. Specifically, the light source exposure order refers to the order in which the light sources are activated during the time-sharing exposure process. By setting a reasonable exposure order, it can be ensured that when acquiring images of each modality, the target area is exposed to consistent lighting conditions, thereby avoiding interference between images of different modalities. The neighboring light source exposure interval refers to the time interval between the exposure of two adjacent light sources during continuous acquisition. The setting of this interval needs to take into account the effect of exposure lag, that is, after the previous light source is exposed, the target area may need some time to recover to a stable state, so as to avoid the next light source. The images generated by exposure cause interference. Based on the set light source exposure order and the exposure interval of the neighboring light sources, specific acquisition rules are formulated to guide the process of time-sharing exposure and continuous acquisition. That is, according to the set light source exposure order, each light source is activated in turn, and the target area is exposed. During the exposure of each light source, the corresponding image data is collected. On the basis of time-sharing exposure, the image data generated by the exposure of each light source is continuously collected to ensure that the image of each modality can be accurately and efficiently collected, so as to realize multi-modal imaging of the target area, so as to provide richer image information, which is helpful to improve the accuracy and reliability of applications such as scratch detection.

[0026] The detection constraint condition setting module 20 is used to set detection constraint conditions, wherein the detection constraint conditions include interference constraint conditions for natural surface textures and optical amplification conditions based on detection requirements.

[0027] Preferably, detection constraints are set to ensure the accuracy and reliability of scratch detection while reducing the possibility of false alarms and missed alarms. The detection constraints usually include interference constraints on the natural texture of the surface and optical amplification conditions based on detection requirements. Specifically, vehicle paint surfaces often have complex natural textures, which can be easily confused with scratches during scratch detection, resulting in false alarms or missed alarms. Therefore, interference constraints on the natural texture of the surface are set, which may include texture feature recognition, which uses image processing technology to identify and extract natural texture features of the vehicle paint surface to distinguish scratches from natural textures, thereby reducing false alarms; texture filtering, which applies a filtering algorithm to smooth the natural texture in the image to reduce its impact on scratch detection, helping to improve the accuracy of scratch detection. Scratch detection often requires the identification of tiny scratches, which requires the device to have sufficient optical magnification capabilities. The optical magnification conditions based on detection requirements usually include optical magnification, that is, according to the size of the scratch and the detection requirements, the appropriate optical magnification is set to ensure that the detailed features of the scratch are clearly captured, thereby improving the accuracy of detection; optical system stability. The stability of the optical system is crucial for scratch detection. It is necessary to ensure that the optical system remains stable during the magnification process to avoid problems such as image blur or distortion; optical system adaptability. Different scratches may require different optical magnification conditions. Therefore, the optical system needs to have a certain degree of adaptability and be able to adjust and optimize according to detection requirements. By using interference constraints on the natural texture of the surface and optical magnification conditions based on detection requirements, false alarms and missed alarms can be effectively reduced, and the accuracy and efficiency of scratch detection can be improved. The scratch detection module construction module 30 is used to construct a scratch detection module, wherein the scratch detection module includes a pre-processing unit based on the detection constraint condition and morphological operation, and a post-connected fusion detection unit.

[0028] Preferably, a scratch detection module is constructed, comprising a preprocessing unit based on the detection constraints and morphological operations and a post-connected fusion detection unit, wherein the preprocessing unit is an important component of the scratch detection module, and is mainly responsible for processing input image data to reduce noise, enhance scratch features, and provide high-quality images for subsequent scratch detection, specifically including image screening based on detection constraints, morphological operations, and image enhancement. First, the input image is screened according to the set detection constraints (such as optical magnification, texture feature recognition, etc.) to ensure that the image quality meets the requirements of scratch detection, and then morphological operations (such as expansion, corrosion, opening operations, closing operations, etc.) are applied to process the image to remove noise, smooth edges, and broken scratches in the image, thereby enhancing scratch features and improving the accuracy of scratch detection. By adjusting contrast and brightness, the scratch features in the image are further enhanced to make the scratches more clearly visible. The fusion detection unit is the core part of the scratch detection module, which is mainly responsible for scratch detection on the preprocessed image, including feature extraction, scratch identification and positioning, and fusion detection. Specifically, image processing techniques (such as edge detection, corner detection, etc.) are used to extract scratch features in the image. These features include the shape, length, width, direction, etc. of the scratches. Based on the extracted scratch features, machine learning algorithms or threshold segmentation methods are used to identify and locate the scratches. By comparing the scratch features with the preset scratch template or threshold, it is determined whether there are scratches in the image, and the position and size of the scratches are determined. Finally, the vehicle scratch detection results of different modes are fused, and the final detection results are output to ensure the accuracy and reliability of scratch detection.

[0029] The grayscale image processing module 40 is used to identify the multimodal image, perform morphological processing and adaptive binarization processing of the grayscale image based on the preprocessing unit, transfer it to the fusion detection unit for multimodal superposition fusion and scratch feature recognition, and integrate and output a scratch detection list.

[0030] Preferably, the collected multimodal images are identified, converted to obtain corresponding grayscale images and input into the scratch detection module, and the preprocessing unit is used to perform morphological processing and adaptive binarization processing on the grayscale images, and then the processing results are transferred to the fusion detection unit for multimodal superposition fusion and scratch feature recognition. Specifically, the multimodal image is a plurality of images of the paint surface of the target vehicle, which may include visible light images, infrared images, ultraviolet images, etc. Each mode provides different characteristic information of the target. The original color multimodal image is converted into a grayscale image to reduce the amount of calculation during processing. For scratch detection, the grayscale information is usually sufficient. The grayscale image is then subjected to morphological processing, such as expansion, corrosion, opening operation, closing operation, etc., to remove noise, enhance scratch features, connect broken scratches, etc., and then the image is localized according to the local characteristics. The threshold is automatically determined based on the feature, and the grayscale image is converted into a binary image. In the binary image, scratches usually appear as connected white or black areas, which is helpful for subsequent scratch identification. The preprocessed images of different modalities are input into the fusion detection unit for superposition and fusion, such as through simple image superposition, weighted average, maximum and minimum fusion, etc. The fused image will contain complementary information from different modalities, which is helpful for more accurate identification of scratches. Finally, on the fused image, image processing technology (such as edge detection, contour extraction, template matching, etc.) is used to identify the scratch features, including the shape, length, width, direction, position, etc. of the scratches, and the identified scratch feature information is integrated into the scratch detection form, which may include a detailed description of the scratch, position marking, size measurement, severity assessment and other information.

[0031] like Figure 2 As shown, the specific configuration of the grayscale image processing module 40 also includes acquiring a multimodal grayscale image; performing morphological processing on the multimodal grayscale image to determine a primary processed image; traversing the primary processed image, performing binarization processing and optical magnification processing based on the detection constraint conditions, and determining a preprocessed image.

[0032] Preferably, the original color image of each modality is converted into a corresponding grayscale image, that is, the color channels red (R), green (G), and blue (B) of the color image are extracted, and then the values ​​of the three channels of red, green, and blue are added according to a certain weight to calculate the grayscale value. The most commonly used weights are 0.299 (R), 0.587 (G), and 0.114 (B) to obtain a multimodal grayscale image; then the multimodal grayscale image is subjected to morphological processing, such as expansion, corrosion, opening operation, closing operation, etc., to remove noise in the image, enhance scratch features, and connect broken scratches. The morphological processing result is used as a primary processing image, and then each pixel or region of the primary processing image is traversed, and the detection constraints (such as the minimum width of the scratch, The primary processed image is adaptively binarized according to the set detection constraints, specifically, the image is converted into an image containing only black and white colors according to the set detection constraints, where scratches are usually manifested as white areas, to ensure the accuracy of the binarization processing and avoid misidentifying noise or other non-scratch areas as scratches. On the basis of the binarization processing, the image is optically magnified to further highlight the scratch features and make the scratches more clearly visible. The optical magnification is determined according to the size of the scratches and the detection requirements to ensure that the scratch features remain clear and recognizable in the magnified image. Finally, the preprocessed image is obtained as the basis for subsequent scratch detection, which is used to extract scratch features, identify scratch positions, etc.

[0033] The specific configuration of the grayscale image processing module 40 also includes: determining the structural element characteristics and the morphological processing scale based on the detection and recognition requirements; setting the operation processing mode, wherein the operation processing mode is determined based on the combination of closing operation and opening operation; and performing morphological processing on the multimodal grayscale image based on the structural element characteristics, the morphological processing scale and the operation processing mode.

[0034] Preferably, based on the detection and recognition requirements, the characteristics of the structural element and the morphological processing scale are determined. Generally speaking, the size and shape of the structural element will affect the effect of the operation to a certain extent. For example, a suitable structural element can be selected according to the characteristics of the image and the need to remove noise. At the same time, in order to avoid the loss of image details or excessive smoothing during the operation, it is best to avoid over-processing during the processing to maintain the clarity and details of the image and limit the processing scale. Specifically, the structural element (also called a kernel or mask) is the basis of the morphological operation and defines the shape and size of the operation. The shape of the structural element can be a rectangle, a circle, an ellipse, a cross, etc., depending on the shape and characteristics of the object to be processed (such as scratches). Selecting an appropriate structural element helps to more accurately identify and extract scratches; the morphological processing scale is usually related to the size of the structural element and determines the scope of influence of the morphological operation. When processing multimodal grayscale images, it is necessary to select an appropriate morphological processing scale based on the width, length and contrast of the scratches. A larger scale may be more suitable for wide scratches or low-contrast scratches, while a smaller scale may be more suitable for thin scratches or high-contrast scratches.

[0035] Preferably, according to the needs of detection and recognition, the closing operation and the opening operation are combined to form a specific operation processing mode, that is, single operation, combined operation, sequence and the like are adaptive, and noise processing is performed. For example, a closing operation can be performed first to connect scratches, and then an opening operation can be performed to remove noise, which helps to reduce interference factors in the image while retaining the scratch features. Among them, the closing operation (dilation first and then erosion) and the opening operation (erosion first and then dilation) are two basic operations in morphological processing. The closing operation helps to connect broken scratches and fill small holes, while the opening operation helps to remove small noise and separate adhered objects. Finally, based on the determined structural element characteristics, morphological processing scale and operation processing mode, the multimodal grayscale image is morphologically processed, including traversing each pixel or region of the image, and performing corresponding operations according to the structural element and the operation processing mode. The processed image will contain clearer and more significant scratch features for subsequent scratch detection and recognition tasks.

[0036] The specific configuration of the grayscale image processing module 40 also includes: performing a binarization process on the image background and texture to determine a primary processing result; based on the interference constraint condition, binarizing the natural texture and the non-natural texture to determine a secondary processing result; traversing the secondary processing result, performing level marking on the non-natural texture, and determining a binary image.

[0037] Preferably, a binarization process is performed on the image background and texture, that is, each pixel in the image is converted into a pixel containing only two possible values ​​(usually 0 and 1 or black and white), usually based on a threshold, pixel values ​​above the threshold are set to white (or 1), and pixels below the threshold are set to black (or 0), and the background area and texture area in the image are regarded as a whole, without distinguishing between natural texture and non-natural texture, to obtain a processing result, that is, a binarized image, at which time each pixel in the image is black and white; the interference constraint condition is used to perform the binarization process on the natural texture and the non-natural texture. Specifically, in the first binarization process, On the basis of the second binarization processing, natural textures (such as leaves, stones and other naturally formed textures) and non-natural textures (such as buildings, man-made objects and other textures) are further distinguished to obtain the secondary processing result, that is, the image after the binarization processing of natural textures and non-natural textures. Finally, each pixel in the image after the second binarization processing is checked, and the identified non-natural textures are further classified or marked. For example, they are divided into different levels according to the complexity of the texture, the contrast with the background and other features, and finally a binary image is obtained, in which the non-natural texture may have been marked or classified, while the entire image still remains in a binary form (that is, it only contains black and white).

[0038] The specific configuration of the grayscale image processing module 40 also includes, based on the optical magnification condition, performing boundary segmentation on the binary image to determine a segmented image, wherein the magnification method includes geometric morphology magnification and contrast enhancement; traversing the segmented image to perform image optical magnification processing.

[0039] Preferably, the determined binary image is subjected to boundary segmentation according to the optical magnification condition, that is, the binary image is further processed, and an edge detection algorithm such as Roberts gradient operator method, Sobel operator method, Canny edge detection method, etc. is used to identify points where the grayscale value in the image changes significantly, thereby identifying and extracting the boundary or contour in the image, and determining the segmented image. For example, an optimal threshold is determined according to the inter-class variance between the foreground and the background, thereby obtaining a segmented image. The segmented image usually contains multiple sub-regions, each of which corresponds to a set of pixels with similar features, representing different objects, backgrounds or texture features in the image; wherein the optical magnification method usually includes geometric magnification and There are two types of contrast enhancement: geometric magnification is to magnify objects or features in the image by changing the size or shape of the image. This magnification method may introduce some distortion or blurring effects, but it can usually be corrected through image processing algorithms. Contrast enhancement: enhances the details or features in the image by adjusting the contrast of the image. This magnification method does not change the actual size of the image, but can make the objects or features in the image clearer and easier to identify. It traverses the segmented image to determine the position and size of each sub-area to ensure that no important image information is lost or distorted during the magnification process. According to the selected magnification method and the segmented image information, each sub-area is magnified one by one to further improve the image quality and readability.

[0040] The specific configuration of the grayscale image processing module 40 further includes: introducing visual calculation Gestalt, setting verification checkpoints; and performing scratch track verification on the scratch detection sheet based on the verification checkpoints.

[0041] Preferably, visual calculation refers to the use of mathematical and computational methods to simulate and analyze visual information, covering image processing, computer vision, pattern recognition, etc. In psychology and cognitive science, Gestalt theory emphasizes that the whole is greater than the sum of its parts, that is, when people perceive objects or scenes, they tend to regard them as a complete and meaningful whole rather than separate parts. For example, in the process of manual inspection, people can use intuition and experience to predict the possible shape of defects, which is the Gestalt ability in psychology. When processing images, we not only focus on single pixels or local features, but also consider how they are combined into complete objects or scenes, that is, according to the specific needs and goals of scratch detection, set the verification points. The card is inspected after output. The verification level may include the verification of the length, width, direction, position and other features of the scratch, as well as the verification of the relationship between the scratch and the background and other objects, which helps to more accurately identify and understand the information in the image; then based on the verification level, the scratch track verification of the scratch detection list is carried out. Specifically, based on the set verification level, the scratch track in the scratch detection list is verified and verified, including checking whether the scratch is continuous, whether it conforms to the expected track shape, whether it is clearly distinguishable from the background or other objects, etc. The scratch track verification ensures the accuracy and reliability of the scratch detection results, which helps to improve the accuracy and efficiency of scratch detection.

[0042] The homology summarization and marking module 50 is used to perform homology summarization and marking on the scratch detection sheet and visualize it on the terminal display interface.

[0043] Furthermore, the specific configuration of the homology summarization and marking module 50 also includes: performing a first summarization and marking on the scratch detection sheet based on the homology of the scratch type; and performing a second summarization and marking on the scratch detection sheet based on the homology of the repair method.

[0044] Preferably, the scratch detection sheets are subjected to homologous inductive marking, that is, the scratch data from the same source or under the same detection conditions are summarized and classified, and corresponding labels are assigned to them, which helps to quickly identify and process scratch data with similar characteristics. The homologous inductive marking mainly includes the first inductive marking of the scratch detection sheets based on the homology of the scratch type, and the second inductive marking of the scratch detection sheets based on the homology of the repair method. Specifically, scratches can be classified according to various factors such as their appearance, formation causes, and degree of influence. For example, in machine vision inspection, scratches can be divided into scratches with obvious grayscale changes (first type of scratches), scratches with no obvious grayscale changes (second type of scratches), and scratches with large grayscale differences and long strip shapes (third type of scratches). According to the scratch type , the scratch inspection form is first summarized and marked, which means that the inspection forms with the same or similar scratch types are classified into one category and assigned a specific mark, which helps to quickly identify and classify different types of scratches; the scratch repair method depends on factors such as the type, location, size and material of the scratch. For example, for minor scratches on the surface of certain materials, it may only take polishing or grinding to repair, while for deep scratches or scratches that affect material properties, more complex repair methods may be required, such as welding, spraying, etc. According to the repair method, the scratch inspection form is secondly summarized and marked, which means that the inspection forms with the same or similar repair requirements are classified into one category and assigned a specific mark, which helps to formulate targeted repair plans and improve repair efficiency and effectiveness.

[0045] Preferably, a reasonable interface layout is designed on the terminal display interface to ensure that the information of the scratch detection list can be clearly and intuitively displayed on the terminal screen, including the display of key information such as the image, location, size, type, and repair method of the scratch, and a visual display is performed, wherein visualization is the use of computer graphics and image processing technology to convert data into graphics or images, display them on the screen and perform interactive processing, so as to display the scratch information, summary marking results and repair solutions in the scratch detection list, etc. At the same time, rich interactive functions are provided, so that users can easily browse, query, edit and export the information of the scratch detection list. For example, detailed information of different scratches can be viewed by clicking or dragging, or the screening function can be used to find the detection list with specific summary marks, so that the scratch detection data can be more effectively managed and processed, and the accuracy and efficiency of scratch detection can be improved.

[0046] Although the present application makes various references to certain components in the apparatus according to the embodiments of the present application, any number of different components may be used and run on a user terminal and / or server, and the various units and components included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be implemented; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0047] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A vehicle paint scratch detection device integrating multimodal imaging, characterized in that: The device comprises: A multimodal image determination module is used to configure a multi-channel ring light source, perform time-sharing exposure and continuous acquisition, and determine a multimodal image, wherein the multimodal image has a light source modality number based on the same target area; A detection constraint condition setting module, used to set detection constraint conditions, wherein the detection constraint conditions include interference constraint conditions for natural surface textures and optical amplification conditions based on detection requirements; A scratch detection module construction module, used to construct a scratch detection module, wherein the scratch detection module includes a pre-processing unit based on the detection constraint conditions and morphological operations, and a fusion detection unit connected to the post-processing unit; A grayscale image processing module, used for identifying the multimodal image, performing morphological processing and adaptive binarization processing of the grayscale image based on the preprocessing unit, transferring the grayscale image to the fusion detection unit for multimodal superposition fusion and scratch feature recognition, and integrating and outputting a scratch detection list; The homology induction marking module is used to perform homology induction marking on the scratch detection sheet and visualize it on the terminal display interface.

2. The vehicle paint scratch detection device integrating multimodal imaging as claimed in claim 1, characterized in that: The time-sharing exposure and continuous acquisition include: Set the light source angle and light source type, and combine them to determine the multi-channel ring light source; Setting the light source exposure order and the neighboring light source exposure interval to determine the acquisition rule, wherein the neighboring light source exposure interval is determined based on the exposure lag effect; Based on the multi-path ring light source and the acquisition rule, time-sharing exposure and continuous acquisition are performed.

3. The vehicle paint scratch detection device integrating multimodal imaging as claimed in claim 1, characterized in that: The method includes performing morphological processing and adaptive binarization processing of the grayscale image based on the preprocessing unit, including: Acquire multimodal grayscale images; Performing morphological processing on the multimodal grayscale image to determine a primary processed image; The primary processed image is traversed, binarization processing and optical magnification processing based on the detection constraint condition are performed, and a pre-processed image is determined.

4. The vehicle paint scratch detection device integrating multimodal imaging as claimed in claim 3, characterized in that: Performing morphological processing on the multimodal grayscale image includes: Determine the characteristics of structural elements and the scale of morphological processing based on detection and recognition requirements; Setting an operation processing mode, wherein the operation processing mode is determined based on a combination of a closing operation and an opening operation; Based on the structural element characteristics, the morphological processing scale and the operation processing mode, morphological processing is performed on the multimodal grayscale image.

5. The vehicle paint scratch detection device integrating multimodal imaging as claimed in claim 3, characterized in that: Performing binarization processing based on the detection constraint condition includes: A binary process is performed on the image background and texture to determine a processing result; Based on the interference constraint condition, binarize the natural texture and the non-natural texture to determine the secondary processing result; The secondary processing results are traversed, the levels of the unnatural textures are marked, and a binary image is determined.

6. The vehicle paint scratch detection device integrating multimodal imaging as claimed in claim 5, characterized in that: Perform optical magnification processing, including: Based on the optical magnification condition, the binary image is subjected to boundary segmentation to determine a segmented image, wherein the magnification method includes geometric magnification and contrast enhancement; The segmented image is traversed to perform image optical magnification processing.

7. The vehicle paint scratch detection device integrating multimodal imaging as claimed in claim 1, characterized in that: After integrated output of scratch inspection list, including: Introduce visual calculus and set verification levels; Based on the verification checkpoint, the scratch track verification is performed on the scratch detection sheet.

8. The vehicle paint scratch detection device integrating multimodal imaging as claimed in claim 1, characterized in that: The scratch test sheet is subjected to homologous inductive labeling, including: Performing a first inductive marking on the scratch detection sheet based on the scratch type homology; Homologous in repair manner, a second inductive marking is performed on said scratch detection sheet.

Images