A metal part defect intelligent detection method and system based on multispectral fusion

Abstract

The present application relates to the technical field of defect detection, and particularly relates to a metal part defect intelligent detection method and system based on multispectral fusion, which comprises the following steps: acquiring a metal part, performing image acquisition operation on the metal part by using a plurality of pre-constructed different wave bands of light, acquiring a plurality of polarization pixels of a target polarization part image, respectively acquiring the gray values of the polarization pixels and a plurality of same-position polarization pixels, calculating a first polarization component and a second polarization component based on the plurality of polarization gray values, calculating the linear polarization gray value corresponding to the polarization pixels based on the first polarization component and the second polarization component, constructing a part spectral image corresponding to the target polarization part image set based on the plurality of linear polarization gray values, performing multispectral fusion operation on the plurality of part spectral images to obtain a fused part image, and identifying the metal part defect based on the fused part image. The present application reduces the interference of metal reflection and achieves intelligent detection of metal part defects through multispectral fusion.

Description

Technical Field

[0001] This invention relates to the field of defect detection technology, and in particular to an intelligent method and system for detecting defects in metal parts based on multispectral fusion. Background Technology

[0002] Metal parts are widely used in industrial manufacturing, and their surface quality directly affects the reliability and service life of the products.

[0003] In existing technologies, defect detection of metal parts mostly relies on manual visual inspection or single-spectrum imaging methods, which suffer from low detection efficiency, high subjectivity, and susceptibility to interference from metal surface reflections. Especially for highly reflective or complex curved parts, traditional image detection methods struggle to effectively extract subtle defects (such as scratches, cracks, and pores), leading to missed detections or misjudgments.

[0004] Therefore, there is a need for intelligent detection of defects in metal parts by reducing metal reflection interference and using multispectral fusion. Summary of the Invention

[0005] This invention provides a method for intelligent detection of defects in metal parts based on multispectral fusion and a computer-readable storage medium. Its main purpose is to achieve intelligent detection of defects in metal parts by reducing metal reflection interference and through multispectral fusion.

[0006] To achieve the above objectives, this invention provides an intelligent defect detection method for metal parts based on multispectral fusion, comprising:

[0007] A metal part is acquired, and an image acquisition operation is performed on the metal part using pre-constructed multiple light of different wavelengths to obtain multiple polarized part image sets. The polarized part image set includes multiple polarized part images with different polarization directions obtained by performing an image acquisition operation on the metal part using light of the same wavelength.

[0008] A target polarization part image set is obtained by sequentially extracting one polarization part image set from multiple polarization part image sets, wherein the target polarization part image set includes multiple target polarization part images;

[0009] Perform the following operations on the image set of the target polarized component:

[0010] Randomly extract a target polarization part image from the target polarization part image set, and perform the following operations on the extracted target polarization part image:

[0011] Obtain multiple polarization pixels from the image of the target polarized part, and perform the following operation on each of the multiple polarization pixels:

[0012] Extract the co-polarized pixels corresponding to the polarization pixels from each of the multiple target polarization part images to obtain multiple co-polarized pixels;

[0013] The grayscale values ​​of polarized pixels and multiple co-polarized pixels are obtained separately to obtain multiple polarization grayscale values;

[0014] The first polarization component and the second polarization component are calculated based on multiple polarization gray values, and the linear polarization gray value corresponding to the polarization pixel is calculated based on the first polarization component and the second polarization component.

[0015] By summing up the linear polarization grayscale values, multiple linear polarization grayscale values ​​corresponding to multiple polarization pixels are obtained. Based on the multiple linear polarization grayscale values, a spectral image of the part corresponding to the target polarization part image set is constructed.

[0016] By summarizing the spectral images of the parts, multiple spectral images of the parts corresponding to multiple sets of polarized part images are obtained;

[0017] A multispectral fusion operation is performed on the spectral images of multiple parts to obtain a fused part image, and defects in metal parts are identified based on the fused part image.

[0018] Optionally, performing a multispectral fusion operation on multiple part spectral images to obtain a fused part image includes:

[0019] Extract one spectral image from multiple part spectral images sequentially to obtain the target part image, and then perform the following operations on the target part image:

[0020] Step A: Perform Gaussian filtering and downsampling on the target part image to obtain a low-resolution image, and identify the resolution of the low-resolution image;

[0021] Repeat step A. When the resolution of the low-resolution image is less than or equal to the preset resolution threshold, summarize the low-resolution images to obtain multiple low-resolution images.

[0022] Upsampling is performed on multiple low-resolution images to obtain multiple original-resolution images;

[0023] The resolutions of the target part image and multiple low-resolution images are obtained respectively, resulting in multiple resolutions. Based on the multiple resolutions, the target part image and multiple original resolution images are sorted in descending order of resolution to obtain a resolution image sequence, wherein the resolution image sequence includes multiple part resolution images.

[0024] Extract one resolution part image from the resolution image sequence one by one to obtain the target resolution image;

[0025] If the extracted target resolution image is the preset end resolution image, then the target resolution image is confirmed as a difference image;

[0026] If the extracted target resolution image is not the end resolution image, then the adjacent resolution image of the target resolution image is obtained. The adjacent resolution image is the part resolution image that is adjacent to the target resolution image and lags behind the target resolution image in the resolution image sequence.

[0027] Calculate the difference image between the target resolution image and adjacent resolution images;

[0028] By summing the difference images, multiple difference images corresponding to the resolution image sequence are obtained;

[0029] Based on the multiple resolutions, the multiple differential images are sorted in descending order of resolution to obtain a differential image sequence, wherein the spectral image of the part corresponds one-to-one with the differential image sequence;

[0030] By summing the difference image sequences, multiple difference image sequences are obtained;

[0031] Image registration is performed on the spectral images of multiple parts based on multiple difference image sequences to obtain multiple registered images of the parts;

[0032] Perform a multispectral fusion operation on the registered images of multiple parts to obtain a fused part image.

[0033] Optionally, the step of performing image registration on multiple part spectral images based on multiple difference image sequences to obtain multiple part registered images includes:

[0034] For each of the multiple difference image sequences, perform the following operation:

[0035] One difference image is extracted sequentially from the difference image sequence to obtain the target difference image. Feature descriptor points are extracted from the target difference image to obtain a set of feature descriptor points, which includes multiple feature descriptor points.

[0036] By summarizing the feature description point sets, multiple feature description point sets corresponding to the difference image sequences are obtained;

[0037] By summarizing multiple feature descriptor point sets, multiple feature descriptor point groups corresponding to multiple difference image sequences are obtained;

[0038] Extract a reference part image from multiple part spectral images, then remove the reference part image from the multiple part spectral images to obtain multiple part images to be registered. Perform the following operations on the reference part image:

[0039] Perform pairwise combination operations on the reference part image and multiple part images to be registered to obtain multiple sets of part images to be registered. Each set of part images to be registered includes a reference part image and a part image to be registered.

[0040] Perform the following operation on each of the multiple sets of part images to be registered:

[0041] Identify the target reference part image and the target image to be registered from the group of images of parts to be registered;

[0042] From multiple feature description point sets, the reference feature description set and the feature description point set to be registered corresponding to the target reference part image and the target reference registration image are identified respectively;

[0043] Multiple reference feature description points of the target reference part image and multiple registration feature description points of the target image to be registered are obtained from the reference feature description set and the registration feature description point set, respectively.

[0044] Extract one reference feature descriptor point from multiple reference feature descriptor points sequentially to obtain the target reference feature point, and then perform the following operations on the target reference feature point:

[0045] Calculate the reference distance between the target reference feature point and each of the multiple feature descriptor points to be registered, obtain multiple reference distances, and extract the feature descriptor point to be registered corresponding to the reference distance with the smallest value among the multiple reference distances to obtain the matching feature point. The target reference feature point and the matching feature point are recorded as a reference matching point pair.

[0046] By summarizing the reference matching point pairs, multiple reference matching point pairs corresponding to the image group of the part to be registered are obtained;

[0047] Multiple pairs of matching points to be registered are obtained based on multiple feature description points to be registered.

[0048] Extract the same matching point pairs from multiple reference matching point pairs and multiple matching point pairs to be registered, and obtain multiple matching point pairs corresponding to the image group of the part to be registered;

[0049] The registration parameter set of the image group of the part to be registered is calculated based on multiple matching point pairs, and the coordinate system mapping operation is performed on the image group of the part to be registered according to the registration parameter set to obtain the mapped part image;

[0050] The mapped part images are summarized to obtain multiple mapped part images corresponding to multiple sets of part images to be registered. The multiple mapped part images and the reference part image are recorded as multiple part registration images.

[0051] Optionally, the calculation of the registration parameter set for the image group of parts to be registered based on multiple matching point pairs includes:

[0052] Obtain multiple reference coordinates and multiple coordinates to be registered for multiple matching point pairs;

[0053] A parametric solution model is constructed based on multiple reference coordinates and multiple coordinates to be registered. The parametric solution model is shown below:

[0054]

[0055] in, Indicates the first of multiple coordinate systems to be registered. The x-coordinate of the coordinates to be registered. Indicates the first of multiple coordinate systems to be registered. The ordinate of the coordinates to be registered. Indicates the first of multiple coordinate systems to be registered. The x-coordinate of the coordinates to be registered. Indicates the first of multiple coordinate systems to be registered. The ordinate of the coordinates to be registered. Represents the first of multiple reference coordinates The x-coordinate of each reference coordinate. Represents the first of multiple reference coordinates The ordinate of each reference coordinate. Represents the first of multiple reference coordinates The x-coordinate of each reference coordinate. Represents the first of multiple reference coordinates The ordinate of each reference coordinate. , , , , , , and All of these are registration parameters;

[0056] Solve the parameter solution model to obtain the registration parameter set.

[0057] Optionally, performing a multispectral fusion operation on multiple registered part images to obtain a fused part image includes:

[0058] Multiple optical bands for acquiring registration images of multiple parts are obtained, and multiple spectral reflectivities of these optical bands are determined. There is a one-to-one correspondence between the registration images of the parts, the spectral reflectivities, and the optical bands for acquisition.

[0059] The spectral reflectance values ​​are compared with the reflectance threshold. If the spectral reflectance is less than or equal to the reflectance threshold, the part registration image corresponding to the spectral reflectance is identified as a low-sensitivity image. If the spectral reflectance is greater than the reflectance threshold, the part registration image corresponding to the spectral reflectance is identified as a high-sensitivity image.

[0060] The low-sensitivity images and high-sensitivity images are summarized separately to obtain multiple low-sensitivity images and multiple high-sensitivity images;

[0061] A multispectral fusion operation is performed on multiple low-sensitivity images and multiple high-sensitivity images to obtain a fused part image.

[0062] Optionally, the step of performing a multispectral fusion operation on multiple low-sensitivity images and multiple high-sensitivity images to obtain a fused part image includes:

[0063] Randomly extract one low-sensitivity image from multiple low-sensitivity images to obtain the target low-sensitivity image, and perform the following operations on the target low-sensitivity image:

[0064] Obtain multiple pixels from the target low-sensitivity image, extract one pixel from each of the multiple pixels to obtain the target pixel, and perform the following operations on the target pixel:

[0065] Obtain the gray value of the target pixel, and extract the gray value of the corresponding pixel of the target pixel from each of the pre-constructed multiple remaining low-sensitivity images to obtain multiple remaining gray values. The multiple remaining low-sensitivity images are multiple low-sensitivity images obtained after removing the target low-sensitivity image from the multiple low-sensitivity images.

[0066] A weighted average operation is performed on the gray value of the target pixel and the multiple remaining gray values ​​based on the multiple spectral reflectances to obtain the average low-sensitivity gray value corresponding to the target pixel.

[0067] By summing the average low-sensitivity gray values, multiple average low-sensitivity gray values ​​corresponding to the target low-sensitivity image are obtained;

[0068] One high-sensitivity image is randomly extracted from multiple high-sensitivity images, and multiple average high-sensitivity grayscale values ​​are obtained based on the extracted high-sensitivity image;

[0069] Based on multiple average low-sensitivity gray values ​​and multiple average high-sensitivity gray values, multiple reflective contrasts are calculated using a pre-constructed reflective contrast calculation formula to perform binary region division on multiple registered part images. The reflective contrast, average low-sensitivity gray value, and average high-sensitivity gray value correspond one-to-one, and the reflective contrast corresponds one-to-one with the pixels of the fused part image in the fused part image.

[0070] Based on multiple reflective contrasts, each part registration image in multiple part registration images is divided into two regions to obtain multiple reflective regions and multiple normal regions. Each part registration image corresponds to one reflective region and one normal region.

[0071] A multispectral fusion operation is performed on multiple reflective areas and multiple normal areas to obtain a fused part image.

[0072] Optionally, the step of performing a multispectral fusion operation on multiple reflective areas and multiple normal areas to obtain a fused part image includes:

[0073] The average gray value of the same pixel in multiple normal regions is calculated to obtain multiple normal average gray values;

[0074] Perform coordinate mapping on multiple normal average gray values ​​to obtain a normal fused image;

[0075] Based on the multiple spectral reflectivities, a sensitivity segmentation operation is performed on multiple reflective regions to obtain a sensitive region set and a stable region set, wherein the sensitive region set includes multiple sensitive regions and the stable region set includes multiple stable regions.

[0076] The target sensitive area is identified from the set of sensitive areas, and multiple sensitive reflective pixels of the target sensitive area are obtained. Then, one sensitive reflective pixel is extracted from the multiple sensitive reflective pixels to obtain the target sensitive pixel. The following operations are performed on the target sensitive pixel:

[0077] In each sensitive region of the sensitive region set, the sensitive co-pixels of the target sensitive pixel are extracted to obtain multiple sensitive co-pixels. The gray values ​​of each sensitive co-pixel and the target sensitive pixel are obtained to obtain multiple target sensitive gray values. The total sensitive reflective weight is calculated based on the multiple target sensitive gray values ​​and the pre-constructed formula for calculating the total sensitive reflective weight.

[0078] The total weight of stable reflection corresponding to the target sensitive pixel is calculated based on the total weight of sensitive reflection, where the sum of the total weight of sensitive reflection and the total weight of stable reflection is 1;

[0079] The total weights of sensitive reflectivity and stable reflectivity are summarized separately to obtain multiple total weights of sensitive reflectivity and multiple total weights of stable reflectivity;

[0080] Multispectral fusion operations are performed on multiple sensitive regions and multiple stable regions based on multiple total weights of total sensitive reflectivity to obtain a reflective fusion image.

[0081] By stitching together the normal fused image and the reflected fused image, a fused part image is obtained.

[0082] Optionally, the formula for calculating the total weight of sensitive reflectivity is as follows:

[0083]

[0084] in, Indicates the total weight of sensitive reflectivity. This represents the minimum value of the pre-constructed total weight range for sensitive reflectivity. This represents the total number of grayscale values ​​sensitive to multiple targets. Represents the first of multiple target-sensitive grayscale values. Index of target sensitive grayscale values This represents a constraint function that restricts values ​​to the range [0,1]. Represents the first of multiple target-sensitive grayscale values. Each target's sensitive grayscale value This represents the average of the multiple normal average grayscale values. This represents the upper limit of multiple normal average grayscale values. This represents the maximum value of the pre-constructed total weight range for sensitive reflectivity.

[0085] Optionally, the step of performing multispectral fusion operations on multiple sensitive regions and multiple stable regions based on multiple total weights of sensitive reflectivity and multiple total weights of stable reflectivity to obtain a reflectivity fusion image includes:

[0086] Extract one total sensitive reflectance weight from multiple total sensitive reflectance weights sequentially to obtain the target total sensitive reflectance weight, and then perform the following operations on the target total sensitive reflectance weight:

[0087] Based on the total target sensitivity weight, extract the second sensitive co-position pixel in each of the multiple surface sensitive regions to obtain multiple second sensitive co-position pixels. Then, calculate multiple reflectivity scores corresponding to the total target sensitivity weight based on the multiple second sensitive co-position pixels. Calculate the sum of the multiple reflectivity scores to obtain the reflectivity sum.

[0088] Calculate the ratio of multiple reflectivity distinctions and reflectivity sums to obtain multiple reflectivity ratios;

[0089] Calculate the product of multiple reflectivity values ​​and the total sensitivity weight of the target to obtain multiple assigned sensitivity weights;

[0090] Obtain the sensitive reflective pixels corresponding to the total sensitive weight of the target, and then obtain the corresponding sensitive pixels;

[0091] Obtain multiple grayscale values ​​of the corresponding sensitive pixel in multiple sensitive regions to obtain multiple corresponding grayscale values;

[0092] A weighted average is performed on multiple assigned sensitive weights and multiple corresponding gray values ​​to obtain the sensitive reflective gray value;

[0093] By summing the sensitive reflective grayscale values, multiple sensitive reflective grayscale values ​​are obtained;

[0094] Extract a stable total reflectance weight from multiple stable total reflectance weights sequentially to obtain the target stable total weight, and then perform the following operations on the target stable total weight:

[0095] Obtain multiple grayscale values ​​corresponding to the target stable total weight in multiple stable regions to obtain multiple target stable grayscale values;

[0096] Calculate the average of multiple target stable gray values ​​to obtain the target stable average value;

[0097] The stable reflective grayscale value is obtained by multiplying the target stable average value and the target stable total weight.

[0098] By summing up the stable reflective grayscale values, multiple stable reflective grayscale values ​​are obtained;

[0099] Coordinate mapping is performed on multiple sensitive reflective grayscale values ​​and multiple stable reflective grayscale values ​​to obtain a reflective fusion image.

[0100] To achieve the above objectives, the present invention also provides an intelligent defect detection system for metal parts based on multispectral fusion, comprising:

[0101] The image acquisition module is used to acquire metal parts. It performs image acquisition operations on the metal parts using pre-constructed multiple light bands of different wavelengths to obtain multiple polarized part image sets. The polarized part image sets include multiple polarized part images with different polarization directions obtained by performing image acquisition operations on the metal parts using light of the same wavelength.

[0102] The polarization fusion module is used to sequentially extract a target polarization part image set from multiple polarization part image sets to obtain a target polarization part image set. The target polarization part image set includes multiple target polarization part images. The following operations are performed on the target polarization part image set: a target polarization part image is randomly extracted from the target polarization part image set, and the following operations are performed on the extracted target polarization part image: multiple polarization pixels of the target polarization part image are obtained, and the following operations are performed on each of the multiple polarization pixels: the corresponding co-polarization pixels are extracted from each of the multiple target polarization part images to obtain multiple co-polarization pixels, the gray values ​​of the polarization pixels and the multiple co-polarization pixels are obtained to obtain multiple polarization gray values, the first polarization component and the second polarization component are calculated based on the multiple polarization gray values, the linear polarization gray value corresponding to the polarization pixel is calculated based on the first polarization component and the second polarization component, the linear polarization gray values ​​are summarized to obtain multiple linear polarization gray values ​​corresponding to multiple polarization pixels, and the part spectral image corresponding to the target polarization part image set is constructed based on the multiple linear polarization gray values.

[0103] The multispectral fusion module is used to summarize the spectral images of parts, obtain multiple spectral images of parts corresponding to multiple polarization part image sets, and perform multispectral fusion operation on multiple part spectral images to obtain fused part images;

[0104] The defect identification module is used to identify defects in metal parts based on fused part images.

[0105] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:

[0106] A memory that stores at least one instruction; and a processor that executes the instructions stored in the memory to implement the aforementioned intelligent detection method for defects in metal parts based on multispectral fusion.

[0107] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one instruction, which is executed by a processor in an electronic device to implement the aforementioned intelligent detection method for defects in metal parts based on multispectral fusion.

[0108] To address the problems described in the background art, this invention acquires metal parts and performs image acquisition operations on the metal parts using pre-constructed multiple light bands of different wavelengths, resulting in multiple polarized part image sets. These polarized part image sets include multiple polarized part images with different polarization directions obtained by performing image acquisition operations on the metal parts using light of the same wavelength band. It is evident that this invention reduces the influence of metal part reflection by fusing multiple polarized part images with different polarization directions. Furthermore, a multispectral fusion operation is performed on the multiple part spectral images to obtain a fused part image. This invention further reduces the influence of reflection through the multispectral fusion operation of multiple part spectral images. Therefore, this invention achieves intelligent detection of defects in metal parts by reducing metal reflection interference and through multispectral fusion. Attached Figure Description

[0109] Figure 1 This is a flowchart illustrating an intelligent defect detection method for metal parts based on multispectral fusion, provided in an embodiment of the present invention.

[0110] Figure 2 A functional block diagram of an intelligent metal part defect detection system based on multispectral fusion provided in an embodiment of the present invention;

[0111] Figure 3 This is a schematic diagram of the structure of an electronic device that implements the intelligent detection method for defects in metal parts based on multispectral fusion, according to an embodiment of the present invention.

[0112] Explanation of reference numerals in the attached figures:

[0113] 10. Electronic device; 11. Processor; 12. Memory; 13. Bus.

[0114] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0115] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0116] This application provides a method for intelligent detection of defects in metal parts based on multispectral fusion. The executing entity of this method includes, but is not limited to, at least one electronic device that can be configured to execute the method provided in this application, such as a server or a terminal. In other words, the method can be executed by software or hardware installed on a terminal device or a server device, and the software may be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0117] Reference Figure 1 The diagram shown is a flowchart illustrating an intelligent detection method for metal part defects based on multispectral fusion according to an embodiment of the present invention. In this embodiment, the intelligent detection method for metal part defects based on multispectral fusion includes:

[0118] S1. Acquire metal parts, and perform image acquisition operations on the metal parts using pre-constructed multiple different wavelength bands of light to obtain multiple polarized part image sets. The polarized part image sets include multiple polarized part images with different polarization directions obtained by performing image acquisition operations on the metal parts using light of the same wavelength band.

[0119] It should be noted that the metal parts referred to are parts manufactured from metal. The multiple different wavelengths of light refer to light of multiple different wavelengths, such as ultraviolet light with a wavelength of 100 nanometers, visible light with a wavelength of 450 nanometers, and infrared light with a wavelength of 800 nanometers. Metal parts may have defects (minor scratches, cracks, etc.) during the manufacturing process. The reflective properties of metal itself can easily prevent the identification of surface defects when using image recognition. For example, minor scratches may not be visible in the acquired image due to reflection. Furthermore, the reflective intensity of metal varies across different wavelengths of light, and different types of defects exhibit different characteristics in different wavelengths. For instance, metals have high reflectivity (high intensity) in visible light, making it impossible to fully identify a specific defect in a polarized image of a metal part using visible light. Conversely, the metal has low reflectivity (low intensity) in infrared light, allowing the identification of areas with high reflectivity in visible light. Therefore, this invention utilizes multispectral fusion technology to reduce the interference caused by the reflective properties of metal itself in defect detection, thereby optimizing the effect of defect detection on metal parts.

[0120] It should be understood that the polarization direction refers to the direction of light acquisition. For example, if a polarization camera (a camera that performs image acquisition on a metal part) is positioned directly opposite the horizontal axis of the metal part as the reference axis, and the image acquisition operation is performed on the metal part from a 30-degree angle, then the polarization direction is the direction that differs from the reference axis by 30 degrees. The image acquired at this time is the polarized part image. In this invention, the polarization direction is selected to be less than 180 degrees. For example, this invention can select four polarization directions: 0 degrees, 45 degrees, 90 degrees, and 135 degrees.

[0121] Specifically, since the direction of metal reflection is fixed in one direction, the present invention can reduce the influence of metal reflection by acquiring multiple polarized part images of the metal part from different polarization directions.

[0122] S2. Sequentially extract a polarization part image set from multiple polarization part image sets to obtain a target polarization part image set. The target polarization part image set includes multiple target polarization part images. Perform the following operation on the target polarization part image set: randomly extract a target polarization part image from the target polarization part image set, and perform the following operation on the extracted target polarization part image: obtain multiple polarization pixels of the target polarization part image.

[0123] It should be noted that the purpose of extracting the target polarization part image set in this invention is to construct a corresponding part spectral image. The part spectral image is constructed based on multiple polarization pixels and multiple linear grayscale values ​​of a target polarization part image in the target image set. This invention randomly extracts a target polarization part image from the target polarization part image set, and based on the extracted target polarization part, fuses other target polarization part images that were not extracted from the target polarization part image set taken under different wavelengths of light. This reduces interference from the reflective properties of the metal itself, thereby reproducing the characteristics of surface defects of the parts contained in the target polarization part image set and realizing intelligent detection of defects in metal parts.

[0124] It is understood that the polarization pixel refers to a single pixel in the image of the target polarized part, and multiple polarization pixels refer to all pixels in the image of the target polarized part.

[0125] S3. Perform the following operation on each of the multiple polarization pixels: extract the corresponding co-polarization pixels from each of the multiple target polarization part images to obtain multiple co-polarization pixels.

[0126] It is clear that the step of extracting the co-position polarized pixels corresponding to the polarized pixels from each of the multiple target polarized component images means: obtaining the polarized pixel coordinates based on the pixel coordinates of the target polarized pixel in the pre-constructed image coordinate system of the target polarized component image, and extracting the pixels corresponding to the polarized pixel coordinates from each of the multiple target polarized component images to obtain multiple co-position polarized pixels. Therefore, the image coordinate systems in the multiple target polarized component images are the same, each co-position polarized pixel in the multiple co-position polarized pixels comes from a different target polarized component image, and the pixel coordinates of the different co-position polarized pixels in their respective target polarized component images are all polarized pixel coordinates.

[0127] S4. Obtain the gray values ​​of the polarized pixel and multiple co-polarized pixels respectively to obtain multiple polarized gray values, and calculate the first polarization component and the second polarization component based on the multiple polarization gray values.

[0128] It should be noted that obtaining the target polarized pixel and the multiple polarization gray values ​​of the multiple corresponding polarized pixels means taking the gray value of the target polarized pixel and the multiple gray values ​​corresponding to the multiple polarized pixels in the multiple polarized part images as multiple polarization gray values.

[0129] Furthermore, the calculation formula for the first polarization component is as follows:

[0130]

[0131] in, Indicates the first polarization component. Indicates the first Each polarization gray value Indicates the first Index of polarization grayscale values This indicates the total number of multiple polarization grayscale values. Represents the first polarized part in multiple polarization part images The angle of polarization direction of the polarization direction of the polarization component image corresponding to each polarization gray value. This represents the cosine function.

[0132] The formula for calculating the second polarization component is as follows:

[0133]

[0134] in, Indicates the second polarization component. This represents the sine function.

[0135] S5. Calculate the linear polarization grayscale value corresponding to the polarization pixel based on the first polarization component and the second polarization component, summarize the linear polarization grayscale values, obtain multiple linear polarization grayscale values ​​corresponding to multiple polarization pixels, and construct the part spectral image corresponding to the target polarization part image set based on the multiple linear polarization grayscale values.

[0136] The detailed formula for calculating the grayscale value of the line offset is shown below:

[0137]

[0138] in, Indicates the grayscale value of the line. Indicates the first polarization component. This represents the second polarization component.

[0139] It should be explained that both the first polarization component and the second polarization component represent the preference of light vibration in different directions. Examples of vibration in different directions include: light vibrating along a direction perpendicular to light propagation, light vibrating along the direction of light propagation, and light vibrating obliquely (i.e., neither along the direction of light propagation nor perpendicular to it). In this invention, a larger and positive first polarization component indicates that the light is more biased towards vibrating in the direction perpendicular to light propagation; otherwise, it indicates that the light is more biased towards vibrating along the direction of light propagation. If the second polarization component is positive, it indicates that the light is biased towards vibrating obliquely to the upper left (if the light source is the origin, and the direction of light propagation is the positive direction and the baseline, then the upper left is the direction in the same direction as the direction of light propagation, and the angle between the two directions is 45 degrees). If it is negative, it indicates that the light is biased towards vibrating obliquely to the upper right (when the light source is the origin, and the direction of light propagation is the positive direction and the baseline, the oblique upper right and the oblique upper left are axially symmetrical with respect to the baseline).

[0140] Furthermore, the reflection of metal creates light with a definite directional vibration, and defects disrupt this preference. Therefore, the larger the linear polarization grayscale value calculated using the first and second polarization components, the more reflective the polarized pixel is; conversely, the smaller the calculated linear polarization grayscale value, the more the polarized pixel is biased towards the absence of reflection, suggesting that the polarized pixel may be a defect in the metal part.

[0141] It is understood that the linearly offset gray value corresponds to the target polarization pixel or the same polarization pixel. The target polarization pixel or the same polarization pixel has a pixel coordinate in the image coordinate system of the polarized part image. Therefore, the construction of the part spectral image based on multiple linearly offset gray values ​​refers to replacing multiple gray values ​​corresponding to the pixel coordinates in the target polarized part image with multiple linearly offset gray values, and then recording the replaced target polarized part image as the part spectral image.

[0142] S6. Summarize the spectral images of the parts to obtain multiple spectral images of the parts corresponding to multiple polarized part image sets. Perform a multispectral fusion operation on the multiple spectral images of the parts to obtain fused part images. Identify defects in metal parts based on the fused part images.

[0143] It should be explained that the present invention reduces the influence of reflection by fusing multispectral part images, so that metal part defects can be clearly identified by existing image recognition algorithms during the inspection of metal parts.

[0144] Specifically, the step of performing a multispectral fusion operation on multiple part spectral images to obtain a fused part image includes:

[0145] Extract one spectral image from multiple part spectral images sequentially to obtain the target part image, and then perform the following operations on the target part image:

[0146] Step A: Perform Gaussian filtering and downsampling on the target part image to obtain a low-resolution image, and identify the resolution of the low-resolution image;

[0147] Repeat step A. When the resolution of the low-resolution image is less than or equal to the preset resolution threshold, summarize the low-resolution images to obtain multiple low-resolution images.

[0148] Upsampling is performed on multiple low-resolution images to obtain multiple original-resolution images;

[0149] The resolutions of the target part image and multiple low-resolution images are obtained respectively, resulting in multiple resolutions. Based on the multiple resolutions, the target part image and multiple original resolution images are sorted in descending order of resolution to obtain a resolution image sequence, wherein the resolution image sequence includes multiple part resolution images.

[0150] Extract one resolution part image from the resolution image sequence one by one to obtain the target resolution image;

[0151] If the extracted target resolution image is the preset end resolution image, then the target resolution image is confirmed as a difference image;

[0152] If the extracted target resolution image is not the end resolution image, then the adjacent resolution image of the target resolution image is obtained. The adjacent resolution image is the part resolution image that is adjacent to the target resolution image and lags behind the target resolution image in the resolution image sequence.

[0153] Calculate the difference image between the target resolution image and adjacent resolution images;

[0154] By summing the difference images, multiple difference images corresponding to the resolution image sequence are obtained;

[0155] Based on the multiple resolutions, the multiple differential images are sorted in descending order of resolution to obtain a differential image sequence, wherein the spectral image of the part corresponds one-to-one with the differential image sequence;

[0156] By summing the difference image sequences, multiple difference image sequences are obtained;

[0157] Image registration is performed on the spectral images of multiple parts based on multiple difference image sequences to obtain multiple registered images of the parts;

[0158] Perform a multispectral fusion operation on the registered images of multiple parts to obtain a fused part image.

[0159] It should be noted that the Gaussian filtering and downsampling operation on the target part image refers to the process of reducing the resolution of the target part image through Gaussian filtering and downsampling to obtain a low-resolution image. Because the grayscale values ​​of multiple pixels in the target part image are continuous and smooth, directly downsampling the target part image would result in the loss of smooth transition pixels between two sampled pixels, causing distortion in the details of the downsampled low-resolution image (i.e., details in the target part image, such as edges, scratches, cracks, and other defects). Therefore, Gaussian filtering must be performed on the target part image before downsampling to make the grayscale values ​​between two sampled pixels closer, ensuring that the downsampled target part image does not produce distortion. Gaussian filtering is a prior art technique and will not be elaborated upon here.

[0160] Furthermore, the purpose of downsampling the Gaussian-filtered target part image is to construct a differential image sequence, which enables coarse-to-fine registration of multiple part spectral images.

[0161] Understandably, when constructing a differential image sequence, there exists a minimum registration resolution (at which multiple part spectral images can be accurately registered without further downsampling). This minimum registration resolution is the resolution threshold preset in this invention. Generally, the minimum registration resolution is determined by the features of multiple part spectral images and historical data on metal part defect detection. The features of the part spectral images refer to the resolution of the part spectral images and the structural features of the metal part. For example, if the metal part is a screw, the structural features could be the screw pitch, screw diameter, screw length-to-diameter ratio, etc.

[0162] It should be understood that performing upsampling on multiple low-resolution images refers to aligning the resolution of the multiple low-resolution images with the resolution of the target part image through upsampling. For example, if the resolution of the first low-resolution image in the first execution of step A is 100 pixels × 100 pixels, and the resolution of the second low-resolution image in the second execution of step A is 50 pixels × 50 pixels, while the resolution of the target part image is 200 pixels × 200 pixels, then after performing upsampling on the multiple low-resolution images, the resolution of both the first and second low-resolution images becomes 200 pixels × 200 pixels.

[0163] It is clear that multiple resolutions refer to the resolutions of the target part image and multiple low-resolution images. The resolution image sequence refers to the sequence obtained by arranging the target part image and multiple original resolution images corresponding to each resolution in descending order of resolution.

[0164] It is understood that the "end resolution image" refers to the last part resolution image in the resolution image sequence. For example, if there exists a resolution image sequence: {first part resolution image, second part resolution image, third part resolution image}, then the end resolution image is the third part resolution image. If the extracted target resolution image is the third part resolution image, then the third part resolution image is identified as the third difference image. If the extracted target resolution image is the first part resolution image, then the adjacent resolution image of the first part resolution image, i.e., the second part resolution image, is obtained.

[0165] Importantly, each part resolution image in the resolution image sequence, except for the target part image, is obtained through Gaussian filtering, downsampling, and upsampling. During Gaussian filtering, downsampling, and upsampling, some details are lost (e.g., texture and grayscale values ​​of each pixel). Therefore, there are differences between multiple part resolution images in the resolution image sequence. This invention calculates the difference image between the target resolution image and adjacent resolution images, and uses the difference image to characterize the lost detail features between two adjacent part resolution images in the resolution image sequence.

[0166] It should be understood that since there is a large amount of interference information in the spectral images of multiple parts, including but not limited to reflections and illumination, while the differential image only retains lost details such as texture and contour, this invention uses the differential image as the basis for registering the spectral images of multiple parts, which can achieve the effect of ignoring interference information. Therefore, this invention uses the differential image for registration, which makes the registration accuracy higher than the accuracy of directly registering the spectral images of multiple parts, avoiding the inability to accurately identify defects in metal parts due to low registration accuracy during subsequent multispectral fusion.

[0167] Specifically, the process of performing a resolution descending order on the multiple differential images based on the multiple resolutions is the same as the process of performing a resolution descending order on the target part image and the multiple original resolution images based on the multiple resolutions, and will not be described again here.

[0168] In detail, the step of performing image registration operation on multiple part spectral images based on multiple difference image sequences to obtain multiple part registered images includes:

[0169] For each of the multiple difference image sequences, perform the following operation:

[0170] One difference image is extracted sequentially from the difference image sequence to obtain the target difference image. Feature descriptor points are extracted from the target difference image to obtain a set of feature descriptor points, which includes multiple feature descriptor points.

[0171] By summarizing the feature description point sets, multiple feature description point sets corresponding to the difference image sequences are obtained;

[0172] By summarizing multiple feature descriptor point sets, multiple feature descriptor point groups corresponding to multiple difference image sequences are obtained;

[0173] Extract a reference part image from multiple part spectral images, then remove the reference part image from the multiple part spectral images to obtain multiple part images to be registered. Perform the following operations on the reference part image:

[0174] Perform pairwise combination operations on the reference part image and multiple part images to be registered to obtain multiple sets of part images to be registered. Each set of part images to be registered includes a reference part image and a part image to be registered.

[0175] Perform the following operation on each of the multiple sets of part images to be registered:

[0176] Identify the target reference part image and the target image to be registered from the group of images of parts to be registered;

[0177] From multiple feature description point sets, the reference feature description set and the feature description point set to be registered corresponding to the target reference part image and the target reference registration image are identified respectively;

[0178] Multiple reference feature description points of the target reference part image and multiple registration feature description points of the target image to be registered are obtained from the reference feature description set and the registration feature description point set, respectively.

[0179] Extract one reference feature descriptor point from multiple reference feature descriptor points sequentially to obtain the target reference feature point, and then perform the following operations on the target reference feature point:

[0180] Calculate the reference distance between the target reference feature point and each of the multiple feature descriptor points to be registered, obtain multiple reference distances, and extract the feature descriptor point to be registered corresponding to the reference distance with the smallest value among the multiple reference distances to obtain the matching feature point. The target reference feature point and the matching feature point are recorded as a reference matching point pair.

[0181] By summarizing the reference matching point pairs, multiple reference matching point pairs corresponding to the image group of the part to be registered are obtained;

[0182] Multiple pairs of matching points to be registered are obtained based on multiple feature description points to be registered.

[0183] Extract the same matching point pairs from multiple reference matching point pairs and multiple matching point pairs to be registered, and obtain multiple matching point pairs corresponding to the image group of the part to be registered;

[0184] The registration parameter set of the image group of the part to be registered is calculated based on multiple matching point pairs, and the coordinate system mapping operation is performed on the image group of the part to be registered according to the registration parameter set to obtain the mapped part image;

[0185] The mapped part images are summarized to obtain multiple mapped part images corresponding to multiple sets of part images to be registered. The multiple mapped part images and the reference part image are recorded as multiple part registration images.

[0186] It is understood that the extraction of feature descriptor points from the target difference image refers to the process of extracting the feature descriptor information of pixels in the target difference image using a pre-constructed ORB feature extraction method, and marking the feature descriptor information on the pixels. Therefore, a feature descriptor point is a pixel in the target difference image that contains feature descriptor information, and the feature descriptor information includes descriptions of features such as pixel coordinates, scale information, and grayscale values. Scale information refers to the textual description of the target difference image to which the feature descriptor point belongs. The ORB feature extraction method is existing technology and will not be described in detail here.

[0187] It should be understood that multiple part images to be registered refer to the remaining multiple part spectral images after removing the reference part image from multiple part spectral images.

[0188] It should be explained that the pairwise combination operation of the reference part image and multiple part images to be registered refers to the process of sequentially extracting one part image to be registered from the multiple part images to be registered and combining it with the reference part image.

[0189] For example, there is a reference part image and multiple part images to be registered, which are a first part image to be registered and a second part image to be registered. After performing pairwise combinations on the reference part image and the multiple part images to be registered, a first part image group to be registered {reference part image, first part image to be registered} and a second part image group to be registered {reference part image, second part image to be registered} are obtained.

[0190] It should be noted that identifying the target reference part image and the target image to be registered from the group of part images to be registered means that the reference part image in the group of part images to be registered is denoted as the target reference part image, and the part image to be registered in the group of part images to be registered is denoted as the target image to be registered.

[0191] Specifically, the feature description point set is the collection of feature description point sets corresponding to each difference image in the difference image sequence. That is, the feature description point set includes multiple feature description point sets, and all multiple feature description point sets originate from the same part spectral image. Since both the target reference part image and the target image to be registered are part spectral images, there exists one feature description point set corresponding to the target reference part image and another corresponding to the target image to be registered. Therefore, the feature description point set of the target reference part image and the feature description point set of the target image to be registered can be identified from the multiple feature description point sets. The feature description point set of the target reference part image is denoted as the reference feature description set, and the feature description point set of the target image to be registered is denoted as the target image to be registered feature description point set.

[0192] Furthermore, there are multiple feature description point sets within the feature description point group set. Each feature description point set corresponds to one difference image. Multiple feature description points in each feature description point set correspond to multiple pixels in the difference image, with one feature description point corresponding to one pixel. The number of pixels in each difference image in the difference image sequence is the same, which is also the same as the number of pixels in the part's spectral image. Moreover, the image coordinate systems of each difference image in the difference image sequence and the part's spectral image are the same. Therefore, each pixel in the target reference part image has pixels with the same pixel coordinates in each difference image in the difference image sequence (the same pixel coordinates can be exemplified by: if the pixel coordinates are (1,1), then in all other difference images...). Pixels with coordinates (1,1) are extracted from all the sub-images (i.e., the pixel coordinates are the same). Therefore, for each pixel in the target reference part image, a corresponding feature descriptor point can be found in all other difference images. This pixel and its related information in the target reference part image are recorded as feature descriptor points. Pixels with the same pixel coordinates in all other difference images and their related information are also recorded as feature descriptor points, resulting in multiple feature descriptor points. Since the pixel coordinates of these multiple feature descriptor points are the same, they can be merged into a single reference feature descriptor point corresponding to one pixel coordinate in the target reference part image. Thus, multiple reference feature descriptor points can be obtained from the set of feature descriptor points for multiple pixels in the target reference part image. Similarly, multiple registration feature descriptor points can be obtained from the set of multiple feature descriptor points for the target image, and the process is consistent with the process of obtaining multiple reference feature descriptor points from the set of multiple feature descriptor points for the target reference part image. This will not be elaborated further here.

[0193] In detail, the reference distance refers to the Hamming distance between the target reference feature point and the feature description point to be registered. It is used to describe the similarity of features between the target reference feature point and the feature description point to be registered. The smaller the reference distance, the higher the similarity of features between the target reference feature point and the feature description point to be registered. This increases the probability that the target reference feature point and the feature description point to be registered correspond to the same position on the metal part. Consequently, when calculating multiple registration parameter sets based on multiple matching point pairs, the calculated registration parameters are more accurate, and the registration accuracy of the multiple part registration images is higher. Therefore, this invention selects the feature description point to be registered corresponding to the smallest reference distance among multiple reference distances as the matching feature point, and records the target reference feature point and the matching feature point as a reference matching point pair.

[0194] It should be explained that before calculating the reference distance between the target reference feature point and each of the multiple feature description points to be registered, the textual descriptions of the target reference feature point and the multiple feature description points to be registered are all converted into binary representations, so that the calculation of the reference distance between the target reference feature point and the multiple feature description points to be registered has a unified benchmark.

[0195] It is clear that the process of calculating multiple registration distances based on multiple registration feature descriptor points is the same as the process of calculating the reference distance between the target reference feature point and each of the multiple registration feature descriptor points, and will not be repeated here. The process of obtaining multiple registration matching point pairs based on multiple registration distances is the same as the process of extracting the registration feature descriptor point corresponding to the smallest reference distance among multiple reference distances to obtain the matching feature point, and will not be repeated here.

[0196] Importantly, the extraction of identical matching point pairs among multiple reference matching point pairs and multiple matching point pairs to be registered refers to finding the intersection of multiple reference matching point pairs and multiple matching point pairs to be registered. Finding the intersection is a prior art technique, and will not be elaborated upon here.

[0197] For example, there exist multiple reference feature description points {first reference feature description point, second reference feature description point, third reference feature description point} and multiple feature description points to be registered {first feature description point to be registered, second feature description point to be registered, third feature description point to be registered}. When searching for multiple matching point pairs based on the multiple reference feature description points, multiple reference matching point pairs are obtained: {first reference feature description point, first feature description point to be registered} and {third reference feature description point, third feature description point to be registered}. The second reference feature description point has no matching feature description point to be registered. Furthermore, when searching for multiple matching point pairs based on the multiple feature description points to be registered, multiple matching point pairs to be registered are obtained: {first feature description point to be registered, first reference feature description point}, {second feature description point to be registered, first reference feature description point}, and {third feature description point to be registered, third reference feature description point}.

[0198] As further exemplified, when searching for multiple matching point pairs based on multiple feature description points to be registered, there exists a first reference feature description point set corresponding to both the first feature description point to be registered and the second feature description point to be registered. However, when searching for multiple matching point pairs based on multiple reference feature description points, the first reference feature description point only corresponds to the first feature description point to be registered. Therefore, in this invention, after performing the operation of finding multiple matching point pairs once based on both the reference feature description point set and the feature description point set to be registered, and then finding the intersection, the final multiple matching point pairs obtained are {first reference feature description point, first feature description point to be registered} and {third reference feature description point, third feature description point to be registered}, where {second feature description point to be registered, first reference feature description point} is a one-way matching error.

[0199] It should be explained that when performing feature matching operations based on multiple reference feature description points and multiple feature description points to be registered, that is, when searching for paired feature points with a feature point (reference feature description point or feature description point to be registered), the search is performed within a preset range centered on the feature point, so as to avoid two points that are far apart in space but have similar features being incorrectly matched together (for example, matching the edge of a defect in a metal part with the edge of the metal part).

[0200] Importantly, multiple matching point pairs correspond to a group of part images to be registered. Therefore, each registration parameter set in the multiple registration parameter sets calculated from the multiple matching point pairs corresponds to a group of part images to be registered. The part images to be registered in the group can be mapped to the image coordinate system of the reference part image using the registration parameter set corresponding to the group, thus achieving image registration between the reference part image and the part images to be registered, providing a positional reference for subsequent multispectral fusion. The mapped part images to be registered are denoted as mapped part images; multiple part images to be registered can then yield multiple mapped part images.

[0201] In detail, the process of performing the coordinate system mapping operation is as follows: according to the pre-constructed horizontal coordinate mapping formula and vertical coordinate mapping formula, multiple two-dimensional coordinates of the part image to be registered are mapped to multiple two-dimensional coordinates under the image coordinate system of the reference part image. The horizontal coordinate mapping formula is as follows:

[0202]

[0203] in, It represents the abscissa of the i-th coordinate to be registered after mapping.

[0204] The formula for mapping the ordinate is shown below:

[0205]

[0206] in, Indicates the first The ordinate of a coordinate system to be registered is the ordinate after mapping.

[0207] Furthermore, the calculation of the registration parameter set for the image group of parts to be registered based on multiple matching point pairs includes:

[0208] Obtain multiple reference coordinates and multiple coordinates to be registered for multiple matching point pairs;

[0209] A parametric solution model is constructed based on multiple reference coordinates and multiple coordinates to be registered. The parametric solution model is shown below:

[0210]

[0211] in, Indicates the first of multiple coordinate systems to be registered. The x-coordinate of the coordinates to be registered. Indicates the first of multiple coordinate systems to be registered. The ordinate of the coordinates to be registered. Indicates the first of multiple coordinate systems to be registered. The x-coordinate of the coordinates to be registered. Indicates the first of multiple coordinate systems to be registered. The ordinate of the coordinates to be registered. Represents the first of multiple reference coordinates The x-coordinate of each reference coordinate. Represents the first of multiple reference coordinates The ordinate of each reference coordinate. Represents the first of multiple reference coordinates The x-coordinate of each reference coordinate. Represents the first of multiple reference coordinates The ordinate of each reference coordinate. , , , , , , and All of these are registration parameters;

[0212] Solve the parameter solution model to obtain the registration parameter set.

[0213] It should be noted that the multiple matching point pairs correspond to a group of part images to be registered. Each matching point pair includes a target reference feature point and a matching feature point. The target reference feature point corresponds to a pixel coordinate in the reference part image, and the matching feature point corresponds to a pixel coordinate in the part image to be registered. The reference coordinate is the two-dimensional coordinate of the pixel coordinate in the reference part image corresponding to the target reference feature point, and the coordinate to be registered is the two-dimensional coordinate of the pixel coordinate in the part image to be registered corresponding to the matching feature point. Multiple matching point pairs can yield multiple reference coordinates and multiple coordinates to be registered.

[0214] For example, there exists a first matching point pair {matching feature points ( , ) = (10, 15), target reference feature point ( , ) = (12, 18)} and the second matching point pair {matching feature points ( , =(20,25), target reference feature point ( , = (22, 28)}, then ( , (10, 15) is the first coordinate to be registered. , (20, 25) is the second coordinate to be registered. , (12, 18) is the first reference coordinate. , )=(22,28) is the second reference coordinate. =10 indicates that the x-coordinate is the first x-coordinate among the multiple coordinates to be registered. =15 represents the ordinate of the first coordinate to be registered among multiple coordinates, and so on. 22 is the x-coordinate of the second reference coordinate in a set of multiple reference coordinates. =28 is the ordinate of the second reference coordinate among multiple reference coordinates.

[0215] It should be explained that the solution of the parameter solution model refers to solving the parameter solution model using the least squares method. The least squares method is an existing technology, and will not be described in detail here.

[0216] Furthermore, the step of performing a multispectral fusion operation on multiple registered part images to obtain a fused part image includes:

[0217] Multiple optical bands for acquiring registration images of multiple parts are obtained, and multiple spectral reflectivities of these optical bands are determined. There is a one-to-one correspondence between the registration images of the parts, the spectral reflectivities, and the optical bands for acquisition.

[0218] The spectral reflectance values ​​are compared with the reflectance threshold. If the spectral reflectance is less than or equal to the reflectance threshold, the part registration image corresponding to the spectral reflectance is identified as a low-sensitivity image. If the spectral reflectance is greater than the reflectance threshold, the part registration image corresponding to the spectral reflectance is identified as a high-sensitivity image.

[0219] The low-sensitivity images and high-sensitivity images are summarized separately to obtain multiple low-sensitivity images and multiple high-sensitivity images;

[0220] A multispectral fusion operation is performed on multiple low-sensitivity images and multiple high-sensitivity images to obtain a fused part image.

[0221] It is understood that multiple light acquisition bands refer to multiple different light bands, such as infrared light at 800 nm and visible light at 450 nm. The spectral reflectance is a pre-existing concept and will not be elaborated upon here.

[0222] It needs to be explained that spectral reflectance is specifically manifested in the different sensitivities of light to reflection at different wavelengths (i.e., the intensity of light reflection differs at different wavelengths).

[0223] It should be noted that the reflectance threshold is a threshold of spectral reflectance determined based on a physical reference. It is used to classify multiple registered part images. By comparing multiple spectral reflectance values ​​with the reflectance threshold, the multiple registered part images are divided into multiple low-sensitivity images and multiple high-sensitivity images. Furthermore, different multispectral fusion operations can be performed based on the different sensitivities to reflection, further reducing the impact of reflection.

[0224] Furthermore, the step of performing a multispectral fusion operation on multiple low-sensitivity images and multiple high-sensitivity images to obtain a fused part image includes:

[0225] Randomly extract one low-sensitivity image from multiple low-sensitivity images to obtain the target low-sensitivity image, and perform the following operations on the target low-sensitivity image:

[0226] Obtain multiple pixels from the target low-sensitivity image, extract one pixel from each of the multiple pixels to obtain the target pixel, and perform the following operations on the target pixel:

[0227] Obtain the gray value of the target pixel, and extract the gray value of the corresponding pixel of the target pixel from each of the pre-constructed multiple remaining low-sensitivity images to obtain multiple remaining gray values. The multiple remaining low-sensitivity images are multiple low-sensitivity images obtained after removing the target low-sensitivity image from the multiple low-sensitivity images.

[0228] A weighted average operation is performed on the gray value of the target pixel and the multiple remaining gray values ​​based on the multiple spectral reflectances to obtain the average low-sensitivity gray value corresponding to the target pixel.

[0229] By summing the average low-sensitivity gray values, multiple average low-sensitivity gray values ​​corresponding to the target low-sensitivity image are obtained;

[0230] One high-sensitivity image is randomly extracted from multiple high-sensitivity images, and multiple average high-sensitivity grayscale values ​​are obtained based on the extracted high-sensitivity image;

[0231] Based on multiple average low-sensitivity gray values ​​and multiple average high-sensitivity gray values, multiple reflective contrasts are calculated using a pre-constructed reflective contrast calculation formula to perform binary region division on multiple registered part images. The reflective contrast, average low-sensitivity gray value, and average high-sensitivity gray value correspond one-to-one, and the reflective contrast corresponds one-to-one with the pixels of the fused part image in the fused part image.

[0232] Based on multiple reflective contrasts, each part registration image in multiple part registration images is divided into two regions to obtain multiple reflective regions and multiple normal regions. Each part registration image corresponds to one reflective region and one normal region.

[0233] A multispectral fusion operation is performed on multiple reflective areas and multiple normal areas to obtain a fused part image.

[0234] It should be explained that the target low-sensitivity image includes multiple pixels, and the pixel is an existing concept, which will not be elaborated here.

[0235] It is clear that the meaning of the co-position pixel of the target pixel and the co-polarized pixel of the polarized pixel is the same, and will not be repeated here.

[0236] Specifically, the weighted averaging operation based on the multiple spectral reflectances of the target pixel and the multiple remaining gray values ​​refers to normalizing the multiple spectral reflectances using the MAX-MIN normalization method, then calculating the product of each normalized spectral reflectance with the gray value of the target pixel and the corresponding gray value among the multiple remaining gray values ​​to obtain multiple reflective gray values, and summing the multiple reflective gray values ​​to obtain the average low-sensitivity gray value.

[0237] In detail, the process of obtaining multiple average high-sensitivity gray values ​​based on multiple high-sensitivity images is the same as the process of obtaining multiple average low-sensitivity gray values ​​based on multiple low-sensitivity images, and will not be repeated here.

[0238] Importantly, the formula for calculating reflective contrast is as follows:

[0239]

[0240] in, Indicates reflective contrast. Indicates the average high-sensitivity grayscale value. This represents the average low-sensitivity grayscale value.

[0241] It should be understood that both the average high-sensitivity grayscale value and the average low-sensitivity grayscale value correspond to the same pixel coordinate in multiple low-sensitivity images and multiple high-sensitivity images. The calculated reflectance contrast characterizes the sensitivity of the same pixel coordinate to reflectance in multiple low-sensitivity images and multiple high-sensitivity images. Furthermore, by comparing the multiple reflectance contrasts of multiple pixel coordinates with a pre-constructed reflectance contrast threshold, multiple registered images of parts can be divided into multiple reflective regions and multiple normal regions.

[0242] It should be noted that the pre-constructed reflectivity contrast threshold was determined by statistical analysis of a large number of sample images (historical image data of metal parts), selecting the critical value that best distinguishes reflective areas from normal areas. If the reflectivity contrast is less than or equal to the reflectivity contrast threshold, the pixel coordinates corresponding to the reflectivity contrast are classified as normal areas. Conversely, if the reflectivity contrast is greater than or equal to the threshold, the pixel coordinates corresponding to the reflectivity contrast are classified as reflective areas.

[0243] In detail, the multispectral fusion operation performed on multiple reflective areas and multiple normal areas to obtain a fused part image includes:

[0244] The average gray value of the same pixel in multiple normal regions is calculated to obtain multiple normal average gray values;

[0245] Perform coordinate mapping on multiple normal average gray values ​​to obtain a normal fused image;

[0246] Based on the multiple spectral reflectivities, a sensitivity segmentation operation is performed on multiple reflective regions to obtain a sensitive region set and a stable region set, wherein the sensitive region set includes multiple sensitive regions and the stable region set includes multiple stable regions.

[0247] The target sensitive area is identified from the set of sensitive areas, and multiple sensitive reflective pixels of the target sensitive area are obtained. Then, one sensitive reflective pixel is extracted from the multiple sensitive reflective pixels to obtain the target sensitive pixel. The following operations are performed on the target sensitive pixel:

[0248] In each sensitive region of the sensitive region set, the sensitive co-pixels of the target sensitive pixel are extracted to obtain multiple sensitive co-pixels. The gray values ​​of each sensitive co-pixel and the target sensitive pixel are obtained to obtain multiple target sensitive gray values. The total sensitive reflective weight is calculated based on the multiple target sensitive gray values ​​and the pre-constructed formula for calculating the total sensitive reflective weight.

[0249] The total weight of stable reflection corresponding to the target sensitive pixel is calculated based on the total weight of sensitive reflection, where the sum of the total weight of sensitive reflection and the total weight of stable reflection is 1;

[0250] The total weights of sensitive reflectivity and stable reflectivity are summarized separately to obtain multiple total weights of sensitive reflectivity and multiple total weights of stable reflectivity;

[0251] Multispectral fusion operations are performed on multiple sensitive regions and multiple stable regions based on multiple total weights of total sensitive reflectivity to obtain a reflective fusion image.

[0252] The normal fused image and the reflected fused image are stitched together to obtain a fused part image. It is understood that the average of the multiple normal grayscale value sets refers to extracting a single normal grayscale value set from the multiple normal grayscale value sets, and performing the following operations on the extracted normal grayscale value set:

[0253] Extract one normal grayscale value from the extracted normal grayscale value set in turn, and perform the following operations on the extracted normal grayscale value:

[0254] Obtain the normal grayscale value of each normal grayscale value at the corresponding position in the set of multiple normal grayscale values ​​to obtain multiple normal grayscale values. Then calculate the average value of the multiple normal grayscale values ​​and the extracted normal grayscale values ​​to obtain the normal average grayscale value. Summarize the normal average grayscale values ​​to obtain multiple normal average grayscale values.

[0255] It should be noted that the calculation of the average gray value of co-position pixels in multiple normal regions refers to extracting pixels with the same pixel coordinates in multiple normal regions and calculating the average gray value of all pixels with the same coordinates, which is the normal average gray value. The number of normal average gray values ​​is the same as the number of pixels in the normal region, and the number of pixels in different normal regions is the same.

[0256] It should be explained that performing coordinate mapping on multiple normal average gray values ​​means replacing multiple gray values ​​of multiple pixel coordinates of a randomly extracted normal region with multiple normal average gray values. Generally, each of the multiple normal regions contains the same number of pixels; therefore, a normal region can be randomly extracted from the multiple normal regions to perform gray value replacement.

[0257] Specifically, each reflective region corresponds to a part registration image, a part registration image corresponds to a collection light band, and a collection light band corresponds to a spectral reflectance. Therefore, the sensitivity division operation based on the multiple spectral reflectances for multiple reflective regions refers to dividing the multiple reflective regions into a sensitive region set and a stable region set according to the spectral reflectances corresponding to the multiple reflective regions. The sensitive region set includes multiple sensitive regions, and the stable region set includes multiple stable regions. The sensitive regions are the regions of the part registration image that are sensitive to reflection, and the stable regions are the regions of the part registration image that are not sensitive to reflection.

[0258] In detail, identifying the target sensitive region from the sensitive region set refers to randomly extracting a target sensitive region from the sensitive region set. The reason for random extraction is the same as the reason for randomly extracting a target polarized component image from the target polarized component image set, and will not be repeated here. Multiple sensitive reflective pixels refer to multiple pixels included in the target sensitive region.

[0259] It is clear that the meaning of the sensitive co-position pixel of the target sensitive pixel and the co-polarized pixel corresponding to the polarization pixel is the same, and will not be repeated here. The target sensitive gray value refers to the gray value of the sensitive co-position pixel or the target sensitive pixel.

[0260] Importantly, the formula for calculating the total weight of the sensitive reflectivity is as follows:

[0261]

[0262] in, Indicates the total weight of sensitive reflectivity. This represents the minimum value of the pre-constructed total weight range for sensitive reflectivity. This represents the total number of grayscale values ​​sensitive to multiple targets. Represents the first of multiple target-sensitive grayscale values. Index of target sensitive grayscale values This represents a constraint function that restricts values ​​to the range [0,1]. Represents the first of multiple target-sensitive grayscale values. Each target's sensitive grayscale value This represents the average of the multiple normal average grayscale values. This represents the upper limit of multiple normal average grayscale values. This represents the maximum value of the pre-constructed total weight range for sensitive reflectivity.

[0263] It should be noted that the upper limit of multiple normal average gray values ​​refers to the maximum value of multiple normal average gray values. The total weight range of sensitive reflectivity is determined by historical image data of metal parts, and it allows for the accurate identification of defects in metal parts by performing multispectral fusion on registered images of multiple parts. The function that restricts values ​​to the range [0,1] is an existing Python function, which will not be elaborated upon here.

[0264] As a further example, if the total weight of sensitive reflectivity is 0.8, then the total weight of stable reflectivity is 0.2.

[0265] Understandably, the purpose of this invention in calculating multiple total weights for sensitive reflectivity and multiple total weights for stable reflectivity is to perform different fusion operations on multiple different regions of multiple part registration images using multiple spectral reflectivities, thereby reducing the impact of reflectivity.

[0266] It is understood that the normal fused image is a fused image of multiple normal areas corresponding to multiple part registration images, and the reflective fused image is a fused image of multiple reflective areas corresponding to multiple part registration images. Therefore, the normal fused image and the reflective fused image can be stitched together by performing fused image stitching through multiple pixel coordinates corresponding to the normal fused image and the reflective fused image.

[0267] Furthermore, the step of performing multispectral fusion operations on multiple sensitive regions and multiple stable regions based on multiple sensitive total weights and multiple stable total weights to obtain a fused reflection image includes:

[0268] Extract one total sensitive reflectance weight from multiple total sensitive reflectance weights sequentially to obtain the target total sensitive reflectance weight, and then perform the following operations on the target total sensitive reflectance weight:

[0269] Based on the total target sensitivity weight, extract the second sensitive co-position pixel in each of the multiple surface sensitive regions to obtain multiple second sensitive co-position pixels. Then, calculate multiple reflectivity scores corresponding to the total target sensitivity weight based on the multiple second sensitive co-position pixels. Calculate the sum of the multiple reflectivity scores to obtain the reflectivity sum.

[0270] Calculate the ratio of multiple reflectivity distinctions and reflectivity sums to obtain multiple reflectivity ratios;

[0271] Calculate the product of multiple reflectivity values ​​and the total sensitivity weight of the target to obtain multiple assigned sensitivity weights;

[0272] Obtain the sensitive reflective pixels corresponding to the total sensitive weight of the target, and then obtain the corresponding sensitive pixels;

[0273] Obtain multiple grayscale values ​​of the corresponding sensitive pixel in multiple sensitive regions to obtain multiple corresponding grayscale values;

[0274] A weighted average is performed on multiple assigned sensitive weights and multiple corresponding gray values ​​to obtain the sensitive reflective gray value;

[0275] By summing the sensitive reflective grayscale values, multiple sensitive reflective grayscale values ​​are obtained;

[0276] Extract a stable total reflectance weight from multiple stable total reflectance weights sequentially to obtain the target stable total weight, and then perform the following operations on the target stable total weight:

[0277] Obtain multiple grayscale values ​​corresponding to the target stable total weight in multiple stable regions to obtain multiple target stable grayscale values;

[0278] Calculate the average of multiple target stable gray values ​​to obtain the target stable average value;

[0279] The stable reflective grayscale value is obtained by multiplying the target stable average value and the target stable total weight.

[0280] By summing up the stable reflective grayscale values, multiple stable reflective grayscale values ​​are obtained;

[0281] Coordinate mapping is performed on multiple sensitive reflective grayscale values ​​and multiple stable reflective grayscale values ​​to obtain a reflective fusion image. It should be noted that the total target sensitivity weight corresponds to the target sensitive pixel, and the total target sensitivity weight corresponds to a pixel coordinate. The second sensitive co-position pixel refers to the pixel whose pixel coordinate corresponding to the total target sensitivity weight is extracted from one of the multiple sensitive regions. Thus, multiple second sensitive co-position pixels can be extracted from the multiple sensitive regions.

[0282] It is clear that the multiple reflectivity scores corresponding to the total target sensitivity weight based on multiple second sensitive co-pixels refer to calculating multiple reflectivity scores corresponding to multiple second sensitive co-pixels using a pre-constructed reflectivity score calculation formula. The formula for calculating the reflectivity score is as follows:

[0283]

[0284] in, Represents the first of multiple second-sensitive co-position pixels The reflectivity of the second sensitive co-pixel. Represents the first of multiple second-sensitive co-position pixels The grayscale value of the second sensitive co-position pixel. This represents the average of multiple grayscale values ​​of the pixel coordinates corresponding to the target-sensitive pixel in each of the multiple stable regions, corresponding to the total target-sensitive weight. It represents the standard deviation of multiple grayscale values ​​of the target sensitive pixel corresponding to the total target sensitivity weight in each of the multiple stable regions.

[0285] It should be noted that obtaining the sensitive reflective pixel corresponding to the total sensitive weight of the target refers to obtaining the sensitive reflective pixel corresponding to the total sensitive weight of the target from multiple sensitive reflective pixels as the corresponding sensitive pixel. The pixel coordinates represented by the corresponding sensitive pixel have corresponding pixel coordinates in multiple sensitive regions. Therefore, the present invention can obtain multiple grayscale values ​​of the corresponding pixel coordinates in multiple sensitive regions as multiple corresponding grayscale values ​​through the pixel coordinates represented by the corresponding sensitive pixel.

[0286] It should be understood that assigning a sensitive weight corresponds to a pixel coordinate of one of the multiple sensitive regions, and the corresponding grayscale value corresponds to a pixel coordinate of one of the multiple sensitive regions.

[0287] Further, it can be understood that the weighted average of the multiple assigned sensitive weights and the multiple corresponding gray values ​​refers to multiplying each of the multiple assigned sensitive weights by its corresponding gray value and then summing them to obtain the sensitive reflective gray value.

[0288] It is clear that the process of obtaining the multiple grayscale values ​​corresponding to the target stable total weight in multiple stable regions is the same as the process of obtaining the sensitive reflective pixels corresponding to the target sensitive total weight and obtaining the multiple grayscale values ​​of the corresponding sensitive pixels in multiple sensitive regions, and will not be elaborated further here. The target stable average value is the average of multiple target stable grayscale values, and the process of performing coordinate mapping on multiple sensitive reflective grayscale values ​​and multiple stable reflective grayscale values ​​is the same as the process of performing coordinate mapping on multiple normal average grayscale values, and will not be elaborated further here.

[0289] To address the problems described in the background art, this invention acquires metal parts and performs image acquisition operations on the metal parts using pre-constructed multiple light bands of different wavelengths, resulting in multiple polarized part image sets. These polarized part image sets include multiple polarized part images with different polarization directions obtained by performing image acquisition operations on the metal parts using light of the same wavelength band. It is evident that this invention reduces the influence of metal part reflection by fusing multiple polarized part images with different polarization directions. Furthermore, a multispectral fusion operation is performed on the multiple part spectral images to obtain a fused part image. This invention further reduces the influence of reflection through the multispectral fusion operation of multiple part spectral images. Therefore, this invention achieves intelligent detection of defects in metal parts by reducing metal reflection interference and through multispectral fusion.

[0290] like Figure 2 The diagram shown is a functional block diagram of an intelligent metal part defect detection system based on multispectral fusion provided in an embodiment of the present invention.

[0291] The intelligent metal part defect detection system 100 based on multispectral fusion described in this invention can be installed in an electronic device. Depending on the functions implemented, the intelligent metal part defect detection system 100 based on multispectral fusion may include an image acquisition module 101, a polarization fusion module 102, a multispectral fusion module 103, and a defect identification module 104. The module described in this invention can also be called a unit, referring to a series of computer program segments that can be executed by the processor of an electronic device and perform a fixed function, stored in the memory of the electronic device.

[0292] The image acquisition module 101 is used to acquire metal parts and perform image acquisition operations on the metal parts using pre-constructed multiple different wavelength bands of light to obtain multiple polarized part image sets. The polarized part image sets include multiple polarized part images with different polarization directions obtained by performing image acquisition operations on the metal parts using light of the same wavelength band.

[0293] The polarization fusion module 102 is used to sequentially extract a polarization part image set from multiple polarization part image sets to obtain a target polarization part image set. The target polarization part image set includes multiple target polarization part images. The following operations are performed on the target polarization part image set: randomly extract a target polarization part image from the target polarization part image set, and perform the following operations on the extracted target polarization part image: obtain multiple polarization pixels of the target polarization part image, and perform the following operations on each of the multiple polarization pixels: extract the co-position polarization pixels corresponding to the polarization pixels from each of the multiple target polarization part images to obtain multiple co-position polarization pixels, obtain the gray values ​​of the polarization pixels and the multiple co-position polarization pixels respectively to obtain multiple polarization gray values, calculate the first polarization component and the second polarization component based on the multiple polarization gray values, calculate the linear polarization gray value corresponding to the polarization pixel based on the first polarization component and the second polarization component, summarize the linear polarization gray values ​​to obtain multiple linear polarization gray values ​​corresponding to the multiple polarization pixels, and construct the part spectral image corresponding to the target polarization part image set based on the multiple linear polarization gray values.

[0294] The multispectral fusion module 103 is used to summarize the spectral images of the parts, obtain multiple spectral images of the parts corresponding to multiple polarization part image sets, and perform a multispectral fusion operation on the multiple spectral images of the parts to obtain a fused part image;

[0295] The defect identification module 104 is used to identify defects in metal parts based on fused part images.

[0296] In detail, the modules in the intelligent metal part defect detection system 100 based on multispectral fusion described in this embodiment of the invention employ the same methods as described above during use. Figure 1The method used is the same as the intelligent detection method for defects in metal parts based on multispectral fusion described in the article, and can produce the same technical effect, so it will not be repeated here.

[0297] like Figure 3 The diagram shown is a structural schematic of an electronic device that implements a method for intelligent detection of defects in metal parts based on multispectral fusion, according to an embodiment of the present invention.

[0298] The electronic device 1 may include a processor 10, a memory 11 and a bus 12, and may also include a computer program stored in the memory 11 and capable of running on the processor 10, such as a program for intelligent detection of defects in metal parts based on multispectral fusion.

[0299] The memory 11 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device 1, such as a plug-in portable hard drive, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device 1. Furthermore, the memory 11 includes both internal storage units and external storage devices of the electronic device 1. The memory 11 can be used not only to store application software and various types of data installed on the electronic device 1, such as the code of a multispectral fusion-based intelligent detection method for metal part defects, but also to temporarily store data that has been output or will be output.

[0300] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., a method for intelligent detection of defects in metal parts based on multispectral fusion) and calls data stored in the memory 11 to perform various functions of the electronic device 1 and process data.

[0301] The bus 12 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus 12 can be divided into an address bus, a data bus, a control bus, etc. The bus 12 is configured to realize the connection and communication between the memory 11 and at least one processor 10, etc.

[0302] Figure 3 Only electronic devices with components are shown; those skilled in the art will understand that... Figure 3 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0303] For example, although not shown, the electronic device 1 may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management system, thereby enabling functions such as charging management, discharging management, and power consumption management through the power management system. The power supply may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0304] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish communication connections between the electronic device 1 and other electronic devices.

[0305] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), and optionally, a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device 1 and to display a visual user interface.

[0306] The intelligent detection method program for metal part defects based on multispectral fusion, stored in the memory 11 of the electronic device 1, is a combination of multiple instructions. When run in the processor 10, it can achieve the following:

[0307] A metal part is acquired, and an image acquisition operation is performed on the metal part using pre-constructed multiple light of different wavelengths to obtain multiple polarized part image sets. The polarized part image set includes multiple polarized part images with different polarization directions obtained by performing an image acquisition operation on the metal part using light of the same wavelength.

[0308] A target polarization part image set is obtained by sequentially extracting one polarization part image set from multiple polarization part image sets, wherein the target polarization part image set includes multiple target polarization part images;

[0309] Perform the following operations on the image set of the target polarized component:

[0310] Randomly extract a target polarization part image from the target polarization part image set, and perform the following operations on the extracted target polarization part image:

[0311] Obtain multiple polarization pixels from the image of the target polarized part, and perform the following operation on each of the multiple polarization pixels:

[0312] Extract the co-polarized pixels corresponding to the polarization pixels from each of the multiple target polarization part images to obtain multiple co-polarized pixels;

[0313] The grayscale values ​​of polarized pixels and multiple co-polarized pixels are obtained separately to obtain multiple polarization grayscale values;

[0314] The first polarization component and the second polarization component are calculated based on multiple polarization gray values, and the linear polarization gray value corresponding to the polarization pixel is calculated based on the first polarization component and the second polarization component.

[0315] By summing up the linear polarization grayscale values, multiple linear polarization grayscale values ​​corresponding to multiple polarization pixels are obtained. Based on the multiple linear polarization grayscale values, a spectral image of the part corresponding to the target polarization part image set is constructed.

[0316] By summarizing the spectral images of the parts, multiple spectral images of the parts corresponding to multiple sets of polarized part images are obtained;

[0317] A multispectral fusion operation is performed on the spectral images of multiple parts to obtain a fused part image, and defects in metal parts are identified based on the fused part image.

[0318] Specifically, the processor 10's implementation method for the above instructions can be found in [reference needed]. Figures 1 to 3 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0319] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or system capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0320] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following:

[0321] A metal part is acquired, and an image acquisition operation is performed on the metal part using pre-constructed multiple light of different wavelengths to obtain multiple polarized part image sets. The polarized part image set includes multiple polarized part images with different polarization directions obtained by performing an image acquisition operation on the metal part using light of the same wavelength.

[0322] A target polarization part image set is obtained by sequentially extracting one polarization part image set from multiple polarization part image sets, wherein the target polarization part image set includes multiple target polarization part images;

[0323] Perform the following operations on the image set of the target polarized component:

[0324] Randomly extract a target polarization part image from the target polarization part image set, and perform the following operations on the extracted target polarization part image:

[0325] Obtain multiple polarization pixels from the image of the target polarized part, and perform the following operation on each of the multiple polarization pixels:

[0326] Extract the co-polarized pixels corresponding to the polarization pixels from each of the multiple target polarization part images to obtain multiple co-polarized pixels;

[0327] The grayscale values ​​of polarized pixels and multiple co-polarized pixels are obtained separately to obtain multiple polarization grayscale values;

[0328] The first polarization component and the second polarization component are calculated based on multiple polarization gray values, and the linear polarization gray value corresponding to the polarization pixel is calculated based on the first polarization component and the second polarization component.

[0329] By summing up the linear polarization grayscale values, multiple linear polarization grayscale values ​​corresponding to multiple polarization pixels are obtained. Based on the multiple linear polarization grayscale values, a spectral image of the part corresponding to the target polarization part image set is constructed.

[0330] By summarizing the spectral images of the parts, multiple spectral images of the parts corresponding to multiple sets of polarized part images are obtained;

[0331] A multispectral fusion operation is performed on the spectral images of multiple parts to obtain a fused part image, and defects in metal parts are identified based on the fused part image.

[0332] In the embodiments provided by this invention, it should be understood that the disclosed devices, systems, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative, and actual implementations may have other classification methods.

[0333] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0334] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0335] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0336] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A smart defect detection method for metal parts based on multispectral fusion, characterized in that, The method includes: A metal part is acquired, and an image acquisition operation is performed on the metal part using pre-constructed multiple light of different wavelengths to obtain multiple polarized part image sets. The polarized part image set includes multiple polarized part images with different polarization directions obtained by performing an image acquisition operation on the metal part using light of the same wavelength. A target polarization part image set is obtained by sequentially extracting one polarization part image set from multiple polarization part image sets, wherein the target polarization part image set includes multiple target polarization part images; Perform the following operations on the image set of the target polarized component: Randomly extract a target polarization part image from the target polarization part image set, and perform the following operations on the extracted target polarization part image: Obtain multiple polarization pixels from the image of the target polarized part, and perform the following operation on each of the multiple polarization pixels: Extract the co-polarized pixels corresponding to the polarization pixels from each of the multiple target polarization part images to obtain multiple co-polarized pixels; The grayscale values ​​of polarized pixels and multiple co-polarized pixels are obtained separately to obtain multiple polarization grayscale values; The first polarization component and the second polarization component are calculated based on multiple polarization gray values, and the linear polarization gray value corresponding to the polarization pixel is calculated based on the first polarization component and the second polarization component. By summing up the linear polarization grayscale values, multiple linear polarization grayscale values ​​corresponding to multiple polarization pixels are obtained. Based on the multiple linear polarization grayscale values, a spectral image of the part corresponding to the target polarization part image set is constructed. By summarizing the spectral images of the parts, multiple spectral images of the parts corresponding to multiple sets of polarized part images are obtained; A multispectral fusion operation is performed on the spectral images of multiple parts to obtain a fused part image, and defects in metal parts are identified based on the fused part image.

2. The intelligent defect detection method for metal parts based on multispectral fusion as described in claim 1, characterized in that, The step of performing a multispectral fusion operation on multiple part spectral images to obtain a fused part image includes: Extract one spectral image from multiple part spectral images sequentially to obtain the target part image, and then perform the following operations on the target part image: Step A: Perform Gaussian filtering and downsampling on the target part image to obtain a low-resolution image, and identify the resolution of the low-resolution image; Repeat step A. When the resolution of the low-resolution image is less than or equal to the preset resolution threshold, summarize the low-resolution images to obtain multiple low-resolution images. Upsampling is performed on multiple low-resolution images to obtain multiple original-resolution images; The resolutions of the target part image and multiple low-resolution images are obtained respectively, resulting in multiple resolutions. Based on the multiple resolutions, the target part image and multiple original resolution images are sorted in descending order of resolution to obtain a resolution image sequence, wherein the resolution image sequence includes multiple part resolution images. Extract one resolution part image from the resolution image sequence one by one to obtain the target resolution image; If the extracted target resolution image is the preset end resolution image, then the target resolution image is confirmed as a difference image; If the extracted target resolution image is not the end resolution image, then the adjacent resolution image of the target resolution image is obtained. The adjacent resolution image is the part resolution image that is adjacent to the target resolution image and lags behind the target resolution image in the resolution image sequence. Calculate the difference image between the target resolution image and adjacent resolution images; By summing the difference images, multiple difference images corresponding to the resolution image sequence are obtained; Based on the multiple resolutions, the multiple differential images are sorted in descending order of resolution to obtain a differential image sequence, wherein the spectral image of the part corresponds one-to-one with the differential image sequence; By summing the difference image sequences, multiple difference image sequences are obtained; Image registration is performed on the spectral images of multiple parts based on multiple difference image sequences to obtain multiple registered images of the parts; Perform a multispectral fusion operation on the registered images of multiple parts to obtain a fused part image.

3. The intelligent defect detection method for metal parts based on multispectral fusion as described in claim 2, characterized in that, The process involves performing image registration on multiple spectral images of multiple parts based on multiple difference image sequences to obtain multiple registered images of the parts, including: For each of the multiple difference image sequences, perform the following operation: One difference image is extracted sequentially from the difference image sequence to obtain the target difference image. Feature descriptor points are extracted from the target difference image to obtain a set of feature descriptor points, which includes multiple feature descriptor points. By summarizing the feature description point sets, multiple feature description point sets corresponding to the difference image sequences are obtained; By summarizing multiple feature descriptor point sets, multiple feature descriptor point groups corresponding to multiple difference image sequences are obtained; Extract a reference part image from multiple part spectral images, then remove the reference part image from the multiple part spectral images to obtain multiple part images to be registered. Perform the following operations on the reference part image: Perform pairwise combination operations on the reference part image and multiple part images to be registered to obtain multiple sets of part images to be registered. Each set of part images to be registered includes a reference part image and a part image to be registered. Perform the following operation on each of the multiple sets of part images to be registered: Identify the target reference part image and the target image to be registered from the group of part images to be registered; From multiple feature description point sets, the reference feature description set and the feature description point set to be registered corresponding to the target reference part image and the target reference registration image are identified respectively; Multiple reference feature description points of the target reference part image and multiple registration feature description points of the target image to be registered are obtained from the reference feature description set and the registration feature description point set, respectively. Extract one reference feature descriptor point from multiple reference feature descriptor points sequentially to obtain the target reference feature point, and then perform the following operations on the target reference feature point: Calculate the reference distance between the target reference feature point and each of the multiple feature descriptor points to be registered, obtain multiple reference distances, and extract the feature descriptor point to be registered corresponding to the reference distance with the smallest value among the multiple reference distances to obtain the matching feature point. The target reference feature point and the matching feature point are recorded as a reference matching point pair. By summarizing the reference matching point pairs, multiple reference matching point pairs corresponding to the image group of the part to be registered are obtained; Multiple pairs of matching points to be registered are obtained based on multiple feature description points to be registered. Extract the same matching point pairs from multiple reference matching point pairs and multiple matching point pairs to be registered, and obtain multiple matching point pairs corresponding to the image group of the part to be registered; The registration parameter set of the image group of the part to be registered is calculated based on multiple matching point pairs, and the coordinate system mapping operation is performed on the image group of the part to be registered according to the registration parameter set to obtain the mapped part image; The mapped part images are summarized to obtain multiple mapped part images corresponding to multiple sets of part images to be registered. The multiple mapped part images and the reference part image are recorded as multiple part registration images.

4. The intelligent defect detection method for metal parts based on multispectral fusion as described in claim 3, characterized in that, The registration parameter set calculated based on multiple matching point pairs for the image group of parts to be registered includes: Obtain multiple reference coordinates and multiple coordinates to be registered for multiple matching point pairs; A parametric solution model is constructed based on multiple reference coordinates and multiple coordinates to be registered. The parametric solution model is shown below: in, Indicates the first of multiple coordinate systems to be registered. The x-coordinate of the coordinates to be registered. Indicates the first of multiple coordinate systems to be registered. The ordinate of the coordinates to be registered. Indicates the first of multiple coordinate systems to be registered. The x-coordinate of the coordinates to be registered. Indicates the first of multiple coordinate systems to be registered. The ordinate of the coordinates to be registered. Represents the first of multiple reference coordinates The x-coordinate of each reference coordinate. Represents the first of multiple reference coordinates The ordinate of each reference coordinate. Represents the first of multiple reference coordinates The x-coordinate of each reference coordinate. Represents the first of multiple reference coordinates The ordinate of each reference coordinate. , , , , , , and All of these are registration parameters; Solve the parameter solution model to obtain the registration parameter set.

5. The intelligent defect detection method for metal parts based on multispectral fusion as described in claim 4, characterized in that, The step of performing a multispectral fusion operation on multiple registered part images to obtain a fused part image includes: Multiple optical bands for acquiring registration images of multiple parts are obtained, and multiple spectral reflectivities of these optical bands are determined. There is a one-to-one correspondence between the registration images of the parts, the spectral reflectivities, and the optical bands for acquisition. The spectral reflectance values ​​are compared with the reflectance threshold. If the spectral reflectance is less than or equal to the reflectance threshold, the part registration image corresponding to the spectral reflectance is identified as a low-sensitivity image. If the spectral reflectance is greater than the reflectance threshold, the part registration image corresponding to the spectral reflectance is identified as a high-sensitivity image. The low-sensitivity images and high-sensitivity images are summarized separately to obtain multiple low-sensitivity images and multiple high-sensitivity images; A multispectral fusion operation is performed on multiple low-sensitivity images and multiple high-sensitivity images to obtain a fused part image.

6. The intelligent defect detection method for metal parts based on multispectral fusion as described in claim 5, characterized in that, The process of performing a multispectral fusion operation on multiple low-sensitivity images and multiple high-sensitivity images to obtain a fused part image includes: Randomly extract one low-sensitivity image from multiple low-sensitivity images to obtain the target low-sensitivity image, and perform the following operations on the target low-sensitivity image: Obtain multiple pixels from the target low-sensitivity image, extract one pixel from each of the multiple pixels to obtain the target pixel, and perform the following operations on the target pixel: Obtain the gray value of the target pixel, and extract the gray value of the corresponding pixel of the target pixel from each of the pre-constructed multiple remaining low-sensitivity images to obtain multiple remaining gray values. The multiple remaining low-sensitivity images are multiple low-sensitivity images obtained after removing the target low-sensitivity image from the multiple low-sensitivity images. A weighted average operation is performed on the gray value of the target pixel and the multiple remaining gray values ​​based on the multiple spectral reflectances to obtain the average low-sensitivity gray value corresponding to the target pixel. By summing the average low-sensitivity gray values, multiple average low-sensitivity gray values ​​corresponding to the target low-sensitivity image are obtained; One high-sensitivity image is randomly extracted from multiple high-sensitivity images, and multiple average high-sensitivity grayscale values ​​are obtained based on the extracted high-sensitivity image; Based on multiple average low-sensitivity gray values ​​and multiple average high-sensitivity gray values, multiple reflective contrasts are calculated using a pre-constructed reflective contrast calculation formula to perform binary region division on multiple registered part images. The reflective contrast, average low-sensitivity gray value, and average high-sensitivity gray value correspond one-to-one, and the reflective contrast corresponds one-to-one with the pixels of the fused part image in the fused part image. Based on multiple reflective contrasts, each part registration image in multiple part registration images is divided into two regions to obtain multiple reflective regions and multiple normal regions. Each part registration image corresponds to one reflective region and one normal region. A multispectral fusion operation is performed on multiple reflective areas and multiple normal areas to obtain a fused part image.

7. The intelligent defect detection method for metal parts based on multispectral fusion as described in claim 6, characterized in that, The process of performing a multispectral fusion operation on multiple reflective areas and multiple normal areas to obtain a fused part image includes: The average gray value of the same pixel in multiple normal regions is calculated to obtain multiple normal average gray values; Perform coordinate mapping on multiple normal average gray values ​​to obtain a normal fused image; Based on the multiple spectral reflectivities, a sensitivity segmentation operation is performed on multiple reflective regions to obtain a sensitive region set and a stable region set, wherein the sensitive region set includes multiple sensitive regions and the stable region set includes multiple stable regions. The target sensitive area is identified from the set of sensitive areas, and multiple sensitive reflective pixels of the target sensitive area are obtained. Then, one sensitive reflective pixel is extracted from the multiple sensitive reflective pixels to obtain the target sensitive pixel. The following operations are performed on the target sensitive pixel: In each sensitive region of the sensitive region set, the sensitive co-pixels of the target sensitive pixel are extracted to obtain multiple sensitive co-pixels. The gray values ​​of each sensitive co-pixel and the target sensitive pixel are obtained to obtain multiple target sensitive gray values. The total sensitive reflective weight is calculated based on the multiple target sensitive gray values ​​and the pre-constructed formula for calculating the total sensitive reflective weight. The total weight of stable reflection corresponding to the target sensitive pixel is calculated based on the total weight of sensitive reflection, where the sum of the total weight of sensitive reflection and the total weight of stable reflection is 1; The total weights of sensitive reflectivity and stable reflectivity are summarized separately to obtain multiple total weights of sensitive reflectivity and multiple total weights of stable reflectivity; Multispectral fusion operations are performed on multiple sensitive regions and multiple stable regions based on multiple total weights of total sensitive reflectivity to obtain a reflective fusion image. By stitching together the normal fused image and the reflected fused image, a fused part image is obtained.

8. The intelligent defect detection method for metal parts based on multispectral fusion as described in claim 7, characterized in that, The formula for calculating the total weight of sensitive reflectivity is as follows: in, Indicates the total weight of sensitive reflectivity. This represents the minimum value of the pre-constructed total weight range for sensitive reflectivity. This represents the total number of grayscale values ​​sensitive to multiple targets. Represents the first of multiple target-sensitive grayscale values. Index of target sensitive grayscale values This represents a constraint function that restricts values ​​to the range [0,1]. Represents the first of multiple target-sensitive grayscale values. Each target's sensitive grayscale value This represents the average of the multiple normal average grayscale values. This represents the upper limit of multiple normal average grayscale values. This represents the maximum value of the pre-constructed total weight range for sensitive reflectivity.

9. The intelligent defect detection method for metal parts based on multispectral fusion as described in claim 8, characterized in that, The process involves performing multispectral fusion operations on multiple sensitive regions and multiple stable regions based on multiple total weights of sensitive and stable reflectivity, respectively, to obtain a reflectivity fusion image, including: Extract one total sensitive reflectance weight from multiple total sensitive reflectance weights sequentially to obtain the target total sensitive reflectance weight, and then perform the following operations on the target total sensitive reflectance weight: Based on the total target sensitivity weight, extract the second sensitive co-position pixel in each of the multiple surface sensitive regions to obtain multiple second sensitive co-position pixels. Then, calculate multiple reflectivity scores corresponding to the total target sensitivity weight based on the multiple second sensitive co-position pixels. Calculate the sum of the multiple reflectivity scores to obtain the reflectivity sum. Calculate the ratio of multiple reflectivity distinctions and reflectivity sums to obtain multiple reflectivity ratios; Calculate the product of multiple reflectivity values ​​and the total sensitivity weight of the target to obtain multiple assigned sensitivity weights; Obtain the sensitive reflective pixels corresponding to the total sensitive weight of the target, and then obtain the corresponding sensitive pixels; Obtain multiple grayscale values ​​of the corresponding sensitive pixel in multiple sensitive regions to obtain multiple corresponding grayscale values; A weighted average is performed on multiple assigned sensitive weights and multiple corresponding gray values ​​to obtain the sensitive reflective gray value; By summing the sensitive reflective grayscale values, multiple sensitive reflective grayscale values ​​are obtained; Extract a stable total reflectance weight from multiple stable total reflectance weights sequentially to obtain the target stable total weight, and then perform the following operations on the target stable total weight: Obtain multiple grayscale values ​​corresponding to the target stable total weight in multiple stable regions to obtain multiple target stable grayscale values; Calculate the average of multiple target stable gray values ​​to obtain the target stable average value; The stable reflective grayscale value is obtained by multiplying the target stable average value and the target stable total weight. By summing up the stable reflective grayscale values, multiple stable reflective grayscale values ​​are obtained; Coordinate mapping is performed on multiple sensitive reflective grayscale values ​​and multiple stable reflective grayscale values ​​to obtain a reflective fusion image.

10. A smart defect detection system for metal parts based on multispectral fusion, characterized in that, The system includes: The image acquisition module is used to acquire metal parts. It uses pre-constructed multiple light bands of different wavelengths to perform image acquisition operations on the metal parts, and obtains multiple polarized part image sets. The polarized part image sets include multiple polarized part images with different polarization directions obtained by performing image acquisition operations on the metal parts with light of the same wavelength. The polarization fusion module is used to sequentially extract a target polarization part image set from multiple polarization part image sets to obtain a target polarization part image set. The target polarization part image set includes multiple target polarization part images. The following operations are performed on the target polarization part image set: a target polarization part image is randomly extracted from the target polarization part image set, and the following operations are performed on the extracted target polarization part image: multiple polarization pixels of the target polarization part image are obtained, and the following operations are performed on each of the multiple polarization pixels: the corresponding co-polarization pixels are extracted from each of the multiple target polarization part images to obtain multiple co-polarization pixels, the gray values ​​of the polarization pixels and the multiple co-polarization pixels are obtained to obtain multiple polarization gray values, the first polarization component and the second polarization component are calculated based on the multiple polarization gray values, the linear polarization gray value corresponding to the polarization pixel is calculated based on the first polarization component and the second polarization component, the linear polarization gray values ​​are summarized to obtain multiple linear polarization gray values ​​corresponding to multiple polarization pixels, and the part spectral image corresponding to the target polarization part image set is constructed based on the multiple linear polarization gray values. The multispectral fusion module is used to summarize the spectral images of parts, obtain multiple spectral images of parts corresponding to multiple polarization part image sets, and perform multispectral fusion operation on multiple part spectral images to obtain fused part images; The defect identification module is used to identify defects in metal parts based on fused part images.

Images