Method and system for detecting furniture special-shaped parts based on machine vision

Abstract

The application discloses a furniture special-shaped part detection method and system based on machine vision, and relates to the technical field of machine vision detection, which comprises the following steps: collecting original image data of a furniture special-shaped part and performing image registration, performing feature enhancement on the registered image data, and generating multi-modal image data; performing decomposition and enhancement processing on the multi-modal image data to obtain a final reflection image and an enhanced topographic image; inputting the final reflection image and the enhanced topographic image into a defect detection network based on an attention mechanism and contrast learning, performing cross-modal feature fusion, and outputting a defect detection result of the furniture special-shaped part; and the application realizes effective differentiation of surface texture and mirror reflection interference of the furniture special-shaped part through multi-modal image acquisition and feature enhancement, polarization light reflection correction, and a cross-modal defect detection network based on an attention mechanism and contrast learning, and solves the problems of difficulty in accurately identifying real defects and high false detection rate and missed detection rate.

Description

Technical Field

[0001] This invention relates to the field of machine vision inspection technology, and more specifically, to a method and system for inspecting irregularly shaped furniture parts based on machine vision. Background Technology

[0002] With the rapid development of the furniture industry, the manufacturing precision and surface quality of irregularly shaped furniture parts have a significant impact on the overall performance and aesthetics of the products. Traditional defect detection of irregularly shaped furniture parts mainly relies on manual visual inspection and simple mechanical measurement methods. However, these methods suffer from low detection efficiency, poor accuracy, and significant susceptibility to human factors, making it difficult to meet the high-precision and high-efficiency quality inspection requirements of the modern furniture industry.

[0003] Furniture wood naturally possesses grain and knots, which, after finishing, often result in strong specular reflections and highlight areas. This leads to texture artifacts and overexposure interference in the inspection images, making it difficult to distinguish between real defects and natural grain, resulting in high false positive and false negative rates. To address these issues, this invention proposes a solution. Summary of the Invention

[0004] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a method and system for detecting irregularly shaped furniture parts based on machine vision. Through multimodal image acquisition and feature enhancement, polarized light reflection correction, and a cross-modal defect detection network based on attention mechanism and contrastive learning, the method effectively distinguishes between surface texture and specular reflection interference of irregularly shaped furniture parts, solving the problems of difficulty in accurately identifying real defects and high false detection and false negative rates.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] Firstly, this application provides a machine vision-based method for detecting irregularly shaped furniture parts. The method includes: acquiring original image data of irregularly shaped furniture parts and performing image registration; performing feature enhancement on the registered image data to generate multimodal image data, wherein the original image data includes color images, polarized light images, and photometric stereoscopic topographic images; performing decomposition and enhancement processing on the multimodal image data to obtain final reflection images and enhanced topographic images; inputting the final reflection images and enhanced topographic images into a defect detection network based on attention mechanisms and contrastive learning to perform cross-modal feature fusion and outputting defect detection results for the irregularly shaped furniture parts.

[0007] In one embodiment, original image data of irregularly shaped furniture parts is acquired and image registration is performed. The registered image data is then enhanced to generate multimodal image data. Specifically, the original image data is downsampled layer by layer to construct an image sequence from low resolution to high resolution. At each resolution, the image sequence is divided into several local blocks, and key feature points are extracted within each local block. Descriptors are generated for the key feature points, and feature matching is performed between corresponding local blocks. Combined with bidirectional matching verification and distance ratio thresholding to eliminate mismatched points, local matching feature point pairs are obtained. Based on the local matching feature point pairs, a preliminary affine transformation matrix is ​​obtained within each local block using a random sampling consensus algorithm. The preliminary affine transformation matrices of each local block are integrated to obtain an initial global affine matrix. With maximizing the mutual information between the reference image and the affine transformed image as the optimization objective, the initial global affine matrix is ​​iteratively updated to output a globally optimized global affine matrix. Geometric transformations are performed on the polarized light image and the photometric stereo image according to the global affine matrix, and they are aligned with the color image in spatial coordinates to obtain registered image data.

[0008] In one embodiment, the registered image data is enhanced to generate multimodal image data. Specifically, the registered image data is decomposed, and the image resolution and image noise level are obtained. Based on the image resolution and image noise level, the spatial standard deviation and intensity standard deviation of the bilateral filter are obtained. Based on the spatial standard deviation and intensity standard deviation, bilateral filtering is performed on each channel to obtain a smoothed image. High-frequency features are extracted from the smoothed image using a high-pass filtering method to output high-pass enhanced image data. The high-pass enhanced image data is fused with the original image data to generate multimodal image data.

[0009] In one embodiment, multimodal image data is decomposed and enhanced to obtain the final reflection image and enhanced topography image. Specifically, the color image is initially decomposed based on multi-scale wavelet transform and luminance-chrominance space transform to obtain low-frequency reflection components and high-frequency texture components; the low-frequency reflection components are corrected based on polarized light images to obtain polarization-corrected reflection components; the multispectral image is decomposed into diffuse reflection components and specular reflection components, and solved using the least squares method to obtain the specular reflection intensity distribution of the corresponding bands; the specular reflection intensity distribution of each band is compared at the pixel level, and... Determine whether a pixel contains specular reflection components; if so, fuse the specular reflection intensity distributions of each band to obtain a multispectral specular reflection component; construct a constraint optimization function based on the polarization-corrected reflection component and the multispectral specular reflection component for iterative optimization to obtain the optimized intrinsic reflection component; enhance the high-frequency texture component based on the surface normal information of the photometric stereo image, highlighting the minute uneven features of the target surface through a shape consistency constraint method to obtain the enhanced texture component; fuse the intrinsic reflection component and the enhanced texture component to generate the final reflection image and enhanced shape image.

[0010] In one embodiment, a color image is initially decomposed based on multi-scale wavelet transform and luminance-chrominance space transform to obtain low-frequency reflection components and high-frequency texture components. Specifically, the process involves: acquiring a color image and obtaining luminance and chrominance components based on luminance-chrominance color space decomposition; performing multi-scale wavelet decomposition on the luminance components to decompose them into low-frequency sub-bands and high-frequency sub-bands in multiple directions; performing multi-scale wavelet decomposition on the chrominance components to extract low-frequency and high-frequency components; constructing preliminary reflection components based on the low-frequency sub-bands and low-frequency components; constructing preliminary texture components based on the high-frequency sub-bands and high-frequency components; and combining the preliminary reflection and preliminary texture components to obtain the low-frequency reflection and high-frequency texture components of the color image.

[0011] In one embodiment, the low-frequency reflection component is corrected based on the polarized light image. Specifically, the following steps are taken: a polarized light image sequence is acquired and paired with the low-frequency reflection component; for each pixel in the polarized light image sequence, the light intensity value at different polarization angles is obtained to form a light intensity variation curve; based on Malus's law, the light intensity is decomposed into diffuse reflection and specular reflection components, and the specular reflection intensity and diffuse reflection intensity of each pixel are solved to generate a specular reflection estimation map; based on the specular reflection estimation map, specular reflection regions and diffuse reflection regions are determined; for pixels determined to be specular reflection regions, a linear attenuation coefficient is obtained; for each pixel, the intensity is suppressed using the linear attenuation method, and the polarization-corrected reflection component is output.

[0012] In one embodiment, the final reflection image and enhanced shape image are input into a defect detection network based on attention mechanism and contrastive learning to perform cross-modal feature fusion and output the defect detection result of the furniture irregular part. Specifically, the final reflection image and enhanced shape image are input into the defect detection network for feature extraction to obtain reflection feature map and shape feature map; a channel attention mechanism is introduced into each frame of the reflection feature map and shape feature map to generate channel weights; the channel weights are combined with the feature map channel by channel to obtain a weighted channel feature map; and a spatial attention mechanism is introduced into the channel weighted feature map to generate spatial attention. Weights are assigned and combined with the channel-weighted feature map to obtain a spatially weighted enhanced feature map. The feature map enhanced by channel attention and spatial attention is input into the contrastive learning module for attention weight optimization, and the optimized attention feature map is output. The optimized attention feature map is aligned in the spatial dimension and fused by channel concatenation to obtain a multimodal feature map. The multimodal feature map is input into the defect discrimination module to output a defect probability map. Threshold segmentation is performed on the defect probability map, and the segmented defect regions are post-processed to output the final defect detection result, which includes defect category, location, and visual annotation map.

[0013] In one embodiment, the defect probability map is segmented by a threshold, and the segmented defect regions are post-processed. Specifically, the defect probability value of each pixel is compared with a preset defect threshold. If the defect probability value of a pixel is greater than or equal to the preset defect threshold, the pixel is determined to belong to a defect region. If the defect probability value of a pixel is less than the preset defect threshold, the pixel is determined to belong to a non-defect region. The segmented defect regions are then post-processed to generate the final defect detection result.

[0014] In one embodiment, the feature map enhanced by channel attention and spatial attention is input into the contrastive learning module for attention weight optimization, and the optimized attention feature map is output. Specifically, positive sample pairs and negative sample pairs are constructed based on the enhanced feature map; for each positive and negative sample pair, the contrastive loss function is calculated and the contrastive loss value is output; the contrastive loss value is backpropagated to the channel attention module and the spatial attention module, and the weights are updated by gradient descent; the optimized attention feature map is output.

[0015] Secondly, this application provides a machine vision-based furniture irregular-shaped component detection system, which includes: an image processing module for acquiring original image data of furniture irregular-shaped components and performing image registration, performing feature enhancement on the registered image data, and generating multimodal image data, wherein the original image data includes color images, polarized light images, and photometric stereoscopic topographic images; an image decomposition and enhancement module for decomposing and enhancing the multimodal image data to obtain the final reflection image and enhanced topographic image; and a furniture irregular-shaped component defect detection module for inputting the final reflection image and enhanced topographic image into a defect detection network based on attention mechanism and contrastive learning, performing cross-modal feature fusion, and outputting the defect detection results of the furniture irregular-shaped components.

[0016] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0017] 1. By employing multimodal image acquisition and precise registration, bilateral filtering and high-pass enhancement feature processing, multi-scale wavelet and luminance-chrominance space decomposition, and polarization-based reflection correction, low-frequency reflection and high-frequency texture are effectively separated, and specular reflection interference is suppressed. At the same time, the surface micro-bumps and texture details are enhanced, making the final generated reflection image and enhanced morphology image both realistic and stable and rich in detail. This significantly improves the accuracy, reliability and robustness of defect detection for irregularly shaped furniture parts, especially under complex material and multi-light conditions, it can still accurately identify minute scratches, cracks and surface anomalies.

[0018] 2. By inputting the reflection image and the enhanced morphology image into a defect detection network based on channel attention, spatial attention, and contrastive learning, fine fusion of cross-modal features is achieved. This not only highlights defect-sensitive channels and potential defect areas and suppresses redundant background information, but also optimizes attention weights through positive and negative sample constraints. This makes the features of the same defect more compact under different modalities or perspectives, while the features of non-defect areas are dispersed. This significantly enhances the discriminative power and robustness of the feature representation, ensuring that the multimodal feature map retains the complementary advantages of reflection and morphology information. This provides a more accurate, stable, and highly discriminative input for defect discrimination, thereby improving the accuracy, reliability, and applicability of defect detection for irregularly shaped furniture parts. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the process for detecting irregularly shaped furniture parts based on machine vision, as provided in an embodiment of this application.

[0020] Figure 2 A schematic diagram of the structure of a machine vision-based furniture irregular part detection system provided in the embodiments of this application. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0022] Reference Figure 1 As shown in the schematic diagram, the method for detecting irregularly shaped furniture parts based on machine vision provided by the present invention includes the following steps:

[0023] S1. Collect the original image data of irregularly shaped furniture parts and perform image registration. Then, perform feature enhancement on the registered image data to generate multimodal image data. The original image data includes color images, polarized light images, and photometric three-dimensional morphology images.

[0024] In this embodiment, the acquisition of raw image data of irregularly shaped furniture components specifically involves: placing the irregularly shaped furniture component to be inspected in an adjustable lighting environment and fixing its spatial orientation; in this fixed state, using a color imaging device to perform an initial acquisition of the irregularly shaped furniture component, obtaining color image data of the irregularly shaped furniture component; based on the color image data, controlling the polarization direction change of the polarization light source, and using an imaging device equipped with a polarization filter to perform multi-angle polarized light imaging of the irregularly shaped furniture component, obtaining polarized light image data of the irregularly shaped furniture component; by adjusting the incident angle of the multi-directional light source, and combining it with the color imaging device to perform photometric stereoscopic imaging of the irregularly shaped furniture component, obtaining a photometric stereoscopic morphological image of the irregularly shaped furniture component.

[0025] This involves collecting and registering the original image data of irregularly shaped furniture components. The original image data includes color images, polarized light images, and photometric three-dimensional topographic images. Specifically:

[0026] The original image data was downsampled layer by layer using a Gaussian pyramid method to construct an image sequence from low resolution to high resolution.

[0027] At each resolution, the image sequence is divided into several local blocks according to a grid of preset size, and key feature points are extracted in each local block using a scale-invariant feature transformation algorithm.

[0028] Descriptors are generated for key feature points, and feature matching is performed between corresponding local blocks using a similarity metric. Mismatched points are eliminated by combining bidirectional matching verification and distance ratio thresholding to obtain locally matched feature point pairs.

[0029] In this process, the similarity distance between feature points in two local blocks is calculated to obtain the nearest neighbor matching point and the second nearest neighbor matching point. The matching points are then filtered using a distance ratio test method. When the ratio of the nearest neighbor distance to the second nearest neighbor distance is less than a preset distance threshold, the matching pair is determined to be a reliable match. Based on this, a bidirectional matching verification strategy is further adopted. For example, it is required that a certain feature point in local block A matches a corresponding point in local block B. At the same time, the same feature point in local block B must also correspond to the same feature point in local block A when performing reverse matching. Only matching pairs that satisfy bidirectional consistency are retained.

[0030] Based on the feature point pairs of local matching, the preliminary affine transformation matrix is ​​obtained in each local block using the random sampling consensus algorithm;

[0031] The initial affine transformation matrices of each local block are integrated. The integration includes assigning weights based on the matching quality of the feature points corresponding to each local block, and performing a weighted average of the affine parameters to obtain the initial global affine matrix.

[0032] With the goal of maximizing the mutual information between the reference image and the affine transformed image, the initial global affine matrix is ​​iteratively updated until a preset number of iterations is reached, at which point the iteration stops. The iteration method includes gradient descent.

[0033] Output the globally optimized global affine matrix;

[0034] Geometric transformations are performed on the polarized light image and the photometric stereo image based on the globally optimized global affine matrix to align them with the color image in spatial coordinates, resulting in registered image data.

[0035] It should be noted that mutual information is an indicator that measures the statistical dependence of grayscale values ​​between two images. A higher mutual information value indicates a stronger dependence in grayscale distribution between the two images, resulting in more accurate alignment. By using a preliminary global affine matrix as a starting point and iteratively adjusting its parameters to maximize the mutual information between the transformed image and the reference image, the optimal global affine matrix is ​​obtained, achieving precise image registration.

[0036] Furthermore, the registered image data undergoes feature enhancement to generate multimodal image data, specifically:

[0037] The registered image data is decomposed by channel, and the image resolution and image noise level of each channel are obtained;

[0038] Among them, image resolution can be obtained by pixel spacing or sampling interval, and image noise level can be obtained by local variance or signal-to-noise ratio (SNR).

[0039] Determine the spatial standard deviation and intensity standard deviation of the bilateral filter based on the image resolution and image noise level;

[0040] Among them, the spatial standard deviation of the bilateral filter is determined by linear mapping of the image resolution within a preset resolution range, and the intensity standard deviation is determined by linear mapping of the image noise level within a preset noise level range.

[0041] Based on the determined spatial standard deviation and intensity standard deviation, bilateral filtering is performed on each channel, and the filtered results of each channel are combined to obtain a smooth image.

[0042] The smoothed image is used to extract high-frequency features based on the high-pass filtering method, and high-pass enhanced image data is output. The high-pass filtering uses the Laplacian operator to select the convolution kernel size and enhancement coefficients to enhance edge, texture and microstructure information.

[0043] The Qualcomm-enhanced image data is fused with the original image data to generate multimodal image data. The fusion method is weighted superposition, channel stitching, or principal component analysis.

[0044] The multimodal image data includes color images, multispectral images, polarized light images, and photometric stereoscopic topography images. Color images are used to characterize the inherent color and texture of the target, multispectral images are used to characterize the spectral reflectance characteristics at different wavelengths, polarized light images are used to suppress specular reflection interference, and photometric stereoscopic topography images are used to characterize the minute geometric shapes of the target's surface.

[0045] It should be noted that the acquisition, registration, and feature enhancement operations on the original image data ensure precise spatial alignment of the color image, polarized light image, and photometric stereoscopic topography image. Bilateral filtering smooths noise while preserving edges, and high-pass filtering extracts high-frequency features, ultimately generating fused multimodal image data. This series of processes fully integrates multimodal information, presenting color, texture, reflectivity, and minute topographic features in a unified manner. This provides high-resolution, low-noise, and feature-rich input data for subsequent defect detection, facilitating accurate identification of surface defects, shape anomalies, or texture discontinuities, thereby improving the accuracy, reliability, and robustness of defect detection.

[0046] S2 decomposes and enhances the multimodal image data to obtain the final reflection image and enhanced morphology image.

[0047] In this embodiment, the multimodal image data is decomposed and enhanced to obtain the final reflection image and enhanced topography image, specifically as follows:

[0048] The color image is initially decomposed based on multi-scale wavelet transform and luminance-chrominance space transform to obtain low-frequency reflection component and high-frequency texture component.

[0049] The low-frequency reflection component is corrected based on the polarized light image to obtain the polarization-corrected reflection component.

[0050] The multispectral image is decomposed into diffuse reflection and specular reflection components by using a dichroic reflectance model. The diffuse reflection and specular reflection components of each pixel are solved by the least squares method to obtain the specular reflection intensity distribution of the corresponding band.

[0051] Among them, using a dichroic reflection model to decompose images of each band is an existing technology, which will not be elaborated on here;

[0052] The specular reflection intensity distribution of each band is compared at the pixel level. The judgment is based on the difference in specular reflection intensity between wavelengths. If the intensity difference is greater than the preset intensity difference threshold, the pixel is determined to contain a specular reflection component.

[0053] The specular reflection intensity distributions of each band are fused to obtain the multispectral specular reflection components;

[0054] A constrained optimization function is constructed based on the polarization-corrected reflection component and the multispectral specular reflection component. Iterative optimization is performed with the goal of minimizing the reflection inconsistency between different modes to obtain the optimized intrinsic reflection component.

[0055] The specific calculation formula for the constraint optimization function is as follows:

[0056]

[0057] In the formula, For the constrained optimization function, To sum the values ​​of all pixels (x, y) in the image, For multispectral specular reflection components, These are the regularization weight coefficients. The regularization constraint function applies prior or constraint conditions to the intrinsic reflection component R of the optimization. This is the polarization-corrected reflection component.

[0058] Based on the surface normal information of the photometric stereo image, the high-frequency texture component is enhanced, and the small concave and convex features of the target surface are highlighted by the topography consistency constraint method to obtain the enhanced texture component.

[0059] For each point on the surface of an object, the surface normal information is a unit vector perpendicular to the tangent plane at that point, used to describe the surface orientation. The surface normal information can directly reflect the surface's unevenness, curvature, scratches, or fine lines.

[0060] The inherent reflection component and the enhanced texture component are fused to generate the final reflection image and enhanced topography image for subsequent defect detection and identification.

[0061] Among them, the enhanced morphology image is an image that enhances the representation of the object's surface features such as small bumps, scratches, and cracks by processing the high-frequency components (details, edges, textures) of the image and combining them with the surface normal information of photometric stereo reconstruction. The intrinsic reflection component refers to the reflection component of the object's surface that best reflects the material's properties under ideal conditions and after removing external interference such as lighting and specular reflection.

[0062] It should be noted that by jointly decomposing and correcting multimodal image data, effective separation between reflective and texture components can be achieved. Furthermore, through polarization constraints, multispectral difference analysis, and photometric stereoscopic morphology enhancement, specular reflection interference is eliminated and minute surface irregularities are highlighted. This provides more realistic, stable, and detailed image data for subsequent defect detection. Its benefits include not only improving the accuracy of identifying minute scratches, cracks, and surface anomalies during defect detection but also enhancing the algorithm's robustness under complex materials and multi-light conditions, thereby significantly improving the reliability and practical value of the detection system.

[0063] Furthermore, based on multi-scale wavelet transform and luminance-chrominance space transform, the color image is initially decomposed to obtain low-frequency reflection components and high-frequency texture components, specifically:

[0064] A color image is acquired and converted from the RGB color space to the luminance-chrominance color space to obtain the luminance component and the chrominance component; wherein, the luminance-chrominance color space is the YCbCr color space, the luminance component is used to characterize illumination and reflection information, and the chrominance component is used to characterize inherent texture and color features;

[0065] The brightness component is decomposed into a multi-scale wavelet decomposition, which divides the brightness component into a low-frequency sub-band and a high-frequency sub-band in multiple directions; wherein the low-frequency sub-band mainly contains reflection and overall illumination information, and the high-frequency sub-band mainly contains edge, texture and detail information.

[0066] The chromaticity components are subjected to multi-scale wavelet decomposition to extract low-frequency and high-frequency components; wherein, the low-frequency components are used to characterize the smooth distribution of color, and the high-frequency components are used to characterize color changes and texture features.

[0067] Based on the low-frequency subband and low-frequency components, a preliminary reflection component is constructed; among which, the smoothing characteristics of low frequency are used to characterize illumination and specular reflection, and to suppress interference with texture details.

[0068] Based on high-frequency subbands and high-frequency components, preliminary texture components are constructed; the local variation characteristics of high frequencies are used to highlight the detailed information and texture structure of the target surface, and to reduce the influence of overall illumination.

[0069] The preliminary reflection component and the preliminary texture component are combined to obtain the preliminary decomposition result of the color image, namely the low-frequency reflection component and the high-frequency texture component.

[0070] Furthermore, the low-frequency reflection component is corrected based on the polarized light image, specifically as follows:

[0071] A polarized light image sequence is acquired, the image sequence including polarized light images acquired at multiple polarization angles; wherein the polarized light images and color images have been spatially registered;

[0072] Pair polarized light image sequences with low-frequency reflection components;

[0073] For each pixel in the polarized light image sequence, the light intensity value at different polarization angles is obtained to form a light intensity variation curve;

[0074] Based on Malus's law, the light intensity is decomposed into diffuse reflection and specular reflection components, and the specular reflection intensity and diffuse reflection intensity of each pixel are solved by least squares fitting.

[0075] The specular reflection intensity is normalized to generate a specular reflection estimation map;

[0076] Based on the specular reflection estimation map, determine the specular reflection area and the diffuse reflection area;

[0077] For pixels identified as specular reflection areas, the local signal-to-noise ratio is obtained and the ratio between it and the maximum signal-to-noise ratio in the image is used to calculate the linear attenuation coefficient.

[0078] The linear attenuation coefficient is calculated by using a pre-set reference attenuation coefficient as a reference value. This reference value is scaled based on the ratio of the signal-to-noise ratio (SNR) of a local pixel to the maximum SNR in the image. The final attenuation coefficient is equal to "the reference attenuation coefficient multiplied by the ratio of the maximum SNR to the local SNR". If the SNR of a pixel's region is low, the attenuation coefficient will be larger, thus providing stronger suppression of specular reflection. Conversely, in regions with high SNR, the attenuation coefficient will be relatively smaller to avoid over-suppression.

[0079] For each pixel, intensity suppression is performed using a linear attenuation method, and the polarization-corrected reflection component is output.

[0080] The polarization-corrected reflection component is specifically calculated using the following formula:

[0081]

[0082] In the formula, The polarization-corrected reflection component. This is the low-frequency reflection component. The linear attenuation coefficient is... The percentage of specular reflection is determined by the ratio of specular reflection intensity to total intensity. This is a non-negativity constraint function used to ensure that the correction result will not be less than 0.

[0083] Malus's law describes the change in intensity of polarized light as a function of angle after passing through a polarizer. In reflected light analysis, the specular reflection component varies with the square of the cosine of the polarization angle, while the diffuse reflection component remains essentially unchanged. Utilizing this law, the total light intensity of a pixel can be decomposed into diffuse and specular reflection components using least-squares fitting.

[0084] It should be noted that by introducing a polarized light image correction mechanism into the low-frequency reflection component, specular reflection and diffuse reflection components can be effectively distinguished and separated. Furthermore, a linear attenuation method based on adaptive signal-to-noise ratio is used to specifically suppress specular reflection, thereby avoiding the loss of texture details due to overprocessing. The advantages are twofold: firstly, it reduces the impact of specular reflection interference on image analysis, improving the realism and stability of the reflection component; secondly, it preserves the natural texture and detail features of the diffuse reflection region, providing higher accuracy and reliability for subsequent defect detection, surface morphology reconstruction, or feature extraction.

[0085] S3 inputs the final reflection image and enhanced morphology image into a defect detection network based on attention mechanism and contrastive learning to perform cross-modal feature fusion and output the defect detection results of irregular furniture parts.

[0086] In this embodiment, the final reflection image and enhanced topography image are input into a defect detection network based on attention mechanism and contrastive learning to perform cross-modal feature fusion and output the defect detection results for irregularly shaped furniture parts, specifically:

[0087] The final reflection image and enhanced morphology image are input into the defect detection network for feature extraction to obtain the reflection feature map and morphology feature map;

[0088] For each frame of the reflection feature map and the shape feature map, a channel attention mechanism is introduced. The statistics of each channel are obtained by global average pooling, and then channel weights are generated by a fully connected network and a sigmoid activation function.

[0089] The channel weights are multiplied by the feature map channel by channel to obtain a weighted channel feature map, which highlights the defect-sensitive channels and suppresses the background redundant channels.

[0090] A spatial attention mechanism is introduced on the channel-weighted feature map. Spatial attention weights are generated through convolution operations and the Sigmoid activation function. The spatial attention weights are multiplied element-wise with the channel-weighted feature map to obtain a spatially weighted enhanced feature map, which highlights potential defect areas and suppresses irrelevant areas.

[0091] The feature maps enhanced by channel attention and spatial attention are input into the contrastive learning module to optimize the attention weights, and the optimized attention feature maps are output.

[0092] The optimized attention feature maps are aligned in the spatial dimension and fused by channel splicing to obtain a multimodal feature map containing reflection information, morphological information and attention-optimized defect-sensitive features. The optimized attention feature map includes optimized reflection feature map and morphological feature map.

[0093] The multimodal feature map is input into the defect discrimination module, which includes multiple convolutional layers for extracting high-level semantic features and spatial texture features; fully connected layers for vectorizing the convolutional output features to achieve global feature integration; and a classifier for outputting a defect probability map or defect category label.

[0094] The defect probability map is segmented by threshold, the segmented defect regions are post-processed, and the final defect detection results are output, including defect category, location and visual annotation map.

[0095] Furthermore, the defect probability map is thresholded and segmented, and the segmented defect regions are then post-processed, specifically as follows:

[0096] The defect probability value of each pixel in the output defect probability map is compared with the preset defect threshold.

[0097] If the defect probability value of a pixel is greater than or equal to the preset defect threshold, then the pixel is determined to belong to the defect area.

[0098] If the defect probability value of a pixel is less than the preset defect threshold, the pixel is determined to belong to a non-defect area.

[0099] Post-processing is performed on the segmented defective regions, including connected component analysis, size calculation, and type labeling;

[0100] The process involves several steps: connected component analysis determines the area, boundary, and center location of each defect region; size calculation records the length, width, or diameter of the defect; and type labeling associates the defect category with probability information to generate the final defect detection result.

[0101] Furthermore, the feature maps enhanced by channel attention and spatial attention are input into the contrastive learning module for attention weight optimization, and the optimized attention feature map is output, specifically:

[0102] Construct positive and negative sample pairs based on enhanced feature maps;

[0103] Positive sample pairs are selected by choosing corresponding features of the same defect region in different modalities or different time frames as positive sample pairs, while negative sample pairs are selected by choosing features of defective regions and non-defective regions, or regions of different defect types as negative sample pairs.

[0104] For each positive and negative sample pair, the InfoNCE loss is used to calculate the contrastive loss function. By calculating the cosine similarity between positive sample pairs, the distance between positive samples is reduced, and the contrastive loss value is output.

[0105] The input to the contrastive loss function is the vectorized representation of the feature blocks, which can be reduced in dimensionality through global average pooling or principal component analysis.

[0106] The contrastive loss value is backpropagated to the channel attention module and the spatial attention module, and the weights are updated by gradient descent. During the optimization process, the attention weights in defective regions are increased and the attention weights in non-defective regions are decreased to improve the robustness of the attention response to defects.

[0107] The output is an optimized attention feature map, providing a stable and reliable input for subsequent cross-modal feature fusion and defect detection.

[0108] It should be noted that by introducing channel attention and spatial attention, defect-sensitive channels and potential defect regions can be highlighted in the multimodal feature map, while suppressing redundant background information and irrelevant regions. This makes the network more sensitive to small or low-contrast defects on the surface of irregularly shaped furniture parts. Further optimization of the attention weights using contrastive learning allows positive and negative sample pairs to constrain the feature space, making the features of the same defect more similar across different modalities or viewpoints, while features of non-defect regions are distanced, thereby enhancing the discriminative power and robustness of the feature representation. The overall effect is that the generated multimodal feature map retains the complementary advantages of reflection and morphology information while highlighting and optimizing defect information, providing a more accurate, stable, and discriminative input for subsequent defect discrimination, thus improving detection accuracy and reliability.

[0109] Reference Figure 2 As shown in the schematic diagram, the furniture irregular shape detection system based on machine vision provided by the present invention includes an image processing module, an image decomposition and enhancement module, and a furniture irregular shape defect detection module, and the modules are connected to each other:

[0110] The image processing module acquires the original image data of irregularly shaped furniture parts and performs image registration. It then enhances the registered image data to generate multimodal image data. The original image data includes color images, polarized light images, and photometric three-dimensional morphology images.

[0111] The image decomposition and enhancement module is used to decompose and enhance multimodal image data to obtain the final reflection image and enhanced morphology image;

[0112] The furniture irregular part defect detection module is used to input the final reflection image and enhanced shape image into a defect detection network based on attention mechanism and contrast learning, perform cross-modal feature fusion, and output the defect detection results of furniture irregular parts.

[0113] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0114] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0115] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0116] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0117] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0118] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A machine vision-based method for detecting irregularly shaped furniture parts, characterized in that, include: The original image data of irregularly shaped furniture parts is collected and image registration is performed. The registered image data is then enhanced to generate multimodal image data. The original image data includes color images, polarized light images, and photometric stereoscopic morphology images. The multimodal image data includes color images, multispectral images, polarized light images, and photometric stereoscopic morphology images. The multimodal image data is decomposed and enhanced to obtain the final reflection image and enhanced topography image, specifically as follows: The color image is initially decomposed based on multi-scale wavelet transform and luminance-chrominance space transform to obtain low-frequency reflection component and high-frequency texture component. The low-frequency reflection component is corrected based on the polarized light image to obtain the polarization-corrected reflection component. The multispectral image is decomposed into diffuse reflection and specular reflection components, and the specular reflection intensity distribution of the corresponding band is obtained by solving the least squares method. The specular reflection intensity distribution of each band is compared at the pixel level, and it is determined whether the pixel contains specular reflection components. If specular reflection is included, the specular reflection intensity distribution of each band is fused to obtain the multispectral specular reflection component; A constrained optimization function is constructed based on the polarization-corrected reflection component and the multispectral specular reflection component, and iterative optimization is performed to obtain the optimized intrinsic reflection component. Based on the surface normal information of the photometric stereo topography image, the high-frequency texture component is enhanced to obtain the enhanced texture component. The intrinsic reflection component and the enhanced texture component are fused to generate the final reflection image and enhanced topography image. The final reflection image and enhanced topography image are input into a defect detection network based on attention mechanism and contrastive learning to perform cross-modal feature fusion and output the defect detection results of irregular furniture parts.

2. The method for detecting irregularly shaped furniture parts based on machine vision according to claim 1, characterized in that, The process of collecting original image data of irregularly shaped furniture components and performing image registration specifically involves: The original image data is downsampled layer by layer to construct an image sequence from low resolution to high resolution; At each resolution, the image sequence is divided into several local blocks, and key feature points are extracted; Descriptors are generated for key feature points, and feature matching is performed between corresponding local blocks. By combining bidirectional matching verification and distance ratio threshold to remove mismatched points, feature point pairs with local matching are obtained. Based on feature point pairs, a preliminary affine transformation matrix is ​​obtained within each local block through a random sampling consensus algorithm, and then integrated to obtain the initial global affine matrix. With the goal of maximizing the mutual information between the reference image and the affine transformed image, the initial global affine matrix is ​​iteratively updated, and the globally optimized global affine matrix is ​​output. Geometric transformations are performed on the polarized light image and the photometric stereo image based on the global affine matrix, and then aligned with the color image in spatial coordinates to obtain the registered image data.

3. The method for detecting irregularly shaped furniture parts based on machine vision according to claim 2, characterized in that, The step of performing feature enhancement on the registered image data to generate multimodal image data specifically involves: The registered image data is decomposed, and the image resolution and image noise level are obtained; Based on the image resolution and image noise level, the spatial standard deviation and intensity standard deviation of the bilateral filter are obtained; Based on the spatial standard deviation and intensity standard deviation, bilateral filtering is performed on each channel to obtain a smooth image; High-frequency features are extracted from the smoothed image using a high-pass filtering method, and high-pass enhanced image data is output. Qualcomm enhanced image data is fused with the original image data to generate multimodal image data.

4. The method for detecting irregularly shaped furniture parts based on machine vision according to claim 1, characterized in that, The preliminary decomposition of the color image based on multi-scale wavelet transform and luminance-chrominance space transform yields low-frequency reflection components and high-frequency texture components, specifically: Acquire a color image and decompose it into luminance and chrominance components based on the luminance-chrominance color space; Multi-scale wavelet decomposition is performed on the brightness component, which is decomposed into low-frequency sub-bands and high-frequency sub-bands in multiple directions. Multi-scale wavelet decomposition is performed on the chromaticity components to extract low-frequency and high-frequency components; Based on the low-frequency subband and low-frequency components, a preliminary reflection component is constructed; Based on high-frequency subbands and high-frequency components, preliminary texture components are constructed; The initial reflection component and the initial texture component are combined to obtain the low-frequency reflection component and the high-frequency texture component of the color image.

5. The method for detecting irregularly shaped furniture parts based on machine vision according to claim 1, characterized in that, The correction of low-frequency reflection components based on polarized light images is specifically as follows: Acquire polarized light image sequences and pair them with low-frequency reflection components; For each pixel in the polarized light image sequence, the light intensity value at different polarization angles is obtained to form a light intensity variation curve; Based on Malus's law, the light intensity is decomposed into diffuse reflection and specular reflection components, and the specular reflection intensity and diffuse reflection intensity of each pixel are solved to generate a specular reflection estimation map. Based on the specular reflection estimation map, determine the specular reflection area and the diffuse reflection area; Obtain the linear attenuation coefficient of pixels identified as specular reflection areas; For each pixel, intensity suppression is performed using a linear attenuation method, and the polarization-corrected reflection component is output.

6. The method for detecting irregularly shaped furniture parts based on machine vision according to claim 1, characterized in that, The final reflection image and enhanced topography image are input into a defect detection network based on attention mechanism and contrastive learning to perform cross-modal feature fusion and output the defect detection results for irregularly shaped furniture parts, specifically as follows: The final reflection image and enhanced morphology image are input into the defect detection network for feature extraction to obtain the reflection feature map and morphology feature map; A channel attention mechanism is introduced for each frame of the reflection feature map and the shape feature map to generate channel weights; The channel weights are combined with the feature maps channel by channel to obtain the weighted channel feature maps; On the channel-weighted feature map, a spatial attention mechanism is introduced to generate spatial attention weights, which are then combined with the channel-weighted feature map to obtain a spatially weighted enhanced feature map. The feature maps enhanced by channel attention and spatial attention are input into the contrastive learning module to optimize the attention weights, and the optimized attention feature maps are output. The optimized attention feature maps are aligned in the spatial dimension and fused by channel concatenation to obtain a multimodal feature map; Input the multimodal feature map into the defect discrimination module and output a defect probability map; The defect probability map is segmented by threshold, the segmented defect regions are post-processed, and the final defect detection result is output. The defect detection result includes defect category, location and visual annotation map.

7. The method for detecting irregularly shaped furniture parts based on machine vision according to claim 6, characterized in that, The threshold segmentation of the defect probability map and the subsequent post-processing of the segmented defect regions are specifically as follows: The defect probability value of each pixel is compared with a preset defect threshold. If the defect probability value of a pixel is greater than or equal to the preset defect threshold, then the pixel is determined to belong to the defect area. If the defect probability value of a pixel is less than the preset defect threshold, the pixel is determined to belong to a non-defect area. The segmented defect areas are post-processed to generate the final defect detection results.

8. The method for detecting irregularly shaped furniture parts based on machine vision according to claim 6, characterized in that, The step of inputting the feature map enhanced by channel attention and spatial attention into the contrastive learning module for attention weight optimization and outputting the optimized attention feature map is as follows: Construct positive and negative sample pairs based on enhanced feature maps; For each pair of positive and negative samples, calculate the contrastive loss function and output the contrastive loss value; The contrast loss value is backpropagated to the channel attention module and the spatial attention module, and the weights are updated by gradient descent to output the optimized attention feature map.

9. A system for detecting irregularly shaped furniture parts using a machine vision-based method as described in any one of claims 1-8, characterized in that, include: The image processing module acquires the original image data of irregularly shaped furniture parts and performs image registration. It then enhances the registered image data to generate multimodal image data. The original image data includes color images, polarized light images, and photometric three-dimensional morphology images. The image decomposition and enhancement module is used to decompose and enhance multimodal image data to obtain the final reflection image and enhanced morphology image; The furniture irregular part defect detection module is used to input the final reflection image and enhanced shape image into a defect detection network based on attention mechanism and contrast learning, perform cross-modal feature fusion, and output the defect detection results of furniture irregular parts.

Images