Method for quality detection of pu artificial leather production line based on multi-modal image recognition

Abstract

The present application relates to the technical field of defect detection, in particular to a PU artificial leather production line quality detection method based on multi-modal image recognition, comprising the following steps: acquiring multi-modal images of the detection area and fusing the response, calibrating the micro-mark blocks and drawing the connection atlas, identifying the aggregation distribution and extracting the edge configuration features, and generating the image quality detection discrimination result. In the present application, by fusing the multi-modal response information of visible light images, infrared thermal images and laser reflection images, the combination distribution relationship is identified and the layer mapping is established, which can realize the accurate positioning of uneven illumination and thermal area edges, identify the directional features of texture changes and calibrate the potential defect blocks, has stronger adaptability and recognition accuracy in the face of various textures and complex working conditions, is more flexible and robust, and significantly improves the integrity, accuracy of image detection and the intuitiveness of result expression.

Description

Technical Field

[0001] This invention relates to the field of defect detection technology, and in particular to a quality inspection method for PU artificial leather production lines based on multimodal image recognition. Background Technology

[0002] The field of defect detection technology involves using image processing, machine learning, and sensor fusion to acquire and analyze images of the surface or internal structure of products during industrial production. This identifies structural anomalies such as cracks, impurities, holes, and breaks that do not meet quality standards. This technology primarily includes the configuration and deployment of image acquisition equipment, the design of multi-source image fusion algorithms, the establishment of image feature extraction methods, the construction of defect pattern recognition models, and the integration and application of online automatic detection systems. With the increasing demands for product quality control in the manufacturing industry, defect detection technology has been widely applied in textiles, metallurgy, electronics, chemicals, automobiles, and other fields, becoming an important support for production automation and intelligence. In particular, the quality inspection method for PU artificial leather production lines involves acquiring images of the artificial leather surface using a single-modal image acquisition device, such as a visible light camera. Then, it employs texture feature analysis methods based on gray-level co-occurrence matrices or geometric feature extraction methods based on edge detection operators for defect localization and classification. These methods rely on fixed angles and uniform light sources, and often use support vector machines or decision tree models to discriminate extracted features, making them unsuitable for the inspection needs of artificial leather products with different texture types under complex lighting and variable working conditions. To overcome the above limitations, a quality inspection method for PU artificial leather production lines based on multimodal image recognition is proposed. This method integrates multimodal image information, including visible light images, infrared thermal images, and laser contour images, and performs joint analysis using a feature-level fusion strategy. It also utilizes convolutional neural networks to achieve end-to-end processing for feature extraction and defect classification.

[0003] Existing technologies use a single visible light image acquisition method to detect defects on the surface of PU artificial leather. This relies on a fixed angle and uniform light source conditions. In scenarios with uneven light distribution or complex texture types, feature extraction is easily interfered with, leading to a decrease in detection accuracy. When using support vector machines or decision tree models, the adaptability of feature dimensions is insufficient, resulting in a sluggish response to blurred boundaries or minor defects. It is difficult to identify hidden edge defects and texture change trends. When there are multiple sources of disturbance or differences in configuration density on the surface of artificial leather products, traditional models cannot reasonably distinguish defect aggregation trends, affecting the accuracy of overall quality assessment and the controllability of risk determination. Summary of the Invention

[0004] To address the shortcomings of existing technologies, such as the susceptibility of feature extraction to interference in scenarios with uneven lighting distribution or complex textures, leading to decreased detection accuracy, and the insufficient adaptability of feature dimensions when using support vector machines or decision tree models, resulting in sluggish responses to blurred boundaries or minor defects, and difficulty in identifying hidden edge defects and texture change trends, this invention provides a quality inspection method for PU artificial leather production lines based on multimodal image recognition. The technical solution is as follows:

[0005] On the one hand, a quality inspection method for PU artificial leather production lines based on multimodal image recognition is provided, the method including:

[0006] S1: Acquire the first frame image of the inspection area of ​​the PU artificial leather production line, and sequentially acquire the edge image under oblique visible light illumination, the temperature domain image in infrared thermal image, and the return image formed by laser reflection. Label and compare the three types of image response phenomena, identify the combination distribution relationship, and generate a multimodal response fusion layer.

[0007] S2: Based on the multimodal response fusion layer, the visible light illumination, infrared thermal intensity and laser emission interval status are judged to generate a detection image response adaptation set;

[0008] S3: Call the detection image response adaptation set, delineate the uneven illumination area and the edge zone of heat distribution as potential micro-trace blocks, identify the directional changes in texture arrangement density, and generate an indentation positioning annotation map;

[0009] S4: Call the indentation positioning annotation map, draw the connection relationship between the centers of the defect area, mark the area blocks with the clustering trend, classify and judge the outline and dense performance characteristics of the cluster blocks, perform layer color covering according to the clustering level, and generate the configuration clustering distribution recognition result.

[0010] As a further embodiment of the present invention, the multimodal response fusion layer includes a texture response structure layer, a thermal edge layer, and a laser reflection response layer; the detection image response adaptation set includes an illumination adaptation parameter set, a thermal adaptation parameter set, and a laser rhythm calibration set; the indentation positioning annotation map includes a micro-trace distribution map, a texture directionality change map, and a grayscale transition feature map; and the configuration clustering distribution recognition result includes a defect connectivity map, a clustering level layering map, and a contour density classification map.

[0011] As a further aspect of the present invention, the steps for obtaining the multimodal response fusion layer are specifically as follows:

[0012] S101: Acquire the first frame image of the inspection area of ​​the PU artificial leather production line, and sequentially collect the gray scale change of the image edge area under oblique visible light illumination, the temperature response state of the position in the infrared thermal image, and the reflectance brightness of the area in the laser reflection image. Based on the coordinate consistency of the pixel position in the image, the sampling data of the three types of images are spatially corresponding processed to generate a position-corresponding image dataset.

[0013] S102: Based on the image dataset corresponding to the location, determine whether there is a continuous boundary in the grayscale change state, whether the temperature response state exhibits heterogeneous changes in a local range, and whether the reflectance brightness is lower than the set recognition benchmark. Mark the location of the multi-type response regions according to the judgment results and generate a composite response intersection location set.

[0014] S103: Call the composite response cross-location set to obtain the grayscale change amplitude, temperature change trend and reflection brightness difference of the location in the three types of images. Based on the superposition distribution relationship between the three types of change values, identify the degree of response performance of the location in the image and generate a multimodal response fusion layer.

[0015] As a further aspect of the present invention, the step of obtaining the detection image response adaptation set specifically comprises:

[0016] S201: By analyzing the spot edge transition regions in the multimodal response fusion layer, the number distribution of spot edge transition positions within a unit area is statistically analyzed. Combining the spatial arrangement characteristics of the thermal zone inflection points in the corresponding area and the brightness change rhythm in the laser return image, the changing trends of the visible light illumination state, infrared thermal intensity expression form, and laser emission interval within the area are determined respectively. The three types of states are marked at the corresponding layer positions to generate a layer state distribution label set.

[0017] S202: Call the layer state distribution annotation set, and for each type of state annotation position, sequentially obtain the grayscale, temperature and brightness jump direction of the adjacent areas in the multimodal response fusion layer, and compare the three types of directional change features at the same position to determine whether there is a response direction consistency deviation. If there is, adjust the coordinates of the spatial projection point at the corresponding position in the layer to generate a detection image response adaptation set.

[0018] As a further aspect of the present invention, the step of obtaining the indentation positioning annotation map specifically includes:

[0019] S301: Call the detection image response adaptation set, extract the pixel light intensity value sequence and the heat pixel continuous value sequence in the layer, filter the overlapping blocks of the light intensity average area and the heat gradient boundary area, calculate the light and heat joint dispersion index, and obtain the coordinate set of micro-trace potential blocks.

[0020] S302: Based on the coordinate set of the micro-trace potential block, the image segment region is located, and the texture arrangement direction angle value, texture spacing density value and continuous gray level difference value are extracted within the region. The texture spacing density value in the texture arrangement direction angle value change segment is determined to be a decreasing trend segment. The position segment of continuous gray level difference is recorded, and the coordinate group of texture segment with consistent decreasing direction is obtained.

[0021] S303: Based on the position index of the coordinate group of the texture segment with consistent decreasing direction in the image, call the original image layer and locate the corresponding image pixel area. After proportionally expanding the boundary of the coordinate group, mark it as the image target annotation layer. Based on the pixel coverage range of the annotation layer, perform block coverage processing on the original image to obtain the indentation positioning annotation map.

[0022] As a further aspect of the present invention, the photothermal joint dispersion index is expressed by the following formula:

[0023] ;

[0024] in, M represents the photothermal joint dispersion index in the i-th layer block. i I represents the total number of pixels in the i-th layer block. ij This represents the light intensity value of the j-th pixel in the i-th layer block. T represents the average light intensity value of the pixels in the i-th layer block. ij This represents the heat value of the j-th pixel in the i-th layer block. W represents the average heat value of pixels in the i-th layer block. ij This represents the weight factor corresponding to the j-th pixel in the i-th layer block.

[0025] As a further aspect of the present invention, the steps for obtaining the configuration clustering distribution identification results are specifically as follows:

[0026] S401: Call the set of marked coordinates of the defect area in the indentation positioning annotation map, extract the coordinate values ​​of the center of the area, the length of the line connecting the centers and the angle of the staggered distribution, calculate the feature distribution difference value, filter the coordinate points of multiple intersecting nodes, mark the boundaries of the area block, and obtain the coordinate map of the defect cluster area block;

[0027] S402: Based on the boundary of the region block defined by the defect clustering region block coordinate map, extract the area value enclosed by the clustering block boundary, the internal defect distribution density value and the cross-coverage ratio of the adjacent area, classify the clustering level according to the combined difference of the boundary enclosed area value and the internal defect distribution density value, and fill the layer with color according to the color mark corresponding to the clustering level to obtain the configuration clustering distribution recognition result.

[0028] As a further aspect of the present invention, the feature distribution difference value is expressed by the following formula:

[0029] ;

[0030] in, x represents the difference in characteristic distribution between the a-th and b-th center coordinate points. a x b y represents the horizontal coordinates of the center points of the a-th and b-th defect regions, respectively. a y b θ represents the vertical coordinates of the center points of the a-th and b-th defect regions, respectively. a θ b L represents the angle values ​​of the staggered distribution of the a-th and b-th defect regions, respectively. a L b These represent the lengths of the lines connecting the a-th and b-th center points, respectively. This represents the sum of the lengths of the lines connecting the centers in the sample, totaling M lines.

[0031] As a further aspect of the present invention, the method further includes step S5:

[0032] S5: Call the defect cluster areas that have been classified in the configuration cluster distribution recognition results, extract the cluster blocks and center offset distribution relationship patterns in the edge area of ​​PU artificial leather, and mark the risk range, structural offset trend and boundary morphology characteristics according to the position distribution, configuration density and boundary overlap, and generate PU artificial leather image quality detection and discrimination results.

[0033] The image quality detection and discrimination results of the PU artificial leather include a risk area distribution map, a structural offset trend map, and a boundary morphology feature map.

[0034] As a further aspect of the present invention, the steps for obtaining the PU artificial leather image quality detection and discrimination result are specifically as follows:

[0035] S501: Call the defect clustering areas marked with the grade in the configuration clustering distribution recognition results, filter the clustering blocks located at the edge of PU artificial leather, record the coordinate contour information and center position point, extract the geometric offset direction and offset distance from the clustering block to the center position of the image, and generate an edge clustering offset relationship pattern.

[0036] S502: Based on the edge cluster offset relationship pattern, obtain the spatial arrangement range of cluster blocks block by block, calculate the number of clusters and the proportion of the area in the local area, and combine and judge the number of clusters, distribution range and interval performance to generate a densely distributed area map.

[0037] S503: Call the dense distribution area map of the configuration, detect the degree of overlap between the boundary point of the area and the original edge line of the PU artificial leather, and combine the trend of boundary extension direction change and boundary continuous morphological characteristics to determine whether the overlapping range presents a stretched or broken shape, and generate a boundary morphological characteristic mark map.

[0038] S504: Call the boundary morphology feature marker map to jointly identify the locations of areas with three types of phenomena: clustering offset, dense configuration, and boundary change. Sequentially mark the risk distribution range, the trend of internal structure offset direction change, and edge morphology features of the corresponding locations to generate PU artificial leather image quality detection and discrimination results.

[0039] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0040] By fusing multimodal response information from visible light images, infrared thermal images, and laser reflection images, and identifying their combined distribution relationships and establishing layer mapping, it can accurately locate uneven illumination and hot zone edges. At the same time, it can identify the directional features of texture changes and mark potential defect blocks. Based on the recognition of aggregation trends, it further extracts the edge distribution and center offset patterns of aggregation areas, and combines dense representation and boundary morphology to complete the annotation of structural offset trends and risk ranges. It has stronger adaptability and recognition accuracy in the face of diverse textures and complex working conditions, is more flexible and robust, and significantly improves the integrity, accuracy and intuitiveness of image detection results. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the workflow of the present invention; Detailed Implementation

[0042] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0043] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0044] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0045] Please see Figure 1This invention provides a quality inspection method for PU artificial leather production lines based on multimodal image recognition. The processing flow of this method may include the following steps:

[0046] S1: Acquire the first frame image of the inspection area of ​​the PU artificial leather production line, and sequentially acquire the edge image under oblique visible light illumination, the temperature domain image in infrared thermal image, and the return image formed by laser reflection. Label and compare the three types of image response phenomena, identify the combination distribution relationship, and generate a multimodal response fusion layer.

[0047] S2: Based on the multimodal response fusion layer, the visible light illumination, infrared thermal intensity and laser emission interval status are judged, spatial correspondence calibration is performed, and a detection image response adaptation set is generated;

[0048] S3: Call the detection image response adaptation set, delineate the uneven lighting area and the edge zone of heat distribution as potential micro-trace blocks, identify the directional changes in texture density, if the texture decreases in the same direction and the gray level becomes gradual, mark the block on the original image and generate an indentation positioning annotation map.

[0049] S4: Call the indentation positioning annotation map, draw the connection relationship between the centers of the defect area, mark the area blocks with the clustering trend, classify and judge the outline and dense expression characteristics of the cluster blocks, perform layer color covering according to the clustering level, and generate the configuration clustering distribution recognition result.

[0050] S5: Call the defect cluster areas that have been classified in the configuration cluster distribution recognition results, extract the cluster blocks and center offset distribution relationship patterns in the edge area of ​​PU artificial leather, and mark the risk range, structural offset trend and boundary morphology characteristics according to the location distribution, configuration density and boundary overlap, and generate PU artificial leather image quality detection and discrimination results.

[0051] The multimodal response fusion layer includes a texture response structure layer, a thermal edge layer, and a laser reflection response layer. The detection image response adaptation set includes an illumination adaptation parameter set, a thermal adaptation parameter set, and a laser rhythm calibration set. The indentation positioning annotation map includes a micro-trace distribution map, a texture directionality change map, and a grayscale transition feature map. The configuration clustering distribution recognition results include a defect connectivity map, a clustering level layering map, and a contour density classification map. The PU artificial leather image quality detection and discrimination results include a risk area distribution map, a structural offset trend map, and a boundary morphology feature map.

[0052] The specific steps for obtaining the multimodal response fusion layer are as follows:

[0053] S101: Acquire the first frame image of the inspection area of ​​the PU artificial leather production line, and sequentially collect the gray scale change of the image edge area under oblique visible light illumination, the temperature response state of the position in the infrared thermal image, and the reflectance brightness of the area in the laser reflection image. Based on the coordinate consistency of the pixel position in the image, the sampling data of the three types of images are spatially corresponding processed to generate a position-corresponding image dataset.

[0054] An industrial camera captures the entire inspection area from a set shooting position. Ambient light intensity calibration is performed before the image sensor initiates exposure. An oblique visible light illumination method is used, utilizing the oblique angle to control the overlap of interference between the illumination path and the reflection path, resulting in clearer edge structures in the inspection area. The image acquisition device captures image frames sequentially over time and performs frame selection in the image buffer, choosing the first frame as the baseline image for subsequent analysis. Edge regions in the first frame are obtained through image cropping, and the grayscale values ​​of each pixel in that region are read in a row-column traversal manner. The trend of grayscale value changes in each row or column is recorded as a sequence, and the grayscale values ​​are simultaneously labeled. The pixel coordinates of the mutation point are then used to synchronously acquire infrared images from the same angle using a thermal imager in the image acquisition device. The infrared thermal imager continuously samples within the detection area, timestamps the thermal imaging data with the first frame image, and locates the thermal response state of each pixel. Simultaneously, the laser reflection image acquisition device scans the detection surface, measures the reflectance brightness of each point in the scanning path, and forms a two-dimensional brightness matrix. After the three types of images record data for the same detection area, the three types of images are spatially matched based on pixel coordinates through image registration, so that each location contains three data values: grayscale, temperature, and brightness, generating a location-corresponding image dataset.

[0055] S102: Based on the location-corresponding image dataset, determine whether there are continuous boundaries in the grayscale change state, whether the temperature response state exhibits heterogeneous changes in a local range, and whether the reflectance brightness is lower than the set recognition benchmark. Mark the positions of multiple response regions according to the judgment results and generate a composite response cross-position set.

[0056] In the image processing module, the channel parsing order of three types of layers—grayscale, thermal imaging, and reflection—is set. Edge regions with continuous grayscale differences are filtered from the grayscale layer. The presence of boundary trends in the image is determined by calculating the variation in grayscale values ​​between neighboring pixels. A search window is then set to perform sliding detection in the image. If most pixels in the window show significant grayscale differences, the current area is considered a grayscale boundary region. Simultaneously, based on the temperature changes at corresponding points in the temperature layer, the temperature values ​​of multiple surrounding pixels are extracted to form a temperature distribution vector. If significant differences in temperature values ​​occur within this region, it is identified as a response region with localized heterogeneous changes. The brightness information of this region in the laser image is checked. If the brightness is significantly lower than the background area, it indicates weak laser energy reflection at that point. Multiple response signals are generated in the three layers. By recording the spatial overlap positions of the response signals, position marking is performed in the image labeling channel, generating a composite response intersection set.

[0057] S103: Call the composite response cross-location set to obtain the grayscale change amplitude, temperature change trend and reflectance difference of the location in the three types of images. Based on the superposition distribution relationship between the three types of change values, identify the degree of response performance of the location in the image and generate a multimodal response fusion layer.

[0058] The algorithm reads the coordinates of each pixel in the composite response intersection set, extracts the corresponding data of that point in the three layers one by one, compares the grayscale channel changes to obtain the grayscale change amplitude, compares the temperature values ​​of the same location at different time points in the thermal imaging channel, extracts the numerical difference of temperature rise or fall, extracts the brightness value of the corresponding location from the laser reflection layer and calculates the difference of the brightness value relative to the brightness benchmark in the overall image, forms a fusion feature vector by channel order for the three types of change data and processes it with a unified numerical standard, determines the positional relationship of each feature value in the overall distribution in the feature set formed by multi-point data through data analysis, classifies the response degree of the detection point into levels, and draws the fusion layer image layer by layer for the pixels of the response level. Each level is represented by a different pixel value, which can intuitively reflect the trend of the comprehensive response strength of different detection areas in the three dimensions of grayscale, thermal imaging and reflection, and generate a multimodal response fusion layer.

[0059] The specific steps for obtaining the image response adaptation set are as follows:

[0060] S201: By analyzing the edge transition regions of speckles in the multimodal response fusion layer, the number distribution of speckle edge transition positions within a unit area is statistically analyzed. Combining the spatial arrangement characteristics of thermal inflection points in the corresponding area and the brightness change rhythm in the laser return image, the changing trends of visible light illumination state, infrared thermal intensity expression form and laser emission interval within the area are determined respectively. The three states are marked at the corresponding layer positions to generate a layer state distribution annotation set.

[0061] The outline of spots formed by gray-level abrupt changes in grayscale images is extracted. A fixed-window sliding method is used to traverse and detect the entire image, statistically analyzing the distribution of abrupt edge counts within a unit area, i.e., the cumulative edge change count in each scanning window. The entire image is divided into equal-area cells by setting an area resolution standard, and the edge abrupt count within each cell is recorded as the basis for spatial density distribution. Simultaneously, thermal inflection points or extreme points are extracted from each region in the thermal image layer, and the spatial arrangement characteristics of these inflection points are calculated, including the distance between them, their arrangement angle, and the direction of change, and these are marked accordingly. The distribution pattern within the cell is then analyzed, and the sequence of brightness values ​​changing with spatial coordinates for each region is extracted from the laser reflection map. By analyzing the periodicity and fluctuation frequency of this sequence, the rhythmic changes in intensity during laser emission and return are inferred. After feature extraction is completed for the three types of layer data, the manifestation of each state is determined. The density of grayscale edge jumps is used to determine the visible light illumination state, the regularity and intensity of thermal inflection points are used to determine the infrared thermal manifestation, and the rhythmic changes in laser brightness reflect the emission interval trend. The above three states are labeled at the corresponding positions in the original layer according to pixel coordinates, generating a layer state distribution label set.

[0062] S202: Call the layer state distribution annotation set, for each type of state annotation position, sequentially obtain the grayscale, temperature and brightness jump direction of the adjacent areas in the multimodal response fusion layer, and compare the three types of directional change features at the same position to determine whether there is a response direction consistency deviation. If there is, adjust the coordinates of the spatial projection point at the corresponding position in the layer to generate the detection image response adaptation set.

[0063] The system sequentially reads the position of each labeled layer and extracts the response data of its surrounding area from the multimodal response fusion layer. A fixed neighborhood width is set within the extraction range. The maximum change direction of the neighborhood grayscale value is calculated in the grayscale channel. The angle of the jump direction is determined based on the pixel grayscale difference gradient. Then, the dominant direction of temperature increase or decrease is identified in the thermal imaging channel within the same neighborhood. The direction of heat flow trend is determined by comparing the difference between the temperature at the center point and the edge points of the neighborhood. Spatial positional relationship analysis is performed on the brightness values ​​in the laser reflection layer to obtain the direction of brightness change from strong to weak or from weak to strong, forming direction vectors for the jump directions in the three types of layers. After direction extraction, the three types of direction vectors are compared at each labeled position. If a significant directional deviation is found between the three, i.e., the angle between any two directions is greater than a certain set standard angle, it is determined to be a response direction consistency deviation area. Then, based on the magnitude of the vector angle between the deviation directions and the priority of the dominant direction, the spatial projection coordinates of this position in the fusion layer are readjusted. The projection point is then slightly corrected along the dominant direction according to the set rules to generate a detection image response adaptation set.

[0064] The specific steps for obtaining the indentation positioning annotation map are as follows:

[0065] S301: Call the detection image response adaptation set, extract the pixel light intensity value sequence and thermal pixel continuous value sequence in the layer, filter the overlapping blocks of the light intensity average area and the thermal gradient boundary area, calculate the light and heat joint dispersion index, and obtain the coordinate set of micro-trace potential blocks.

[0066] The dispersion index for combined photothermal and optical processes is calculated using the following formula:

[0067] ;

[0068] in, M represents the photothermal joint dispersion index in the i-th layer block. i I represents the total number of pixels in the i-th layer block. ij This represents the light intensity value of the j-th pixel in the i-th layer block. T represents the average light intensity value of the pixels in the i-th layer block. ij This represents the heat value of the j-th pixel in the i-th layer block. W represents the average heat value of pixels in the i-th layer block. ij This represents the weight factor corresponding to the j-th pixel in the i-th layer block;

[0069] Formula calculation logic: The Euclidean spatial scale of joint dispersion is achieved by subtracting two terms, squaring them, adding them, and taking the square root. Modulation is performed by multiplying by local weights. The dimensionality is ensured by averaging the total number of pixels and taking the absolute value. The overall logic measures the dispersion of regional photothermal characteristics by weighted averaging of the covariance of grayscale and heat.

[0070] The photothermal joint dispersion index represents the degree of co-variation between the grayscale light intensity value and the heat value of pixels in a layer block. It is used to measure the overall fluctuation of light and heat signals in the region. The larger the index value, the more severe the deviation between light intensity and heat in the region, indicating the presence of structural anomalies or local heat source characteristics.

[0071] Parameter meaning and calculation logic:

[0072] M i This represents the number of pixels contained in the i-th layer block;

[0073] I ij It is the grayscale intensity value of the j-th pixel in the i-th layer block, which is acquired by an infrared image sensor and is measured in GrayLevel.

[0074] It is the average pixel light intensity value of the layer block, according to Calculated;

[0075] T ij This is the heat value of the corresponding pixel, in °C, collected by a thermal imager;

[0076] It is the average value of the heat value within the block, calculated as follows: ;

[0077] W ij Let be the weight value of the j-th pixel. The weight value is calculated from the absolute value of the ratio of the gray-level change rate to the thermal gradient change rate among the pixels above, below, left, and right of this pixel.

[0078] ;

[0079] Where N(j) represents the set of neighboring pixels of the j-th pixel, and ϵ is a minimal constant term to avoid the denominator being zero;

[0080] To verify the actual calculation process used in the formula, the following example data is constructed;

[0081] Table 1: Sampling data of pixel light intensity and heat within layer blocks:

[0082]

[0083] As shown in Table 1, the light intensity and heat values ​​of 5 pixels in the layer block were collected by an image device. The weight value was calculated based on the ratio of grayscale and heat difference between each pixel and its neighbors. For example, the total grayscale neighborhood difference of pixel 1 was 20, and the total heat neighborhood difference was 17.4. The weight value was calculated using the formula: W i1 =|20 / 17.4|≈1.15;

[0084] Calculate the mean light intensity and mean heat:

[0085] ;

[0086] ;

[0087] Next, calculate the joint deviation:

[0088] ;

[0089] ;

[0090] ;

[0091] ;

[0092] ;

[0093] Substitute into the formula to calculate:

[0094] ;

[0095] The results show that the photothermal joint dispersion index is 4.079, indicating that the light intensity and heat intensity in the current layer block have a moderate degree of coupling dispersion, which suggests that there is a structural thermal anomaly in this area, making it suitable as a basis parameter for subsequent screening of overlapping blocks.

[0096] The advantage of the formula lies in introducing a weight W, which represents the ratio of local grayscale to heat of a pixel. ij By combining the weighted summation of joint biases in Euclidean space, the sensitivity and block differentiation of photothermal coupling anomaly identification are improved; and the ability to focus on weak thermal trace areas and extract potential anomalies is achieved in the whole.

[0097] S302: Based on the coordinate set of the micro-trace potential block, the image segment region is located, and the texture arrangement direction angle value, texture spacing density value and continuous gray level difference value are extracted within the region. The texture spacing density value is determined to be decreasing in the segment where the texture arrangement direction angle value changes. The position segment of continuous gray level difference is recorded, and the coordinate group of texture segment with consistent decreasing direction is obtained.

[0098] In the fused image, corresponding image segments are extracted, and the texture structure within each segment is parameterized. A gradient direction analysis algorithm is used to calculate the angle of the texture arrangement direction, obtaining the directional angle values ​​of the main texture directions within the region. Then, the spacing density value of the texture arrangement is calculated by combining this with the periodic structure of gray-level changes between pixels. This value reflects the spatial distribution density of adjacent texture peaks or valleys. A continuous gray-level difference sequence formed by pixel gray-level values ​​in this region is extracted. Through continuous change trend analysis, characteristic segments with unidirectional increasing or decreasing gray-level values ​​are identified. The previously extracted directional angle values ​​are then analyzed. The segment is scanned, and in the interval where the direction and angle change trend exists, the corresponding texture spacing density value sequence is extracted. It is then determined whether the spacing density forms a decreasing trend between adjacent sampling points. If the density value continues to decrease within a certain sampling range, the segment is considered to show a texture structure aggregation trend. If the continuous gray level difference value within the trend segment shows a significant shift or a stable decrease, the position is recorded as the target segment with consistent texture direction and decreasing density. The coordinate boundary information of the image segment that meets this combination condition is organized into a coordinate group of texture segments with consistent decreasing direction, and the coordinate group of texture segments with consistent decreasing direction is obtained.

[0099] S303: Based on the position index of the texture segment coordinate group with consistent decreasing direction in the image, call the original image layer and locate the corresponding image pixel area. After proportionally expanding the boundary of the coordinate group, mark it as the image target annotation layer. Based on the pixel coverage range of the annotation layer, perform block coverage processing on the original image to obtain the indentation positioning annotation map.

[0100] Based on the position index information of the coordinate group in the fused image, the corresponding image pixel area is located back in the original image layer. The coordinate group of the fused layer is mapped to the original image space coordinates using an index table or pixel mapping relationship. Then, each coordinate boundary is expanded outward by a scaling factor. A fixed number of pixels or a set percentage is selected to expand the original area range, covering the potential extension area outside the micro-trace boundary. An image annotation layer is set within the expanded boundary range, and the pixels in this area are marked. This image target annotation layer exists as a separate layer and retains the integrity of the original image. In the image processing operation, the annotation layer is used as an image overlay template to perform block overlay operation on the corresponding area in the original image, and the overlay area is given a uniform layer identifier code or color fill to obtain the indentation positioning annotation map.

[0101] The specific steps for obtaining the configuration clustering distribution identification results are as follows:

[0102] S401: Call the set of marked coordinates of the defect area in the indentation positioning annotation map, extract the coordinate values ​​of the center of the area, the length of the line connecting the centers and the angle of the staggered distribution, calculate the feature distribution difference value, filter the coordinate points of multiple intersecting nodes, mark the boundaries of the area block, and obtain the coordinate map of the defect cluster area block;

[0103] The characteristic distribution difference value is calculated using the following formula:

[0104] ;

[0105] in, x represents the difference in characteristic distribution between the a-th and b-th center coordinate points. a x b y represents the horizontal coordinates of the center points of the a-th and b-th defect regions, respectively. a y b θ represents the vertical coordinates of the center points of the a-th and b-th defect regions, respectively. a θ b L represents the angle values ​​of the staggered distribution of the a-th and b-th defect regions, respectively. a L b These represent the lengths of the lines connecting the a-th and b-th center points, respectively. The sum of the lengths of the lines connecting the centers in the sample represents the total number of M lines.

[0106] The calculation logic of the formula: The first part is the Euclidean distance calculation term. The first part is used to obtain the spatial distance between center points a and b in a two-dimensional plane (xy) coordinate system; the second part is the normalized weight term of the staggered distribution angle difference. The first part linearly maps the absolute angle difference and then superimposes it onto the spatial distance to enhance the ability to describe the interference of angles on spatial relationships; the second part normalizes the length difference. This ratio is used to characterize the relative differences of the lines connecting each point in the overall length distribution. The larger the ratio, the higher the volatility of the lines connecting individual nodes and the more unstable the regional distribution. Through the organic combination of these three types of participants, multi-dimensional feature fusion is achieved while ensuring geometric rationality.

[0107] Parameter description and acquisition process:

[0108] x a =124.7mm, x b =129.5mm: The horizontal coordinates of the center points of regions a and b are obtained by the coordinate positioning algorithm in the visual image recognition device. The positioning basis is the centroid position of the center point of the indentation area contour.

[0109] y a =208.2mm, y b =204.8mm: This is the vertical coordinate value extracted from the above coordinates, and the center of gravity is also located by an image recognition algorithm;

[0110] θ a =37.2°, θ b=52.6°: This is the angle between the direction of the indentation stripes within the region and the reference reference direction (usually the horizontal axis). It is extracted by image grayscale gradient quantization, and its quantization standard is the angle value of the direction with the largest grayscale change along the main texture direction.

[0111] L a =6.3mm, L b =7.1mm: This is the actual length of the line connecting the centers of the regions, calculated using Euclidean distance;

[0112] M=12: represents the total length of the center line connecting the detection regions, which is collected from 12 pairs of region pairs in the entire defect clustering candidate region (extracted from the image dataset one pair at a time).

[0113] Substituting the above values ​​into the formula, the calculation is as follows:

[0114] ;

[0115] The results show that the feature distribution difference value is 6.3975. This result indicates that, under the current implementation conditions, the two indentation regions a and b have a moderate degree of difference in dimensions such as center distribution, directional characteristics, and distance intensity. This value will be used as a quantitative indicator input for subsequent defect cluster screening.

[0116] The advantage of this formula lies in its ability to effectively reflect directional trends and connectivity differences between regions beyond geometric proximity by integrating the difference in intersection angles, the normalized length ratio, and geometric distance. This provides a multi-dimensional quantitative basis for determining regional clustering. This is especially true when |L is introduced. a -L b | / ∑L k The addition of the proportional term can highlight the abnormal connection behavior in the local structure, enhancing the formula's ability to identify "irregular clustering" phenomena.

[0117] Table 2. Characteristic parameters of indentation area:

[0118]

[0119] Table 2 lists the center coordinates, angle values, and connection lengths of two sets of key indentation areas. The data was obtained through image processing and pairing calculations.

[0120] The results were compared with the established benchmark: based on the differences in regional characteristic distributions in relevant research. If the value is greater than 5.5, the two regions are considered to be weakly coupled structures that are not directly related. Therefore, the value obtained in this calculation is 6.3975, which exceeds the threshold. It is inferred that there is no obvious tendency for defects to cluster between the two regions, and they need to be removed in the cluster analysis.

[0121] S402: Based on the boundary of the region block delineated by the defect clustering region block coordinate map, extract the area value enclosed by the clustering block boundary, the internal defect distribution density value and the cross-coverage ratio of the adjacent area. Based on the magnitude of the combined difference between the boundary enclosed area value and the internal defect distribution density value, the clustering level is divided. The layer is then filled with color according to the color mark corresponding to the clustering level to obtain the configuration clustering distribution recognition result.

[0122] Geometric area calculation is performed on each region block, and the bounded area value of the region in the image is counted. The total number of pixels with marked defects inside the region is accumulated, and the defect distribution density value in the region block is calculated in area units. The ratio between the number of defect points and the area is obtained as a density parameter. At the same time, a neighboring buffer is set around each clustered region block. The degree of overlap between the defect distribution in the neighboring region and the inside of the target block is compared. The cross-over ratio is calculated as a quantitative basis for the defect extension trend. The bounded area value of each cluster block and its internal defect density value are used to form a parameter pair. They are matched sequentially in the layer according to the coordinate index, and the magnitude of their combined difference is calculated. That is, the magnitude of the area expansion required to increase the density in different regions. The clustered regions are divided into multiple clustering level segments by the relative size of the combined difference. The level results are mapped and matched with the preset color code spectrum. Different colors are assigned to the clustered regions in the original layer according to the clustering level to obtain the configuration clustering distribution recognition results.

[0123] The specific steps for obtaining the image quality detection and discrimination results of PU artificial leather are as follows:

[0124] S501: Call the defect cluster areas with marked levels in the configuration cluster distribution recognition results, filter the cluster blocks located at the edge of PU artificial leather, record the coordinate contour information and center position point, extract the geometric offset direction and offset distance from the cluster block to the center position of the image, and generate the edge cluster offset relationship pattern.

[0125] Clusters located at the edge of the PU artificial leather image are selected. Specifically, a fixed pixel width is set inward from the image boundary as the selection range. The coordinate range of the clusters is spatially overlapped with the edge region. If the center coordinate point or contour boundary of the cluster overlaps with the image edge region, it is considered an edge cluster. The complete contour coordinate set of each edge cluster is recorded, and the coordinates of the geometric center point of each cluster are calculated for subsequent geometric positional relationship analysis. Then, using the image center point as a reference point, a straight line is drawn connecting the center point of each cluster to the image center. The direction angle and line length of the connecting line are measured to obtain the offset direction and offset distance of the cluster relative to the image center. The contour, center point, direction angle, and distance data of the edge clusters are summarized and arranged according to the distribution position of each cluster at the image edge to form an edge cluster offset relationship pattern.

[0126] S502: Based on the edge cluster offset relationship pattern, the spatial arrangement range of cluster blocks is obtained block by block, the number of clusters and the proportion of the region in the local area are calculated, and the cluster number, distribution range and interval performance are combined and judged to generate a densely distributed area map.

[0127] The actual position data of the clustered blocks in the image coordinate system are read block by block. The arrangement boundaries in the horizontal and vertical directions are counted to form the spatial coverage of each clustered area. Multiple local area windows are delineated proportionally according to the overall size of the image. The number of clustered blocks contained in each local window is calculated. At the same time, the ratio between the actual occupied area of ​​the clustered blocks and the total area of ​​the local area is recorded to obtain the clustering ratio of the area. Then, the center point distances between the clustered blocks are compared item by item, and the minimum spacing, average spacing and maximum spacing are counted to form an interval description of the clustering interval performance. By jointly comparing the number of clusters, the area ratio and the interval performance, the areas with prominent clustering degree are identified and marked as densely distributed configuration areas. The coordinate contours of the areas are integrated and output to form a densely distributed configuration area map.

[0128] S503: Call the dense distribution area map of the configuration, detect the degree of overlap between the boundary point of the area and the original edge line of the PU artificial leather, and combine the trend of boundary extension direction change and boundary continuous morphological characteristics to determine whether the overlapping area presents a stretched or broken shape, and generate a boundary morphological characteristic mark map.

[0129] The contour point set of each region is analyzed, and the spatial overlap between its outer boundary curve and the original edge line of the PU artificial leather is compared. The shortest distance between the boundary point and the original edge line is used for judgment. If the distance is within the preset pixel error range, the two are considered to overlap. The boundary extension trend is analyzed in combination with the arrangement direction of the boundary points. The directional angle change value of continuous boundary segments is calculated to analyze whether the boundary line has a tendency to extend and deform. At the same time, the continuity index of the boundary curve is extracted, and the number of boundary breakpoints, the maximum break spacing and the average connection length are set to identify whether there are boundary break morphological characteristics. The geometric characteristics, continuity and directional changes of the boundary of each cluster region are comprehensively analyzed, and the corresponding boundary is marked as boundary stretching morphology, boundary break morphology or conventional boundary morphology, generating a boundary morphology feature marking map.

[0130] S504: Call the boundary morphology feature marker map to jointly identify the location of areas with three types of phenomena: clustering shift, dense configuration and boundary change. Then, mark the risk distribution range, the trend of internal structure shift direction and edge morphology features of the corresponding location in sequence to generate PU artificial leather image quality detection and discrimination results.

[0131] Spatial joint identification is performed on regions in the image that exhibit three types of morphology: clustering offset, dense configuration, and boundary changes. Image regions with all three types of features are extracted by pixel coordinate intersection. The spatial coverage of risk distribution is marked for each composite feature region. The texture arrangement trend within the region is extracted and the relative position change trajectory of its center coordinate in multiple layers is analyzed to form a trend curve of the change direction of internal structural offset. Then, the edge morphology is finely classified and marked as different types of boundary change labels, such as protruding edges, cracks, and bends. For each region with the three types of clustering features, risk level labels, offset direction arrows, and boundary morphology description information are embedded in sequence to completely mark the risk micro-defect regions and their causal structures, providing image basis for subsequent process control or finished product screening, and forming the image quality detection and discrimination results of PU artificial leather.

[0132] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A PU artificial leather production line quality detection method based on multi-modal image recognition, characterized in that, The method comprises the following steps: S1: acquiring the first frame image of the PU artificial leather production line detection area, sequentially collecting the edge image under oblique visible light irradiation, the temperature field image in the infrared thermal image, and the return image formed by laser reflection, marking and comparing the three types of image response phenomena, identifying the combined distribution relationship, and generating a multi-modal response fusion layer; S2: based on the multi-modal response fusion layer, judging the visible light illumination, infrared thermal intensity, and laser emission interval state to generate a detection image response adaptation set; specifically: S201: through the spot edge jump region in the multi-modal response fusion layer, counting the number distribution of spot edge jump positions in the unit area, combining the spatial arrangement characteristics of the heat zone turning points in the corresponding area and the brightness change rhythm in the laser return image, respectively judging the visible light illumination state, infrared thermal intensity form and laser emission interval trend in the region, marking the three states in the corresponding layer position to generate a layer state distribution marking set; S202: calling the layer state distribution marking set, for each state marking position, sequentially acquiring the gray level, temperature and brightness jump direction of the adjacent region in the multi-modal response fusion layer, and comparing and processing the three types of direction change characteristics at the same position to judge whether there is a consistent deviation in the response direction, if there is, adjusting the coordinates of the spatial projection point in the corresponding position in the layer to generate a detection image response adaptation set; S3: calling the detection image response adaptation set, delimiting the uneven light area and the thermal distribution edge band as the micro-mark potential block, identifying the directional change of the texture arrangement density, and generating a pressure mark positioning marking map; specifically: S301: calling the detection image response adaptation set, extracting the pixel light intensity value sequence and the thermal pixel continuous value sequence in the layer, screening the overlapping block of the light intensity average value region and the thermal gradient boundary region, calculating the light-thermal joint dispersion index, and obtaining the micro-mark potential block coordinate set; S302: based on the image segment area positioned by the micro-mark potential block coordinate set, extracting the texture arrangement direction angle value, texture spacing density value and continuous gray scale difference value in the region, judging the interval where the texture spacing density value decreases in the texture arrangement direction angle value change section, recording the position paragraph of the continuous gray scale difference, and obtaining the direction consistent decreasing texture section coordinate group; S303: according to the position index of the direction consistent decreasing texture section coordinate group in the image, calling the original image layer and positioning the corresponding image pixel region, marking the image target marking layer after the boundary of the coordinate group is proportionally expanded, performing block coverage processing on the original image according to the pixel coverage range of the marking layer, and obtaining the pressure mark positioning marking map; S4: calling the pressure mark positioning marking map, drawing a graph of the connection relationship between the defect area centers, marking the area block with an aggregation trend, classifying and judging the outline and dense performance characteristics of the aggregation block, performing layer color overlay according to the aggregation level, and generating a configuration aggregation distribution identification result.

2. The PU artificial leather production line quality detection method based on multi-modal image recognition according to claim 1, characterized in that, The multi-modal response fusion layer includes a texture response structure layer, a thermal domain edge layer, and a laser reflection response layer. The detection image response adaptation set includes an illumination adaptation parameter set, a thermal adaptation parameter set, and a laser rhythm calibration set. The indentation positioning label image includes a micro-mark distribution map, a texture directionality change map, and a gray-scale transition feature map. The configuration aggregation distribution recognition result includes a defect connection map, an aggregation level hierarchical map, and a contour density classification map. 3.The PU artificial leather production line quality detection method based on multi-modal image recognition of claim 1, characterized in that, The acquisition step of the multi-modal response fusion layer is specifically as follows: S101: A first image of a PU artificial leather production line detection area is acquired. The gray-scale change of the image edge region under oblique visible light illumination, the temperature response state of the pixel position in the infrared thermal image, and the reflection brightness degree of the region in the laser reflection image are sequentially collected. According to the coordinate consistency of the pixel position in the image, the sampling data of the three types of images are spatially corresponded to generate a position corresponding image data set; S102: Based on the position corresponding image data set, it is determined whether the gray-scale change state has a continuous boundary, whether the temperature response state presents inhomogeneous change in a local range, and whether the reflection brightness degree is lower than a set recognition reference. According to the determination result, the positions of the multi-class response regions are marked to generate a composite response intersection position set; S103: The composite response intersection position set is called to acquire the gray-scale change amplitude, temperature change trend, and reflection brightness difference corresponding to the position in the three types of images. According to the superposition distribution relationship between the three types of change values, the response performance degree of the position in the image is recognized to generate a multi-modal response fusion layer.

4. The PU artificial leather production line quality detection method based on multi-modal image recognition according to claim 1, characterized in that, The light-heat combined dispersion index is calculated by the following formula: ； wherein, represents the joint photothermal dispersion index in the i-th layer block, M i represents the total number of pixels in the i-th layer block, represents the light intensity value of the j-th pixel point in the i-th layer block, represents the average value of the pixel light intensity value in the i-th layer block, T ij represents the heat value of the j-th pixel point in the i-th layer block, represents the average value of the pixel heat value in the i-th layer block, W ij represents the weight factor corresponding to the j-th pixel point in the i-th layer block. 5.The PU artificial leather production line quality detection method based on multi-modal image recognition of claim 1, characterized in that, The acquisition step of the configuration aggregation distribution recognition result is specifically as follows: S401: The annotation coordinate set of the defect region in the indentation positioning label image is called to extract the region center coordinate value, the center-to-center line length value, and the staggered distribution angle value. The feature distribution difference value is calculated. A plurality of intersection node coordinate points are screened, and the region block is boundary marked to acquire a defect aggregation region block coordinate map; S402: Based on the region block boundary defined by the defect aggregation region block coordinate map, the aggregation block boundary surrounding area value, the internal defect distribution density value, and the intersection coverage ratio of the adjacent region are extracted. The aggregation level is divided according to the combined difference amplitude of the boundary surrounding area value and the internal defect distribution density value. The layer is color-filled and covered according to the corresponding color scale of the aggregation level to acquire a configuration aggregation distribution recognition result.

6. The PU artificial leather production line quality detection method based on multi-modal image recognition according to claim 5, characterized in that, The feature distribution difference value is calculated by the following formula: ； wherein, represents the characteristic distribution difference value between the a-th and b-th center coordinate points, x a , x b respectively represent the coordinate values of the a-th and b-th defect region center points in the horizontal direction, y a , y b respectively represent the coordinate values of the a-th and b-th defect region center points in the vertical direction, θ a , θ b respectively represent the a-th and b-th defect region staggered distribution angle values, L a , L b respectively represent the length values of the lines connecting the a-th and b-th center points, represents the sum of the length values of the lines connecting the centers in the sample, a total of M lines.

7. The PU artificial leather production line quality detection method based on multi-modal image recognition according to claim 1, characterized in that, The method further includes the following S5 step: S5: The defect aggregation region of the configuration aggregation distribution recognition result that has been divided into levels is called to extract the aggregation block and the center offset distribution relationship pattern in the PU artificial leather edge zone. According to the position distribution, the configuration density, and the boundary overlap performance, the risk range, the structure offset trend, and the boundary morphology feature are labeled to generate a PU artificial leather image quality detection discrimination result; The PU artificial leather image quality detection discrimination result includes a risk area distribution map, a structure offset trend map, and a boundary morphology feature map. 8.The PU artificial leather production line quality detection method based on multi-modal image recognition of claim 7, characterized in that, The acquisition step of the PU artificial leather image quality detection discrimination result is specifically as follows: S501: Call the defect clustering areas marked with the grade in the configuration clustering distribution recognition results, filter the clustering blocks located at the edge of PU artificial leather, record the coordinate contour information and center position point, extract the geometric offset direction and offset distance from the clustering block to the center position of the image, and generate an edge clustering offset relationship pattern. S502: Based on the edge cluster offset relationship pattern, obtain the spatial arrangement range of cluster blocks block by block, calculate the number of clusters and the proportion of the area in the local area, and combine and judge the number of clusters, distribution range and interval performance to generate a densely distributed area map. S503: Call the dense distribution area map of the configuration, detect the degree of overlap between the boundary point of the area and the original edge line of the PU artificial leather, and combine the trend of boundary extension direction change and boundary continuous morphological characteristics to determine whether the overlapping range presents a stretched or broken shape, and generate a boundary morphological characteristic mark map. S504: Call the boundary morphology feature marker map to jointly identify the locations of areas with three types of phenomena: clustering offset, dense configuration, and boundary change. Sequentially mark the risk distribution range, the trend of internal structure offset direction change, and edge morphology features of the corresponding locations to generate PU artificial leather image quality detection and discrimination results.

Images