PU artificial leather production line quality detection method based on multi-modal image identification

Abstract

The invention relates to the technical field of defect detection, in particular to a PU artificial leather production line quality detection method based on multi-modal image identification, which comprises the following steps: acquiring a multi-modal image of a detection area and fusing response, calibrating a micro-mark block and drawing a connection atlas, identifying aggregation distribution and extracting edge configuration features, and generating an image quality detection judgment result. According to the method, the multi-modal response information of the visible light image, the infrared thermal image and the laser reflection image is fused, the combination distribution relation of the images is recognized, and the layer mapping is established, so that accurate positioning of uneven illumination and the edge of the hot area can be realized, and meanwhile, the directional characteristics of texture change are recognized, and potential defect blocks are calibrated; the method has higher adaptability and recognition precision in the environment of various textures and complex working conditions, is more flexible and has enhanced robustness, and the integrity and accuracy of image detection and the intuition of result expression are remarkably improved.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of defect detection, in particular to a PU artificial leather production line quality detection method based on multi-modal image recognition. BACKGROUND

[0002] The technical field of defect detection involves using image processing, machine learning, and sensor fusion to collect and analyze images of product surfaces or internal structures during industrial production processes to identify structural abnormalities such as cracks, impurities, holes, and faults that do not meet quality standards. This technical field mainly includes the configuration and deployment of image acquisition devices, the design of multi-source image fusion algorithms, the establishment of image feature extraction methods, the construction of defect pattern recognition models, and the integration and application of online automatic detection systems. As manufacturing industries continue to demand higher product quality control, defect detection technology has been widely applied in textiles, metallurgy, electronics, chemical industry, automobile industry, and other fields, becoming an important support means for production automation and intelligentization. The PU artificial leather production line quality detection method refers to acquiring images of artificial leather surfaces using single modal image acquisition devices such as visible light cameras, then using texture feature analysis methods based on gray level co-occurrence matrix or geometric feature extraction methods based on edge detection operators for defect positioning and classification recognition. This method relies on fixed angle and uniform light source shooting conditions, and mostly uses support vector machines or decision tree models to distinguish extracted features, making it difficult to adapt to the detection needs of different texture type artificial leather products under complex lighting and variable working conditions. To overcome the above limitations, a PU artificial leather production line quality detection method based on multi-modal image recognition is proposed. This method fuses multi-modal image information including visible light images, infrared thermal images, and laser profile images, uses feature level fusion strategy for joint analysis, and uses convolutional neural networks for end-to-end processing of feature extraction and defect classification.

[0003] Existing technologies use single visible light image acquisition methods for defect detection on the surface of PU artificial leather, relying on fixed angle and uniform light source conditions. In uneven lighting or complex texture scenarios, feature extraction is easily disturbed, leading to decreased detection accuracy. When using support vector machines or decision tree models, the adaptability of feature dimensions is insufficient, making the response to fuzzy or small defects sluggish, and it is difficult to identify edge hidden defects and texture change trends. When the surface of artificial leather products has multiple source disturbances or differences in structure density, traditional models cannot reasonably distinguish the defect aggregation trend, affecting the accuracy of overall quality assessment and the controllability of risk judgment. SUMMARY

[0004] In order to solve the uneven illumination distribution or the complex texture type scene in the prior art, the feature extraction is easy to be disturbed, which leads to the decrease of detection accuracy, when the support vector machine or decision tree model is adopted, the adaptability of feature dimension is insufficient, so that the response to the fuzzy or tiny defect is slow, it is difficult to identify the edge hidden defect and the texture change trend, when the surface of the artificial leather product appears multi-source disturbance or configuration density difference, the traditional model cannot realize the reasonable distinction of the defect aggregation trend, which affects the accuracy of the overall quality evaluation and the controllability of the risk judgment, the present application provides a PU artificial leather production line quality detection method based on multi-modal image recognition. The technical solution is as follows: In one aspect, a PU artificial leather production line quality detection method based on multi-modal image recognition is provided, which comprises: S1: acquiring the first image of the PU artificial leather production line detection area, sequentially collecting the edge image under oblique visible light irradiation, the temperature domain image in the infrared thermal image, and the return image formed by laser reflection, marking and comparing the three types of image response phenomena, and identifying the combined distribution relationship to generate a multi-modal response fusion layer; S2: based on the multi-modal response fusion layer, judging the visible light illumination, infrared thermal intensity and laser emission interval state to generate a detection image response adaptation set; S3: calling the detection image response adaptation set, delimiting the uneven illumination area and the thermal distribution edge zone as a micro-mark potential block, identifying the directional change of texture arrangement density, and generating a pressure mark positioning label image; S4: calling the pressure mark positioning label image, drawing a graph of the connection relationship between the defect area centers, marking the area block with aggregation trend, classifying and judging the outline and dense performance characteristics of the aggregation block, layering and coloring according to the aggregation level to generate a configuration aggregation distribution recognition result.

[0005] As a further scheme of the present application, the multi-modal response fusion layer comprises a texture response structure layer, a thermal domain edge layer and a laser reflection response layer, the detection image response adaptation set comprises an illumination adaptation parameter set, a thermal sensing adaptation parameter set and a laser rhythm calibration set, the pressure mark positioning label image comprises a micro-mark distribution graph, a texture directional change graph and a gray scale transition feature graph, and the configuration aggregation distribution recognition result comprises a defect connection graph, an aggregation level hierarchical graph and an outline dense classification graph.

[0006] As a further scheme of the present application, the acquisition step of the multi-modal response fusion layer is specifically: S101: Obtain the first frame image of the PU artificial leather production line detection area, sequentially collect the gray scale change of the image edge region under oblique visible light irradiation, the temperature response state of the position in the infrared thermal image and the reflection brightness degree of the region in the laser reflection image, according to the coordinate consistency of the pixel position in the image, the sampling data of the three types of images are processed in space, and a position corresponding image data set is generated; S102: Based on the position corresponding image data set, judge whether the gray scale change state exists continuous boundary, whether the temperature response state presents inhomogeneous change in local range, whether the reflection brightness degree is lower than the set recognition reference, mark the position of the multi-class response region according to the judgment result, and generate a composite response cross position set; S103: Call the composite response cross position set, obtain the gray scale change amplitude, temperature change trend and reflection brightness difference corresponding to the position in the three types of images, identify the response performance degree of the position in the image according to the superposition distribution relationship between the three types of change values, and generate a multi-modal response fusion layer.

[0007] As a further scheme of the present application, the detection image response adaptation set acquisition step is specifically: S201: Through the spot edge jump region in the multi-modal response fusion layer, the number distribution of the spot edge jump position in the unit area is counted, the spatial arrangement characteristics of the hot area turning point in the corresponding region and the brightness change rhythm in the laser return image are combined, the visible light illumination state, the infrared heat intensity form and the laser emission interval change trend in the region are judged respectively, the three types of state are marked in the corresponding layer position, and a layer state distribution marking set is generated; S202: Call the layer state distribution marking set, for each type of state marking position, sequentially obtain the gray scale, temperature and brightness jump direction of the adjacent region in the multi-modal response fusion layer, and compare the three types of direction change characteristics at the same position, judge whether there is a response direction consistency deviation, if there is, adjust the coordinate of the space projection point in the corresponding position of the layer, and generate a detection image response adaptation set.

[0008] As a further scheme of the present application, the acquisition step of the indentation positioning marking graph is specifically: S301: Call the detection image response adaptation set, extract the pixel light intensity value sequence and the thermal pixel continuous value sequence in the layer, screen the overlapping block of the light intensity average value region and the thermal gradient boundary region, calculate the light-thermal joint dispersion index, obtain the micro mark potential block coordinate set; S302: Based on the image segment area positioned by the micro-mark potential block coordinate set, texture arrangement direction angle value, texture interval density value and continuous gray scale difference value in the area are extracted, a section with a decreasing trend of texture interval density value in the texture arrangement direction angle value change section is judged, a position section of continuous gray scale difference is recorded, and a direction consistent decreasing texture section coordinate group is obtained; S303: According to the position index of the direction consistent decreasing texture section coordinate group in the image, the original image layer is called and the corresponding image pixel area is positioned, the coordinate group boundary is proportionally expanded, and then the image target marking layer is calibrated, the original image is block covered according to the pixel coverage range of the marking layer, and the indentation positioning marking image is obtained.

[0009] As a further scheme of the present application, the photo-thermal combined dispersion index adopts the following formula: ; Among them, represents the photo-thermal combined dispersion index in the i-th layer block, M i represents the total number of pixels in the i-th layer block, I ij represents the light intensity value of the j-th pixel point in the i-th layer block, represents the average value of the pixel light intensity value in the i-th layer block, T ij represents the heat value of the j-th pixel point in the i-th layer block, represents the average value of the pixel heat value in the i-th layer block, W ij represents the weight factor corresponding to the j-th pixel point in the i-th layer block.

[0010] As a further scheme of the present application, the obtaining step of the conformation aggregation distribution recognition result is specifically: S401: The marking coordinate set of the defect area in the indentation positioning marking image is called, the region center coordinate value, the center line length value and the staggered distribution angle value are extracted, the feature distribution difference value is calculated, a plurality of intersection node coordinate points are screened, the boundary of the region block is marked, and the defect aggregation region block coordinate atlas is obtained; S402: Based on the region block boundary drawn by the defect aggregation region block coordinate atlas, the aggregation block boundary surrounding area value, the internal defect distribution density value and the intersection coverage ratio of the adjacent region are extracted, the aggregation level is divided according to the combination difference value amplitude of the boundary surrounding area value and the internal defect distribution density value, the image layer is filled and covered according to the corresponding color scale of the aggregation level, and the conformation aggregation distribution recognition result is obtained.

[0011] As a further scheme of the present application, the feature distribution difference value adopts the following formula: ; Among them, represents the feature distribution difference value between the a th and b th center coordinate points, x a , x b , y a , y b , θ a , θ b , L a , L b , L represents the length value of the line connecting the a th and b th center points, L

[0012] As a further scheme of the present application, the method further comprises a step S5: S5: calling the defect aggregation area distinguished by levels in the configuration aggregation distribution identification result, extracting the aggregation block and the center offset distribution pattern in the edge zone of the PU artificial leather, marking the risk range, the structure offset trend and the boundary shape characteristics according to the position distribution, the configuration density and the boundary overlap performance, and generating the PU artificial leather image quality detection discrimination result; The PU artificial leather image quality detection discrimination result comprises a risk area distribution map, a structure offset trend map and a boundary shape characteristic map.

[0013] As a further scheme of the present application, the acquisition step of the PU artificial leather image quality detection discrimination result is specifically: S501: calling the defect aggregation area marked by levels in the configuration aggregation distribution identification result, screening the aggregation block located at the edge position of the PU artificial leather, recording the coordinate contour information and the center position point, extracting the geometric offset direction and the offset distance of the aggregation block to the image center position, and generating an edge aggregation offset relationship pattern; S502: based on the edge aggregation offset relationship pattern, acquiring the position arrangement range of the aggregation block in space block by block, calculating the aggregation number and the area proportion in the local area, and combining the aggregation number, the distribution range and the interval performance to generate a configuration dense distribution area map; S503: calling the configuration dense distribution area map, detecting the coincidence degree of the area boundary point and the original edge line of the PU artificial leather, and combining the boundary extension direction change trend and the boundary continuous shape characteristics to judge whether the overlap range presents a stretching or breaking form, and generating a boundary shape characteristic marking map; S504: Call the boundary morphology feature label map, jointly identify the region position of the three phenomena of aggregation deviation, structure density and boundary change, label the risk distribution range, internal structure deviation direction change trend and edge morphology feature content of the corresponding position in turn, and generate PU artificial leather image quality detection discrimination result.

[0014] The technical scheme provided by the embodiment of the present application brings at least the following beneficial effects: By fusing the multi-modal response information of visible light images, infrared thermal images and laser reflection images, identifying the combined distribution relationship and establishing the layer mapping, the uneven illumination and the edge of the thermal area can be accurately positioned, the directional features of the texture changes are identified and the potential defect blocks are calibrated, the edge distribution and center deviation pattern of the aggregation area are further extracted on the basis of the aggregation trend identification, the structure deviation trend and the risk range are labeled in combination with the dense performance and the boundary morphology, and in the face of various textures and complex working conditions, the adaptability and identification accuracy are higher, the flexibility is higher and the robustness is enhanced, and the integrity, accuracy and intuitiveness of the image detection result are significantly improved. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 The workflow of the present application is shown in the figure; DETAILED DESCRIPTION

[0016] The technical scheme in the present application will be described below with reference to the accompanying drawings.

[0017] In the embodiments of the present application, the words such as "example", "for example" are used to represent as an example, illustration or explanation. Any embodiment or design scheme described as "example" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present application, the meaning expressed by "and / or" can be both, or can be one of the two.

[0018] In order to make the technical problems, technical schemes and advantages of the present application more clear, the following will be described in detail with reference to the accompanying drawings and specific embodiments.

[0019] Please refer to Figure 1 The embodiment of the present application provides a PU artificial leather production line quality detection method based on multi-modal image recognition. The processing flow of the method can include the following steps: S1: Acquire the first image of the PU artificial leather production line detection area, sequentially collect the edge map under oblique visible light irradiation, the temperature domain map in the infrared thermal image, and the return map formed by laser reflection, label and compare the three types of image response phenomena, identify the combined distribution relationship, and generate a multi-modal response fusion layer; S2: judging the visible light illumination, infrared thermal intensity and laser emission interval state based on the multi-modal response fusion layer, performing spatial corresponding calibration, and generating a detection image response adaptation set; S3: calling the detection image response adaptation set, delineating the uneven light area and thermal distribution edge band as a micro mark potential block, identifying the directional change of texture arrangement density, if the texture decreases in the same direction and the gray scale changes slowly, marking the block on the original image, and generating a indentation positioning annotation map; S4: calling the indentation positioning annotation map, drawing a map of the connection relationship between the defect area centers, marking the area block with an aggregation trend, classifying and judging the contour and dense performance characteristics of the aggregation block, layering and coloring according to the aggregation level, and generating a configuration aggregation distribution identification result; S5: calling the defect aggregation area with distinguished levels in the configuration aggregation distribution identification result, extracting the aggregation block and the center offset distribution relationship pattern in the PU artificial leather edge zone, and according to the position distribution, configuration density and boundary overlap performance, marking the risk range, structure offset trend and boundary shape characteristics, and generating a PU artificial leather image quality detection discrimination result; The multi-modal response fusion layer includes a texture response structure layer, a thermal domain edge layer, and a laser reflection response layer. The detection image response adaptation set includes an illumination adaptation parameter set, a thermal adaptation parameter set, and a laser rhythm calibration set. The indentation positioning annotation map includes a micro mark distribution map, a texture directionality change map, and a gray scale transition feature map. The configuration aggregation distribution identification result includes a defect connection map, an aggregation level layering map, and a contour dense classification map. The PU artificial leather image quality detection discrimination result includes a risk area distribution map, a structure offset trend map, and a boundary shape feature map.

[0020] The acquisition steps of the multi-modal response fusion layer are as follows: S101: acquiring the first image of the PU artificial leather production line detection area, sequentially collecting the gray scale change of the image edge area under oblique visible light illumination, the temperature response state of the position in the infrared thermal image, and the reflection brightness degree of the area in the laser reflection image, and performing spatial corresponding processing on the sampling data of the three types of images according to the coordinate consistency of the pixel positions in the image, to generate a position corresponding image data set; The industrial camera is used to take a shot of the whole detection area from a set shooting position, and the ambient light intensity calibration operation is completed before the image sensor starts exposure, the oblique visible light is used for lighting, the interference coincidence between the irradiation path and the reflection path is controlled by using the oblique angle, so that the edge structure of the detection area is clearer, the image acquisition device captures image frames in time sequence, and frame selection operation is performed in the image buffer, the first frame image is selected as the subsequent analysis reference image, the edge area in the first frame image is obtained through image cropping operation, and the gray scale values of each pixel in the area are read in row and column traversal mode, the change trend of the gray scale values in each row or column is recorded as a sequence, and the pixel coordinates of the gray scale mutation points are marked synchronously, the thermal imager in the image acquisition device synchronously collects infrared images from the same angle, the infrared thermal imaging device continuously samples in the detection area, the thermal imaging data is timestamped with the first frame image, the thermal response state of each pixel point is located, and the laser reflection image acquisition device scans the detection surface, measures the reflection brightness of each point in the scanning path, and forms a two-dimensional brightness matrix, after the three kinds of images record data of the same detection area, the three kinds of images are matched and matched in space according to the pixel coordinates through image registration operation, so that each position contains gray scale, temperature and brightness three kinds of data values, and the position corresponding image data set is generated.

[0021] S102: Based on the position corresponding image data set, whether the gray scale change state exists continuous boundary, whether the temperature response state presents inhomogeneous change in local range, whether the reflection brightness degree is lower than the set recognition reference, according to the judgment result, the position of the multi-class response area is marked, and the composite response cross position set is generated. In the image processing module, the channel analysis order of the three kinds of gray scale, thermal image and reflection layers is set, the edge area with continuous gray scale difference is selected from the gray scale layer, whether there is a boundary trend in the image is judged by calculating the change amplitude of the gray scale values between adjacent pixel points, a search window is set to slide in the image for detection, if most of the pixels in the window have obvious gray scale difference, it is considered that the current area is a gray scale boundary area, according to the temperature change of the position points in the temperature layer, the temperature values of the surrounding multiple pixel points are extracted to form a temperature distribution vector, if the temperature values in the area have obvious difference, it is judged that there is a response area with local inhomogeneous change, the brightness information of the area in the laser image is checked, if the brightness is obviously lower than that of the background area, it is indicated that the laser energy reflection at the point is weak, multiple response signals are formed in the three kinds of layers, the spatial coincidence position of the response signals is recorded, the position is identified in the image marking channel, and the composite response cross position set is generated.

[0022] S103: Call the composite response cross-position set to obtain the corresponding gray scale change amplitude, temperature change trend and reflection brightness difference of the position in the three types of images, according to the superposition distribution relationship between the three types of change values, identify the response performance degree of the position in the image, and generate a multi-modal response fusion layer; Read each pixel coordinate position in the composite response cross-position set, extract the corresponding data of the point in the three layers one by one, compare the change values of the gray channel to obtain the gray scale change amplitude, compare the temperature values of the same position at different time points in the thermal image channel to extract the numerical difference of the temperature rise or fall, extract the brightness value of the corresponding position from the laser reflection layer and calculate the difference of the brightness value relative to the brightness reference in the overall image, form a fusion feature vector according to the channel order of the three types of change data and perform unified numerical standard processing, determine the position relationship of each feature value in the overall distribution in the feature set formed by the multi-point data through data analysis, grade the response degree of the detection point through this way, and draw the fusion layer image by layering the pixel points of the response level, each level is represented by different pixel values, which can directly reflect the comprehensive response strength change trend of different detection areas in the three dimensions of gray scale, thermal image and reflection, and generate a multi-modal response fusion layer.

[0023] The acquisition steps of the detection image response adaptation set are specifically: S201: Through the spot edge jump region in the multi-modal response fusion layer, count the number distribution of the spot edge jump position in the unit area, combine the spatial arrangement characteristics of the hot zone turning point in the corresponding area and the brightness change rhythm in the laser return image, respectively judge the visible light illumination state, infrared heat intensity form and laser emission interval change trend in the area, mark the three types of states in the corresponding layer position, and generate a layer state distribution marking set. The contour of the spot outer edge shape formed by extracting the gray level mutation boundary in the gray level layer image is detected by fixed window sliding method, the number distribution of the jump edge in the unit area region is counted, that is, the number value of the accumulated edge change in each scanning window, the whole image is divided into equal area cells by setting the area resolution standard, and the edge jump count in each cell is recorded as the basis of spatial density distribution, the inflection point or extreme point of the thermal area in each region of the thermal image layer is extracted, the spatial arrangement characteristics of the turning point are calculated, including the distance between the turning points, the arrangement angle, the change direction, etc., and the distribution form in the corresponding cell is marked, then the sequence of the brightness value of each region in the laser reflection map changing with the spatial coordinates is extracted, the periodicity and fluctuation frequency of the sequence are analyzed to infer the strong and weak rhythm changes existing in the laser emission return process, after the feature extraction of the three types of layer data is completed, the performance form of each type of state is judged, the gray level edge jump density is judged as the visible light illumination state, the regularity and intensity of the thermal area inflection point are used to judge the infrared thermal sensing performance form, and the laser brightness rhythm change reflects the emission interval trend, the above three types of states are marked in the original layer at the corresponding positions according to the pixel coordinates, and a layer state distribution marking set is generated.

[0024] S202: Call the layer state distribution marking set, mark the position for each type of state, sequentially obtain the gray level, temperature and brightness jump direction of the adjacent region in the multi-modal response fusion layer, and compare and process the three types of direction change characteristics at the same position, judge whether there is a response direction consistency deviation, if there is, adjust the coordinate of the spatial projection point in the corresponding position in the layer, and generate a detection image response adaptation set; The layer position of each marked state is read in sequence, and the response data of the surrounding region thereof is extracted from the multi-modal response fusion layer, a fixed neighborhood width is set in the extraction range, the maximum change direction of the neighborhood gray value is calculated in the gray channel, the angle of the jump direction is judged according to the pixel gray difference gradient, the dominant direction of the temperature value rising or falling is identified in the same neighborhood in the thermal imaging channel, the heat flow trend direction is determined by comparing the difference between the center point temperature and the neighborhood points, the spatial position relationship of the brightness value in the laser reflection layer is analyzed, the change direction of the brightness value from strong to weak or from weak to strong is obtained, the direction vectors of the jump directions in the three types of layers are formed, after the direction extraction is completed, the three types of direction vectors at each marked position are compared and processed, if a significant direction deviation is found between any two types of directions, that is, the included angle between the two types of directions is greater than a certain set standard angle, it is determined that it is a response direction consistency deviation region, and then according to the vector included angle size and the priority of the dominant direction, the spatial projection coordinates of the position in the fusion layer are re-adjusted, the projection point is slightly adjusted in position along the dominant direction according to the set rule, and a detection image response adaptation set is generated.

[0025] The acquisition step of the indentation positioning mark map is specifically: S301: call the detection image response adaptation set, extract the pixel light intensity value sequence and the thermal pixel continuous value sequence in the layer, filter the overlapping block of the light intensity average value region and the thermal gradient boundary region, calculate the light-thermal joint dispersion index, and obtain the micro-mark potential block coordinate set; The light-thermal joint dispersion index is calculated by the following formula: ; Wherein, represents the light-thermal joint dispersion index in the ith layer block, M i represents the total number of pixels in the ith layer block, I ij represents the light intensity value of the jth pixel point in the ith layer block, represents the average value of the pixel light intensity value in the ith layer block, T ij represents the thermal value of the jth pixel point in the ith layer block, represents the average value of the pixel thermal value in the ith layer block, W ij represents the weight factor corresponding to the jth pixel point in the ith layer block; The formula calculation logic: the Euclidean space scale measurement of joint dispersion is realized by subtracting two items, squaring, adding and taking the square root, and is modulated by multiplying the local weight, and the dimension consistency is ensured by taking the average according to the total number of pixels and taking the absolute value. The overall logic realizes the measurement of the regional light-thermal feature dispersion degree through the weighted average of the common variation degree of gray scale and thermal value; The light-thermal joint dispersion index represents the cooperative variation degree between the gray scale light intensity value and the thermal value of the pixels in the layer block, and is used to measure the comprehensive fluctuation of the light and thermal signals in the region. The larger the index value is, the more intense the deviation of the light intensity and the thermal value in the region is, and there is a structural abnormality or local heat source feature; Parameter meaning and calculation logic: M i represents the number of pixel points contained in the ith layer block; I ij is the gray scale light intensity value of the jth pixel in the ith layer block, which is obtained by an infrared image sensor and has a unit of GrayLevel; is the average value of the layer block pixel light intensity value, which is calculated according to ; T ij is the thermal value of the corresponding pixel point, which has a unit of ℃ and is collected by a thermal imager; is the in-block average value of the thermal value, which is calculated in the following manner: ; W ijW(j) is the weight value of the jth pixel, which is calculated by the absolute value of the ratio of the gray level change rate and the thermal gradient change rate between the pixel and its four adjacent pixels, that is, ; wherein N(j) represents the neighborhood pixel set of the jth pixel, and ε is a minimum constant term to avoid zero denominator; To verify the actual calculation process of the formula, the following embodiment data is constructed. Table 1 Pixel light intensity and thermal sampling data table in layer block:

[0026] As shown in Table 1, the light intensity values and thermal values of five pixels in the layer block are collected by the image device, wherein the weight value is calculated by the gray level and thermal difference ratio of each pixel and its neighborhood, such as the gray level neighborhood difference sum of pixel 1 is 20, and the thermal neighborhood difference sum is 17.4, and the weight value is calculated according to the formula: W i1 =|20 / 17.4|≈1.15; Calculate the average light intensity and thermal mean value: ; ; Then the joint deviation is calculated: ; ; ; ; ; Substitute the formula to calculate: ; The result shows that the light-thermal joint dispersion index is 4.079, and the light intensity and thermal in the current layer block exist moderate coupling dispersion degree on the whole, which shows that there is structural thermal trace anomaly in this area, which is suitable as the basis parameter for subsequent screening of overlapping blocks. The advantage of the formula is that by introducing the pixel local gray level and thermal ratio weight W ij , combined with the weighted sum of Euclidean root space joint deviation, the sensitivity and block discrimination of light-thermal coupling anomaly recognition are improved; in the whole, the high-precision focusing and potential anomaly point extraction ability of weak thermal mark area are realized.

[0027] S302: Based on the micro mark potential block coordinate set positioning, the image segment area is located, the texture arrangement direction angle value, the texture interval density value and the continuous gray scale difference value in the area are extracted, the texture interval density value in the texture arrangement direction angle value change section is judged to be in the decreasing trend interval, the position section of the continuous gray scale difference is recorded, and the direction consistent decreasing texture section coordinate group is obtained; In the fusion image, the corresponding image segment area is intercepted, the texture structure in each segment is processed, the angle of the texture arrangement direction is calculated by the gradient direction analysis algorithm, the direction angle value of the main texture direction presented in the region is obtained, the interval density value of the texture arrangement is calculated combined with the periodic structure of the pixel gray scale change, the value reflects the spatial distribution density of the adjacent texture peak or valley, the continuous gray scale difference value sequence formed by the pixel gray scale value in the region is extracted, the feature paragraph of the gray scale in one-way increasing or decreasing is identified through the continuity change trend analysis, the direction angle value change section extracted before is scanned, the texture interval density value sequence is extracted in the interval where the direction angle change exists, and whether the interval density forms a decreasing trend between adjacent sampling points is judged. If the density value continuously decreases within a certain sampling range, it is considered that the section shows the texture structure aggregation trend. If the continuous gray scale difference value in the trend paragraph is significantly offset or stably decreasing, the position is recorded as the target section of the texture direction consistent and density decreasing. The coordinate boundary information of the image segment meeting the combination condition is organized to form the direction consistent decreasing texture section coordinate group, and the direction consistent decreasing texture section coordinate group is obtained.

[0028] S303: According to the position index of the direction consistent decreasing texture section coordinate group in the image, the original image layer is called and the corresponding image pixel area is positioned, the coordinate group boundary is proportionally expanded and marked as the image target marking layer, the original image is processed by block coverage according to the pixel coverage range of the marking layer, and the indentation positioning marking image is obtained. According to the position index information of the coordinate group in the fusion image, the corresponding image pixel area is located in the original image layer, the fusion layer coordinate group is mapped to the original image space coordinate by using the index table or pixel mapping relationship, each coordinate boundary is expanded by a proportion factor, a fixed number of pixels or a set percentage is selected to expand the original area range, the potential extension area outside the indentation boundary is covered, the image marking layer is set in the expanded boundary range, and the pixels in the area are marked and labeled. The image target marking layer exists in the form of a separate layer and retains the integrity of the original image. In the image processing operation, the marking layer is used as an image coverage template to perform block coverage operation on the corresponding area in the original image, and the coverage area is given a uniform layer identification code or color filling. The indentation positioning marking image is obtained.

[0029] The acquisition steps of the conformation aggregation distribution recognition result are specifically: S401: Call the annotation coordinates of the defect region in the indentation positioning labeled map, extract the region center coordinate value, center line length value and staggered distribution angle value, calculate the feature distribution difference value, screen the node coordinate points of multiple intersections, and mark the boundary of the region block to obtain the defect aggregation region block coordinate map; The feature distribution difference value is calculated by the following formula: ; Wherein, represents the feature distribution difference value between the a th and b th center coordinate point, x a , x b respectively represent the coordinate value of the a th and b th defect region center point in the horizontal direction, y a , y b respectively represent the coordinate value of the a th and b th defect region center point in the vertical direction, θ a , θ b respectively represent the a th and b th defect region staggered distribution angle value, L a , L b respectively represent the length value of the line between the a th and b th center point, represents the sum of the length of the center line in the sample, a total of M lines; The calculation logic of the formula: the first part is the Euclidean distance calculation term , which is used to obtain the spatial distance of the center points a and b in the two-dimensional plane (xy) coordinate system; The second part is the normalized weight term of the staggered distribution angle difference , which is used to enhance the interference description ability of the angle to the spatial relationship after linear mapping of the absolute angle difference; The third part is the length difference normalization term , which is used to describe the relative difference of the line between the points in the overall length distribution. The larger the ratio is, the higher the fluctuation of the line between the individual nodes is, and the more unstable the region distribution is. Through the organic combination of the three types of participation terms, multi-dimensional feature fusion is realized while ensuring the geometric rationality; Parameter description and acquisition process: x a = 124.7 mm, x b = 129.5 mm: the horizontal direction coordinates of the region a and b center points are obtained by the coordinate positioning algorithm in the visual image recognition device. The positioning basis is the centroid position of the indentation region contour center point; y a = 208.2 mm, y b = 204.8 mm: the vertical direction coordinate value extracted in the above coordinates is also realized by the image recognition algorithm. The center of gravity is positioned; θ a = 37.2°, θb = 52.6°: the angle between the direction of indentation stripe in the region and the reference direction (usually the horizontal axis), which is quantified by the direction gradient of gray level in the image, and the quantization standard is the angle value of the maximum direction of gray level change along the main texture direction; L a = 6.3mm, L b = 7.1mm: the actual length of the line between the centers of the regions, which is obtained by using the Euclidean distance calculation; , M = 12: represents the total length of the center line between the region pairs, which is collected from 12 groups of region pair lines in the entire defect cluster candidate region (extracted from the image data set pair by pair); The above values are substituted into the formula as follows: ; The results show that the feature distribution difference value is 6.3975, which indicates that under the current implementation conditions, the two indentation regions a and b have moderate differences in center distribution, direction feature, distance strength, etc. This value will be used as a quantitative index input for subsequent defect aggregation screening; The benefits of the formula are that by integrating the staggered angle difference, length normalization ratio and geometric distance into one, it effectively reflects the direction trend and connectivity difference between regions other than geometric proximity, providing a multi-dimensional quantitative basis for region aggregation judgment. Especially after introducing the |L a -L b | / ∑L k ratio term, it can highlight the abnormal connection behavior in local structure and enhance the formula's ability to identify "irregular aggregation" phenomena; Table 2: Indentation region feature parameter table:

[0030] Table 2 lists the center coordinates, angle values and connection lengths of two groups of key indentation regions, which are obtained by image processing and pairing calculation; The results are compared with the set reference: according to related field research, if the feature distribution difference value between regions is greater than 5.5, it is considered that the two regions belong to weakly coupled structures that are not directly related, so the value obtained in this calculation is 6.3975, which has exceeded the threshold value, and it is inferred that the two regions do not have obvious defect aggregation tendency, and need to be excluded in the aggregation analysis.

[0031] S402: Based on the region block boundary drawn by the defect aggregation region block coordinate atlas, the intersection coverage ratio of the aggregation block boundary surrounding area value, the internal defect distribution density value and the adjacent region is extracted, the aggregation level is divided according to the combination difference amplitude of the boundary surrounding area value and the internal defect distribution density value, the layer is filled and covered according to the corresponding color scale of the aggregation level, and the configuration aggregation distribution recognition result is obtained; The geometric area calculation operation is performed on each region block, the boundary surrounding area value in the image is calculated, the total number of pixel points in the region is accumulated, the defect distribution density value in the region block is calculated according to the area unit, the ratio between the defect point number and the area is obtained as the density parameter, and the adjacent buffer zone is set outside each aggregation region block. The coverage degree of the defect distribution in the adjacent region and the target block is compared, the intersection coverage ratio is calculated as the quantitative basis of the defect extension trend, and the surrounding area value of each aggregation block and the internal defect density value constitute a parameter pair. In the layer, the coordinate index is matched in turn, the combination difference amplitude is calculated, that is, the amplitude of the area expansion required for the density improvement in different regions. The aggregation region is divided into multiple aggregation level sections through the relative size of the combination difference, the level result is mapped and matched with the preset color scale atlas, different colors are filled in the aggregation region in the original layer according to the aggregation level, and the configuration aggregation distribution recognition result is obtained.

[0032] The acquisition steps of the PU artificial leather image quality detection and discrimination result are specifically: S501: Call the defect aggregation region marked in the configuration aggregation distribution recognition result, filter the aggregation blocks located at the edge position of the PU artificial leather, record the coordinate contour information and the center position point, extract the geometric offset direction and offset distance of the aggregation block to the image center position, and generate an edge aggregation offset relationship pattern; The aggregation blocks in the edge region of the PU artificial leather image are filtered, and the image boundary is set to a fixed pixel width as the filtering range. The coordinate range of the aggregation block and the edge region are judged for spatial overlap. If the center coordinate point or the contour boundary of the aggregation block overlaps with the image edge region, it is regarded as an edge aggregation block. The complete contour coordinate point set of the edge aggregation block is recorded, and the geometric center point coordinates of each aggregation block are calculated for subsequent geometric position relationship analysis. The center point of each aggregation block is connected to the image center in turn with the image center point as the reference datum point. The direction angle and line segment length of the connection line are measured to obtain the offset direction and offset distance of the aggregation block relative to the image center. The contour, center point, direction angle and distance data of the edge aggregation block are uniformly summarized and arranged in the pattern according to the distribution position of each aggregation block in the image edge to form an edge aggregation offset relationship pattern.

[0033] S502: Based on the edge aggregation offset relationship pattern, the position arrangement range of the aggregation block in space is obtained block by block, the aggregation number in the local area is calculated, and the aggregation number, distribution range and interval performance are combined to judge, and the configuration dense distribution area map is generated; The actual position data of the aggregation block in the image coordinate system is read block by block, the arrangement boundary in the horizontal and vertical directions is counted, the spatial coverage range of each aggregation area is formed, and a plurality of local area windows are demarcated in proportion according to the overall size of the image, the number of aggregation blocks contained in each local window is calculated, the proportion value between the actual occupied area of the aggregation block and the total area of the local area is recorded, the aggregation proportion of the area is obtained, and the center point distance between the aggregation blocks is compared item by item, the minimum distance, the average distance and the maximum distance are counted, and the interval description of the aggregation interval performance is formed. By jointly comparing the aggregation number, the area proportion and the interval performance, the area with prominent aggregation degree is identified, which is marked as the configuration dense distribution area, and the coordinate contour of the area is integrated to form the configuration dense distribution area map.

[0034] S503: Call the configuration dense distribution area map, detect the coincidence degree of the region boundary point and the PU artificial leather original edge line, and combine the boundary extension direction change trend and the boundary continuous form feature to judge whether the overlapping range presents stretching or breaking form, and generate the boundary form feature marking map; Each region contour point set is analyzed, the spatial overlap degree of its outer boundary curve and the PU artificial leather original edge line is compared, the shortest distance between the boundary point and the original edge line is judged, if the distance is within the preset pixel error range, it is considered that the two overlap, and the boundary extension trend is analyzed by combining the arrangement direction of the boundary point, the direction angle change value of the continuous boundary segment is calculated, whether the boundary line has extension deformation tendency is analyzed, and the continuity index of the boundary curve is extracted, the boundary breakpoint number, the maximum breaking distance and the average connection length are set to identify whether there is a boundary breaking form feature. The geometric characteristics, continuity and direction change of the boundary of each aggregation area are comprehensively analyzed, and the corresponding are marked as boundary stretching form, boundary breaking form or regular boundary form, and the boundary form feature marking map is generated.

[0035] S504: Call the boundary form feature marking map, jointly identify the region position of the three phenomena of aggregation offset, configuration dense and boundary change, mark the risk distribution range, the offset direction change trend of the internal structure and the edge form feature content of the corresponding position in turn, and generate the PU artificial leather image quality detection judgment result; The three types of morphological regions of the existing aggregation deviation, configuration density and boundary change in the image are identified in space, the image region with the three types of features is extracted according to the pixel coordinate intersection, the spatial coverage range of the risk distribution of the composite feature region is marked respectively, the texture arrangement trend in the region is extracted and the relative position change track of the center coordinates in multiple layers is analyzed, the change trend curve of the internal structure deviation direction is formed, the edge morphology is finely classified and processed, and the boundary change labels such as sudden edge, crack and bending are marked, the risk level label, the deviation direction arrow and the boundary morphology description information are embedded into each region with the three types of aggregation features in turn, the risk micro-defect region and the cause structure are completely marked, the image basis is provided for subsequent process control or product screening, and the PU artificial leather image quality detection and discrimination result is formed.

[0036] The above merely illustrates the specific embodiments of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A quality inspection method for PU artificial leather production line based on multimodal image recognition, characterized in that: The following steps are involved: S1: Acquire the first frame image of the inspection area of ​​the PU artificial leather production line, and sequentially collect the edge image under oblique visible light illumination, the temperature domain image from the infrared thermal image, and the return image formed by laser reflection. The three types of image response phenomena are annotated and compared, and the combination distribution relationship is identified to generate a multimodal response fusion layer; S2: Based on the multimodal response fusion layer, judging the visible light illumination, infrared thermal intensity and laser emission interval status, and generating a detection image response adaptation set; S3: calling the detection image response adaptation set, demarcating the uneven illumination area and the heat distribution edge band as potential micro-mark blocks, identifying the directional change of texture arrangement density, and generating an indentation positioning annotation map; S4: Call the indentation positioning annotation map, draw a map of the connection relationship between the centers of the defect areas, mark the area blocks with clustering trends, classify and judge the contours and dense performance characteristics of the clustered blocks, perform color separation and overlay of the layers according to the clustering level, and generate the configuration clustering distribution recognition results.

2. The PU artificial leather production line quality inspection method based on multimodal image recognition according to claim 1, characterized in that: The multimodal response fusion layer includes a texture response structure layer, a thermal domain edge layer, and a laser reflection response layer; the detection image response adaptation set includes an illumination adaptation parameter set, a thermal adaptation parameter set, and a laser rhythm calibration set; the indentation positioning annotation map includes a micro-trace distribution map, a texture directionality change map, and a grayscale transition feature map; the configuration aggregation distribution recognition result includes a defect connection map, an aggregation level layered map, and a contour dense classification map.

3. The PU artificial leather production line quality inspection method based on multimodal image recognition according to claim 1, characterized in that: The steps for obtaining the multimodal response fusion layer are specifically as follows: S101: Acquire the first frame image of the inspection area of ​​the PU artificial leather production line, and sequentially collect the grayscale changes in the edge area of ​​the image under oblique visible light illumination, the temperature response state of the position in the infrared thermal image, and the reflection brightness of the area in the laser reflection image. Based on the coordinate consistency of the pixel positions in the image, perform spatial correspondence processing on the sampling data of the three types of images to generate a position correspondence image dataset; S102: Based on the position-corresponding image dataset, determining whether there is a continuous boundary in the grayscale change state, whether the temperature response state shows an inhomogeneous change in a local range, and whether the reflective brightness is lower than a set recognition benchmark; marking the positions of multiple types of response areas according to the determination results, and generating a composite response cross position set; S103: Call the composite response cross position set to obtain the grayscale change amplitude, temperature change trend and reflection brightness difference corresponding to the position in the three types of images, identify the response performance degree of the position in the image based on the superposition distribution relationship between the three types of change values, and generate a multimodal response fusion layer.

4. The PU artificial leather production line quality inspection method based on multimodal image recognition according to claim 3, characterized in that: The steps for obtaining the detection image response adaptation set are specifically as follows: S201: Counting the number of spot edge jump locations within a unit area based on the spot edge jump areas in the multimodal response fusion layer. Combining the spatial arrangement characteristics of the hot zone turning points in the corresponding area and the brightness change rhythm in the laser return image, the visible light illumination state, infrared thermal intensity expression, and laser emission interval change trends in the area are determined. The three states are annotated at the corresponding layer locations to generate a layer state distribution annotation set. S202: Call the layer state distribution annotation set, and for each type of state annotation position, obtain the grayscale, temperature, and brightness jump directions of adjacent areas in the multimodal response fusion layer in turn, and compare the three types of directional change features at the same position to determine whether there is a response direction consistency deviation. If so, adjust the coordinates of the spatial projection point at the corresponding position in the layer to generate a detection image response adaptation set.

5. The PU artificial leather production line quality inspection method based on multimodal image recognition according to claim 4, characterized in that: The steps for obtaining the indentation positioning annotation map are specifically as follows: S301: calling the detection image response adaptation set, extracting the pixel intensity value sequence and the thermal pixel continuous value sequence in the layer, screening the overlapping blocks of the light intensity average value area and the thermal gradient boundary area, calculating the light and thermal joint dispersion index, and obtaining the micro-trace potential block coordinate set; S302: Based on the image segment region located by the micro-trace potential block coordinate set, extract the texture arrangement direction angle value, texture spacing density value, and continuous grayscale difference value within the region, determine the interval where the texture arrangement direction angle value changes and the texture spacing density value shows a decreasing trend, record the position segment of the continuous grayscale difference, and obtain the coordinate group of the texture segment with a consistent decreasing direction; S303: Based on the position index of the direction-consistent decreasing texture segment coordinate group in the image, call the original image layer and locate the corresponding image pixel area, expand the coordinate group boundary in equal proportion and calibrate it as the image target annotation layer, perform block coverage processing on the original image according to the pixel coverage range of the annotation layer, and obtain the indentation positioning annotation map.

6. The PU artificial leather production line quality inspection method based on multimodal image recognition according to claim 5, characterized in that: The light-thermal combined dispersion index adopts the following formula: ； in, Represents the light-heat joint dispersion index in the i-th layer block, M i Represents the total number of pixels in the i-th layer block, I ij Represents the light intensity value of the jth pixel in the i-th layer block, Represents the average value of the pixel intensity value in the i-th layer block, T ij Represents the heat value of the jth pixel in the i-th layer block, Represents the average value of the pixel heat value in the i-th layer block, W ij Represents the weight factor corresponding to the j-th pixel in the i-th layer block.

7. The PU artificial leather production line quality inspection method based on multimodal image recognition according to claim 5, characterized in that: The steps for obtaining the configuration aggregation distribution recognition result are specifically as follows: S401: Calling the annotated coordinate set of the defect area in the indentation positioning annotation map, extracting the coordinate value of the area center, the length value of the line connecting the centers, and the staggered distribution angle value, calculating the characteristic distribution difference value, screening the coordinate points of multiple intersecting nodes, and marking the boundaries of the area blocks where they are located to obtain the coordinate map of the defect cluster area blocks; S402: Based on the area block boundaries defined by the defect cluster area block coordinate map, extract the area value of the cluster block boundary enclosed, the internal defect distribution density value and the cross coverage ratio of the adjacent area, divide the clustering level according to the combined difference amplitude of the boundary enclosed area value and the internal defect distribution density value, and perform color separation and filling coverage on the layer according to the color code corresponding to the clustering level to obtain the configuration cluster distribution recognition result.

8. The PU artificial leather production line quality inspection method based on multimodal image recognition according to claim 7, characterized in that: The characteristic distribution difference value adopts the following formula: ； in, Represents the difference in feature distribution between the ath and bth center coordinate points, x a 、x b Respectively represent the horizontal coordinate values ​​of the center points of the ath and bth defect areas, y a 、y b Respectively represent the vertical coordinate values ​​of the center points of the ath and bth defect areas, θ a ,θ b Respectively represent the staggered distribution angle values ​​of the ath and bth defect areas, L a 、L b Respectively represent the length of the line between the ath and bth center points, Represents the sum of the lengths of the lines between centers in the sample, totaling M lines.

9. The PU artificial leather production line quality inspection method based on multimodal image recognition according to claim 1, characterized in that: The method further comprises step S5: S5: Calling the graded defect cluster areas in the configuration cluster distribution recognition result, extracting the cluster blocks and center offset distribution relationship patterns at the edge of the PU artificial leather, marking the risk range, structural offset trend and boundary morphological characteristics based on the position distribution, configuration density and boundary overlap, and generating the PU artificial leather image quality detection and judgment results; The PU artificial leather image quality detection and discrimination results include a risk area distribution map, a structure deviation trend map, and a boundary morphology feature map.

10. The PU artificial leather production line quality inspection method based on multimodal image recognition according to claim 9, characterized in that: The steps for obtaining the PU artificial leather image quality detection and judgment results are specifically as follows: S501: Calling the defect cluster area with the marked level in the configuration cluster distribution recognition result, screening the cluster blocks located at the edge of the PU artificial leather, recording the coordinate contour information and the center position point, extracting the geometric offset direction and offset distance from the cluster blocks to the center position of the image, and generating an edge cluster offset relationship pattern; S502: Based on the edge clustering offset relationship pattern, obtain the position arrangement range of the cluster blocks in space one by one, calculate the number of clusters and the area proportion in the local area, make a combined judgment on the number of clusters, distribution range and interval performance, and generate a configuration dense distribution area map; S503: calling the densely distributed configuration area map, detecting the degree of overlap between the area boundary points and the original edge line of the PU artificial leather, and combining the boundary extension direction change trend and the boundary continuous morphological characteristics to determine whether the overlapping range presents a stretched or broken morphology, and generating a boundary morphological feature marker map; S504: Call the boundary morphological feature marking map to jointly identify the regional locations where three types of phenomena exist: clustered offset, dense configuration, and boundary change. The risk distribution range of the corresponding locations, the offset direction change trend of the internal structure, and the edge morphological feature content are marked in sequence to generate the PU artificial leather image quality detection and judgment results.

Images