Residue detection method and system in closed steel drums based on visual inspection

Abstract

The present invention discloses a method and system for detecting residues in closed-end steel drums based on visual inspection. This method belongs to the interdisciplinary field of industrial nondestructive testing and computer vision. The method includes acquiring multi-angle images through dynamic light field control to generate high dynamic range images; applying a semi-global matching algorithm to construct a dense three-dimensional point cloud, fusing infrared heat absorption coefficients with visible light texture roughness to generate a three-dimensional model with material attributes; filtering out potential targets in non-residue areas based on structural features, and combining point cloud height variance with HSV hue gradient analysis to generate physical state discrimination parameters; training a convolutional neural network using a voxelized model and annular weld feature template to output an anti-interference residue segmentation mask; and expanding the image into a two-dimensional projection using cylindrical coordinates, and calculating the residue weight using a density mapping library. The present invention utilizes dynamic light field control and multispectral fusion technology, combined with three-dimensional point cloud reconstruction and a deep learning algorithm, to achieve accurate detection and quantitative analysis of residues in closed-end steel drums.

Description

Technical Field

[0001] The present invention relates to the technical field of intersection of industrial nondestructive testing and computer vision, and in particular to a method and system for detecting residues in closed steel drums based on visual detection. Background Art

[0002] The method for detecting residues in closed steel drums eliminates reflective interference from the inner wall of the steel drum by dynamically adjusting multi-angle light sources, and reconstructs a three-dimensional model with material attributes by combining HDR imaging and infrared / visible light dual-channel data. After intelligently identifying welds and reinforcing rib structures to eliminate interference areas, the residue is segmented using AI. Finally, the three-dimensional residual area cylinder is expanded into a two-dimensional projection, and the weight of the residue is accurately calculated using a pre-calibrated density-area mapping function.

[0003] Existing technologies mainly use visible light or depth cameras to obtain two-dimensional images or sparse point clouds, and combine them with texture segmentation or height mapping to achieve residue identification. Some solutions introduce infrared imaging to assist analysis.

[0004] These existing methods have the following technical defects: the accuracy of two-dimensional image segmentation is significantly reduced due to interference from strong reflective areas and welds inside the steel drum; three-dimensional modeling based on sparse point clouds is difficult to integrate multi-spectral material information, resulting in misjudgment of residue properties; existing technologies lack the ability to accurately unfold and spatially project complex surface structures, affecting the reliable estimation of the physical weight of the residue. Summary of the Invention

[0005] To solve the above problems, the present invention adopts dynamic light field control and multispectral fusion technology, combined with three-dimensional point cloud reconstruction and deep learning algorithm, to achieve accurate detection and quantitative analysis of residues in closed steel drums.

[0006] The above objectives can be achieved through the following solutions:

[0007] A method and system for detecting residues in closed steel drums based on visual inspection include collecting multi-angle images through dynamic light field control to generate high dynamic range images; applying a semi-global matching algorithm to construct a dense three-dimensional point cloud, fusing infrared thermal absorption coefficient and visible light texture roughness to generate a three-dimensional model with material attributes; filtering out potential targets in non-residue areas based on structural features, and generating physical state discrimination parameters by combining point cloud height variance and HSV hue gradient analysis; training a convolutional neural network using a voxelized model and annular weld feature template to output an interference-resistant residue segmentation mask; and expanding it into a two-dimensional projection image using cylindrical coordinates, and calculating the residue weight in combination with a density mapping library.

[0008] Optionally, the method of obtaining multi-angle light source feedback parameters inside a closed steel drum and using the multi-angle light source feedback parameters to generate dynamic light field distribution parameters includes: real-time monitoring of the position distribution of high-reflective areas in the original image set to obtain spatial coordinate mapping data of high-reflective areas; based on the spatial coordinate mapping data of the high-reflective areas, lowering the light intensity threshold of the corresponding orientation light source group; based on preset near-infrared band compensation parameters and visible light band compensation parameters, dynamic energy superposition is performed in combination with the light intensity threshold to generate dynamic light field distribution parameters.

[0009] Optionally, performing a semi-global matching algorithm on the high dynamic range image to obtain a dense three-dimensional point cloud model, and extracting material reflection properties based on the dense three-dimensional point cloud model includes: performing a semi-global matching algorithm on the high dynamic range image to obtain a dense three-dimensional point cloud model; separating the infrared reflection channel and the visible light reflection channel based on the dense three-dimensional point cloud model; mapping the intensity value of the infrared reflection channel to obtain the material thermal absorption coefficient; extracting the visible light reflection channel to obtain texture roughness; and fusing the material thermal absorption coefficient with the texture roughness to generate material reflection properties.

[0010] Optionally, the use of the three-dimensional reconstruction model with material attributes to obtain the potential residue target area includes: performing curvature skeleton extraction on the three-dimensional reconstruction model with material attributes to identify the linear distribution pattern of the reinforcement; performing multi-scale gradient analysis on the preset weld edge to construct an annular weld feature template; performing a Boolean subtraction operation on the linear distribution pattern and the annular weld feature template to output the filtered potential residue target area.

[0011] Optionally, the local variance characteristics and HSV color space gradient distribution analysis of the potential residue target area to generate residue material discrimination parameters includes: calculating the standard deviation of the height values ​​within the point cloud neighborhood of the potential residue target area to generate a flow morphology index of the liquid residue; extracting the hue mutation gradient along the normal direction of the residue edge in the HSV color space gradient distribution to generate a crack distribution index of the solid residue; correlating the flow morphology index of the liquid residue with the crack distribution index of the solid residue to output the residue material discrimination parameters.

[0012] Optionally, the convolutional neural network joint training of the residue material discrimination parameters and outputting the residue segmentation result mask includes: converting the three-dimensional reconstructed model with material attributes into a voxel grid input into the convolutional neural network; injecting the annular weld feature template into the convolutional neural network as a dynamic weighted mask; adjusting the category weight coefficient according to the residue material discrimination parameters; and jointly training the category weight coefficient with the convolutional neural network to output the residue segmentation result mask.

[0013] Optionally, based on the residue segmentation result mask, the three-dimensional reconstructed model with material attributes is unfolded to obtain a two-dimensional plane projection image, including: establishing a cylindrical coordinate system of the inner surface of the steel barrel with the center of the barrel bottom as the coordinate origin; unfolding the cylinder along the circumferential direction and maintaining the radial height mapping relationship; projecting the residue segmentation result mask onto the unfolded cylindrical coordinate system to generate a two-dimensional plane projection image.

[0014] Optionally, the calculation based on the two-dimensional plane projection image and the output of the residue weight data include: constructing a minimum circumscribed polygon of the residue outline in the two-dimensional plane projection image; calculating the physical projection area of ​​the minimum circumscribed polygon based on a preset pixel size-actual size conversion coefficient; and outputting the residue weight data based on the density parameter mapping library and the physical projection area.

[0015] Optionally, the method further includes: constructing a steel barrel structure archive based on the three-dimensional reconstructed model with material attributes; associating the residue weight data with the barrel body identification code of the steel barrel structure archive to generate risk analysis data including a residue distribution heat map and a safety level assessment report.

[0016] Based on the same inventive concept, the present invention also provides a residue detection system in closed steel barrels based on visual inspection, the system comprising: a light source feedback parameter acquisition module, used to obtain multi-angle light source feedback parameters inside the closed steel barrel, and generate dynamic light field distribution parameters using the multi-angle light source feedback parameters; a dynamic light field generation module, used to obtain an original image set using the dynamic light field distribution parameters, and perform fisheye distortion correction and HDR synthesis processing on the original image set to generate a high dynamic range image; a three-dimensional point cloud modeling module, used to perform a semi-global matching algorithm on the high dynamic range image to obtain a dense three-dimensional point cloud model, and extract material reflection properties based on the dense three-dimensional point cloud model; a material fusion reconstruction module, used to fuse the material reflection properties with the three-dimensional coordinate data of the dense three-dimensional point cloud model to generate a A three-dimensional reconstruction model; a potential residue positioning module, used to use the three-dimensional reconstruction model with material attributes to obtain a potential residue target area; a material feature analysis module, used to use the three-dimensional reconstruction model with material attributes to obtain a potential residue target area, perform local variance feature and HSV color space gradient distribution analysis on the potential residue target area, and generate residue material discrimination parameters; an intelligent segmentation module, used to perform convolutional neural network joint training on the residue material discrimination parameters, and output a residue segmentation result mask; a three-dimensional unfolding projection module, used to unfold the three-dimensional reconstruction model with material attributes based on the residue segmentation result mask, and obtain a two-dimensional plane projection map; a weight quantification module, used to calculate according to the two-dimensional plane projection map in combination with a preset density parameter mapping library, and output residue weight data.

[0017] Compared with the prior art, the present invention has the following advantages:

[0018] 1. This invention effectively suppresses the high reflectivity of the steel drum's internal metal surface by integrating multi-angle light source feedback and dynamic light field control technology. Furthermore, it combines fisheye distortion correction and HDR synthesis processing to generate high-quality images. A semi-global matching algorithm is used to construct a dense 3D point cloud model, achieving submillimeter-level 3D reconstruction. This enables more precise positioning of potential residue targets, avoids the error accumulation associated with traditional visual inspection on complex geometric surfaces, and significantly reduces false detection rates, ensuring high reliability of inspection results in industrial applications.

[0019] 2. This invention employs a dual-modal feature analysis strategy, utilizing both the thermal absorption coefficient of the infrared reflectance channel and the texture roughness of the visible light reflectance channel to extract surface material properties. Furthermore, local variance feature analysis combined with HSV color space gradient distribution technology simultaneously quantifies the flow pattern of liquid materials and the crack distribution of solid materials. This fusion of physical and visual features enables the system to adaptively identify diverse residue types, such as oil stains, solid sediments, and chemical slime, without the need for manual parameter presetting, addressing the poor generalization capabilities of traditional detection tools for heterogeneous materials.

[0020] 3. This invention unfolds a 3D model with material attributes into a 2D projection. Combining this with a pre-set density parameter mapping library, it accurately converts pixel area to actual physical projection area, ultimately outputting residue weight data. This data is then linked to the barrel identification code in the structural archive to automatically generate a residue distribution heat map and safety level report. This feature not only provides real-time quantitative indicators to support container cleaning decisions but also significantly simplifies the user process, avoiding errors caused by subjective factors in manual estimation.

[0021] 4. This invention utilizes a convolutional neural network to jointly train material characteristics and integrates real-time light source feedback parameters with annular weld feature templates to achieve adaptive segmentation mask calculation. This method, combined with dynamic light field balancing and intelligent algorithms, significantly shortens the inspection cycle and reduces reliance on professional human intervention. This makes the system easy to integrate into assembly line applications, significantly reducing the cost and workload of container cleaning and maintenance in industrial environments.

[0022] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0024] Figure 1 The figure is a flow chart of a method for detecting residues in closed steel drums based on visual inspection according to an embodiment of the present invention.

[0025] Figure 2 This is a dynamic light field control diagram of an embodiment of the present invention.

[0026] Figure 3 This is a material reflection property extraction diagram according to an embodiment of the present invention.

[0027] Figure 4 This is a diagram of steel drum inner wall structural feature recognition and filtering according to an embodiment of the present invention.

[0028] Figure 5 This is a material surface feature analysis diagram of an embodiment of the present invention.

[0029] Figure 6 It is a cylindrical expansion projection diagram of an embodiment of the present invention.

[0030] Figure 7 2 is a schematic structural diagram of a closed-end steel drum residue detection system based on visual inspection according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0032] Reference Figure 1 One embodiment of the present invention proposes a method for detecting residues in closed steel drums based on visual inspection. Dynamic light field control and multispectral fusion technology, combined with three-dimensional point cloud reconstruction and deep learning algorithm, can achieve accurate detection and quantitative analysis of residues in closed steel drums.

[0033] The method of this embodiment specifically includes:

[0034] Obtaining multi-angle light source feedback parameters inside the closed steel drum, and generating dynamic light field distribution parameters using the multi-angle light source feedback parameters;

[0035] Obtaining an original image set using the dynamic light field distribution parameters, and performing fisheye distortion correction and HDR synthesis processing on the original image set to generate a high dynamic range image;

[0036] performing a semi-global matching algorithm on the high dynamic range image to obtain a dense three-dimensional point cloud model, and extracting material reflection properties based on the dense three-dimensional point cloud model;

[0037] fusing the material reflection attribute with the three-dimensional coordinate data of the dense three-dimensional point cloud model to generate a three-dimensional reconstructed model with material attributes;

[0038] Obtaining a potential residue target area using the three-dimensional reconstructed model with material attributes;

[0039] The three-dimensional reconstruction model with material attributes is used to obtain a potential residue target area, and local variance characteristics and HSV color space gradient distribution analysis are performed on the potential residue target area to generate residue material discrimination parameters;

[0040] Performing convolutional neural network joint training on the residue material discrimination parameters, and outputting a residue segmentation result mask;

[0041] Expanding the three-dimensional reconstructed model with material attributes based on the residue segmentation result mask to obtain a two-dimensional plane projection image;

[0042] According to the two-dimensional plane projection diagram, combined with a preset density parameter mapping library, calculation is performed to output the residue weight data.

[0043] Specifically, dynamic light field control is as follows Figure 2 As shown in the figure, when acquiring the original image set based on the dynamic light field distribution parameters, it is first necessary to adjust the illumination angle and brightness value of multiple groups of controllable light sources according to the preset light source spatial distribution pattern and intensity parameters, and synchronously trigger the wide-angle industrial camera to collect multiple sets of original image data under different exposure parameters. To eliminate the inherent distortion of the lens, the original image is geometrically corrected using the pre-calibrated fisheye lens internal parameter matrix and distortion coefficient, with a focus on correcting the image edge distortion caused by radial distortion. In view of the strong reflection and shadow areas on the inner wall of the steel drum, the corrected images with different exposure amounts at the same viewing angle are subjected to HDR fusion processing based on a weighting function. Pixel-level radiometry reconstruction is performed according to the linear response range of the image brightness, and the final output is a high dynamic range image that retains complete light and dark details.

[0044] Optionally, obtaining multi-angle light source feedback parameters inside the closed steel drum and generating dynamic light field distribution parameters using the multi-angle light source feedback parameters includes:

[0045] Performing real-time monitoring of the position distribution of the high-reflective area in the original image set to obtain spatial coordinate mapping data of the high-reflective area;

[0046] Based on the spatial coordinate mapping data of the high-reflective area, lowering the light intensity threshold of the corresponding azimuth light source group;

[0047] Based on the preset near-infrared band compensation parameters and visible light band compensation parameters, dynamic energy superposition is performed in combination with the light intensity threshold to generate dynamic light field distribution parameters.

[0048] Specifically, first, in the collected original image set, by identifying the connected areas whose pixel brightness values ​​exceed the preset highlight threshold, the high-reflection areas are located in real time, and the pixel coordinates of the center points of these highlight areas are converted into the corresponding high-reflection area spatial coordinate mapping data using the camera internal and external parameter calibration results; secondly, according to the high-reflection area spatial coordinate mapping data, the azimuth angle interval to which it belongs is accurately located, and a control instruction is sent to the light source group corresponding to the interval to proportionally reduce the light intensity threshold of the light source group according to the highlight intensity; finally, the preset near-infrared band compensation parameters and visible light band compensation parameters are read, wherein the near-infrared band compensation parameters are compensation values ​​adjusted in combination with the ambient temperature, and the visible light band compensation parameters are compensation coefficients adjusted according to the ambient light intensity. The compensation parameters are dynamically weighted and summed with the light intensity threshold set in real time by the light source controller to generate the final dynamic light field distribution parameters, wherein the weighted superposition formula is expressed as:

[0049] ,

[0050] In this formula, Represents the light intensity threshold of the corresponding light source group after real-time reduction, in lux; Represents the preset visible light band compensation parameter, which is a dimensionless coefficient measured based on the ambient light intensity; Represents the preset near-infrared band compensation parameter, which is a dimensionless coefficient measured by the ambient temperature sensor; is the weight coefficient of the visible light band, is the weight coefficient of the near-infrared band, where , used to balance the contribution of the two bands to the light field; Represents the final output dynamic light field distribution parameter, in lux. The light source controller receives this command and instantly adjusts the output intensity of the light source group.

[0051] Exemplarily, this step is implemented at the inspection station of an industrial steel drum recycling station. A closed steel drum to be inspected is fixed on a rotating platform, and there are many oil residues and metal welds on its inner wall. Under the initial light source setting, the weld area on the inner surface of the barrel appears overexposed. The system collects an image sequence through the brightness of the first set of basic light sources, identifies the pixel coordinates of the three overexposed areas, and calculates and converts them into three-dimensional coordinates on the inner wall of the steel drum through the multi-eye vision geometric model. The system determines that two of the highlight areas are located in the irradiation direction numbered as light source group 3 and light source group 7. After the instruction is issued, the light intensity threshold of light source group 3 is dynamically adjusted from 1000 lux to 700 lux, and the threshold of light source group 7 is adjusted from 1000 lux to 750 lux. At the same time, the ambient temperature of the workstation is 25 degrees Celsius, which triggers the preset Parameter 0.95, the ambient light intensity triggers the preset Parameter 1.05. Setting , The dynamic light field distribution parameters for light source group 3 were calculated to be 0.6 × 700 × 1.05 + 0.4 × 700 × 0.95 = 686 + 266 = 952 lux; for light source group 7, the calculated values ​​were 0.6 × 750 × 1.05 + 0.4 × 750 × 0.95 = 472.5 + 285 = 757.5 lux. The light source controller configured the output energy accordingly and recaptured the image. Overexposure in the weld area was effectively suppressed, and the barrel wall details and residue were clearly imaged.

[0052] Optionally, performing a semi-global matching algorithm on the high dynamic range image to obtain a dense three-dimensional point cloud model, and extracting material reflection properties based on the dense three-dimensional point cloud model includes:

[0053] Performing a semi-global matching algorithm on the high dynamic range image to obtain a dense three-dimensional point cloud model;

[0054] Separate infrared reflection channel and visible light reflection channel based on dense 3D point cloud model;

[0055] Mapping the intensity value of the infrared reflection channel to obtain the material heat absorption coefficient;

[0056] Extracting the visible light reflection channel to obtain texture roughness;

[0057] The material heat absorption coefficient is combined with the texture roughness to generate a material reflection property.

[0058] Specifically, the material reflection property extraction is as follows Figure 3 As shown in the figure, the difference between infrared and visible light reflection characteristics is used to extract the surface material parameters of the object, thereby enhancing the accuracy of material analysis; the technical effect is reflected in the effective distinction between the material of the object and the residual matter, significantly improving the discrimination robustness and anti-interference ability. The specific operation is as follows: after obtaining the high dynamic range image, the semi-global matching algorithm is executed to process the image and calculate and generate a dense three-dimensional point cloud model; based on the dense three-dimensional point cloud model, the infrared reflection channel intensity value is separated according to the preset spectral channel segmentation parameters. and the visible light reflection channel intensity value ;in Collected by a near-infrared camera and normalized to a value range, The intensity value of the infrared reflection channel is collected by the visible light camera and normalized in the same way; Application mapping relationship:

[0059] ,

[0060] Generate material heat absorption coefficient , The preset calibration coefficients are obtained through material database training. Indicates heat absorption capacity; intensity value of visible light reflection channel Calculate the standard deviation of texture roughness within the point cloud neighborhood window ;The material heat absorption coefficient Standard deviation of texture roughness Perform fusion processing, the fusion formula is:

[0061] ,

[0062] in is the weight coefficient of the visible light band, is the weight coefficient of the near-infrared band, satisfying And defined by system initialization, Output the result of material reflection properties; finally, accurate generation of material reflection properties is achieved for subsequent 3D reconstruction.

[0063] For example, a closed steel drum with oily residue attached to the inner wall was processed on an industrial scrap steel drum refurbishment line. The high dynamic range image showed that there were spots of unknown residue on the bottom of the drum. The system performed a semi-global matching algorithm on the image to obtain a dense 3D point cloud model containing the 3D coordinates and reflection intensities of the point cloud points; the infrared channel and the visible light channel were separated according to the preset spectral segmentation parameters, and the average intensity of the spot area was measured. The value is 0.75, normalized to the range 0-1, average The value is 0.65; call The calibration factor is 0.833 for oily materials. =0.833×0.75=0.62475; while calculating the point cloud neighborhood window The standard deviation is 0.21; setting , , calculated by the fusion formula The system outputs the material reflection property value that matches the preset oily material threshold and is successfully identified as residue.

[0064] Optionally, obtaining a potential residue target area using the three-dimensional reconstruction model with material attributes includes:

[0065] Performing curvature skeleton extraction on the three-dimensional reconstructed model with material attributes to identify the linear distribution pattern of the reinforcement ribs;

[0066] Perform multi-scale gradient analysis on the preset weld edge to construct a circular weld feature template;

[0067] A Boolean subtraction operation is performed on the linear distribution pattern and the annular weld feature template to output a filtered potential residue target area.

[0068] Specifically, the identification and filtering of the inner wall structural features of the steel drum are as follows: Figure 4 As shown in the figure, first, the Gaussian curvature distribution of the model surface is calculated and the curvature continuous point set is extracted to form a curvature skeleton graph, where the curvature skeleton extraction formula is:

[0069] ,

[0070] and is the surface principal curvature value, which is obtained by differential geometry calculation of point cloud normal vector, and the unit is the reciprocal of millimeter; in the skeleton graph, the connected domains with curvature values ​​greater than the preset threshold and arranged in a straight line are identified and marked as the linear distribution mode of the reinforcement; secondly, the preset weld geometric characteristic parameters are loaded, and three different scales are used. The operator performs gradient amplitude detection on the weld edge and constructs a weld gradient feature pyramid. The layers of the gradient pyramid are weighted fused to generate a circular weld feature template containing weld width and position features. Finally, the three-dimensional space occupied by the linear distribution pattern and the circular weld feature template is converted into a binary voxel matrix and a Boolean subtraction operation is performed:

[0071] ,

[0072] in is the set of all voxels of the original model, is the set of voxels corresponding to the linear distribution pattern, is the voxel set corresponding to the annular weld feature template; the non-empty voxel area after the output operation is the potential residue target area after filtering.

[0073] For example, a chemical recycling steel drum had transverse ribs and circumferential welds on its inner wall, and solidified asphalt residue adhered to the bottom. The system read the 3D reconstructed model with material attributes and performed curvature calculations, identifying four continuous linear protrusions with longitudinal curvature values ​​exceeding a threshold. The widths fluctuated within an 8mm range, matching the pre-set rib features. It also detected a circular high-gradient band in the middle of the drum, which multi-scale gradient analysis confirmed as a 4mm-wide weld feature. After generating a weld voxel template, the system removed the linear rib and circumferential weld voxels from the overall voxel space. After filtering, only the irregularly shaped protrusions on the bottom of the drum remained, with their surface curvature exhibiting no directional pattern. These protrusions were identified as potential residue targets for subsequent analysis.

[0074] Optionally, performing local variance feature and HSV color space gradient distribution analysis on the potential residue target area to generate residue material discrimination parameters includes:

[0075] Calculating the standard deviation of height values ​​within the point cloud neighborhood of the potential residue target area to generate a flow morphology index of the liquid residue;

[0076] Extracting a hue mutation gradient along the normal direction of the residue edge in the HSV color space gradient distribution to generate a crack distribution index of the solid residue;

[0077] The flow morphology index of the liquid residue is correlated with the crack distribution index of the solid residue, and a residue material discrimination parameter is output.

[0078] Specifically, the surface characteristics of the material are analyzed as follows: Figure 5 As shown, first calculate the standard deviation of the height value in the point cloud neighborhood in a spherical space with a radius of 5mm , the formula is:

[0079] ,

[0080] In the formula The first The height value of each point is obtained by laser ranging and the unit is millimeter; is the average height of the neighborhood; is the number of point clouds in the neighborhood; when The flow morphology index of liquid residue is generated when the preset threshold value of 0.8 mm is exceeded :

[0081] ,

[0082] R is the neighborhood radius of 5mm as a normalization factor. At the same time, in the HSV color space, within the 10mm width of the residue edge, the hue value is sampled every 2mm along the edge normal direction. , calculate the absolute value of the hue gradient of adjacent sampling points :

[0083] ,

[0084] right The maximum value of the sequence is used to obtain the crack distribution index of the solid residue :

[0085] ,

[0086] Finally, the residue material discrimination parameters are generated by linearly weighted fusion of flow morphology index and crack distribution index. :

[0087] ,

[0088] and is the preset weight coefficient and , The value range is set from 0 to 1 to identify different material types.

[0089] For example, when inspecting a steel drum that was not thoroughly cleaned after shipping lubricating oil, the system detected a residual area with a diameter of 15 cm at the bottom of the drum. , calculated ; HSV analysis measured the maximum hue gradient correspond ; Set liquid weight , solid weight ,get =0.7×0.24+0.3×0.08=0.192. At the same time, another crystal residue was detected on the side wall. calculate , correspond ,get =0.7×0.06+0.3×0.35=0.147. The combined distribution characteristics of the values ​​are used to determine that the bottom of the barrel is liquid oil residue and the side wall is solid crystallization, and different cleaning strategies are triggered.

[0090] Optionally, the performing convolutional neural network joint training on the residue material discrimination parameters and outputting a residue segmentation result mask includes:

[0091] Converting the three-dimensional reconstructed model with material attributes into a voxel grid and inputting it into a convolutional neural network;

[0092] Injecting the annular weld feature template into a convolutional neural network as a dynamic weighted mask;

[0093] Adjusting the category weight coefficient according to the residue material discrimination parameter;

[0094] The class weight coefficients are jointly trained with a convolutional neural network to output a residue segmentation result mask.

[0095] Specifically, the 3D reconstruction model with material attributes is first converted into a voxel grid, and each voxel stores the position coordinates and material discrimination parameters. and reflection intensity data. Using three-dimensional The network architecture processes voxel data. The network input layer is configured with a three-channel tensor of 256×256×64 size, corresponding to height, material, and infrared features. After the circular weld feature template is converted into a binary mask, the weld position information is injected into the third convolutional layer of the network through the feature map weighting operation. Dynamically adjust the loss function weights to strengthen the learning of blurred areas of the material. The network finally outputs a segmentation mask with a resolution of 512×512. The activation function implements binary classification.

[0096] For example, when inspecting a 200L steel drum used to transport chemical raw materials, the system discretizes the 3D model into 1.57 million voxels and inputs them into the network. The weld seam recognition module accurately marks the two circumferential welds on the drum body and suppresses features in the middle of the network. Material analysis shows that the bottom attachment area of ​​the drum is Value 0.73 (viscous resin), sidewall The value is 0.41 (crystallization precipitation), and the corresponding loss weights are set to 1.28 and 1.11. After training, the output mask accurately separates the continuous thin film of resin from the granular crystal deposits, especially maintaining the continuity of the segmentation boundary at the intersection of the weld on the bottom of the barrel.

[0097] Optionally, the unfolding of the three-dimensional reconstructed model with material attributes based on the residue segmentation result mask to obtain a two-dimensional plane projection image includes:

[0098] Establish a cylindrical coordinate system on the inner surface of the steel drum with the center of the drum bottom as the coordinate origin;

[0099] Expand the cylinder along the circumferential direction and maintain the radial height mapping relationship;

[0100] The residue segmentation result mask is projected onto the unfolded cylindrical coordinate system to generate a two-dimensional plane projection image.

[0101] Specifically, the cylindrical expansion projection is as follows Figure 6 As shown, first take the center of the barrel bottom Establish a cylindrical coordinate system for the origin, The shaft extends along the axis of the barrel. represents the radial distance, Represents the circumferential angle; the expansion transformation formula is defined as:

[0102] ,

[0103]

[0104] in is the coordinate system axis, is the coordinate system axis, is the radius of the inner wall of the barrel, and the actual value is obtained by laser ranging; for each voxel point in the segmentation mask ),when When it is regarded as a valid point on the inner wall, it is mapped to the plane coordinate system And retain its material attribute value; the overlapping projection area is processed by the nearest neighbor interpolation algorithm. When multiple voxels are projected to the same The voxel data closest to the barrel wall is retained when calculating the coordinates; the final two-dimensional projection image is a grayscale image containing material attributes, the brightness represents the height of the residue, the chromaticity channel encodes the material type, and the spatial resolution is maintained. .

[0105] For example, when testing a 210L steel drum filled with polymer, the system measures the drum radius . Automatically exclude the barrel flange area during expansion processing and central area During circumferential expansion, the remaining annular polymer film at the barrel bottom is precisely expanded into a uniformly wide ribbon, while the sidewall sag marks exhibit a gradient height distribution. Compared to traditional perspective projection, this method generates a 2D image in which all geometric features maintain their actual scale, without distortion or magnification of the barrel bottom.

[0106] Optionally, the calculating according to the two-dimensional plane projection diagram in combination with a preset density parameter mapping library to output the residue weight data includes:

[0107] Constructing the minimum circumscribed polygon of the residue outline in the two-dimensional plane projection image;

[0108] Calculating the physical projection area of ​​the minimum circumscribed polygon according to a preset pixel size-actual size conversion coefficient;

[0109] Based on the density parameter mapping library and combined with the physical projection area, the residue weight data is output.

[0110] Specifically, first, in the two-dimensional plane projection image generated by the residue segmentation result mask, the convex hull algorithm is used to traverse the vertex set of the mask contour, and the outermost vertices are connected to form a closed polygon. This polygon must satisfy the geometric constraints of containing all residue pixels and having the smallest area, and is defined as the minimum circumscribed polygon. Secondly, the preset pixel size and actual size conversion coefficient is called. This coefficient is obtained through the camera calibration process, that is, the ratio of the actual size of the known calibration plate to the image pixel size. Multiply the pixel area of ​​the minimum circumscribed polygon by the square of the conversion coefficient to obtain the physical projection area. The formula is:

[0111] ,

[0112] Where, is the pixel area, is the pixel size conversion coefficient, is the physical projection area. Finally, query the density parameter mapping library, which uses the residue material discrimination parameter as the index to store density interval data and calculate the current residue density value through linear interpolation. , combined with the average residue height obtained from physical projection area and material analysis , output residue weight :

[0113] ,

[0114] in The value is calculated based on the mean height of the point cloud neighborhood, and the unit conversion coefficient This method is used to unify the millimeter and cubic centimeter dimensions. It accurately quantifies the area through geometric contours and combines material-adaptive density mapping and thickness analysis to reliably calculate the weight of residues.

[0115] For example, when inspecting a closed steel drum containing solidified asphalt, the system obtains a two-dimensional projection image showing that the residual area at the bottom of the drum is 1,200,000. is 0.08, calculate the physical projection area ,680. The residue discrimination parameter measured by material analysis is 0.92, and the density library matches the density of asphalt materials. ; Average thickness obtained by point cloud height analysis , the final calculated weight .

[0116] Optionally, the method further includes:

[0117] Building a steel drum structure archive based on the three-dimensional reconstructed model with material attributes;

[0118] The residue weight data is associated with the barrel identification code of the steel barrel structure archive to generate risk analysis data including a residue distribution heat map and a safety level assessment report.

[0119] Specifically, firstly, based on the three-dimensional reconstruction model with material attributes, the structural characteristic parameters of the steel drum, including the drum radius, height, number of reinforcement ribs and distribution position of the welds, are extracted to build a steel drum structure archive, where the structural characteristic parameters are calculated from the point cloud data using a geometric measurement algorithm; secondly, a unique drum identification code is assigned to each steel drum, which is generated by combining the production line number, inspection timestamp and material type code; then, the residue weight data is associated with the drum identification code and stored; then, based on the distribution density of the residue in the two-dimensional plane projection map, the Gaussian kernel density estimation algorithm is used to generate the residue distribution heat map, and the calculation formula is: :

[0120] ,

[0121] in Represents any coordinate point in the projection graph, For the The center coordinates of the residue regions, is the weight coefficient after normalization of the weight data of the area, The bandwidth parameter is adaptively calculated according to the size of the projection image, and the three-level gradient of the output heat map corresponds to low, medium and high risk areas respectively; finally, combined with the heat absorption coefficient and crack distribution index in the material reflection properties, a report containing cleaning priority and transportation risk level is output through the preset safety assessment rule matrix. The rows of the assessment rule matrix correspond to the residue type, and the columns correspond to the density classification of the heat map. The matrix elements are quantified and generated by industry safety standards.

[0122] For example, when a chemical plant inspected a batch of steel drums containing epoxy resin, the system measured the drum body radius as 300mm and the height as 900mm from the 3D reconstructed model, identified four longitudinal reinforcement ribs and two annular welds, generated a structural file and marked it with an identification code. The projection image showed a residue weight of 150g at the bottom of the barrel, with scattered debris on the sidewalls totaling 30g. The thermal map calculation set the bandwidth parameter to 10% of the barrel diameter, and the Gaussian kernel density indicated that the barrel bottom was a high-temperature risk area. Combined with material analysis, the residue was confirmed to be a highly viscous resin. The safety assessment matrix determined that this barrel required priority handling and prohibited long-distance transportation.

[0123] Based on the same inventive concept, Figure 7 As shown, the present invention also provides a residue detection system in a closed steel drum based on visual detection, the system comprising:

[0124] A light source feedback parameter acquisition module is used to obtain multi-angle light source feedback parameters inside the closed steel drum, and generate dynamic light field distribution parameters using the multi-angle light source feedback parameters;

[0125] A dynamic light field generation module is used to obtain an original image set using the dynamic light field distribution parameters, and perform fisheye distortion correction and HDR synthesis processing on the original image set to generate a high dynamic range image;

[0126] a 3D point cloud modeling module, configured to perform a semi-global matching algorithm on the high dynamic range image to obtain a dense 3D point cloud model, and extract material reflectance properties based on the dense 3D point cloud model;

[0127] A material fusion and reconstruction module, configured to fuse the material reflection properties with the three-dimensional coordinate data of the dense three-dimensional point cloud model to generate a three-dimensional reconstructed model with material properties;

[0128] A potential residue positioning module is used to obtain a potential residue target area using the three-dimensional reconstruction model with material attributes;

[0129] A material feature analysis module uses the three-dimensional reconstruction model with material attributes to obtain a potential residue target area, performs local variance feature and HSV color space gradient distribution analysis on the potential residue target area, and generates residue material discrimination parameters;

[0130] An intelligent segmentation module, configured to perform convolutional neural network joint training on the residue material discrimination parameters and output a residue segmentation result mask;

[0131] a three-dimensional unfolding projection module, configured to unfold the three-dimensional reconstructed model with material attributes based on the residue segmentation result mask to obtain a two-dimensional plane projection image;

[0132] The weight quantification module is used to calculate according to the two-dimensional plane projection diagram in combination with a preset density parameter mapping library and output the residue weight data.

[0133] It should be noted that the electrical connections between the above-mentioned units do not necessarily mean direct connections of lines. Indirect connections are applicable to the embodiments of the present invention as long as the purpose of the present invention is achieved. The above description is only an exemplary embodiment of the present invention and is not intended to limit the scope of the present invention.

[0134] That is, any equivalent changes and modifications made according to the teachings of the present invention are still within the scope of the present invention. Those skilled in the art will readily conceive of other embodiments of the present invention after considering the disclosure of the specification and practical truths. This application is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary technical means in the art not described herein.

Claims

1. A method for detecting residues in closed steel drums based on visual inspection, characterized in that: The method comprises: Obtaining multi-angle light source feedback parameters inside the closed steel drum, and generating dynamic light field distribution parameters using the multi-angle light source feedback parameters; Obtaining an original image set using the dynamic light field distribution parameters, and performing fisheye distortion correction and HDR synthesis processing on the original image set to generate a high dynamic range image; performing a semi-global matching algorithm on the high dynamic range image to obtain a dense three-dimensional point cloud model, and extracting material reflection properties based on the dense three-dimensional point cloud model; fusing the material reflection attribute with the three-dimensional coordinate data of the dense three-dimensional point cloud model to generate a three-dimensional reconstructed model with material attributes; The method comprises the following steps: utilizing the three-dimensional reconstructed model with material attributes to obtain a potential residue target area, comprising: extracting a curvature skeleton of the three-dimensional reconstructed model with material attributes to identify a linear distribution pattern of the reinforcement ribs; performing a multi-scale gradient analysis on a preset weld edge to construct a circular weld feature template; performing a Boolean subtraction operation on the linear distribution pattern and the circular weld feature template to output a filtered potential residue target area; Performing local variance feature and HSV color space gradient distribution analysis on the potential residue target area to generate residue material discrimination parameters, including: calculating the standard deviation of height values ​​within the point cloud neighborhood of the potential residue target area to generate a flow morphology index of the liquid residue; Extracting a hue mutation gradient along the normal direction of the residue edge in the HSV color space gradient distribution to generate a crack distribution index of the solid residue; correlating the flow morphology index of the liquid residue with the crack distribution index of the solid residue to output a residue material discrimination parameter; Performing convolutional neural network joint training on the residue material discrimination parameters, and outputting a residue segmentation result mask; Expanding the three-dimensional reconstructed model with material attributes based on the residue segmentation result mask to obtain a two-dimensional plane projection image; According to the two-dimensional plane projection diagram, combined with a preset density parameter mapping library, calculation is performed to output the residue weight data.

2. The method for detecting residues in closed steel drums based on visual inspection according to claim 1, characterized in that: The step of obtaining multi-angle light source feedback parameters inside the closed steel drum and generating dynamic light field distribution parameters using the multi-angle light source feedback parameters includes: Performing real-time monitoring of the position distribution of the high-reflective area in the original image set to obtain spatial coordinate mapping data of the high-reflective area; Based on the spatial coordinate mapping data of the high-reflective area, lowering the light intensity threshold of the corresponding azimuth light source group; Based on the preset near-infrared band compensation parameters and visible light band compensation parameters, dynamic energy superposition is performed in combination with the light intensity threshold to generate dynamic light field distribution parameters.

3. The method for detecting residues in closed steel drums based on visual inspection according to claim 1, characterized in that: The performing of a semi-global matching algorithm on the high dynamic range image to obtain a dense three-dimensional point cloud model, and extracting material reflection properties based on the dense three-dimensional point cloud model includes: Performing a semi-global matching algorithm on the high dynamic range image to obtain a dense three-dimensional point cloud model; Separate infrared reflection channel and visible light reflection channel based on dense 3D point cloud model; Mapping the intensity value of the infrared reflection channel to obtain the material heat absorption coefficient; Extracting the visible light reflection channel to obtain texture roughness; The material heat absorption coefficient is combined with the texture roughness to generate a material reflection property.

4. The method for detecting residues in closed steel drums based on visual inspection according to claim 1, characterized in that: The convolutional neural network joint training of the residue material discrimination parameters and outputting the residue segmentation result mask comprises: Converting the three-dimensional reconstructed model with material attributes into a voxel grid and inputting it into a convolutional neural network; Injecting the annular weld feature template into a convolutional neural network as a dynamic weighted mask; Adjusting the category weight coefficient according to the residue material discrimination parameter; The class weight coefficients are jointly trained with a convolutional neural network to output a residue segmentation result mask.

5. The method for detecting residues in closed steel drums based on visual inspection according to claim 1, characterized in that: The three-dimensional reconstruction model with material attributes is expanded based on the residue segmentation result mask to obtain a two-dimensional plane projection image, including: Establish a cylindrical coordinate system on the inner surface of the steel drum with the center of the drum bottom as the coordinate origin; Expand the cylinder along the circumferential direction and maintain the radial height mapping relationship; The residue segmentation result mask is projected onto the unfolded cylindrical coordinate system to generate a two-dimensional plane projection image.

6. The method for detecting residues in closed steel drums based on visual inspection according to claim 1, characterized in that: The calculating according to the two-dimensional plane projection diagram and combining with a preset density parameter mapping library to output the residue weight data includes: Constructing the minimum circumscribed polygon of the residue outline in the two-dimensional plane projection image; Calculating the physical projection area of ​​the minimum circumscribed polygon according to a preset pixel size-actual size conversion coefficient; Based on the density parameter mapping library and combined with the physical projection area, the residue weight data is output.

7. The method for detecting residues in closed steel drums based on visual inspection according to claim 1, characterized in that: The method further comprises: Building a steel drum structure archive based on the three-dimensional reconstructed model with material attributes; The residue weight data is associated with the barrel identification code of the steel barrel structure archive to generate risk analysis data including a residue distribution heat map and a safety level assessment report.

8. A system for detecting residues in closed steel drums based on visual inspection, applied to a method for detecting residues in closed steel drums based on visual inspection as claimed in any one of claims 1 to 7, characterized in that: The system comprises: A light source feedback parameter acquisition module is used to obtain multi-angle light source feedback parameters inside the closed steel drum, and generate dynamic light field distribution parameters using the multi-angle light source feedback parameters; A dynamic light field generation module is used to obtain an original image set using the dynamic light field distribution parameters, and perform fisheye distortion correction and HDR synthesis processing on the original image set to generate a high dynamic range image; a 3D point cloud modeling module, configured to perform a semi-global matching algorithm on the high dynamic range image to obtain a dense 3D point cloud model, and extract material reflectance properties based on the dense 3D point cloud model; A material fusion and reconstruction module, configured to fuse the material reflection properties with the three-dimensional coordinate data of the dense three-dimensional point cloud model to generate a three-dimensional reconstructed model with material properties; a potential residue positioning module, configured to obtain a potential residue target area using the three-dimensional reconstructed model with material attributes, comprising: performing curvature skeleton extraction on the three-dimensional reconstructed model with material attributes to identify the linear distribution pattern of the reinforcement; performing multi-scale gradient analysis on the preset weld edge to construct an annular weld feature template; performing a Boolean subtraction operation on the linear distribution pattern and the annular weld feature template to output a filtered potential residue target area; The material feature analysis module uses the three-dimensional reconstruction model with material attributes to obtain a potential residue target area, performs local variance feature and HSV color space gradient distribution analysis on the potential residue target area, and generates residue material discrimination parameters, including: calculating the standard deviation of height values ​​within the point cloud neighborhood of the potential residue target area to generate a flow morphology index of liquid residues; extracting a hue mutation gradient along the normal direction of the residue edge in the HSV color space gradient distribution to generate a crack distribution index of solid residues; correlating the flow morphology index of the liquid residue with the crack distribution index of the solid residue to output the residue material discrimination parameters; An intelligent segmentation module, configured to perform convolutional neural network joint training on the residue material discrimination parameters and output a residue segmentation result mask; A three-dimensional unfolding projection module, configured to unfold the three-dimensional reconstructed model with material attributes based on the residue segmentation result mask to obtain a two-dimensional plane projection image; The weight quantification module is used to calculate according to the two-dimensional plane projection diagram in combination with a preset density parameter mapping library and output the residue weight data.

Images