A method and system for visually detecting the processing quality of a hot-rolled bearing steel pipe end

Abstract

The application provides a hot-rolled bearing steel pipe end processing quality visual detection method and system, the method comprises the following steps: collecting the asynchronous event stream of the steel pipe end, the end face and the outer wall depth point cloud and the surface temperature field data; reconstructing the event stream by using the space-time domain cumulative function to obtain a clear pipe end edge image, eliminating heat wave refraction distortion and motion blur; registering the edge image and the depth point cloud at the pixel level, constructing a multi-modal fusion feature, and removing false defects with only gray changes according to the depth and gray gradient difference; inputting the effective geometric feature and the temperature field into a physical information neural network, and inversely calculating the normal temperature cold-state geometric size by taking the heat conduction equation as a constraint; finally, comparing the cold-state size with the tolerance, outputting an unqualified signal and recording information for the steel pipe with an out-of-tolerance or crack defect, otherwise, determining the qualification. The application effectively solves the problems of high-temperature interference, oxide skin misjudgment and lack of hot-state compensation in pipe end detection.

Description

Technical Field

[0001] This application relates to the field of steel pipe quality inspection technology, specifically to a visual inspection method and system for the processing quality of hot-rolled bearing steel pipe ends. Background Technology

[0002] The quality of the pipe end processing of hot-rolled bearing steel tubes directly affects the assembly accuracy and fatigue life of the bearings. Currently, visual inspection of the pipe end processing quality of hot-rolled bearing steel tubes mainly relies on manual sampling or machine vision inspection based on traditional industrial cameras.

[0003] However, the inventors discovered that the surface temperature of the rolled steel pipe reaches over 800℃, and the heat wave causes changes in the air refractive index, resulting in severe distortion and motion blur in images captured by traditional global shutter cameras, making it difficult to obtain clear pipe end edges. The iron oxide scale covering the surface of hot-rolled steel pipes has similar grayscale characteristics to defects such as cracks and scratches. Traditional algorithms based on grayscale thresholds or Hough space are prone to misreporting the oxide scale as a defect, leading to a large number of invalid re-inspections. Existing technologies mostly perform inspections after the steel pipe has cooled, making it impossible to achieve full online inspection in the hot state. Furthermore, there is a lack of compensation mechanisms for the effects of thermal expansion, making it difficult to accurately predict the dimensions of the cold-state finished product based on the hot-state inspection results. Traditional 2D vision cannot obtain depth information of defects, making it difficult to distinguish between surface scratches and deep cracks, affecting the determination of defect levels.

[0004] Therefore, there is an urgent need for a visual inspection method for the processing quality of hot-rolled bearing steel pipe ends that can resist high-temperature interference, eliminate oxide scale defects, and achieve hot-to-cold dimensional mapping. Summary of the Invention

[0005] In view of this, this application provides a visual inspection method and system for the processing quality of hot-rolled bearing steel pipe ends, in order to solve the technical problems existing in the pipe end inspection of hot-rolled bearing steel pipes, such as high temperature interference, misjudgment of oxide scale, and lack of thermal compensation.

[0006] Firstly, this application provides a visual inspection method for the processing quality of hot-rolled bearing steel pipe ends, including: An event camera is used to collect asynchronous event streams at the end of the steel pipe, a spectral confocal sensor array is used to collect depth point clouds of the end face and outer wall of the pipe, and an infrared thermal imager is used to collect temperature field data of the pipe end surface. For the acquired event stream, a spatiotemporal accumulation function is used to reconstruct a clear image of the pipe end edge, eliminating refractive distortion and motion blur caused by heat waves; The reconstructed pipe edge image is registered pixel-level with the acquired depth point cloud to construct a multimodal feature fusion vector. Based on the difference between the depth gradient and the gray-level gradient, areas with only gray-level changes and no significant depth changes are identified as false defects and removed. The geometric features obtained after the rejection process and the collected temperature field data are input into a pre-trained physical information neural network. Using the heat conduction equation as a physical constraint, the cold-state geometric dimensions of the steel pipe are calculated in reverse when it is cooled to room temperature. The obtained cold-state geometric dimensions are compared with the preset tolerance range. If they exceed the tolerance range or there are cracks with a depth exceeding the threshold, an unqualified signal is output and the location and type of the defect are recorded; otherwise, a qualified signal is output.

[0007] Secondly, this application provides a visual inspection system for the processing quality of hot-rolled bearing steel pipe ends, comprising: The multimodal data synchronous acquisition module is used to acquire asynchronous event streams at the end of the steel pipe using an event camera, acquire depth point clouds of the end face and outer wall of the pipe using a spectral confocal sensor array, and acquire temperature field data of the pipe end surface using an infrared thermal imager. The spatiotemporal decoupling and image reconstruction module is used to reconstruct a clear pipe end edge image from the acquired event stream using a spatiotemporal accumulation function, eliminating refractive distortion and motion blur caused by heat waves. The feature fusion and false defect removal module is used to perform pixel-level registration between the reconstructed pipe end edge image and the acquired depth point cloud, construct a multimodal feature fusion vector, and determine and remove regions with only gray-level changes and no significant depth changes as false defects based on the difference between depth gradient and gray-level gradient. The physical information neural network thermal deformation compensation module is used to input the geometric features obtained after the rejection process and the collected temperature field data into the pre-trained physical information neural network, and use the heat conduction equation as a physical constraint to invert and calculate the cold-state geometric dimensions of the steel pipe when it is cooled to room temperature. The quality judgment and output module is used to compare the obtained cold-state geometric dimensions with the preset tolerance range. If the dimensions exceed the tolerance range or there are cracks with a depth exceeding the threshold, an unqualified signal is output and the location and type of the defect are recorded; otherwise, a qualified signal is output.

[0008] The technical solution provided in this application fundamentally avoids the inherent problems of blurred imaging and low contrast in traditional cameras under high-temperature heat waves by introducing an event camera and a spectral confocal sensor, achieving clear imaging and accurate ranging of the edges of high-temperature objects. Through multimodal feature fusion and gradient difference analysis, it can effectively distinguish between harmless surface oxide scale and harmful deep cracks and scratches, avoiding a large number of production line downtimes and manual re-inspections caused by false defects, and significantly improving the accuracy and reliability of the detection results. Combining an infrared thermal imager and a physical information neural network, a size prediction model based on the heat conduction mechanism is established. This not only enables detection under hot conditions but also accurately predicts whether the dimensions of the finally delivered cold-state products are up to standard, solving the uncertainty brought about by thermal expansion and contraction to quality control and filling a technological gap. Integrating the advantages of 2D vision and 3D measurement, it can detect geometric defects such as edge defects and end face tilting, and identify and quantify the depth information of cracks and scratches, achieving a comprehensive and accurate assessment of pipe end quality. Attached Figure Description

[0009] Figure 1 A schematic diagram of a visual inspection method for the processing quality of hot-rolled bearing steel pipe ends is shown. Figure 2 A schematic diagram of a visual inspection system for the processing quality of hot-rolled bearing steel pipe ends is shown. Detailed Implementation

[0010] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0011] The hot-rolled bearing steel pipe production line targeted in this application has extremely stringent process requirements. Bearing steel pipes, such as those made of GCr15 steel, require subsequent cold working processes like upsetting and turning after hot rolling to form bearing rings. Cracks, folds, pits, or dimensional deviations at the pipe ends will lead to tool breakage, workpiece scrap, and even premature fatigue failure of the bearing during service. Therefore, the inspection station is located between the hot rolling mill exit and the cooling bed inlet, where the steel pipe temperature is still within the austenitizing temperature range, approximately 800℃ to 950℃.

[0012] The air near the surface of the high-temperature steel pipe is intensely heated, forming a turbulent boundary layer with a refractive index that changes drastically with time and space. In traditional area array cameras, the light path is continuously disturbed during exposure, leading to nonlinear distortion and dynamic blurring in the image. Edge positions can drift by tens of pixels between adjacent frames, making precise sub-millimeter-level measurements impossible. The iron oxide scale generated during hot rolling is sheet-like or layered, and its peeling edges and thickness variations produce gradient responses in 2D grayscale images that are highly similar to the image features of real cracks and scratches. Relying solely on grayscale thresholds or edge detection operators will result in a large number of over-inspections, forcing frequent production line shutdowns for manual visual verification. The linear expansion coefficient of the steel pipe is not a constant at high temperatures, and due to the complex geometry of the pipe ends and uneven heat conduction in contact with the rolls, its temperature field distribution exhibits significant three-dimensional non-uniformity. Traditional methods using a single temperature value for linear compensation cannot accurately predict ellipticity, outer diameter, and wall thickness deviations in the cold state, leading to contradictions such as "passing hot inspection but failing cold inspection" or "exceeding tolerances in hot inspection but actually passing cold inspection."

[0013] To achieve online full inspection under the above working conditions, a multi-physics field synchronous sensing platform was constructed.

[0014] The event camera unit employs a high-resolution (e.g., 1280x720 pixels) event vision sensor with a high dynamic range of >120dB and timestamp accuracy down to the microsecond level. The event vision sensor has a resolution of 1280x720 pixels. A narrow-band infrared filter is added to the front of the lens to suppress ambient visible light interference and enhance the edge contrast of the high-temperature object's own radiation. The center wavelength is matched to the peak thermal radiation of the steel pipe, such as 850nm or 940nm. At least two event cameras are arranged orthogonally or at a certain angle at the pipe end to ensure complete capture of edge events even with chamfered or angled pipe ends.

[0015] The spectral confocal sensor array employs a ring-shaped mechanical structure, consisting of a high-rigidity, internally water-cooled ring frame to withstand high-temperature radiation. N spectral confocal measurement probes, evenly distributed along the circumference, are mounted within the inner ring of the frame; N is determined by the diameter of the steel pipe. Each probe utilizes the principle of axial chromatic aberration. After passing through a dispersive objective lens, white light of different wavelengths is focused at different positions on the optical axis. When the measured surface is at the focal point of a certain wavelength of light, that wavelength is efficiently reflected back to the spectrometer, and the distance can be accurately determined by analyzing the peak wavelength of the spectrum. Its greatest advantage lies in its insensitivity to the tilt, material, color, and reflectivity of the measured surface, allowing direct penetration through oxide layer gaps to measure the contour of the metal substrate. The ring array is mounted on a high-precision servo translation stage, which, after the steel pipe reaches the detection position, feeds axially along the pipe end to complete a spiral or progressive scan of the pipe end face and outer wall within a certain length.

[0016] The infrared thermal imager employs a mid-wave infrared (3μm~5μm) or long-wave infrared (8μm~14μm) cooled thermal imager to obtain high-sensitivity temperature resolution for high-temperature targets, with a noise equivalent temperature difference of <20mK. An integrated online blackbody calibration source performs two-point calibration of the thermal imager at each steel pipe inspection gap to compensate for detector temperature drift, ensuring an absolute temperature measurement accuracy better than ±1% or ±2℃.

[0017] The master clock source, employing a GPS-synchronized IEEE 1588 precision time protocol synchronizer or a master clock pulse generated by an FPGA, sends hardware trigger signals to all sensors. The data acquisition architecture is as follows: the event camera pushes the event stream to the edge computing server in real time via a 10 Gigabit fiber optic network card; the spectral confocal sensor transmits depth data via CameraLink or CoaXPress interfaces; and the infrared thermal imager transmits the temperature matrix via GigE Vision. All data packets are timestamped by hardware before being loaded into memory.

[0018] See Figure 1 As shown, this application provides a visual inspection method for the processing quality of hot-rolled bearing steel pipe ends, including the following steps.

[0019] S1. An event camera is used to collect asynchronous event streams at the end of the steel pipe. A spectral confocal sensor array is used to collect depth point clouds of the end face and outer wall of the pipe. At the same time, an infrared thermal imager is used to collect temperature field data of the pipe end surface.

[0020] Specifically, the moment the steel pipe enters the inspection station via the roller conveyor and is gripped by the V-shaped positioning gripper, an external photoelectric sensor triggers the synchronization controller. The controller simultaneously sends pulse signals to the event camera, the spectral confocal sensor array, and the infrared thermal imager. A circular buffer is established based on the event camera's timestamp, storing in real-time the asynchronous event stream output by the event camera, the contour data points output by the spectral confocal sensor, and the temperature frames output by the infrared thermal imager. Due to the significant differences in their output frequencies—the event camera has an equivalent event rate >10MHz, the confocal sensor has a line contour rate >4kHz, and the infrared thermal imager has a frame rate of approximately 100Hz—time interpolation alignment can be used to ensure that each subsequent fusion calculation unit corresponds to the same physical instant. The time interpolation alignment method will be described later and will not be repeated here.

[0021] S2. For the acquired event stream, a spatiotemporal cumulative function is used to reconstruct a clear image of the pipe end edge, eliminating refractive distortion and motion blur caused by heat waves.

[0022] Specifically, since the refraction caused by heat waves is spatiotemporally dynamic, traditional integral images will inevitably become blurred at high temperatures. This step selects a sliding time window, the value of which is adaptive to the speed of the steel pipe movement and the intensity of the heat wave, typically between 10ms and 50ms. A spatiotemporal accumulation function is applied to the event stream within the window. The core idea of ​​the adaptive threshold function is: events occurring within a pixel's neighborhood in a very short time (e.g., 100μs) are only amplified if they exhibit a consistent polarity and spatiotemporal coherence; while heat wave noise events are temporally random and spatially isolated, thus being significantly attenuated by the adaptive threshold function. The final reconstructed image is a binary or grayscale edge image with sharp edges and a clean background, accurately reflecting the physical boundary of the pipe end at the end of the window.

[0023] S3. Perform pixel-level registration between the reconstructed pipe edge image and the acquired depth point cloud to construct a multimodal feature fusion vector. Based on the difference between the depth gradient and the gray-level gradient, regions with only gray-level changes and no significant depth changes are identified as false defects and removed.

[0024] Specifically, firstly, pixel-level registration is performed. Using the known geometric relationship of the circular array and camera calibration parameters, the depth point cloud is mapped onto the image coordinate system of the event camera through perspective projection transformation and polynomial distortion correction, thus achieving a one-to-one correspondence between 2D pixels and 3D spatial points.

[0025] Secondly, a fusion vector is constructed and a decision is made. A sliding window (e.g., 5x5 pixels) is defined on the image, and the mean of the Laplacian response within the window is calculated as the texture change. In the corresponding local region of the depth point cloud, a microplane is fitted and its normal vector change is calculated to approximate the radial depth gradient.

[0026] The judgment logic is as follows: a genuine surface crack has a slight height difference between the metal on both sides due to plastic deformation, and the depth gradient will reach the millimeter level; while oxide scale patches, although having high grayscale contrast, have a smooth height change relative to the substrate at the micrometer level, and the depth gradient is much smaller than the preset threshold, such as 0.1 mm. Based on this, false defect areas are accurately eliminated.

[0027] S4. Input the geometric features obtained after the rejection process and the collected temperature field data into the pre-trained physical information neural network, and use the heat conduction equation as a physical constraint to invert and calculate the cold-state geometric dimensions of the steel pipe when it is cooled to room temperature.

[0028] Specifically, after removing spurious defects, precise geometric feature parameters of the pipe end are extracted from the depth point cloud, such as outer diameter ellipticity, end face tilt angle, wall thickness, and the depth / length / width of the actual defects. These parameters constitute the thermal observation vector. Simultaneously, the temperature matrix provided by the infrared thermal imager, corrected for convective-radiative boundary conditions according to the heat conduction equation, serves as input. The pre-trained PINN model receives the thermal observation vector and temperature matrix. Its network structure contains multiple fully connected hidden layers, but crucially, its loss function incorporates steel-specific thermal properties, including thermal conductivity, specific heat capacity, density, and coefficient of linear expansion. During forward propagation, the network not only fits the thermal observation vector but also calculates the temperature change field corresponding to the cold-state geometry and compares it with the solution results of the heat conduction equation. Through gradient descent backpropagation, the network weights are adjusted to a state where they can accurately "remember" the complex nonlinear relationship between the current temperature field and geometric deformation. Finally, the network outputs the cold-state geometry, which is the predicted cold-state geometry, such as a cold state of 25℃.

[0029] S5. Compare the obtained cold geometric dimensions with the preset tolerance range. If the dimensions exceed the tolerance range or there are cracks with a depth exceeding the threshold, output an unqualified signal and record the defect location and type; otherwise, output a qualified signal.

[0030] Specifically, the predicted cold-state geometric dimensions are compared with the tolerance zones specified in GB / T or ASTM standards. The judgment rule is a logical "OR". If any cold-state dimension (such as outer diameter or wall thickness) exceeds the tolerance range, it is judged as unqualified; if there is a crack defect with a depth exceeding the specified threshold (such as 0.3 mm), it is judged as unqualified even if the dimension is qualified. The output not only includes qualified / unqualified signals, but also pushes detailed defect maps and digital measurement reports to the host computer MES via OPC UA or Profinet protocol. These reports include the 3D coordinates, dimensions, and type of defects, which are used to guide subsequent optimized sawing or finishing processes.

[0031] This application, by introducing an event camera and a spectral confocal sensor, fundamentally avoids the inherent problems of blurred imaging and low contrast in traditional cameras under high-temperature heat waves, achieving clear imaging and accurate ranging of the edges of high-temperature objects. Through multimodal feature fusion and gradient difference analysis, it can effectively distinguish between harmless surface oxide scale and harmful deep cracks and scratches, avoiding numerous production line downtimes and manual re-inspections caused by false defects, significantly improving the accuracy and reliability of inspection results. Combining an infrared thermal imager and a physical information neural network, a size prediction model based on the heat conduction mechanism was established. This not only enables inspection under hot conditions but also accurately predicts whether the dimensions of the final delivered cold-state product are up to standard, solving the uncertainty brought about by thermal expansion and contraction to quality control and filling a technological gap. Integrating the advantages of 2D vision and 3D measurement, it can detect geometric defects such as edge defects and end face tilting, and identify and quantify the depth information of cracks and scratches, achieving a comprehensive and accurate assessment of pipe end quality.

[0032] Further, in step S2, the asynchronous event stream output by the event camera is accumulated according to a preset time window, and each event is represented as... , Indicating the direction of brightness change, in event cameras, +1 (ON event) typically indicates that the brightness of that pixel has increased by more than a threshold compared to the previous event; -1 (OFF event) indicates that the brightness has decreased by more than a threshold. The edges of the tube often transition from the high-temperature, bright tube body to the relatively low-temperature, dark background, therefore OFF events dominate at the edges. Utilizing this polarity prior, the edge direction can be predicted during reconstruction.

[0033] In time interval Within, for each pixel position Calculate net event density: in, Time interval The set of events within, This is the Dirac function. It is an idealized impulse function used mathematically to "place" discrete, asynchronous event points onto a continuous image plane. In practical digital implementations, it is equivalent to a polarity-weighted summation of corresponding pixel coordinates in a two-dimensional histogram. For example, within a time window... Inside, pixels The integral value at this point can be the total count of all ON events within this window minus the count of OFF events.

[0034] Then, the net event density is filtered by an adaptive threshold function. The adaptive threshold is dynamically set according to the statistical distribution of event density within the time window to remove random noise events caused by heat waves. The final reconstructed tube edge image, i.e., the spatial-domain accumulation function, is: in, For the reconstructed edge image, This is an adaptive threshold function used to remove random noise events caused by heat waves.

[0035] The adaptive threshold of the function is set to the mean of the event density. Double the standard deviation, The value range is 2 to 3. The function is not a simple constant threshold, but a spatiotemporal filter. Its implementation can be described as follows: for each newly arrived event, instead of directly accumulating, its spatiotemporal neighborhood is first retrieved, for example, a 3x3 spatial region centered on the pixel, and a temporal neighborhood spanning the past 500 μs. The support of events within this neighborhood is calculated; if the number of events of the same polarity within the neighborhood exceeds a certain dynamic threshold, then... Output the raw value of the event, indicating that the event was generated by a continuously moving edge; if the events in the neighborhood are sparse and disordered, The output is then 0 or a very small value, thus suppressing the effect of this event on the final integral image. . contributions.

[0036] Time window The selection of the time window is crucial. If the window is too long, edge blurring caused by pipe movement will be integrated into the image; if the window is too short, the number of accumulated events will be insufficient, resulting in broken edges. The time window in this application... The speed of the steel pipe within the camera's field of view is dynamically determined. For example, the axial speed of the steel pipe can be obtained through an external encoder. Then the integration window length ,in, The allowable edge ghosting tolerance length is set to 0.1 mm. This dynamic integration strategy ensures that the reconstructed edge image has consistent spatial sharpness regardless of the production line speed.

[0037] This application precisely defines the conversion process from asynchronous data streams to 2D images, making the reconstruction process quantifiable and reproducible. An adaptive threshold function is explicitly introduced to remove random noise from heat waves, a key technique for event cameras to output clear edge images in high-temperature scenarios. Compared to simple fixed threshold processing, this method is more adaptable and produces cleaner edge extraction. Calculations are performed only at the pixel location where the event occurs; this data-driven approach avoids the repetitive scanning of all pixels common in traditional cameras, providing an algorithmic foundation for online real-time processing.

[0038] Further, in step S3, the edge image reconstructed in step S2 is... Compared with the depth point cloud acquired in step S1 Pixel-level registration is performed to associate both grayscale and depth information at each pixel location. Then, the grayscale gradient magnitude at each pixel location is calculated. and absolute value of radial depth gradient The grayscale gradient magnitude is calculated using the Laplacian operator to characterize texture changes; the absolute value of the radial depth gradient characterizes the degree of geometric shape change along the radial direction of the tube end. The Laplacian operator uses discrete convolution kernels to reconstruct the edge image. Filtering is performed. Since the edges of the image reconstructed by the event camera are already very sharp, the Laplacian operator exhibits a "positive-negative" zero-crossing characteristic on both sides of the edge. Taking its absolute value, both real cracks and pseudo-defect oxide scale boundaries will produce large values. The "radial" of the radial depth gradient is defined with the steel pipe axis as the center. The calculation process first locates the 3D point corresponding to the image pixel in the depth point cloud, then takes a small increment along the radial direction to calculate the depth difference. Because the Z-axis resolution of the spectral confocal sensor can reach sub-micron level, it can accurately capture depth changes caused by cracks and scratches, even if the change is only tens of micrometers.

[0039] Finally, the defect type is determined by the multimodal feature fusion vector: Use a dual-threshold decision logic to eliminate false defects: in, This is a genuine defect. It is a pseudo-defect; The depth gradient threshold ranges from 0.02 mm / mm to 0.10 mm / mm. This is the grayscale gradient threshold, with a value range of 20 to 50 (8-bit grayscale).

[0040] For adaptive weighting coefficients, satisfying , It's not a fixed constant, but rather a function that adaptively adjusts based on the tube-end temperature fed back from the infrared thermal imager. This is because higher temperatures result in stronger thermal radiation, leading to higher contrast (i.e., the signal-to-noise ratio of the grayscale gradient) in the event camera image; therefore, it's possible to appropriately reduce... Weighting; meanwhile, oxide scale growth is more active at high temperatures, leading to increased interference from pseudo-defects, thus requiring an increase in depth gradient. The weighting relies more on reliable 3D information for judgment.

[0041] When the grayscale gradient magnitude exceeds the grayscale gradient threshold and the absolute value of the radial depth gradient exceeds the depth gradient threshold, the pixel location is determined to have a real defect; when the grayscale gradient magnitude exceeds the grayscale gradient threshold but the absolute value of the radial depth gradient does not exceed the depth gradient threshold, the pixel location is determined to be a false defect and is removed.

[0042] For example, the edges of the oxide layer create a strong contrast in the image. However, its thickness is "thin" relative to macroscopic geometry, and its depth gradient exhibits small fluctuations rather than abrupt, step-like changes. For example, the depth variation rate at the edge of an oxide layer with a thickness of 20 μm is much smaller than that of a scratch with a depth of 100 μm; therefore, it can be classified as... This refers to a pseudo-defect in oxide scale.

[0043] For example, when a crack tears open the metal matrix, forming a V-shaped or U-shaped groove, the depth gradient will inevitably be significant. Meanwhile, in event camera imaging, the brightness on both sides of the crack exhibits contrast due to differences in emissivity to thermal radiation or geometric occlusion. Therefore, it is determined to be This refers to actual cracks / scratches.

[0044] For example, it has obvious depth changes. It will also cause changes in brightness and darkness in 2D images, and therefore will also be judged as a real defect.

[0045] This application, by decoupling the analysis of 2D texture information and 3D geometric information, provides a rigorous quantitative basis for distinguishing between oxide scale that has "color difference but no depth" and true defects that have both "color difference and depth." This avoids the problem of traditional algorithms where a single grayscale threshold is easily affected by lighting and oxide scale color. Even if oxide scale appears as a dark "defect" in the image, it will be logically filtered out as long as it does not cause a change in depth, greatly enhancing the detection algorithm's anti-interference ability and robustness. The clear logical expression makes the judgment process no longer a "black box," facilitating engineers' understanding and optimization of threshold parameters. and To adapt to different steel grades and working conditions.

[0046] Furthermore, the thermal geometric dimension observations obtained after step S3 are processed... The surface temperature field of the pipe end, which is synchronously acquired by the infrared thermal imager in step S1. As network input, a multi-layer fully connected Physics-Informed Neural Network (PINN) is constructed. This network takes the thermal geometry and temperature field as input and predicts the cold geometry. The output is the network's training loss function, which is a weighted sum of data fitting terms and physical constraint terms.

[0047] The loss function of the physical information neural network is: The first term is the data fitting term, which minimizes the error between the predicted cold-state size and the actual cold-state size, which is obtained through offline calibration. The second term is the physical constraint term, which uses the heat conduction equation as a regularization constraint to force the network output to conform to the laws of thermodynamics. The thermal conductivity of the steel pipe is... The weight coefficients for the physical constraint terms are used. By minimizing this loss function, the network is trained. After training, the cold-state geometric dimensions of the steel pipe when cooled to room temperature can be calculated by inputting the hot-state detection data.

[0048] Specifically, It is not a simple measured size in the hot state; it is an estimated cold state size after preliminary correction for thermal expansion. The calculation method is as follows: .in, It is the current hot size measured by spectral confocal measurement. It is the average linear expansion coefficient. It is the average temperature at the pipe end. This provides an initial, coarse regression target for network training, preventing the network from failing to converge in the early stages of training due to overly complex physical constraints. It acts like an "anchor point." The neural network is based on the input temperature field The final cold-state size is directly predicted through its internal weights and biases.

[0049] The second term is directly derived from the derivation of Fourier's law of heat conduction and the energy conservation equation. For transient heat conduction in a solid without an internal heat source, the temperature field... The rate of change is proportional to its second spatial derivative (Laplace operator), with the proportionality constant being the thermal diffusivity. The second term concerns the corresponding changes in geometric dimensions as the temperature field changes. The value calculated using the automatic differentiation mechanism of the neural network represents the cooling contraction rate "learned" by the network. It is the known thermal conductivity of the steel (which can vary with temperature). It is derived by performing second-order difference calculations on the temperature matrix acquired by the infrared thermal imager in space, representing the theoretically expected heat flux density and cooling trend under the current temperature distribution. When the network predicts the contraction rate... It deviates from the physical laws determined by the temperature field. When this happens, the value of the second term of the loss function increases, and the network is "punished" through backpropagation, forcing it to adjust its internal weights until its prediction results reach a balance between data fitting and physical laws.

[0050] The choice of value directly affects model performance. If If the value is too small, PINN degenerates into a regular neural network with poor extrapolation ability; if... If the value is too large, the model becomes too rigid and unable to fit the complex nonlinearities of the real world, such as the effects of latent heat of phase transitions. An adaptive adjustment strategy is adopted: in the early stages of training, a smaller value is used. (e.g., 0.01) allows the network to quickly fit the data; as the number of training iterations increases, the value is gradually increased. For example, by increasing the value by 0.01 every 100 rounds, with an upper limit of 1.0, the physical laws are gradually "injected" into the model, which can effectively prevent the network from getting stuck in local optima.

[0051] This application embeds the physical law of heat conduction into the training of a neural network, enabling the model's prediction process to not only rely on data fitting but also follow the fundamental physical laws governing the cooling process of materials. Compared to purely data-driven models (such as linear regression and ordinary neural networks), PINN can provide more accurate predictions that are closer to physical reality when dealing with marginal operating conditions where training data coverage is insufficient. Physical constraints act as a powerful form of prior knowledge, allowing the network to learn the correct "cooling kinetics" even with limited training samples. This avoids the need for cumbersome calibration of a large amount of data on steel pipes of different specifications and initial temperatures, thus improving the model's generalization ability to different operating conditions. By adjusting the weight coefficient λ, the importance of data fitting and physical constraints can be flexibly balanced, allowing the model to be optimized based on the quality of on-site data.

[0052] Further, the output event stream model of the event camera in step S1 is as follows: each pixel in the event camera independently detects the change in logarithmic brightness over time. When the absolute value of the change exceeds an adaptively adjustable contrast detection threshold, the pixel outputs an event, and each event is represented as follows: ,in, The polarity of the brightness change; contrast detection threshold. It is not a fixed value, but rather based on the surface temperature field of the pipe end synchronously acquired by the infrared thermal imager. Real-time adaptive adjustment, its adjustment function is: In the formula, The baseline threshold at room temperature. For temperature sensitivity coefficient, Take a positive value, ranging from 0.01 to 0.05 / ℃. This is the ambient reference temperature, such as 25℃.

[0053] The output conditions of the event camera can be uniformly expressed as: in, For pixels In time Logarithmic brightness change at the location; event camera only at Time output event When the temperature at the pipe end rises, causing heat waves that increase background brightness fluctuations, The corresponding increase allows for the adjustment of background brightness fluctuations caused by heat waves. Active suppression retains only valid events caused by real edge changes.

[0054] Specifically, each pixel of the event camera operates independently, continuously monitoring the logarithmic change in photocurrent. .when At that time, the pixel is reset. And output the event. In high-temperature scenarios, heat waves cause changes in the air refractive index, resulting in continuous, minute fluctuations in background radiance with a frequency of extremely high (>1kHz) and an amplitude of approximately 5% to 15%. If If fixed to a small value (such as 10% typical at room temperature), the heat wave will cause the event camera to output a massive amount of useless noise events in an "avalanche" fashion, instantly drowning out the signal representing the real edge and exhausting the transmission bandwidth.

[0055] Brightness fluctuations caused by heat waves With temperature gradient It is proportional to the square of the value. The higher the temperature, the more... The larger the amplitude, the greater the contrast threshold. To allow the sensor to "ignore" these fluctuations, the contrast threshold must be... Upgraded to higher The level. According to the Stefan-Boltzmann law, the radiant power of an object is proportional to the fourth power of its temperature. The amplitude of background brightness fluctuations is related to the change in radiant power, therefore its relationship with temperature is also approximately exponential or a high-power relationship. An exponential function is used. It can fit this nonlinear growth very well.

[0056] Offline experiments were conducted to determine the noise event rate of an event camera at different fixed thresholds by heating a standard blackbody to different temperatures and placing it in front of a fan that could generate controllable heat waves. Curve fitting was then used to find a method that could keep the noise event rate at an acceptable level. and The relationship, thereby determining .

[0057] Raising the threshold won't miss weak, real defect signals because genuine defects like cracks and scratches cause a step-like, one-time, and significant change in brightness at their edges (for example, the brightness change from a shiny metal surface to the interior of a dull crack can exceed 50%). This large change... Even at the threshold Even after the increase, it can still far exceed the threshold, thus being reliably triggered. This mechanism achieves a clever "high-pass filtering" effect: retaining large-amplitude edge signals while filtering out small-amplitude background noise.

[0058] The air turbulence caused by heat waves can lead to rapid but small fluctuations in background brightness. By linking the threshold to temperature, the trigger threshold is automatically increased when the steel pipe temperature is extremely high or the heat wave is intense. This allows background fluctuations below a threshold to be directly ignored, preventing invalid event output. This proactively suppresses noise at the sensor source, rather than passively filtering it later, significantly improving the signal-to-noise ratio of the event stream. Because a large number of useless heatwave noise events are not triggered, the amount of data output by the event camera is significantly reduced, lowering data transmission bandwidth and the computational burden on backend processing algorithms—crucial for high-speed production lines requiring real-time processing. This mechanism enables the event camera to dynamically adjust its sensitivity based on the actual temperature state of the target, achieving an optimal balance between sensitive capture of high-temperature target edges (producing significant brightness changes) and effective suppression of heatwave interference (producing minor brightness changes).

[0059] Furthermore, the number of probes in the spectral confocal sensor array in step S1 is determined by the following formula: in, The nominal outer diameter of the steel pipe. This refers to the effective measurement width of a single probe in the circumferential direction at the tube end, and the effective measurement linewidth of a single spectral confocal probe. Due to the limitations of the probe's measurement angle (such as a specular reflection angle within ±15°), the arc length that it can effectively receive reflected light on a curved tube wall is limited. The values ​​need to be determined by actual scanning of the standard sample column. The overlap rate of the measurement areas of adjacent probes. Range of values The probes are arranged at equal angular intervals along the circumference of the pipe end, and the angular interval between the probes in the circumferential direction is... , No. The measurement center angle of each probe is . This is a safety redundancy to ensure seamless splicing. If If the probes are aligned correctly, their fields of view will be exactly connected. However, any slight installation error or movement of the steel pipe will cause measurement "gaps". The overlap ratio can ensure coverage without increasing costs due to too many probes.

[0060] To ensure seamless stitching of depth point clouds across the entire circumference, measurement data from adjacent probes must satisfy continuity constraints within overlapping areas: in, For the first Depth measurement values ​​of each probe in polar coordinates This is a preset alignment error tolerance, used to ensure seamless stitching of depth point clouds across the entire circumference, typically ranging from 0.5 μm to 2 μm. If a discontinuity at a certain location exceeds this tolerance, the area is automatically marked as a suspected defect or probe calibration anomaly.

[0061] The implementation of continuity constraints in overlapping regions is the core of practical point cloud stitching algorithms. First, coarse registration is performed based on the known probe installation angles. , will the Point cloud data from each probe Rotate to the global coordinate system using a rigid body transformation. Then, in the overlapping region... Internal fine-grained registration is performed to find corresponding point pairs. Traditional ICP finds the nearest point, while the constraint here is enforced. This term is added as a penalty term to the objective function. It should be slightly greater than the Z-axis repeatability of the spectral confocal sensor. For example, if the repeatability is 1 μm, then... It can be set to 3μm to 5μm. If the difference is found to be consistently greater than 1 in the overlapping region... This will trigger an alarm, indicating that the probe may be loose or there may be abnormal reflection on the surface of the steel pipe, such as large pieces of peeling oxide scale.

[0062] This application, by precisely calculating the number and layout angle of the probes, physically ensures the complete acquisition of the 360° geometric topography of the pipe end, eliminating blind spots in the detection. The continuity constraint of the overlapping area is crucial to ensuring that point cloud data acquired by multiple independent probes can be accurately stitched into a unified, smooth 3D model. It is not only an algorithmic requirement for data stitching but also a physical calibration criterion during sensor installation and calibration. Only by meeting this constraint can the final depth point cloud model truly reflect the continuous topography of the pipe end, avoiding misjudging smooth surfaces as step-like defects due to stitching misalignment.

[0063] Furthermore, the time alignment of the multimodal data in step S1 is achieved through an interpolation function; Timestamps output by the event camera Based on this, cubic spline interpolation is used to interpolate the spectral confocal depth data. and infrared temperature data Interpolate to the same time axis; for depth data, in Synchronization value at time The following interpolation function is given: Spectral confocal depth data and infrared temperature data exist The synchronization value at time t is obtained through cubic spline interpolation: in, for The coefficients of the four nearest original sampling times Determined by minimizing the sum of squared interpolation errors: Similarly, the synchronized value of temperature data Obtained using the same method: Similarly, coefficient Determined by minimizing the sum of squared interpolation errors; Frequency of synchronous trigger clock source The Nyquist sampling theorem must be satisfied: in, These are the highest sampling frequencies of the event camera, the spectral confocal sensor, and the infrared thermal imager, respectively. Maximum allowed time resolution; trigger signal delay The time synchronization time must be less than 10μs to ensure that the time synchronization accuracy of the multimodal data is better than 100μs. If the synchronization accuracy exceeds this range, an automatic alarm will be triggered and the detection will be paused until recalibration is completed.

[0064] Specifically, linear interpolation assumes that the data changes linearly between two sampling points, which introduces significant sawtooth errors when the signal changes drastically. For steel pipe inspection, the depth data changes abruptly when crossing the defect edge; linear interpolation "smooths out" this abrupt change, leading to an underestimation of the defect depth. Cubic spline interpolation, on the other hand, constructs a cubic polynomial between two sampling points, ensuring the continuity of the first and second derivatives at the sampling points. This results in a smoother interpolation curve, better preserving the original characteristics of the signal, especially high-frequency details at the defect edge. The coefficients are obtained by solving a tridiagonal matrix equation system. This ensures the second-order smoothness of the overall curve.

[0065] Require Assuming the highest equivalent event rate of the event camera is 10MHz and the line profile output frequency of the spectral confocal sensor is 4kHz (i.e., =4000), the infrared thermal imager has a frame rate of 100Hz (i.e. =100), then =5ms. Therefore, the synchronization clock frequency is... At least 200Hz is required. However, in practical engineering, to ensure sufficient interpolation accuracy, frequencies much higher than this are used, such as 10kHz. Excessively high frequencies... There are no additional benefits, only increased costs. If the steel pipe moves at a speed of 2 m / s (2000 mm / s), a 10 μs time error results in a spatial misalignment of only 2000 mm / s · 10 μs = 0.02 mm. This order of magnitude is far smaller than the size of typical defects (e.g., above 0.1 mm) and also smaller than the spot diameter of a spectral confocal sensor (approximately 20 μm to 50 μm). Therefore, the trigger signal delay... The accuracy of less than 10μs ensures that the spatial registration error caused by time asynchrony is negligible, meeting the requirements of high-precision detection.

[0066] To achieve such high-precision synchronization, an FPGA is typically used as the main control unit. The hardware logic inside the FPGA can directly respond to external trigger signals and send synchronization pulses to all sensors with a deterministic delay on the order of nanoseconds. At the same time, the FPGA receives data streams from each sensor and uses its internal high-precision counter to add a hardware timestamp of the arrival time to each data packet, thereby avoiding the non-deterministic delays caused by software-layer scheduling.

[0067] During inspection, the steel pipe is in motion. If different sensors are not synchronized, the data they collect will correspond to different physical positions at the pipe end. This application ensures, through high-precision hardware synchronization and software interpolation algorithms, that every pixel or data point participating in subsequent multimodal fusion precisely corresponds to the state of the steel pipe end at the same time and position. This is the physical basis for achieving accurate registration and gradient comparison analysis. Precise time alignment is directly related to the accuracy of defect location marking. Microsecond-level time synchronization errors mean that on high-speed production lines, defect location errors can be controlled at the sub-millimeter or even micrometer level, which is crucial for subsequent precise removal or repair processes.

[0068] See Figure 2 As shown, this application also provides a visual inspection system for the processing quality of hot-rolled bearing steel pipe ends, which may include the following modules.

[0069] The multimodal data synchronous acquisition module is used to acquire asynchronous event streams at the end of the steel pipe using an event camera, acquire depth point clouds of the end face and outer wall of the pipe using a spectral confocal sensor array, and acquire temperature field data of the pipe end surface using an infrared thermal imager. The spatiotemporal decoupling and image reconstruction module is used to reconstruct a clear pipe end edge image from the acquired event stream using a spatiotemporal accumulation function, eliminating refractive distortion and motion blur caused by heat waves. The feature fusion and false defect removal module is used to perform pixel-level registration between the reconstructed pipe end edge image and the acquired depth point cloud, construct a multimodal feature fusion vector, and determine and remove regions with only gray-level changes and no significant depth changes as false defects based on the difference between depth gradient and gray-level gradient. The physical information neural network thermal deformation compensation module is used to input the geometric features obtained after the rejection process and the collected temperature field data into the pre-trained physical information neural network, and use the heat conduction equation as a physical constraint to invert and calculate the cold-state geometric dimensions of the steel pipe when it is cooled to room temperature. The quality judgment and output module is used to compare the obtained cold-state geometric dimensions with the preset tolerance range. If the dimensions exceed the tolerance range or there are cracks with a depth exceeding the threshold, an unqualified signal is output and the location and type of the defect are recorded; otherwise, a qualified signal is output.

[0070] The visual inspection system for the processing quality of hot-rolled bearing steel pipe ends provided in this application can be applied to the above-mentioned visual inspection method for the processing quality of hot-rolled bearing steel pipe ends. For relevant details, please refer to the above-mentioned method implementation method. Its implementation principle and technical effect are similar, and will not be repeated here.

[0071] It should be noted that the visual inspection system for the processing quality of hot-rolled bearing steel pipe ends provided in this application is only illustrated by the above-mentioned division of functional modules / units. In practical applications, the above functions can be assigned to different functional modules / units as needed, that is, the internal structure of the visual inspection system for the processing quality of hot-rolled bearing steel pipe ends can be divided into different functional modules / units to complete all or part of the functions described above. Furthermore, the visual inspection method for the processing quality of hot-rolled bearing steel pipe ends provided in the above-described method implementation is based on the same concept as the visual inspection system for the processing quality of hot-rolled bearing steel pipe ends; the specific implementation process is detailed in the above-described method implementation and will not be repeated here.

Claims

1. A visual inspection method for the processing quality of hot-rolled bearing steel pipe ends, characterized in that, include: An event camera is used to collect asynchronous event streams at the end of the steel pipe, a spectral confocal sensor array is used to collect depth point clouds of the end face and outer wall of the pipe, and an infrared thermal imager is used to collect temperature field data of the pipe end surface. For the acquired event stream, a spatiotemporal accumulation function is used to reconstruct a clear image of the pipe end edge, eliminating refractive distortion and motion blur caused by heat waves; The reconstructed pipe edge image is registered pixel-level with the acquired depth point cloud to construct a multimodal feature fusion vector. Based on the difference between the depth gradient and the gray-level gradient, areas with only gray-level changes and no significant depth changes are identified as false defects and removed. The geometric features obtained after the rejection process and the collected temperature field data are input into a pre-trained physical information neural network. Using the heat conduction equation as a physical constraint, the cold-state geometric dimensions of the steel pipe are calculated in reverse when it is cooled to room temperature. The obtained cold-state geometric dimensions are compared with the preset tolerance range. If they exceed the tolerance range or there are cracks with a depth exceeding the threshold, an unqualified signal is output and the location and type of the defect are recorded; otherwise, a qualified signal is output. The spatiotemporal accumulation function is: in, This is the asynchronous event stream output by the event camera. Indicates the direction of brightness change. Time interval The set of events within, For the Dirac function, An adaptive threshold function is used to remove random noise events caused by heat waves; The output event stream model of the event camera is as follows: in, For pixels In time The logarithmic change in brightness at that location; The contrast detection threshold is adaptively adjusted in real time based on the tube end temperature field acquired by the infrared thermal imager. The adjustment function is as follows: in, The baseline threshold at room temperature. For temperature sensitivity coefficient, For ambient reference temperature; event cameras only operate at... Time output event This allows for the control of background brightness fluctuations caused by heat waves. Active inhibition.

2. The visual inspection method for the processing quality of hot-rolled bearing steel pipe ends according to claim 1, characterized in that, The multimodal feature fusion vector is: in, For grayscale images, use the Laplacian operator to characterize texture changes; The radial depth gradient characterizes changes in geometric shape. These are adaptive weighting coefficients; The decision logic for eliminating pseudo-defects is as follows: in, This is a genuine defect. It is a pseudo-defect. For depth gradient threshold, This is the grayscale gradient threshold.

3. The visual inspection method for the processing quality of hot-rolled bearing steel pipe ends according to claim 2, characterized in that, The loss function of the physical information neural network is: in, These are observations of the thermal geometry. The cold-state geometry predicted by the network. The temperature field acquired by the infrared thermal imager. The thermal conductivity of the steel pipe is... These are the weighting coefficients for the physical constraint terms.

4. The visual inspection method for the processing quality of hot-rolled bearing steel pipe ends according to claim 3, characterized in that, The number of probes in a spectral confocal sensor array is determined by the following formula: in, The nominal outer diameter of the steel pipe. The effective measurement width of a single probe in the circumferential direction at the tube end. The overlap rate of the measurement areas of adjacent probes, with a value range of... The angular intervals between each probe in the circumferential direction are: , No. The measurement center angle of each probe is Its depth measurement value The overlapping area continuity constraint must be satisfied with adjacent probes: in, This is a preset alignment error tolerance, used to ensure seamless stitching of depth point clouds across the entire circumference.

5. The visual inspection method for the processing quality of hot-rolled bearing steel pipe ends according to claim 4, characterized in that, The time alignment of the multimodal data is achieved through an interpolation function; Timestamps output by the event camera Based on spectral confocal depth data and infrared temperature data exist The synchronization value at time t is obtained through cubic spline interpolation: in, for The coefficients of the four nearest original sampling times Determined by minimizing the sum of squared interpolation errors: The frequency of the synchronous trigger clock source satisfies: In the formula, These are the highest sampling frequencies of the event camera, the spectral confocal sensor, and the infrared thermal imager, respectively. Maximum allowed time resolution; trigger signal delay The time synchronization time must be less than 10μs to ensure that the time synchronization accuracy of multimodal data is better than 100μs.

6. A visual inspection system for the processing quality of hot-rolled bearing steel pipe ends, characterized in that, The system employs the visual inspection method for the processing quality of hot-rolled bearing steel pipe ends as described in claim 1, comprising: The multimodal data synchronous acquisition module is used to acquire asynchronous event streams at the end of the steel pipe using an event camera, acquire depth point clouds of the end face and outer wall of the pipe using a spectral confocal sensor array, and acquire temperature field data of the pipe end surface using an infrared thermal imager. The spatiotemporal decoupling and image reconstruction module is used to reconstruct a clear pipe end edge image from the acquired event stream using a spatiotemporal accumulation function, eliminating refractive distortion and motion blur caused by heat waves. The feature fusion and false defect removal module is used to perform pixel-level registration between the reconstructed pipe end edge image and the acquired depth point cloud, construct a multimodal feature fusion vector, and determine and remove regions with only gray-level changes and no significant depth changes as false defects based on the difference between depth gradient and gray-level gradient. The physical information neural network thermal deformation compensation module is used to input the geometric features obtained after the rejection process and the collected temperature field data into the pre-trained physical information neural network, and use the heat conduction equation as a physical constraint to invert and calculate the cold-state geometric dimensions of the steel pipe when it is cooled to room temperature. The quality judgment and output module is used to compare the obtained cold-state geometric dimensions with the preset tolerance range. If the dimensions exceed the tolerance range or there are cracks with a depth exceeding the threshold, an unqualified signal is output and recorded.

Images