A monitoring control system for an in-situ cured liner process based on machine vision

Abstract

This invention discloses a monitoring and control system for in-situ curing lining processes based on machine vision, belonging to the field of machine vision technology. Specifically, it includes: acquiring multi-band images of the inner wall through a surrounding array of active illumination imaging units; calculating and generating a multispectral response coefficient distribution map reflecting the chemical state of the resin; and spatially aligning and overlaying this map with a temperature field distribution map acquired by a thermal imager to identify non-uniform curing potential areas whose spectral characteristics are within the gel state range but whose temperature deviates from a preset window. Based on the spatial morphology and temperature deviation direction of these potential areas, it generates illumination parameter adjustment commands for the active illumination imaging units and power adjustment commands for the heating units, implementing coordinated control and feeding back the range of action of the commands and changes in the potential areas on a three-dimensional model. This invention achieves real-time identification and precise spatial intervention of mismatch defects during the curing process of pipe linings.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and more specifically to a monitoring and control system for an in-situ curing liner process based on machine vision. Background Technology

[0002] In-situ curing lining repair technology is a trenchless pipeline repair method. It involves inserting a resin-impregnated hose into the pipeline to be repaired, and then using heating and pressurization to cure the resin in situ, forming a new lining layer that tightly adheres to the old pipeline. This technology is widely used in municipal engineering, petrochemicals, and other fields due to its rapid construction, minimal environmental impact, and significant social benefits. During this process, the curing state of the resin directly determines the final mechanical properties and service life of the lining layer; therefore, real-time and precise monitoring and control of the curing process is crucial to ensuring repair quality.

[0003] Currently, monitoring of this process primarily relies on discrete temperature sensors and visible light imaging technology. Temperature sensors are deployed on the inner wall of the pipe or in the heating medium to acquire temperature data at localized points, indirectly inferring the progress of the resin curing reaction. Simultaneously, some technologies employ single-view cameras to capture images of the pipe's inner wall, assessing the resin's state by analyzing changes in visual features such as color and texture. These techniques all provide monitoring information about the curing process from a single physical dimension (temperature or vision), forming the basis of existing technologies.

[0004] However, existing discrete temperature sensors cannot acquire the complete temperature field distribution of the pipe's inner wall, making it difficult to detect areas of localized overheating or undercooling, and their spatial resolution is limited. Furthermore, single-dimensional monitoring (temperature only or image only) cannot capture the coupling relationship between the chemical state (such as gel formation) and the physical field (such as temperature) during resin curing. For example, temperature data alone may not be sufficient to identify situations where the temperature meets the standard but the resin has not fully gelled due to material inhomogeneity or insufficient light; while visible light images alone may not be able to identify potential defect areas where visual characteristics conform to the gel state but the actual temperature has deviated from the ideal reaction window. This problem of single-dimensional, spatially discontinuous, and uncorrelated monitoring information makes it difficult for existing technologies to accurately identify potential defect areas that may affect the final curing uniformity in real time, thus limiting further improvements in the precision of the repair process control. Summary of the Invention

[0005] The purpose of this invention is to provide a monitoring and control system for the in-situ curing lining process based on machine vision, solving the following technical problems:

[0006] Existing monitoring methods, due to limited spatial resolution and lack of multi-dimensional information fusion, cannot simultaneously acquire high-resolution temperature field distribution and spectral information reflecting chemical state during the pipeline lining curing process. This makes it difficult to identify potential non-uniform curing areas caused by temperature and reaction process mismatch in real time and accurately, thus limiting the control precision of the repair process.

[0007] The objective of this invention can be achieved through the following technical solutions: A monitoring and control system for in-situ curing lining processes based on machine vision, comprising: The synchronous surround acquisition module is used to acquire synchronous inner wall images of multiple active illumination imaging units arranged around the inside of the pipe. The active illumination imaging unit includes independently modulated multi-band light sources and image sensors. The multispectral distribution generation module is used to calculate the spectral response coefficients of the resin surface in the field of view of each active illumination imaging unit to light of each wavelength band. Combined with the position coordinates of the active illumination imaging unit, a multispectral response coefficient distribution map is generated on the surface of the pipe 3D model. The field map fusion and alignment module is used to spatially align and overlay the multispectral response coefficient distribution map and the pipe inner wall temperature field distribution map acquired by the thermal imager on the surface of the pipe three-dimensional model. The non-uniform region identification module is used to identify regions in the superimposed distribution map whose spectral response coefficients are within the gel-state characteristic range but whose temperature values ​​deviate from the preset curing temperature window, and these regions are defined as non-uniform curing potential regions. The dual-mode instruction generation module is used to generate illumination parameter adjustment instructions for the active illumination imaging unit and power adjustment instructions for the heating unit based on the spatial morphology and temperature deviation direction of the non-uniform curing potential area. The instruction execution and feedback module is used to send lighting parameter adjustment instructions and power adjustment instructions to the corresponding units, and to mark the scope of the instruction execution and the changes in the non-uniform curing potential zone on the updated 3D model surface of the pipeline.

[0008] As a further aspect of the present invention: the specific process of generating a multispectral response coefficient distribution map on the surface of the three-dimensional model of the pipeline in the multispectral distribution generation module is as follows: Extract the grayscale values ​​of pixels on the resin surface region in the image acquired by each active illumination imaging unit under illumination by light sources of different wavelengths, calculate the ratio of the grayscale value of each pixel in each wavelength to the factory-calibrated brightness value of the corresponding wavelength light source, and use it as the single-band spectral response coefficient of that pixel. Based on the internal calibration parameters of the active illumination imaging unit, the two-dimensional coordinates of the pixel points are converted into three-dimensional coordinates of the surface of the pipe three-dimensional model. For each triangular facet on the surface of the pipe three-dimensional model, all active illumination imaging units covering this triangular facet are searched. The single-band spectral response coefficients calculated by each active illumination imaging unit are weighted and averaged, with the weight being the line-of-sight angle and distance from the active illumination imaging unit to the triangular facet. The weighted average single-band spectral response coefficients of each triangular facet are combined in each band to form the multispectral response coefficient vector of that triangular facet. The multispectral response coefficient vectors of all triangular facets constitute the multispectral response coefficient distribution map covering the inner wall of the pipe.

[0009] As a further aspect of the present invention: the specific process of spatial alignment and data overlay in the field map fusion alignment module is as follows: Acquire the temperature field distribution map of the inner wall of the pipe collected by an independent thermal imager array; each data point of the temperature field distribution map contains three-dimensional coordinates and temperature value, establish the mapping relationship between the coordinates of the center point of the triangular facet on the surface of the three-dimensional model of the pipe and the coordinates of the data points in the temperature field distribution map, and assign the temperature value of the nearest data point to the corresponding triangular facet; For triangular facets where the nearest temperature data point is not found, spatial interpolation is performed using the temperature values ​​of adjacent triangular facets to complete the temperature value. After assignment and interpolation, each triangular facet simultaneously possesses a multispectral response coefficient vector and a temperature value attribute. The multispectral response coefficient vector and temperature value attribute of each triangular facet are stored together to form a superimposed distribution map data structure.

[0010] As a further aspect of the present invention: the specific process for identifying potential non-uniform curing regions in the non-uniform region identification module is as follows: Preset a reference range and temperature window range for the multispectral response coefficient vector corresponding to the ideal gel state of the resin. Traverse each triangular facet in the superimposed distribution map and determine whether the multispectral response coefficient vector of the triangular facet falls within the reference range. For triangular facets whose multispectral response coefficient vectors fall within the reference range, determine whether the temperature value of the triangular facet is within the temperature window range, and mark triangular facets whose multispectral response coefficient vectors fall within the reference range but whose temperature values ​​deviate from the temperature window range as initially selected triangular facets. Adjacent initial triangular patches in the cluster space form independent connected regions. The geometric center, spatial extension size, and statistical distribution characteristics of temperature values ​​within each connected region are calculated. Connected regions that meet the preset spatial size conditions and whose internal temperature statistical distribution has the same deviation direction are defined as non-uniform curing potential regions.

[0011] As a further aspect of the present invention: the process of defining the connected regions that meet the preset spatial size conditions and whose internal temperature statistical distribution has the same deviation direction as the non-uniform curing potential region is as follows: Calculate the volume of the connected region. If the volume of the connected region is greater than the preset volume threshold, it is determined that the spatial size condition is met. Calculate the average temperature value of all triangular facets in the connected region. If the average value is lower than the lower limit of the temperature window, the temperature deviation direction is below the window direction. If the average value is higher than the upper limit of the temperature window, the temperature deviation direction is above the window direction. When the volume of a connected region is greater than a preset volume threshold, and the temperature deviation direction within the connected region is consistent, the connected region is defined as a non-uniform curing potential region.

[0012] As a further aspect of the present invention: the specific process of generating lighting parameter adjustment instructions in the dual-mode instruction generation module is as follows: For a non-uniform curing potential region, analyze the specific band component values ​​in the multispectral response coefficient vector of the non-uniform curing potential region that cause it to fall into the gel state characteristic range, query the band configuration table of multi-band light sources in the active illumination imaging unit, and determine the corresponding independently controllable band light source. Based on the temperature deviation direction of the non-uniform curing potential region, the spectral adjustment direction is determined. When the temperature is below the lower limit of the temperature window, the spectral adjustment direction corresponds to the spectral characteristics of liquid resin in subsequent imaging; when the temperature is above the upper limit of the temperature window, the spectral adjustment direction corresponds to the spectral characteristics of solid resin in subsequent imaging. Based on the spectral adjustment direction, calculate the illumination intensity ratio of one or more specific band light sources that need to be enhanced or weakened, and generate illumination parameter adjustment instructions that include the active imaging unit identifier, target band, and intensity adjustment ratio.

[0013] As a further aspect of the present invention: the specific process of generating the heating unit power adjustment command in the dual-mode command generation module is as follows: Based on the spatial geometric center coordinates of the non-uniform curing potential zone, locate one or more nearest heating units in the layout diagram of the heating units inside the pipe, and determine the number of affected heating units based on the spatial extension dimensions of the non-uniform curing potential zone. Based on the statistical distribution characteristics of temperature values ​​and the direction of temperature deviation within the non-uniform curing potential zone, the overall temperature correction is calculated, and the overall temperature correction is distributed to each identified heating unit according to the distance, forming the individual power adjustment amount for each heating unit. The individual power adjustment amount is converted into a power adjustment command format that the heating unit controller can recognize. The power adjustment command includes the heating unit identifier and the power adjustment value.

[0014] As a further aspect of the present invention: the specific process by which the scope of the instruction execution and the changes in the non-uniform solidification potential zone are marked on the updated 3D pipeline model surface in the instruction execution and feedback module is as follows: After the lighting parameter adjustment command and power adjustment command have been executed for one control cycle, the synchronous inner wall image is re-acquired and updated multispectral response coefficient distribution map and temperature field distribution map are generated to identify new non-uniform curing potential areas. Compare the spatial positions of the non-uniform solidification potential areas identified after the update with those identified in the previous control cycle. On the surface of the three-dimensional model of the pipeline, use the first visual symbol to mark the newly emerging non-uniform solidification potential areas, and use the second visual symbol to mark the non-uniform solidification potential areas whose range has shrunk or disappeared. The spatial range affected by all lighting parameter adjustment commands and power adjustment commands within the current control cycle is highlighted using third-vision symbols.

[0015] The beneficial effects of this invention are: This invention simultaneously acquires multi-band active illumination images and temperature field data, generating and fusing multispectral response coefficient distribution maps and temperature field distribution maps on the surface of a 3D pipeline model. This constructs a high-resolution spatially coupled data field that simultaneously includes spectral chemical state and physical temperature attributes. Based on this, the technical solution can identify non-uniform solidification potential areas where the spectral response indicates a gel state but the temperature value deviates from the window, according to a preset gel-state spectral characteristic range and solidification temperature window. This achieves precise spatial positioning of "state-temperature" mismatch areas that cannot be detected by traditional single-dimensional monitoring methods. Furthermore, based on the specific spatial morphology and temperature deviation direction of the potential area, targeted illumination parameter adjustment commands and heating power adjustment commands are generated to modulate the spectral excitation conditions of imaging units at specific locations and correct local heating intensity, respectively. This anomaly identification and dual-mode collaborative control mechanism based on multi-dimensional fusion perception directly binds control commands to the identified specific anomaly type and spatial location, thereby achieving closed-loop control of the solidification process from global monitoring to precise local intervention, improving the control accuracy and result uniformity of the entire repair process. Attached Figure Description

[0016] The invention will now be further described with reference to the accompanying drawings.

[0017] Figure 1 This is a system schematic diagram of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Please see Figure 1 As shown, this invention is a monitoring and control system for an in-situ curing lining process based on machine vision, comprising: The synchronous surround acquisition module is used to acquire synchronous inner wall images of multiple active illumination imaging units arranged around the inside of the pipe. The active illumination imaging unit includes independently modulated multi-band light sources and image sensors. The multispectral distribution generation module is used to calculate the spectral response coefficients of the resin surface in the field of view of each active illumination imaging unit to light of each wavelength band. Combined with the position coordinates of the active illumination imaging unit, a multispectral response coefficient distribution map is generated on the surface of the pipe 3D model. The field map fusion and alignment module is used to spatially align and overlay the multispectral response coefficient distribution map and the pipe inner wall temperature field distribution map acquired by the thermal imager on the surface of the pipe three-dimensional model. The non-uniform region identification module is used to identify regions in the superimposed distribution map whose spectral response coefficients are within the gel-state characteristic range but whose temperature values ​​deviate from the preset curing temperature window, and these regions are defined as non-uniform curing potential regions. The dual-mode instruction generation module is used to generate illumination parameter adjustment instructions for the active illumination imaging unit and power adjustment instructions for the heating unit based on the spatial morphology and temperature deviation direction of the non-uniform curing potential area. The instruction execution and feedback module is used to send lighting parameter adjustment instructions and power adjustment instructions to the corresponding units, and to mark the scope of the instruction execution and the changes in the non-uniform curing potential zone on the updated 3D model surface of the pipeline.

[0020] In a preferred embodiment of the present invention, the specific process of generating a multispectral response coefficient distribution map on the surface of the pipe three-dimensional model in the multispectral distribution generation module is as follows: Multiple active illumination imaging units (AIMs) are deployed around the inside of the pipe to acquire synchronous images of the inner wall. For example, in a pipe repair scenario with an inner diameter of 600 mm, eight AIMs can be evenly arranged along the circumference of the pipe. Each imaging unit integrates four narrow-band light sources with independently modulated luminous intensity, whose center wavelengths are located in the 450 nm blue band, 550 nm green band, 650 nm red band, and 850 nm near-infrared band, respectively. It is also equipped with a high dynamic range industrial image sensor. For each acquired image of the inner wall, the pixel area covered by the resin surface needs to be segmented from the image. This segmentation process can be based on the inherent reflectivity difference between the resin material and the pipe inner wall background at specific wavelengths, achieved by setting a grayscale threshold. Subsequently, for each pixel in the resin surface area, the grayscale value generated under individual illumination by the four different wavelength light sources is extracted. These grayscale values ​​are integers between 0 and 255, reflecting the reflectivity of the resin surface to that wavelength of light.

[0021] To eliminate the influence of slight differences in the luminous intensity of the light sources of different imaging units and to establish a standardized metric that can be compared across units, the original grayscale values ​​need to be normalized. Specifically, the measured grayscale value of a pixel in a certain wavelength band is divided by the factory-calibrated luminance value of the corresponding active illumination imaging unit's light source in that wavelength band. This calibrated luminance value is pre-measured and stored using a spectroradiometer under standard laboratory conditions, and the unit is typically watts per square meter. The ratio obtained through this division is defined as the single-band spectral response coefficient of that pixel in this specific wavelength band. This coefficient is a dimensionless value that more purely expresses the response characteristics of the resin material itself to light in that wavelength band, decoupled from the absolute intensity of the light source.

[0022] The next step is to unify the image coordinates of all pixels into the same three-dimensional coordinate system of the pipeline. Each active illumination imaging unit undergoes precise intrinsic and extrinsic parameter calibration before deployment. Intrinsic parameters include focal length, principal point coordinates, and lens distortion coefficients; extrinsic parameters define the precise position and orientation of the imaging unit's optical center in the pre-defined world coordinate system, i.e., the pipeline's three-dimensional coordinate system. Using these calibration parameters, through a back-projection model, the two-dimensional row and column coordinates of each pixel in the image can be converted into a three-dimensional ray originating from the imaging unit's optical center, passing through the pixel, and pointing into the pipeline. This ray intersects the surface of the pipeline's three-dimensional model, pre-constructed through laser scanning or design drawings; the intersection point represents the three-dimensional coordinates of that pixel. The surface of the pipeline's three-dimensional model is typically discretized from hundreds of thousands of tiny triangular facets.

[0023] For each triangular facet on the surface of the 3D model of the pipe, the system needs to find all active illumination imaging units (AIMs) whose fields of view can cover that facet. This is achieved by determining whether the line of sight from the optical center of the imaging unit to the center of the facet is unobstructed. Assume a specific facet is observed simultaneously by three AIMs. Then, for this facet, each imaging unit that observes it will provide a set of four single-band spectral response coefficients. However, due to different observation angles and distances, these coefficients from different viewpoints will differ. To fuse multi-view information and obtain a more robust estimate, these coefficients need to be weighted and averaged. The weights are set based on two geometric factors: first, the line of sight angle, i.e., the cosine of the angle between the line of sight from the imaging unit to the facet and the normal direction of the facet; the smaller the angle, the greater the weight, because a direct view better represents the true properties of surface reflection; second, the spatial distance; the closer the distance, the greater the weight, and the reciprocal of the square of the distance is usually used as one of the weighting factors. Multiply these two factors and normalize them so that the sum of the weights of all imaging units covering the triangular facet is 1, thus obtaining the final weight of each imaging unit. Then, for each band, multiply the single-band spectral response coefficients from different imaging units by their respective weights and sum them to obtain the weighted average single-band spectral response coefficient of the triangular facet in that band.

[0024] Finally, the weighted average single-band spectral response coefficients calculated for a triangular facet across all four bands are arranged in ascending order of wavelength, forming a 4-dimensional feature vector. This is the multispectral response coefficient vector of the triangular facet. Each triangular facet on the surface of the pipe's 3D model is processed using this method to calculate such a vector. Organizing the vectors of all the triangular facets according to their spatial relationships creates a multispectral response coefficient distribution map that completely covers the inner surface of the pipe, with each triangular facet accompanied by a 4-dimensional spectral vector. This distribution map serves as the foundation for subsequent condition identification.

[0025] In another preferred embodiment of the present invention, the specific process of spatial alignment and data overlay in the field map fusion alignment module is as follows: The acquisition of the temperature field distribution map relies on a separate sensor array: multiple thermal imagers arranged inside the pipe. For example, a thermal imager ring can be arranged every 500 mm along the pipe's axial direction, with four thermal imager probes evenly distributed on each ring, forming a thermal imager array. Each thermal imager probe, while collecting temperature data, also records the three-dimensional spatial coordinates corresponding to that temperature data point through built-in ranging or in conjunction with an external positioning system. Therefore, the thermal imager array outputs a dense point cloud dataset, where each data point contains three coordinate values ​​and the temperature value measured at those coordinates. The temperature value is typically in degrees Celsius, with an accuracy of up to 0.5 degrees Celsius. This point cloud dataset constitutes the original temperature field distribution map of the pipe's inner wall.

[0026] The first step in fusion alignment is establishing a spatial coordinate mapping relationship. The surface of the pipe's 3D model is composed of triangular facets, each with its own 3D coordinates of its center point. Simultaneously, the temperature field distribution map consists of a large number of discrete temperature data points. To assign a temperature value to each triangular facet, a "representative point" needs to be found for each facet in the temperature point cloud. The most common method is to use a nearest neighbor search algorithm. Specifically, for the center point of any triangular facet, the Euclidean spatial distance from it to all data points in the temperature field distribution map is calculated. After traversing and comparing, the temperature value of the nearest temperature data point is directly assigned to this triangular facet as its initial temperature attribute. This process ensures that each triangular facet obtains a temperature value from physical measurement that is spatially closest.

[0027] However, due to the limited deployment density of thermal imagers or localized obstructions, some triangular facets may lack temperature data points within a certain radius. For these facets, spatial interpolation algorithms are needed to estimate their temperature values. A typical approach is linear interpolation. Centering on the facet to be interpolated, multiple adjacent facets whose temperature values ​​have been successfully obtained using the nearest neighbor method are identified, for example, eight adjacent facets. Then, a weighted average is calculated based on the temperature values ​​of these adjacent facets and the distance from their center points to the center of the facet to be interpolated. The closer the adjacent facets are, the greater their contribution to the interpolation result. Through this interpolation calculation, the temperature attributes of all facets on the surface of the entire 3D pipeline model can be completed, forming a continuous, void-free temperature field coverage.

[0028] After assigning and interpolating the temperatures of all triangular facets, each facet possesses two types of attributes: a multispectral response coefficient vector (a 4-dimensional array) obtained from the multispectral distribution generation module, and a temperature value (a single-precision floating-point number) obtained from the fusion of the temperature field distribution map. For ease of subsequent processing, these two attributes need to be stored together. A data structure can be created for each facet in computer memory or a database. This structure contains a unique identifier for the facet, its vertex coordinates, normal vector, and two key data fields: a 4-bit floating-point array to store the multispectral response coefficient vector, and a single-precision floating-point number to store the temperature value. All facet structures are stored sequentially or indexed by their spatial location, ultimately forming a complete dataset containing geometric, spectral, and temperature information. This dataset is the overlaid distribution map data structure, achieving pixel-level fusion of spectral and temperature information on the same spatial basis, providing a precise data foundation for subsequent identification of regions where the spectrum and temperature do not match.

[0029] In another preferred embodiment of the present invention, the specific process of identifying the non-uniform curing potential area in the non-uniform area identification module is as follows: First, two key technical benchmarks need to be established: one is the reference range of the multispectral response coefficient vector corresponding to the ideal gel state of the resin, and the other is the ideal curing reaction temperature window range. These benchmark ranges are not arbitrarily set, but are based on prior material experiments and process verification. For example, for a commonly used epoxy resin lining material, standard spectral response data at four wavelengths (450 nm, 550 nm, 650 nm, and 850 nm) are collected through curing experiments under controlled laboratory conditions when the resin is in a completely ideal gel state. Statistical analysis of repeated experimental data allows for the determination of a multidimensional numerical range. For example, at 450 nm, the normalized response coefficient may be concentrated between 0.15 and 0.25; at 850 nm, it may be concentrated between 0.45 and 0.55. These four wavelength ranges together constitute a four-dimensional "box"-shaped reference range. Simultaneously, based on resin curing reaction kinetics studies, the temperature range where the gelation reaction is most uniform and complete is determined, such as 80°C to 85°C; this range is then set as the temperature window range.

[0030] The identification process begins with traversing the superimposed distribution map data structure. This data structure contains the geometric information, multispectral response coefficient vector, and temperature value of each triangular facet on the surface of the 3D model of the pipe. The traversal operation involves sequentially accessing each triangular facet. For the currently accessed triangular facet, its stored multispectral response coefficient vector of length 4 is read first. Then, each component value in the vector is compared with the upper and lower limits of the preset reference range for the corresponding band. If all four component values ​​of a vector fall within the preset range of their respective bands, the multispectral response coefficient vector of that triangular facet is determined to fall within the gel-state reference range. For example, if the vector value of a triangular facet is [0.18, 0.30, 0.22, 0.49], and it is compared with the preset ranges of the four bands [0.15-0.25, 0.28-0.35, 0.18-0.26, 0.45-0.55], if each condition is met, this step is passed.

[0031] For triangular facets identified through spectral analysis, temperature conditions are then assessed. The temperature value of the facet is read and compared to a preset temperature window range. This temperature window range is typically defined by a lower limit and an upper limit, for example, a lower limit of 80 degrees Celsius and an upper limit of 85 degrees Celsius. If the temperature value of the facet is greater than or equal to the lower limit and less than or equal to the upper limit, it is considered to be within the ideal temperature window. If the temperature value is less than 80 degrees Celsius or greater than 85 degrees Celsius, it is considered to deviate from the temperature window range. Only triangular facets whose multispectral response coefficient vectors fall within the gel-state reference range but whose temperature values ​​deviate from the temperature window range are marked as "initial selection triangular facets." This means that these regions exhibit gel-state characteristics in their spectral features, but their ambient temperature is unsuitable for the stable formation or further development of this state, constituting potential anomalies.

[0032] The initially selected triangular facets are discretely distributed in 3D space. To identify anomalous regions with practical engineering significance rather than sporadic noise points, spatial clustering analysis is required. The clustering algorithm employs a region growing method based on spatial adjacency relationships. Specifically, starting with any unclassified initially selected triangular facet, it is used as a seed for a new connected region. All adjacent triangular facets sharing edges with this facet are checked; if any of these adjacent facets is also an initially selected facet, it is included in the current connected region. Then, using the newly included facet as the new base point, all its adjacent facets are recursively checked, and initially selected facets that meet the criteria are included. This process continues until all the adjacent facets of the boundary facets of the current connected region are no longer initially selected facets, thus forming an independent connected region where all internal facets are spatially adjacent to each other. Afterward, the algorithm searches for new seeds among the remaining unclassified initially selected facets, repeating the above process until all initially selected facets are assigned to a connected region. Ultimately, this may result in several connected regions of varying sizes and shapes.

[0033] For each identified connected region, a series of geometric and physical characteristic parameters need to be calculated for further screening. First, its geometric center is calculated, which is the arithmetic mean of the coordinates of the center points of all triangular facets within the connected region. This three-dimensional coordinate point represents the center position of the region. Second, its spatial extension dimensions are calculated. A practical method is to calculate the maximum span of the connected region in the three main directions of the pipe: axial, circumferential, and radial. For example, by finding the maximum and minimum values ​​of the coordinates of all triangular facet vertices in the pipe's axial direction, the difference between the two is the axial length. The same method can be used to calculate the circumferential arc length and radial thickness. Furthermore, the volume of the connected region needs to be calculated. Since the region is composed of a set of triangular facets, its total volume can be approximated as the sum of the volumes of the tiny triangular prisms corresponding to all the constituent facets. Multiplying the area of ​​each triangular facet by its tiny thickness in the pipe's radial direction yields the volume unit contributed by that facet; summing all units gives the total volume. For example, a connected region might be calculated to have a volume of 0.8 cubic centimeters. Simultaneously, the statistical distribution characteristics of the temperature values ​​within the region are analyzed, including calculating the average, standard deviation, maximum, and minimum values ​​of the temperature values ​​for all triangular facets.

[0034] After obtaining the aforementioned characteristic parameters, the final definition judgment is performed. The judgment is based on a combination of two preset conditions: spatial size condition and temperature deviation direction consistency condition. The spatial size condition typically sets a volume threshold, such as 0.5 cubic centimeters, to filter out regions that are too small or may be insignificant due to measurement noise or minor inconsistencies. If the volume of a connected region is greater than this preset volume threshold, it is considered to meet the spatial size condition.

[0035] The consistency condition for temperature deviation direction needs to be determined step by step. First, calculate the average temperature value of all triangular faces within the connected region. Then, compare this average value with a preset temperature window range. If the average value is lower than the lower limit of the temperature window (80 degrees Celsius), the overall temperature deviation direction of the connected region is determined to be "below the window direction"; if the average value is higher than the upper limit of the temperature window (85 degrees Celsius), the deviation direction is determined to be "above the window direction". Second, it is necessary to check whether the temperature deviation direction within the region is consistent. This can be determined by analyzing the distribution of temperature values ​​within the region. For example, for a region with an average temperature of 76 degrees Celsius, check whether the temperature values ​​of all its triangular faces are all below 80 degrees Celsius, or whether most of them are below 80 degrees Celsius and only a very few are slightly above 80 degrees Celsius but still below the window. If some triangular faces within the region have temperatures significantly below the lower limit while others are above the lower limit or even within the window range, it may mean that the region does not meet the "consistency" requirement. Consistency can be determined by calculating the proportion of triangular facets with temperature values ​​below the lower limit to the total number of triangular facets in the region. If this proportion exceeds a high threshold, such as 90%, it is determined that the internal deviation direction is consistent.

[0036] A connected region will only be formally defined as a "non-uniform curing potential region" if it simultaneously meets both of the following conditions: First, the volume of the connected region is greater than a preset volume threshold of 0.5 cubic centimeters; second, the temperature deviation direction within the connected region is clear and consistent, i.e., the overall average temperature is below 80 degrees Celsius or above 85 degrees Celsius, and the temperature values ​​of the vast majority of the triangular facets within it show the same deviation direction. For example, a connected region with a volume of 1.2 cubic centimeters, an average temperature of 77 degrees Celsius, and 95% of its constituent triangular facets having a temperature below 80 degrees Celsius will be identified as a "non-uniform curing potential region below the window direction." Similarly, a region with a volume of 0.9 cubic centimeters and an average temperature of 87 degrees Celsius will be identified as a "non-uniform curing potential region above the window direction." These precisely defined and located potential regions are the direct basis for the subsequent targeted control by the dual-mode instruction generation module.

[0037] In another preferred embodiment of the present invention, the specific process of generating the lighting parameter adjustment command in the dual-mode command generation module is as follows: First, the process of generating illumination parameter adjustment instructions begins with an in-depth analysis of a specific non-uniform curing potential region. This potential region has been precisely defined by the preceding module, and its key attributes include spatial location, geometric dimensions, and the multispectral response coefficient vector and temperature value of each triangular facet within the region. The first step in generating instructions is to analyze representative multispectral response coefficient vectors within this potential region. For example, the arithmetic mean of the multispectral response coefficient vectors of all triangular facets within the region can be calculated to obtain an average vector representing the overall spectral state of the region. Assuming this average vector is [0.18, 0.30, 0.22, 0.49], corresponding to four predefined sampling bands. Next, each component of this average vector is compared one by one with a preset reference range characterizing the ideal gel state of the resin. This reference range is a four-dimensional numerical space, for example, the first band is 0.15 to 0.25, the second band is 0.28 to 0.35, the third band is 0.18 to 0.26, and the fourth band is 0.45 to 0.55. The purpose of the analysis is to determine which specific band components cause the average vector to fall within this gel-state characteristic range. In this example, all four component values ​​are within their respective intervals. However, the key is to identify which component values, while falling within the range, deviate from the "optimal" center value of the ideal gel state at that band, or whose values ​​indicate that the region is on the edge of a gel state.

[0038] After completing the spectral analysis, it is necessary to query the hardware configuration database of the active illumination imaging unit, namely the band configuration table. This configuration table records the specific parameters of the multi-band light sources equipped in each active illumination imaging unit, including the center wavelength, half-width, maximum illumination intensity, and most importantly, whether each light source supports independent control. For example, the configuration table shows that active illumination imaging units No. 3 and No. 7, covering the potential area, are each equipped with four LED light sources with center wavelengths of 450 nm, 550 nm, 650 nm, and 850 nm, respectively, and the driving current of each LED can be independently adjusted via digital signals. Based on this, the specific actuators that can be used to implement intervention, namely the specific band light sources in these two imaging units, are determined.

[0039] Next, based on the temperature deviation direction of the non-uniform curing potential region, the macroscopic target of spectral adjustment, i.e., the spectral adjustment direction, is determined. The temperature deviation direction is determined by the preceding module; for example, "below the window direction" means that the average temperature of this region is lower than the lower limit of the temperature window set by the curing process, assuming the lower limit is 80 degrees Celsius. According to the material optical properties knowledge base, when the temperature is low, the resin tends to remain in or closer to the liquid stage, and its standard spectral response coefficient vector in the fully liquid state has a quantifiable difference from that in the gel state. Therefore, the spectral adjustment direction at this time is: by changing the illumination conditions, to make the spectral response characteristics of this potential region in the next image acquisition cycle closer to the pre-stored "liquid resin" spectral model. Conversely, if the temperature deviation direction is "above the window direction," for example, the average temperature reaches 88 degrees Celsius, which is higher than the upper limit of 85 degrees Celsius, then the adjustment direction is to make the spectral response closer to the "solid resin" spectral model. These two comparison models are also stored in the material optical properties knowledge base in the form of four-dimensional vectors.

[0040] Finally, based on the determined spectral adjustment direction, the specific illumination intensity ratio calculation is performed. This calculation essentially involves "deriving" the average spectral vector of the current potential region from the target model vector. In practice, each target band needs to be processed individually. For example, for the 850 nm near-infrared band, the value of this component in the current potential region's average vector is 0.49, while the reference value for the target liquid resin model in this band is 0.38. To make the measured value closer to 0.38, theoretically, whether the illumination intensity of the 850 nm light source needs to be increased or decreased depends on the physical relationship between the spectral response coefficient, reflectivity, and light source intensity. Under a simplified linear response assumption, it might be necessary to decrease the intensity of the 850 nm light source. The intensity adjustment ratio calculation introduces a scaling factor, for example, adjusting the light source intensity to 80% of the original calibration value. The calculation process comprehensively considers and weighs the adjustment needs of all four bands, ultimately generating a set of explicit, executable numerical instructions. This instruction is for adjusting illumination parameters. Its content structure includes several key fields: the identification of the active imaging unit, such as "Unit_3" and "Unit_7"; the target wavelength, such as "850nm"; and the intensity adjustment ratio, such as "0.8". This instruction is encapsulated into a specific data packet format and is ready to be sent to the corresponding active illumination imaging unit controller.

[0041] In the dual-mode instruction generation module, the specific process of generating the heating unit power adjustment instruction is as follows: First, the three-dimensional geometric center coordinates of the potential area are obtained. In a pipe model with an array of heating units, a digitized layout diagram of the heating units exists. This diagram records the precise installation coordinates and number of each independently controllable heating unit in the three-dimensional space of the pipe. For example, five rows are arranged along the pipe axis, with six units distributed circumferentially in each row, totaling 30 heating units. Each unit has the coordinates of the center point of its effective range. The first step in generating the instruction is to perform a spatial search on this layout diagram, using the geometric center of the potential area as the origin, to calculate the Euclidean distance from this center point to the center point of each heating unit. Then, the units are sorted from nearest to farthest from this distance. A preset distance threshold, such as 150 mm, is used to initially filter out all heating units with a distance less than this threshold as a candidate set of "nearby heating units."

[0042] Next, based on the spatial extent of the non-uniform curing potential zone, the number of heating units ultimately affected needs to be determined from the candidate set. The dimensions of the potential zone, as previously mentioned, include its axial and circumferential spans. For example, a potential zone might have an axial length of 200 mm and a circumferential arc length of 100 mm. Therefore, the heating unit layout diagram needs to cover an area roughly centered on this region and slightly larger than its own extent. Through spatial range matching, heating units whose center point falls within this coverage area are selected from the neighboring heating unit candidate set. Assuming three heating units are ultimately determined, numbered H05, H06, and H11, they will be included in the list of targets affected by the adjustment command.

[0043] The core step in command generation is calculating the power adjustment. This requires detailed statistical distribution characteristics of temperature values ​​within the non-uniform curing potential zone and their clearly defined temperature deviation direction. Statistical characteristics include the zone's average temperature, temperature standard deviation, minimum and maximum temperatures, etc. For example, the zone's average temperature is 76 degrees Celsius, the standard deviation is 1.5 degrees Celsius, and the deviation direction is below the lower limit of 80 degrees Celsius. The overall temperature correction needs to compensate for the gap between the current average value and the target value, while considering the efficiency and uniformity of heat transfer. The target value is typically set as the median of the temperature window, such as 82.5 degrees Celsius. Therefore, the initial overall temperature requirement is an increase of 6.5 degrees Celsius. This temperature requirement needs to be converted into an energy requirement, i.e., the overall temperature correction. This conversion process relies on a thermal model, pre-calibrated through thermal simulation or experiments, for the current piping environment and materials. This model can estimate the approximate energy input required to raise the temperature of a specific volume area by a certain degree.

[0044] Then, the calculated total energy demand, i.e., the total temperature correction, is allocated to the previously determined heating units based on distance and influence capacity. The allocation follows a weighted principle, with the weighting factor primarily depending on two factors: first, the distance from the center of the heating unit to the geometric center of the potential zone (closer units have higher weights); and second, the rated power and heat flux distribution characteristics of the heating unit itself. For example, heating unit H05, being the closest, might be allocated 40% of the energy demand; H06, next, 35%; and H11, being the furthest, 25%. Based on the allocation ratio and the total energy demand, the additional power increase or decrease required for each heating unit can be calculated, i.e., the individual power adjustment. If the temperature deviation direction is higher than the window direction, the calculated individual power adjustment is negative, indicating a need to reduce power.

[0045] Finally, the calculated individual power adjustment for each heating unit is converted into an instruction format that the heating unit controller can directly recognize and execute. This format is typically a specific industrial communication protocol data frame. The converted power adjustment instruction contains a clear heating unit identifier, such as "Heater_ID:H05", and a specific power adjustment value, such as "Power_Adjustment:+15.6W". This instruction, together with the aforementioned lighting parameter adjustment instruction, constitutes a complete dual-mode collaborative control strategy for this specific non-uniform curing potential area, and is sent to its respective actuator unit for execution.

[0046] In another preferred embodiment of the present invention, the specific process of marking the change in the range of action of the command execution and the non-uniform solidification potential zone on the updated three-dimensional pipe model surface in the instruction execution and feedback module is as follows: After one control cycle of the lighting parameter adjustment command and power adjustment command, the active lighting imaging unit and thermal imager array deployed inside the pipeline are retried to acquire synchronous inner wall images and temperature data at the current moment. This raw data is immediately fed into the processing pipeline, undergoing real-time computation by the multispectral distribution generation module, the field map fusion and alignment module, and the non-uniform region identification module. This series of computations ultimately produces a snapshot of the state based on the latest data: an updated multispectral response coefficient distribution map, a temperature field distribution map, and a new round of non-uniform curing potential region identification results. This new list of potential regions and their set of spatial coordinates constitute the benchmark for feedback comparison.

[0047] Next, the module performs a precise spatial location comparison between the set of non-uniform solidification potential regions identified in the current control cycle and the set of potential regions identified in the previous control cycle, stored in memory. The core of this comparison is determining the presence, persistence, change, or disappearance status of each potential region. To achieve this comparison, the module calculates a unique spatial signature for each identified potential region, such as a hash combination of its three-dimensional geometric center coordinates and volume. By comparing the signatures in the two cycle sets, the state transitions of each region can be determined. Specifically, if a potential region's signature appears in the current set but is not found in the historical set of the previous cycle, the region is classified as a "newly emerging non-uniform solidification potential region." Conversely, if a signature exists in the historical set but cannot be found in the current set, it means that the region may have disappeared or merged with other regions. For signatures found in both sets, the difference in their spatial extent is further calculated, such as comparing their volume or axial span. If the current volume is less than 70% of the historical volume, it is classified as a "non-uniform solidification potential region with a significantly reduced extent."

[0048] After completing the status determination, the module begins dynamic annotation on the pipeline's 3D model visualization interface used for real-time monitoring. All graphical annotation operations are performed based on the coordinate system of the pipeline's 3D digital model surface. For each "newly identified non-uniform solidification potential area," the module marks its spatial geometric center using a first visual symbol. For example, the first visual symbol might be a red hollow ring, the size of which can be scaled proportionally to the volume of the potential area. For each "non-uniform solidification potential area that has shrunk or disappeared," a second visual symbol marks its last known geometric center location. The second visual symbol might be a blue solid triangle, used to indicate an area that once existed but has been effectively intervened in.

[0049] Simultaneously, the module needs to highlight the actual spatial range of action of all issued control commands within the current control cycle. These ranges are not directly identified physical areas, but rather expected influence areas derived from the issued command parameters. For example, a power increase command for heating unit H05, combined with the known heat flow distribution model of that heating unit, can calculate a three-dimensional thermal influence range boundary. A lighting adjustment command for the 850 nm band of imaging unit 3 can calculate the area of ​​the pipe's inner wall covered by special illumination based on the unit's field of view. The module merges these three-dimensional spatial ranges derived from different commands to obtain a comprehensive three-dimensional model of the range of action. On the visualization interface, a third visual symbol, such as a semi-transparent green highlight fill, is used to overlay and render this comprehensive range of action on the corresponding position on the inner surface of the pipe's three-dimensional model. All these visual symbols—red rings, blue triangles, and green highlight areas—are accompanied by brief numerical labels displaying their corresponding potential area numbers or command sequence numbers, collectively forming a dynamic graph that intuitively shows the spatiotemporal relationship between control actions and state evolution.

[0050] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. A monitoring and control system for in-situ curing lining process based on machine vision, characterized in that, include: The synchronous surround acquisition module is used to acquire synchronous inner wall images of multiple active illumination imaging units arranged around the inside of the pipe. The active illumination imaging unit includes independently modulated multi-band light sources and image sensors. The multispectral distribution generation module is used to calculate the spectral response coefficients of the resin surface in the field of view of each active illumination imaging unit to light of each wavelength band. Combined with the position coordinates of the active illumination imaging unit, a multispectral response coefficient distribution map is generated on the surface of the pipe 3D model. include: Extract the grayscale values ​​of pixels on the resin surface region in the image acquired by each active illumination imaging unit under illumination by light sources of different wavelengths, calculate the ratio of the grayscale value of each pixel in each wavelength to the factory-calibrated brightness value of the corresponding wavelength light source, and use it as the single-band spectral response coefficient of that pixel. Based on the internal calibration parameters of the active illumination imaging unit, the two-dimensional coordinates of the pixel points are converted into three-dimensional coordinates of the surface of the pipe three-dimensional model. For each triangular facet on the surface of the pipe three-dimensional model, all active illumination imaging units covering this triangular facet are found. The single-band spectral response coefficients calculated by each active illumination imaging unit are weighted and averaged, with the weights being the viewing angle and distance from the active illumination imaging unit to the triangular facet. The weighted average single-band spectral response coefficients of each triangular facet in each band are combined to form the multispectral response coefficient vector of the triangular facet. The multispectral response coefficient vectors of all triangular facets constitute a multispectral response coefficient distribution map covering the inner wall of the pipe. The field map fusion and alignment module is used to spatially align and overlay the multispectral response coefficient distribution map and the pipe inner wall temperature field distribution map acquired by the thermal imager on the surface of the pipe three-dimensional model. include: Each data point in the temperature field distribution map contains three-dimensional coordinates and a temperature value. A mapping relationship is established between the coordinates of the center point of the triangular facet on the surface of the three-dimensional pipe model and the coordinates of the data points in the temperature field distribution map. The temperature value of the nearest data point is assigned to the corresponding triangular facet. For triangular facets where no nearest temperature data point is found, spatial interpolation is performed using the temperature values ​​of adjacent triangular facets to complete the temperature value. After assignment and interpolation, each triangular facet simultaneously has a multispectral response coefficient vector and a temperature value attribute. The multispectral response coefficient vector and temperature value attribute of each triangular facet are stored together to form the superimposed distribution map data structure. The non-uniform region identification module is used to identify regions in the superimposed distribution map whose spectral response coefficients are within the gel-state characteristic range but whose temperature values ​​deviate from the preset curing temperature window, and these regions are defined as non-uniform curing potential regions. The dual-mode instruction generation module is used to generate illumination parameter adjustment instructions for the active illumination imaging unit and power adjustment instructions for the heating unit based on the spatial morphology and temperature deviation direction of the non-uniform curing potential area. The instruction execution and feedback module is used to send lighting parameter adjustment instructions and power adjustment instructions to the corresponding units, and to mark the scope of the instruction execution and the changes in the non-uniform curing potential zone on the updated 3D model surface of the pipeline.

2. The monitoring and control system for the in-situ curing lining process based on machine vision according to claim 1, characterized in that, In the field map fusion and alignment module, an independent thermal imager array is used to collect the temperature field distribution map of the inner wall of the pipe.

3. The monitoring and control system for the in-situ curing lining process based on machine vision according to claim 1, characterized in that, The specific process for identifying potential non-uniform curing regions in the non-uniform region identification module is as follows: Preset a reference range and temperature window range for the multispectral response coefficient vector corresponding to the ideal gel state of the resin. Traverse each triangular facet in the superimposed distribution map and determine whether the multispectral response coefficient vector of the triangular facet falls within the reference range. For triangular facets whose multispectral response coefficient vectors fall within the reference range, determine whether the temperature value of the triangular facet is within the temperature window range, and mark triangular facets whose multispectral response coefficient vectors fall within the reference range but whose temperature values ​​deviate from the temperature window range as initially selected triangular facets. Adjacent initial triangular patches in the cluster space form independent connected regions. The geometric center, spatial extension size, and statistical distribution characteristics of temperature values ​​within each connected region are calculated. Connected regions that meet the preset spatial size conditions and whose internal temperature statistical distribution has the same deviation direction are defined as non-uniform curing potential regions.

4. The monitoring and control system for the in-situ curing lining process based on machine vision according to claim 3, characterized in that, The process of defining the connected regions that meet the preset spatial size conditions and whose internal temperature statistical distribution has the same deviation direction as non-uniform curing potential regions is as follows: Calculate the volume of the connected region. If the volume of the connected region is greater than the preset volume threshold, it is determined that the spatial size condition is met. Calculate the average temperature value of all triangular facets in the connected region. If the average value is lower than the lower limit of the temperature window, the temperature deviation direction is below the window direction. If the average value is higher than the upper limit of the temperature window, the temperature deviation direction is above the window direction. When the volume of a connected region is greater than a preset volume threshold, and the temperature deviation direction within the connected region is consistent, the connected region is defined as a non-uniform curing potential region.

5. The monitoring and control system for the in-situ curing lining process based on machine vision according to claim 1, characterized in that, The specific process of generating lighting parameter adjustment commands in the dual-mode command generation module is as follows: For a non-uniform curing potential region, analyze the specific band component values ​​in the multispectral response coefficient vector of the non-uniform curing potential region that cause it to fall into the gel state characteristic range, query the band configuration table of multi-band light sources in the active illumination imaging unit, and determine the corresponding independently controllable band light source. Based on the temperature deviation direction of the non-uniform curing potential region, the spectral adjustment direction is determined. When the temperature is below the lower limit of the temperature window, the spectral adjustment direction corresponds to the spectral characteristics of liquid resin in subsequent imaging; when the temperature is above the upper limit of the temperature window, the spectral adjustment direction corresponds to the spectral characteristics of solid resin in subsequent imaging. Based on the spectral adjustment direction, calculate the illumination intensity ratio of one or more specific band light sources that need to be enhanced or weakened, and generate illumination parameter adjustment instructions that include the active imaging unit identifier, target band, and intensity adjustment ratio.

6. The monitoring and control system for in-situ curing lining process based on machine vision according to claim 1, characterized in that, The specific process of generating heating unit power adjustment commands in the dual-mode command generation module is as follows: Based on the spatial geometric center coordinates of the non-uniform curing potential zone, locate one or more nearest heating units in the layout diagram of the heating units inside the pipe, and determine the number of affected heating units based on the spatial extension dimensions of the non-uniform curing potential zone. Based on the statistical distribution characteristics of temperature values ​​and the direction of temperature deviation within the non-uniform curing potential zone, the overall temperature correction is calculated, and the overall temperature correction is distributed to each identified heating unit according to the distance, forming the individual power adjustment amount for each heating unit. The individual power adjustment amount is converted into a power adjustment command format that the heating unit controller can recognize. The power adjustment command includes the heating unit identifier and the power adjustment value.

7. The monitoring and control system for in-situ curing lining process based on machine vision according to claim 1, characterized in that, In the instruction execution and feedback module, the specific process of marking the change in the effective range of instruction execution and the non-uniform solidification potential zone on the updated 3D pipeline model surface is as follows: After the lighting parameter adjustment command and power adjustment command have been executed for one control cycle, the synchronous inner wall image is re-acquired and updated multispectral response coefficient distribution map and temperature field distribution map are generated to identify new non-uniform curing potential areas. By comparing the spatial location of the non-uniform curing potential area identified after the update with that identified in the previous control cycle, the newly emerging non-uniform curing potential area is marked on the surface of the pipeline 3D model using a first visual symbol, and non-uniform curing potential areas whose range has shrunk or disappeared are marked using a second visual symbol; the spatial range affected by all lighting parameter adjustment commands and power adjustment commands in the current control cycle is highlighted using a third visual symbol.

Images