Pump machine metal shell size detection method based on image analysis

Abstract

The invention belongs to the technical field of industrial measurement, and discloses a pump machine metal shell size detection method based on image analysis, which comprises the following steps: synchronously acquiring image data of a pump machine metal shell in a visible light wave band and a near-infrared wave band; the contrast ratio of image data is optimized through a dynamic exposure control technology, and a high-dynamic-range multispectral image is obtained; three-dimensional geometric information of the pump machine metal shell is obtained, registration fusion is carried out on the three-dimensional geometric information and the high-dynamic-range multispectral image, a dense three-dimensional point cloud model with multispectral textures is constructed, and a multi-frequency heterodyne algorithm is adopted to process a high-reflection area in the dense three-dimensional point cloud model; the method comprises the following steps: collecting a multi-angle reflection image of a pump machine metal shell by rotating a linear polarizer, calculating a Stokes vector to extract a diffuse reflection component, carrying out diffuse reflection component constraint and brightness suppression on a high-reflection area based on a CIE-Lab color space, and outputting a dense three-dimensional point cloud model after texture enhancement; and the size detection precision of the pump machine metal shell is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial measurement, and more particularly to a method for detecting the size of a metal shell of a pump based on image analysis. Background Art

[0002] The patent with the patent publication number CN104165591A discloses a fully automatic optical size detection method and system for metal shells, which includes an image management module, a data management module, a system management module and a main control module; the image management module includes an image acquisition submodule, an image enhancement submodule, a feature recognition submodule, an image registration submodule, an image fusion submodule, an edge extraction submodule, an image filtering submodule and an identification control submodule; the data management module includes a data operation submodule, a data storage submodule and a data transmission submodule; the system management module includes an I / O submodule and a system maintenance submodule; the main control module includes an index detection submodule, a signal conversion submodule and a communication control submodule; the above-mentioned functional modules and submodules are interconnected through a data transmission bus or various conversion circuits and control circuits to realize data communication. The present invention is a high-precision, high-automation platform that meets the needs of serving molds, high-tech and other industries.

[0003] The existing method for detecting the size of the metal casing of a pump has the following defects:

[0004] In the method for detecting the size of the metal casing of a pump, multi-frequency heterodyne assumes that the surface of the object is a Lambertian reflector or a modelable phase response body, but the metal curved surface is mostly non-uniform specular reflection, and the light may completely deviate from the image at certain angles, resulting in serious information loss; at this time, heterodyne recovery is difficult to rely on principles to restore the true depth. Traditional structured light relies on the assumption of Lambertian reflection. On metal or specular reflective objects, the pattern will be saturated or completely lost, causing serious phase jumps or depth blind spots. Although the heterodyne method can improve anti-interference performance, its premise is that the target surface is a modelable reflector. Low-frequency envelope distortion or inability to decode may occur in the specular reflection area. Existing technologies simply merge multi-source data in a maximum confidence selection or global weighting manner, which is difficult to adapt to various boundary conditions; in highly reflective areas, visible light images are prone to saturation and distortion, affecting the quality of point cloud three-dimensional reconstruction;

[0005] Existing structured light or multi-frequency heterodyne systems are prone to problems such as highlight saturation and texture bleaching on metal surfaces, resulting in the lack of effective texture information in some areas of the texture map after 3D reconstruction, and overexposure of brightness in some areas, which affects the true restoration and causes serious distortion of the 3D point cloud texture mapping; most traditional methods can only perform segmentation based on brightness thresholds or simple reflection characteristics, and cannot accurately separate high-reflection areas and diffuse reflection areas, especially in high-brightness spots and blurred edges, which are prone to misjudgment.

[0006] In view of this, the present invention proposes a method for detecting the size of the metal shell of a pump based on image analysis to solve the above problems. Summary of the Invention

[0007] In order to overcome the above defects of the prior art and to achieve the above object, the present invention provides the following technical solution: A method for detecting the size of the metal shell of a pump based on image analysis, including:

[0008] S1. Synchronously collect image data of the metal shell of the pump in the visible light band and the near-infrared band; optimize the contrast of the image data through dynamic exposure control technology to obtain a high-dynamic-range multi-spectral image;

[0009] S2. Obtain the three-dimensional geometric information of the metal shell of the pump, register and fuse it with the high-dynamic-range multi-spectral image to construct a dense three-dimensional point cloud model with multi-spectral texture, and use the multi-frequency heterodyne algorithm to process the highly reflective areas in the dense three-dimensional point cloud model;

[0010] S3. Collect multi-angle reflection images of the metal shell of the pump by rotating a linear polarizer, calculate the Stokes vector to extract the diffuse reflection component, and perform diffuse reflection component constraint and brightness suppression on the highly reflective area based on the CIE-Lab color space to output a dense three-dimensional point cloud model with enhanced texture;

[0011] S4. Combine the three-dimensional geometric information of the metal shell of the pump and the dense three-dimensional point cloud model with enhanced texture to extract the regional size features of the metal shell of the pump;

[0012] S5. Compare the regional size features with the preset regional size features, automatically calculate the size error, obtain the geometric error index, and generate real-time alarm information and an SPC report to the size detection intelligent terminal according to the geometric error index.

[0013] Preferably, the method for obtaining the image data in the visible light band and the near-infrared band includes:

[0014] Configure two spectral sensitive cameras to respectively collect the visible light band and the near-infrared band, adopt a coaxial optical path design, and use a beam splitter prism to guide light of different bands to the corresponding spectral sensitive cameras respectively to achieve multi-band imaging in the same field of view;

[0015] Configure a synchronous trigger controller to perform unified timing control on the spectral sensitive cameras, and use a wide-spectrum LED array covering the visible light band and the near-infrared band range; configure a light source modulation control circuit to dynamically control the light emission intensity and timing of the wide-spectrum LED array in different bands according to the working state of the spectral sensitive cameras;

[0016] By means of an external hardware trigger signal, all spectral sensitive cameras perform image acquisition at the same time point. According to the reflection characteristics of different bands, the exposure time and gain parameters are set respectively; the image data of the metal shell of the pump in the visible light band and the near-infrared band are respectively cached in the DMA cache connected to the data interface of the spectral sensitive camera, and a unified timestamp is bound to each frame of image;

[0017] The internal and external parameter matrices between different spectral sensitive cameras are obtained by using the factory calibration method. Geometric transformation is applied to unify the image data in the visible light band and the near-infrared band into the same spatial coordinate system, and then the image data with spatio-temporal synchronization and spatial registration is obtained.

[0018] Preferably, the method for obtaining the high-dynamic range multi-spectral image includes:

[0019] Perform brightness histogram analysis on the acquired image data, and combine the image gradient and texture features. Use the region segmentation algorithm to segment the image data, and preset the first brightness threshold and the second brightness threshold; if the image brightness is less than the preset first brightness threshold, it is determined as a low-brightness region; if the image brightness is greater than or equal to the preset first brightness threshold and less than the preset second brightness threshold, it is determined as a medium-brightness region; if the image brightness is greater than the preset second brightness threshold, it is determined as a high-brightness region;

[0020] Identify the low-brightness region, medium-brightness region and high-brightness region. For different brightness regions, calculate their local contrast and saturation indexes respectively, dynamically set the exposure time and gain parameters adapted to the region, and perform sub-region exposure adjustment; according to the set regional exposure parameters, perform image sampling under different exposure settings for the same field of view; and based on the local feature response, perform HDR fusion processing on the images of each region through the local dynamic weight strategy to generate a high-dynamic range image with the optimal regional contrast;

[0021] Based on the spectral response characteristics, identify the highly reflective metal regions in the image data and detect them by the brightness threshold method; combine the transmission characteristics and reflection suppression ability of the near-infrared band image to perform pixel-level compensation on the high-brightness regions in the visible light band image; use the halo suppression filter to enhance the texture and suppress the highlights of the high-brightness regions to obtain a high-dynamic range multi-spectral image.

[0022] Preferably, the three-dimensional geometric information of the metal shell of the pump includes the spatial coordinate point cloud data, key structural feature data, surface curvature information and geometric dimension parameter data on the surface of the metal shell of the pump.

[0023] Preferably, the method for constructing the dense three-dimensional point cloud model with multi-spectral texture includes:

[0024] For each three-dimensional point in the same spatial coordinate system, according to the internal and external parameter matrices between different spectrally sensitive cameras, map it to the multi-spectral image plane to obtain a projection function from the point cloud space to the image space; based on this projection function, use the pinhole camera model to project each three-dimensional point to the two-dimensional coordinate position of the corresponding multi-spectral image, and extract the multi-spectral texture value of the image pixel at this position;

[0025] If the two-dimensional coordinate position is a non-integer coordinate, use bilinear interpolation to extract the multi-spectral texture value from the multi-spectral image; complete the conversion from the spatial geometric point to the multi-spectral texture point to form a texture point cloud with multi-channel image attributes; perform spectral normalization processing on the texture point cloud, and use an octree indexing structure for dense organization to form a dense three-dimensional point cloud model with multi-spectral texture by combining all texture point clouds.

[0026] Preferably, the method for processing the highly reflective area in the dense three-dimensional point cloud model includes:

[0027] Preset a visible light band texture value threshold and a brightness gradient change rate threshold. If the visible light band texture value corresponding to any position point in the dense three-dimensional point cloud model is greater than the preset visible light band texture value threshold, and the brightness gradient change rate is greater than the preset brightness gradient change rate threshold, then mark this point as a highly reflective point, and define the area formed by all highly reflective points as the highly reflective area;

[0028] Introduce a multi-frequency coded light source mechanism, and project m preset sets of spatial frequency patterns in sequence through a structured light projection device to form a spatial frequency set ; for the highly reflective area, use multi-frequency heterodyne to recover the phase, and calculate the difference between spatial frequency pairs to construct a low-frequency envelope-assisted decoding; based on the spatial frequency set in and the spatial frequency pair combination, generate a heterodyne synthesis phase ;

[0029] For different spatial frequency combinations, perform weighted fusion on different heterodyne synthesis phases to obtain the final phase ; after obtaining the final phase, convert it to the depth coordinate in the dense point cloud through the phase-depth inverse solution function;

[0030] Adopt near-infrared texture compensation and multi-spectral fusion rules to repair the position points where the visible light band texture value is greater than the preset visible light band texture value threshold. When there are blind spots in the near-infrared structured light or it is limited by the occlusion reflection angle, introduce a TOF lidar for supplementary measurement, and calculate the depth value corresponding to any position point in the dense three-dimensional point cloud model based on the pulse flight time;

[0031] For each three-dimensional point position in the dense three-dimensional point cloud model, multi-source depth fusion is performed using a piecewise fusion function based on the confidence of different depth sources, and finally a dense three-dimensional point cloud model with restored depth information is obtained.

[0032] Preferably, the method for extracting the diffuse reflection component includes:

[0033] Install a linear polarizer and set the polarizer to rotate continuously at different angles. Collect the reflection image of the metal shell of the pump at each angle. For each pixel point in the multi-angle reflection images, construct the Stokes vector of this pixel point through the collected multi-angle reflection images, and further calculate the degree of polarization and polarization angle of each pixel point through the Stokes vector. Preset a degree-of-polarization threshold, extract the area corresponding to the pixel points with a degree of polarization less than the preset degree-of-polarization threshold, and mark it as the diffuse reflection component.

[0034] Preferably, the method for obtaining the texture-enhanced dense three-dimensional point cloud model includes:

[0035] Based on the multi-angle reflection images, extract pixel-level polarization information and construct a polarization vector to calculate the degree of polarization of each position point , identify and extract the diffuse reflection region according to the preset degree-of-polarization threshold to generate a binary mask ; Convert the multi-angle reflection images to the CIE-Lab color space. In the CIE-Lab color space, use the diffuse reflection mask to suppress the brightness of the highly reflective region. For each position point , whose brightness value is , construct a brightness regulation function under the constraint of diffuse reflection, and reduce the brightness of the highly reflective region through the brightness regulation function;

[0036] Convert the processed multi-angle reflection images in the CIE-Lab color space back to the RGB color space and re-project them into the dense three-dimensional point cloud model, and reassign texture values to each point cloud, thereby forming a texture-enhanced dense three-dimensional point cloud model.

[0037] Preferably, the method for obtaining the regional size characteristics of the metal shell of the pump includes:

[0038] Use the K-nearest neighbor statistical filtering algorithm to identify and remove the outliers in the texture-enhanced dense three-dimensional point cloud model, and combine the moving least squares method to smooth the surface of the dense three-dimensional point cloud;

[0039] Apply the region growing and RANSAC algorithms to automatically segment the texture-enhanced dense three-dimensional point cloud model, decompose the metal shell of the pump into N geometric regions, obtain the basic geometric parameter data of each geometric region and the regional size characteristics of the metal shell of the pump, and output them in the form of a structured vector.

[0040] Preferably, the geometric error indicators include linear dimension error, angular error, form and position error, surface feature error, and boundary profile error.

[0041] Technical effects and advantages of the method for detecting the size of the metal shell of a pump based on image analysis according to the present invention:

[0042] Through the visible light-infrared-TOF three-stage compensation mechanism, the present invention realizes high-robust depth recovery in complex reflection scenarios; compared with a single structured light, the recovery rate of the solution in highly reflective or occluded areas is greatly improved; by setting confidence thresholds and corresponding processing strategies for each type of depth source, the inadaptability caused by a global unified strategy is avoided; multi-source point cloud data can achieve adaptive fusion under the condition of inconsistent quality, improving the overall point cloud consistency and accuracy. By introducing a near-infrared texture substitution compensation mechanism in the overexposed area of visible light, the visual performance and registration effect of the three-dimensional model are ensured; compared with traditional methods, this method can obtain a higher texture reduction degree on the strongly reflective surface. It can be adapted to a variety of industrial materials and has high stability in environments such as production lines, light interference, and complex object morphologies.

[0043] Through the diffuse reflection mask calculated based on polarization information, highly reflective areas are effectively identified, and brightness suppression is implemented in the CIE-Lab color space, so that the brightness of the highly reflective areas drops below the set threshold, suppressing specular artifacts and significantly enhancing the retention of texture details. During the texture assignment stage of the point cloud, re-projection assignment is performed based on the processed multi-angle images to ensure the registration accuracy between the texture-enhanced image and the three-dimensional geometric structure, achieving high consistency between the texture map and the three-dimensional structure. By identifying the reflection type through the polarization mechanism, it is not affected by external factors such as color and material, and can stably extract highly reflective areas under different lighting and viewing conditions, effectively improving the adaptability and robustness of texture enhancement processing. The three-dimensional point cloud model after brightness suppression and texture enhancement significantly improves the visualization effect and edge sharpness, enabling subsequent algorithms such as region segmentation, structure extraction, and boundary recognition to run stably on highly reflective surfaces. Description of the Drawings

[0044] Figure 1 It is a schematic flow chart of the method for detecting the size of the metal shell of a pump based on image analysis;

[0045] Figure 2 It is a schematic structural diagram of the system for detecting the size of the metal shell of a pump based on image analysis;

[0046] Figure 3 It is a schematic flow chart of the method for extracting the diffuse reflection component provided by the present invention. Detailed Embodiments

[0047] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0048] Embodiment 1

[0049] Please refer to Figure 1 and Figure 3 shown in the figure. Embodiment 1 further illustrates the method for detecting the size of the metal shell of a pump based on image analysis proposed by the present invention, including:

[0050] With the continuous improvement of the intelligent level of industrial manufacturing, the detection of the structural integrity and geometric dimension accuracy of the metal shells of key equipment such as pump machines is gradually shifting from manual sampling inspection to non-contact automated detection means. Among them, the size detection method based on image analysis and three-dimensional reconstruction has been widely used in the detection scenarios of complex surfaces, free-form surfaces, etc. due to its advantages such as non-destructiveness, high precision, and strong adaptability.

[0051] Currently, the commonly used non-contact three-dimensional reconstruction technologies include structured light measurement method, multi-frequency heterodyne coding method, shape light projection method, etc. These methods generally rely on the Lambert reflection model, that is, it is assumed that the surface of the object to be measured exhibits isotropic diffuse reflection to the incident light, so as to ensure that the optical coding pattern can be stably projected onto the object surface and accurately collected by the imaging system. However, the metal shells of pump machines are mostly made of metal materials, and the surface has strong specular reflection characteristics. Its reflection behavior significantly deviates from the ideal Lambert body, showing non-uniformity and anisotropy. On such surfaces, the incident light may be strongly specularly reflected, resulting in the pattern information at certain angles completely deviating from the imaging path, thus causing serious information loss, pattern saturation, or even pattern disappearance.

[0052] Especially when using the multi-frequency heterodyne coding method, although it has strong anti-noise performance and coding and decoding robustness, its recovery accuracy depends on the target surface having a modelable phase response. When the target area is a highly specular reflection area, the coding envelope frequency often distorts in these areas, and the low-frequency envelope signal is blurred or undecodable, resulting in the inability to recover the depth information. In addition, the structured light method is prone to phenomena such as phase jump, reflection overexposure, and texture bleaching in the metal mirror area, which not only affects the integrity of the surface geometry reconstruction but also causes subsequent texture mapping distortion and inaccurate size analysis.

[0053] For the highly reflective areas on the surface of the metal shell, the existing technologies usually adopt multi-source fusion strategies for compensation, such as maximum confidence selection, local weighted average, etc. However, most of these methods are uniformly processed based on global or semi-global features, and it is difficult to adapt to complex conditions such as blurred boundaries and drastic changes in local reflections, resulting in unstable texture fusion effects, inaccurate boundary recognition, and severe distortion of the point cloud structure. At the same time, in highly reflective areas, visible light images often lose key texture details due to overexposure, causing obvious artifacts such as gray blocks, light spots, and floating white bands in the process of texture mapping of the three-dimensional point cloud model, directly affecting the accuracy and stability of dimension detection.

[0054] Therefore, in order to effectively solve the above problems, the present invention proposes a method for detecting the size of the metal shell of a pump based on image analysis, including:

[0055] S1. Synchronously collect the image data of the metal shell of the pump in the visible light band and the near-infrared band; optimize the contrast of the image data through dynamic exposure control technology to obtain a high-dynamic-range multi-spectral image;

[0056] S2. Obtain the three-dimensional geometric information of the metal shell of the pump, register and fuse it with the high-dynamic-range multi-spectral image to construct a dense three-dimensional point cloud model with multi-spectral texture, and use the multi-frequency heterodyne algorithm to process the highly reflective areas in the dense three-dimensional point cloud model;

[0057] S3. Collect multi-angle reflection images of the metal shell of the pump by rotating a linear polarizer, calculate the Stokes vector to extract the diffuse reflection component, and perform diffuse reflection component constraint and brightness suppression on the highly reflective areas based on the CIE-Lab color space to output a dense three-dimensional point cloud model with enhanced texture;

[0058] S4. Combine the three-dimensional geometric information of the metal shell of the pump and the dense three-dimensional point cloud model with enhanced texture to extract the regional size features of the metal shell of the pump;

[0059] S5. Compare the regional size features with the preset regional size features, automatically calculate the size error, obtain the geometric error index, and generate real-time alarm information and an SPC report to the size detection intelligent terminal according to the geometric error index.

[0060] The method for obtaining the image data in the visible light band and the near-infrared band includes:

[0061] Configure two spectral sensitive cameras to collect the visible light band and the near-infrared band respectively, adopt a coaxial optical path design, and use a beam splitter prism to guide the light of different bands to the corresponding spectral sensitive cameras respectively to achieve multi-band imaging in the same field of view;

[0062] Configure a synchronous trigger controller to perform unified timing control on the spectral sensitive cameras, and use a wide-spectrum LED array covering the visible light band and the near-infrared band range; configure a light source modulation control circuit to dynamically control the light emission intensity and timing of the wide-spectrum LED array in different bands according to the working state of the spectral sensitive cameras;

[0063] Through an external hardware trigger signal, make all spectral sensitive cameras perform image acquisition at the same time point. According to the reflection characteristics of different bands, set the exposure time and gain parameters respectively; cache the image data of the pump metal shell in the visible light band and the near-infrared band in the DMA cache connected to the data interface of the spectral sensitive camera, and bind a unified time stamp to each frame of image;

[0064] Use the factory calibration method to obtain the internal and external parameter matrices between different spectral sensitive cameras, and apply geometric transformation to unify the image data in the visible light band and the near-infrared band into the same spatial coordinate system, so as to obtain image data with spatio-temporal synchronization and spatial registration.

[0065] The method for obtaining a high-dynamic range multi-spectral image includes:

[0066] Perform brightness histogram analysis on the collected image data, and combine image gradients and texture features. Use a region segmentation algorithm to segment the image data, and preset a first brightness threshold and a second brightness threshold; if the image brightness is less than the preset first brightness threshold, it is determined as a low-brightness region; if the image brightness is greater than or equal to the preset first brightness threshold and less than the preset second brightness threshold, it is determined as a medium-brightness region; if the image brightness is greater than the preset second brightness threshold, it is determined as a high-brightness region;

[0067] Identify the low-brightness region, medium-brightness region and high-brightness region. For different brightness regions, calculate their local contrast and saturation indexes respectively, dynamically set the exposure time and gain parameters adapted to the region, and perform sub-region exposure adjustment; according to the set region exposure parameters, perform image sampling under different exposure settings for the same field of view; and based on the local feature response, perform HDR fusion processing on the images of each region through a local dynamic weight strategy to generate a high-dynamic range image with the optimal regional contrast;

[0068] Based on the spectral response characteristics, identify the highly reflective metal regions in the image data and detect them by the brightness threshold method; combine the transmission characteristics and reflection suppression ability of the near-infrared band image to perform pixel-level compensation on the high-brightness regions in the visible light band image; use a halo suppression filter to enhance the texture and suppress the highlights of the high-brightness regions to obtain a high-dynamic range multi-spectral image.

[0069] The three-dimensional geometric information of the metal shell of the pump includes the spatial coordinate point cloud data on the surface of the metal shell of the pump, key structural feature data, surface curvature information, and geometric dimension parameter data.

[0070] The key structural feature data includes flange surfaces, bolt hole positions, flanges, and seal groove recesses; the surface curvature information includes the radius of curvature of each area of the metal shell of the pump; the geometric dimension parameter data includes the length, width, height, hole diameter, flatness, roundness, and position tolerance of the metal shell of the pump.

[0071] The method for constructing a dense three-dimensional point cloud model with multi-spectral texture includes:

[0072] For each three-dimensional point in the same spatial coordinate system, according to the internal and external parameter matrices between different spectral-sensitive cameras, map it to the multi-spectral image plane to obtain the projection function from the point cloud space to the image space; based on this projection function, use the pinhole camera model to project each three-dimensional point to the two-dimensional coordinate position of the corresponding multi-spectral image, and extract the multi-spectral texture value of the image pixel at this position.

[0073] If the two-dimensional coordinate position is a non-integer coordinate, use bilinear interpolation to extract the multi-spectral texture value from the multi-spectral image; complete the conversion from the spatial geometric point to the multi-spectral texture point to form a texture point cloud with multi-channel image attributes; perform spectral normalization processing on the texture point cloud, use the octree indexing structure for dense organization, and compose all the texture point clouds into a dense three-dimensional point cloud model with multi-spectral texture.

[0074] The method for processing the highly reflective area in the dense three-dimensional point cloud model includes:

[0075] Preset the visible light band texture value threshold and brightness gradient change rate threshold. If the visible light band texture value corresponding to any position point in the dense three-dimensional point cloud model is greater than the preset visible light band texture value threshold, and the brightness gradient change rate is greater than the preset brightness gradient change rate threshold, then mark this point as a highly reflective point, and define the area formed by all highly reflective points as the highly reflective area.

[0076] Introduce a multi-frequency encoded light source mechanism, and sequentially project m preset spatial frequency patterns through the structured light projection device to form a spatial frequency set ;

[0077] In actual acquisition, due to the high reflection characteristics of the surface in some areas, the pattern is saturated or the phase jumps, and there are significant errors in the directly obtained phase map; therefore, use the multi-frequency heterodyne method to restore the stable phase.

[0078] For the highly reflective area, use multi-frequency heterodyne to restore the phase, and construct a low-frequency envelope-assisted decoding by calculating the difference between spatial frequency pairs , construct a low-frequency envelope-assisted decoding; based on the spatial frequency set The spatial frequency in and the spatial frequency Combined in pairs to generate the heterodyne synthesis phase ; ; where represents the phase value at the position point under the spatial frequency ; represents the phase value at the position point under the spatial frequency ; and represent the indices of the spatial frequencies and satisfy ; represents the total number of spatial frequencies; represents the abscissa of the position point; represents the ordinate of the position point;

[0079] For different combinations of spatial frequencies, different heterodyne synthesis phases are weighted and fused to obtain the final phase ; After obtaining the final phase (recovering the reliable phase), it is converted into the depth coordinates in the dense point cloud through the phase-depth inverse solution function ; where represents the depth value corresponding to the position point ; represents the phase-depth conversion constant; represents the reference spatial frequency;

[0080] In high-reflectivity areas, the reflected light may be too strong, resulting in overexposure of the texture (such as brightness, color) in some areas of the image, that is, the brightness values of some pixels exceed the representable range (greater than the preset visible light band texture value threshold). Such areas will lack detailed information and affect the quality of the point cloud.

[0081] Adopt near-infrared texture compensation and multispectral fusion rules to repair the position points where the visible light band texture value is greater than the preset visible light band texture value threshold ; where represents the fused texture value; represents the near-infrared image texture value; represents the visible light image texture value; represents the fusion coefficient, which controls the weight ratio of near-infrared and visible light. According to the expert experience method, the value range of

[0082] The multi - frequency heterodyne assumes that the object surface is a Lambert reflector or a phase - response body that can be modeled. However, most metal curved surfaces are non - uniform specular reflectors, and light may completely deviate from imaging at certain angles, resulting in serious information loss. At this time, it is difficult to rely on the principle of heterodyne recovery to obtain the true depth.

[0083] In the case where it is completely impossible to recover in the highly reflective area, an incoherent light source or other sensing mechanisms are introduced; a TOF lidar is introduced to complete the depth information, and finally it is integrated into the main point cloud through multi - source fusion; the problem of depth information loss or error accumulation caused by the highly reflective area of the metal in the dense three - dimensional point cloud model is processed.

[0084] When the near - infrared structured light has blind spots or is limited by the occlusion reflection angle, a TOF lidar is introduced for supplementary measurement, and the depth value corresponding to any position point in the dense three - dimensional point cloud model is calculated based on the pulse flight time. ; where represents the speed of light; represents the laser round - trip time difference;

[0085] For each three - dimensional point position in the dense three - dimensional point cloud model, according to the confidence levels of different depth sources, a piece - wise fusion function is used for multi - source depth fusion, and finally a dense three - dimensional point cloud model with recovered depth information is obtained.

[0086] The piece - wise fusion function is:

[0087] ; where represents the depth value after multi - source depth fusion; represents the depth value measured by the near - infrared structured light; represents the depth value measured by the visible light; represents the depth value measured by the TOF laser; represents the confidence - weighted average value of the three depth values; represents the confidence level of the near - infrared structured light; represents the confidence level of the visible light; represents the confidence level of the TOF laser; represents the preset confidence - level threshold of the near - infrared structured light; represents the preset confidence - level threshold of the visible light; represents the preset confidence - level threshold of the TOF laser; represents the current position point where the confidence levels of the three depth sources of near - infrared, visible light, and TOF laser at the current position point have not reached any of the preset thresholds , , ; In this case, all single - depth sources are unreliable, so a multi - source weighted average method is adopted; according to the expert experience method, , , The value ranges of all are between 0 and 1; .

[0088] It should be noted that this piecewise function dynamically selects the most reliable data source according to the quality evaluation (i.e., confidence) of three depth measurement methods. The priority is as follows: the near-infrared structured light has the highest priority; if it fails, it degrades to visible light structured light; if the first two both fail, it further degrades to TOF lidar; if the confidence of all sensors is insufficient, all information is fused; this hierarchical processing mechanism conforms to the cognitive model in industrial-level 3D reconstruction scenarios, preferentially using sensing means with high precision and strong detail performance, and degrading step by step to ensure the balance between integrity and robustness.

[0089] The following problems existing in the prior art are solved: Traditional structured light relies on the Lambertian reflection assumption. On metal or specularly reflecting objects, the pattern will saturate or be completely lost, resulting in serious phase jumps or depth blind spots. Although the heterodyne method can improve anti-interference ability, its premise is that the target surface is a reflectance body that can be modeled (such as diffuse reflection), and low-frequency envelope distortion or inability to decode is likely to occur in the specular reflection area. Most of the prior art simply combines multi-source data in a way of selecting the maximum confidence or global weighting, which is difficult to adapt to various boundary conditions; in high-reflectance areas, visible light images are prone to saturation distortion, affecting the quality of point cloud 3D reconstruction;

[0090] The beneficial effects compared with the prior art are as follows: Through the visible light-infrared-TOF three-stage compensation mechanism, high-robust depth recovery in complex reflection scenarios is achieved; compared with single structured light, the recovery rate of the scheme in high-reflection or occlusion areas is greatly improved; by setting confidence thresholds and corresponding processing strategies for each type of depth source, the inadaptability caused by the global unified strategy is avoided; multi-source point cloud data can achieve adaptive fusion under the condition of inconsistent quality, improving the overall point cloud consistency and accuracy. The near-infrared texture substitution compensation mechanism is introduced in the overexposed area of visible light, ensuring the visual performance and registration effect of the 3D model; compared with traditional methods, this method can obtain higher texture restoration degree on strongly reflective surfaces. It can be adapted to a variety of industrial materials and has high stability in environments such as production lines, light interference, and complex object morphologies.

[0091] The methods for extracting the diffuse reflection component include:

[0092] Install a linear polarizer and set the polarizer to rotate continuously at different angles, and collect the reflection images of the metal shell of the pump at each angle; for each pixel point in the multi-angle reflection images, construct the Stokes vector of this pixel point through the collected multi-angle reflection images, and further calculate the polarization degree and polarization angle of each pixel point through the Stokes vector; preset a polarization degree threshold, and extract the area corresponding to the pixel points with polarization degrees less than the preset polarization degree threshold and mark it as the diffuse reflection component.

[0093] The method for obtaining a dense three-dimensional point cloud model with enhanced texture includes:

[0094] Based on multi-angle reflection images, extract pixel-level polarization information, and construct polarization vectors to calculate the polarization degree of each position point , identify and extract the diffuse reflection region according to a preset polarization degree threshold, and generate a binary mask ; convert the multi-angle reflection images to the CIE-Lab color space. In the CIE-Lab color space, use the diffuse reflection mask to suppress the brightness of the specular reflection region. For each position point , its brightness value is , construct a brightness regulation function under the constraint of diffuse reflection, and reduce the brightness of the specular reflection region through the brightness regulation function;

[0095] The brightness regulation function is ; where represents the brightness value of the position point after brightness suppression; represents that this position point belongs to the diffuse reflection region; represents that this position point belongs to the specular reflection region; represents the brightness suppression coefficient, which is used to control the suppression degree of the brightness of the specular reflection region. According to the expert experience method, ranges from 0 to 1;

[0096] It should be noted that the specular reflection region (specular reflection) usually appears saturated and bright in the image. By multiplying for suppression, the pseudo-highlight is reduced. The diffuse reflection region does not need to be adjusted, and the real texture is retained for restoration. The piecewise function form is simple and efficient, which is conducive to fast hardware implementation and embedded deployment; the boundary continuity is retained to prevent sudden changes between the specular reflection and non-specular reflection regions.

[0097] Convert the processed multi-angle reflection images in the CIE-Lab color space back to the RGB color space, and re-project them into the dense three-dimensional point cloud model, and re-assign texture values to each point cloud, so as to form a dense three-dimensional point cloud model with enhanced texture.

[0098] Solve the following problems existing in the prior art: In the existing structured light or multi-frequency heterodyne systems, problems such as high-light saturation and texture bleaching are likely to occur on the metal surface, resulting in no effective texture information in some regions of the texture map after three-dimensional reconstruction, overexposure of some regions in brightness, affecting the true restoration, and causing serious distortion of the three-dimensional point cloud texture mapping; most traditional methods can only be segmented based on brightness thresholds or simple reflection characteristics, and cannot accurately separate the specular reflection region and the diffuse reflection region, especially in the case of high-light spots and blurred edges, it is easy to misjudge.

[0099] Advantages over the prior art: Through the diffuse reflection mask calculated based on polarization information, the highly reflective areas can be effectively identified, and brightness suppression is implemented in the CIE-Lab color space, reducing the brightness of the highly reflective areas to within the set threshold, suppressing specular artifacts, and significantly enhancing the retention of texture details. During the texture assignment stage of the point cloud, re-projection and assignment are performed based on the processed multi-angle images to ensure the registration accuracy between the texture-enhanced image and the three-dimensional geometric structure, achieving high consistency between the texture map and the three-dimensional structure. By using the polarization mechanism to identify the reflection type, it is not affected by external factors such as color and material, and can stably extract highly reflective areas under different lighting and viewing conditions, effectively improving the adaptability and robustness of texture enhancement processing. The three-dimensional point cloud model after brightness suppression and texture enhancement significantly improves the visualization effect and edge sharpness, enabling subsequent algorithms such as region segmentation, structure extraction, and boundary recognition to run stably on highly reflective surfaces.

[0100] The method for obtaining the regional size characteristics of the metal shell of the pump includes:

[0101] The K-nearest neighbor statistical filtering algorithm is used to identify and remove outliers in the dense three-dimensional point cloud model after texture enhancement, improving the point cloud density and stability. Combining the moving least squares method to smooth the surface of the dense three-dimensional point cloud, obtaining a smoother surface effect of the point cloud, providing a good basis for subsequent feature extraction;

[0102] Using the region growing and RANSAC algorithms, the dense three-dimensional point cloud model after texture enhancement is automatically segmented, and the metal shell of the pump is decomposed into N geometric regions, such as planes, cylindrical surfaces, spherical surfaces, edges, holes, etc., obtaining the effect of clearly dividing different geometric structure regions of the metal shell of the pump; for the segmented geometric regions, the basic geometric parameter data of each geometric region is obtained by geometric parameter fitting; the basic geometric parameter data includes: linear dimensions (such as the distance between two parallel planes, hole center distance, length, width, height of a single region, etc.), angular parameters (the angle between two planes, the inclination deviation of the inclined plane, the parallelism or perpendicularity error between the edge and the reference axis), and curve / surface parameters (radius of the circular arc surface, radius of curvature of the free-form surface, surface roughness, etc.), obtaining the effect of accurately calculating the basic geometric parameters of each region; and the regional size characteristics of the metal shell of the pump are output in the form of a structured vector.

[0103] The geometric error indicators include linear dimension error, angular error, form and position error, surface feature error, and boundary profile error.

[0104] The linear dimension error includes length error, width error, height error, hole distance error, and thickness error; the angular error includes the angle error between two planes, the inclination deviation of the inclined plane, and the parallelism or perpendicularity error between the edge and the reference axis;

[0105] Geometric errors include flatness error, straightness error, roundness error, cylindricity error, coaxiality error, and symmetry error; surface feature errors include curvature radius deviation and surface warping error; boundary profile errors include boundary offset error and corner arc deviation.

[0106] The preset polarization degree threshold is set by the staff. Different polarization degrees are collected at the end, and the average value of multiple polarization degrees is taken as the preset polarization degree threshold; similarly, the first preset brightness threshold, the second preset brightness threshold, the preset visible light band texture value threshold, and the preset brightness gradient change rate threshold are set.

[0107] In this embodiment, through the visible light-infrared-TOF three-stage compensation mechanism, high-robust depth recovery in complex reflection scenarios is achieved; compared with a single structured light, the recovery rate of the scheme in high-reflection or occluded areas is greatly improved; by setting confidence thresholds and corresponding processing strategies for each type of depth source, the inadaptability caused by a global unified strategy is avoided; multi-source point cloud data can be adaptively fused under the condition of inconsistent quality, improving the overall point cloud consistency and accuracy. In the overexposed area of visible light, a near-infrared texture substitution compensation mechanism is introduced to ensure the visual performance and registration effect of the 3D model; compared with traditional methods, this method can obtain a higher texture reduction degree on the strong light reflection surface. It can be adapted to a variety of industrial materials and has high stability in environments such as production lines, light interference, and complex object morphologies.

[0108] Through the diffuse reflection mask calculated based on polarization information, the high-reflection area can be effectively identified, and brightness suppression is implemented in the CIE-Lab color space, so that the brightness of the high-reflection area drops below the set threshold, suppressing the specular artifact and significantly enhancing the retention of texture details. During the point cloud texture assignment stage, re-projection assignment is performed based on the processed multi-angle images to ensure the registration accuracy of the texture-enhanced image and the 3D geometric structure, achieving high consistency between the texture map and the 3D structure. By identifying the reflection type through the polarization mechanism, it is not affected by external factors such as color and material, and the high-reflection area can be stably extracted under different lighting and viewing conditions, effectively improving the adaptability and robustness of the texture enhancement process. The 3D point cloud model after brightness suppression and texture enhancement significantly improves the visualization effect and edge sharpness, enabling subsequent algorithms such as region segmentation, structure extraction, and boundary recognition to run stably on the high-reflection surface.

[0109] Embodiment 2

[0110] Please refer to Figure 2 As shown, for the parts not described in detail in this embodiment, refer to the description content of Embodiment 1. A pump metal shell size detection system based on image analysis is provided, including:

[0111] The multispectral imaging module synchronously acquires the image data of the metal shell of the pump in the visible light band and the near-infrared band; optimizes the contrast of the image data through dynamic exposure control technology to obtain a high-dynamic-range multispectral image;

[0112] The structured light three-dimensional scanning module is used to obtain the three-dimensional geometric information of the metal shell of the pump, register and fuse it with the high-dynamic-range multispectral image, construct a dense three-dimensional point cloud model with multispectral texture, and process the highly reflective area in the dense three-dimensional point cloud model using the multi-frequency heterodyne algorithm;

[0113] The adaptive specular reflection suppression module collects the multi-angle reflection images of the metal shell of the pump by rotating a linear polarizer, calculates the Stokes vector to extract the diffuse reflection component, and performs diffuse reflection component constraint and brightness suppression on the highly reflective area based on the CIE-Lab color space, and outputs a dense three-dimensional point cloud map with enhanced texture;

[0114] The error index generation module combines the three-dimensional geometric information of the metal shell of the pump and the dense three-dimensional point cloud map with enhanced texture to extract the regional dimension features of the metal shell of the pump;

[0115] The on-line dimension detection module compares the regional dimension features with the preset regional dimension features, automatically calculates the dimension error, obtains the geometric error index, and generates a real-time alarm message and an SPC report to the dimension detection intelligent terminal according to the geometric error index.

[0116] The above formulas are all dimensionless and take their numerical calculations. The formulas are obtained by collecting a large amount of data for software simulation to get a formula closest to the actual situation. The preset parameters and threshold selection in the formulas are set by those skilled in the art according to the actual situation.

[0117] The above is only the preferred embodiment of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions falling within the idea of the present invention belong to the protection scope of the present invention. It should be noted that for those of ordinary skill in the art in this technical field, several improvements and refinements made without departing from the principle of the present invention should also be regarded as within the protection scope of the present invention.

Claims

1. A method for detecting the size of the metal shell of a pump based on image analysis, characterized in that, Including: S1. Synchronously collect the image data of the metal shell of the pump machine in the visible light band and the near-infrared band; optimize the contrast of the image data through dynamic exposure control technology to obtain a high-dynamic-range multi-spectral image; S2. Obtain the three-dimensional geometric information of the metal shell of the pump machine, register and fuse it with the high-dynamic-range multi-spectral image, construct a dense three-dimensional point cloud model with multi-spectral texture, and process the highly reflective area in the dense three-dimensional point cloud model using the multi-frequency heterodyne algorithm; S3. Collect the multi-angle reflection images of the metal shell of the pump machine by rotating a linear polarizer, calculate the Stokes vector to extract the diffuse reflection component, and perform diffuse reflection component constraint and brightness suppression on the highly reflective area based on the CIE-Lab color space to output the dense three-dimensional point cloud model with enhanced texture; S4. Combine the three-dimensional geometric information of the metal shell of the pump machine and the dense three-dimensional point cloud model with enhanced texture to extract the regional dimension features of the metal shell of the pump machine; S5. Compare the regional dimension features with the preset regional dimension features, automatically calculate the dimension error, obtain the geometric error index, and generate real-time alarm information and an SPC report to the dimension detection intelligent terminal according to the geometric error index.

2. The method for detecting the size of the metal shell of a pump based on image analysis according to claim 1, characterized in that The method for obtaining the image data in the visible light band and the near-infrared band includes: Configure two spectral-sensitive cameras to respectively collect the visible light band and the near-infrared band, adopt a coaxial optical path design, and use a beam splitter prism to guide the light of different bands to the corresponding spectral-sensitive cameras respectively to achieve multi-band imaging in the same field of view; Configure a synchronous trigger controller to perform unified timing control on the spectral-sensitive cameras, and use a wide-spectrum LED array covering the visible light band and the near-infrared band range; configure a light source modulation control circuit to dynamically control the light emission intensity and timing of the wide-spectrum LED array in different bands according to the working state of the spectral-sensitive cameras; Through an external hardware trigger signal, make all spectral-sensitive cameras collect images at the same time point, set the exposure time and gain parameters respectively according to the reflection characteristics of different bands; cache the image data of the metal shell of the pump machine in the visible light band and the near-infrared band in the DMA cache connected to the data interface of the spectral-sensitive camera respectively, and bind a unified time stamp to each frame of image; Use the factory calibration method to obtain the internal and external parameter matrices between different spectral-sensitive cameras, and apply geometric transformation to unify the image data in the visible light band and the near-infrared band into the same spatial coordinate system, and then obtain the image data with spatio-temporal synchronization and spatial registration.

3. The method for detecting the size of the metal shell of the pump based on image analysis according to claim 2, wherein The method for obtaining the high-dynamic-range multi-spectral image includes: Perform brightness histogram analysis on the collected image data, and combine the image gradient and texture features, and use a region segmentation algorithm to segment the image data, and preset a first brightness threshold and a second brightness threshold; if the image brightness is less than the preset first brightness threshold, it is determined as a low-brightness area; if the image brightness is greater than or equal to the preset first brightness threshold and less than the preset second brightness threshold, it is determined as a medium-brightness area; if the image brightness is greater than the preset second brightness threshold, it is determined as a high-brightness area; Identify low-brightness regions, medium-brightness regions, and high-brightness regions. For different brightness regions, calculate their local contrast and saturation metrics respectively, dynamically set the exposure time and gain parameters adapted to the regions, and perform sub-region exposure adjustment; according to the set regional exposure parameters, sample images under different exposure settings for the same field of view; and based on the local feature response, perform HDR fusion processing on the images of each region through a local dynamic weight strategy to generate a high-dynamic-range image with optimal regional contrast. Based on the spectral response characteristics, identify the highly reflective metal regions in the image data and detect them by the brightness threshold method; combine the transmission characteristics and reflection suppression ability of the near-infrared band image to perform pixel-level compensation on the high-brightness regions in the visible light band image; use a halo suppression filter to enhance the texture and suppress the highlight of the high-brightness regions to obtain a high-dynamic-range multi-spectral image.

4. The method for detecting the size of the metal shell of a pump based on image analysis according to claim 3, wherein, The three-dimensional geometric information of the metal shell of the pump includes the spatial coordinate point cloud data, key structural feature data, surface curvature information, and geometric dimension parameter data on the surface of the metal shell of the pump.

5. The method for detecting the size of the metal shell of a pump based on image analysis according to claim 4, wherein The method for constructing the dense three-dimensional point cloud model with multi-spectral texture includes: For each three-dimensional point in the same spatial coordinate system, according to the internal and external parameter matrices between different spectral-sensitive cameras, map it to the multi-spectral image plane to obtain the projection function from the point cloud space to the image space; based on this projection function, use the pinhole camera model to project each three-dimensional point to the two-dimensional coordinate position of the corresponding multi-spectral image, and extract the multi-spectral texture value of the image pixel at this position. If the two-dimensional coordinate position is a non-integer coordinate, use bilinear interpolation to extract the multi-spectral texture value from the multi-spectral image; complete the conversion from the spatial geometric point to the multi-spectral texture point to form a texture point cloud with multi-channel image attributes; perform spectral normalization processing on the texture point cloud, and use an octree indexing structure for dense organization to form a dense three-dimensional point cloud model with multi-spectral texture by combining all the texture point clouds.

6. The method for detecting the size of the metal shell of a pump based on image analysis according to claim 5, wherein The method for processing the highly reflective regions in the dense three-dimensional point cloud model includes: Preset the texture value threshold in the visible light band and the brightness gradient change rate threshold. If the texture value in the visible light band corresponding to any position point in the dense three-dimensional point cloud model is greater than the preset texture value threshold in the visible light band, and the brightness gradient change rate is greater than the preset brightness gradient change rate threshold, then mark this point as a highly reflective point, and define the region formed by all the highly reflective points as the highly reflective region. Introduce a multi-frequency encoded light source mechanism, and sequentially project m sets of preset spatial frequency patterns through a structured light projection device to form a spatial frequency set ; For high-reflectivity regions, use multi-frequency heterodyne to recover the phase, and calculate the difference between spatial frequency pairs , construct a low-frequency envelope-assisted decoding; Based on the spatial frequencies in the spatial frequency set and the combination of spatial frequency pairs, generate a heterodyne synthesis phase ; For different combinations of spatial frequencies, different heterodyne synthesis phases are weighted and fused to obtain the final phase. After obtaining the final phase, it is converted into depth coordinates in the dense point cloud through the phase-depth inverse solution function. Adopt the near-infrared texture compensation and multi-spectral fusion rule to repair the position points with the texture value in the visible light band greater than the preset texture value threshold in the visible light band. When there is a blind area in the near-infrared structured light or it is limited by the occlusion reflection angle, introduce a TOF lidar for supplementary measurement, and calculate the depth value corresponding to any position point in the dense three-dimensional point cloud model based on the pulse flight time. For each three-dimensional point position in the dense three-dimensional point cloud model, perform multi-source depth fusion using a piecewise fusion function according to the confidence levels of different depth sources, and finally obtain a dense three-dimensional point cloud model with restored depth information.

7. The method for detecting the size of the metal shell of a pump based on image analysis according to claim 6, characterized in that The method for extracting the diffuse reflection component includes: Install a linear polarizer and set it to rotate continuously at different angles. Collect the reflection images of the pump's metal shell at each angle. For each pixel point in the multi-angle reflection images, construct the Stokes vector of this pixel point through the collected multi-angle reflection images, and further calculate the degree of polarization and polarization angle of each pixel point through the Stokes vector. Preset a degree-of-polarization threshold, extract the area corresponding to the pixel points with a degree of polarization less than the preset degree-of-polarization threshold, and mark it as the diffuse reflection component.

8. The method for detecting the size of the metal shell of a pump based on image analysis according to claim 7, wherein, The method for obtaining the texture-enhanced dense three-dimensional point cloud model includes: Extract pixel-level polarization information based on multi-angle reflection images, and construct polarization vectors to calculate the degree of polarization at each position point , identify and extract the diffuse reflection region according to a preset degree of polarization threshold, and generate a binary mask ; Convert the multi-angle reflection image to the CIE-Lab color space. In the CIE-Lab color space, use the diffuse reflection mask to suppress the brightness of the highly reflective region. For each position point , its brightness value is , construct a brightness regulation function under diffuse reflection constraints, and reduce the brightness of the highly reflective region through the brightness regulation function; Convert the multi-angle reflection images in the processed CIE-Lab color space back to the RGB color space and re-project them into the dense three-dimensional point cloud model, and re-assign texture values to each point cloud, thereby forming a texture-enhanced dense three-dimensional point cloud model.

9. The method for detecting the size of the metal shell of a pump based on image analysis according to claim 8, wherein The method for obtaining the regional dimension characteristics of the pump's metal shell includes: Use the K-nearest neighbor statistical filtering algorithm to identify and remove the outliers in the texture-enhanced dense three-dimensional point cloud model, and combine the moving least squares method to smooth the surface of the dense three-dimensional point cloud. Apply the region growing and RANSAC algorithms to automatically segment the texture-enhanced dense three-dimensional point cloud model, decompose the pump's metal shell into N geometric regions, obtain the basic geometric parameter data of each geometric region and the regional dimension characteristics of the pump's metal shell, and output them in the form of a structured vector.

10. The method for detecting the size of the metal shell of a pump based on image analysis according to claim 9, wherein, The geometric error indicators include linear dimension error, angular error, form and position error, surface feature error, and boundary contour error.

Images