Pointer instrument reading recognition method and system based on structured light depth information

Abstract

The present application relates to a pointer instrument reading recognition method and system based on structured light depth information, belonging to the technical field of machine vision. By calibrating the structured light projector and the camera, the calibration parameters are obtained; the structured light pattern is projected to the surface of the dial of the measured pointer instrument, and the structured light image is synchronously collected; the structured light information is extracted according to the structured light image, the three-dimensional reconstruction is carried out combined with the calibration parameters, and the three-dimensional point cloud of the dial area is generated; based on the three-dimensional point cloud, the height difference between the pointer and the dial plane is used to segment the pointer area, and the pointer point cloud is obtained; the main direction of the pointer is extracted from the pointer point cloud, the pointer angle is calculated combined with the center position of the dial and the reference direction of the zero scale, and the pointer angle is mapped to the reading according to the instrument range. The present application obtains the dial depth information through the structured light three-dimensional reconstruction, directly segments the pointer in the three-dimensional space by using the physical height difference, and realizes the high-precision recognition of the pointer instrument in the complex environment.

Description

Technical Field

[0001] This invention relates to a method and system for recognizing pointer instrument readings based on structured light depth information, belonging to the field of machine vision technology. Background Technology

[0002] Pointer-type instruments (such as SF6 densitometers and pressure gauges) are still widely used in industrial fields due to their simple structure, high reliability, low cost, and suitability for harsh environments. Especially in power systems, SF6 densitometers are key instruments for monitoring the pressure of insulating gases, and the accuracy of their readings directly affects the safe operation of substation equipment. Traditional instrument reading methods rely on manual inspection and recording, which is not only labor-intensive and inefficient, but also highly susceptible to subjective factors and prone to misjudgment, making it difficult to meet the development needs of unmanned substation inspections.

[0003] With the rapid development of machine vision technology, automatic meter reading recognition methods based on image processing have gradually become a research hotspot. Most existing methods acquire dial images using visible light cameras, then utilize image processing or deep learning algorithms to extract pointer position and scale information, thereby calculating the meter reading. However, such two-dimensional image-based methods face numerous challenges in practical applications: outdoor substations experience drastic lighting changes such as strong direct sunlight, shadows, and backlighting, leading to localized overexposure or underexposure of the dial; SF6 densitometers are constantly exposed to harsh climatic conditions such as sunlight, rain, and ultraviolet radiation, causing the transparent cover to age and yellow, and the white base paint inside the dial to oxidize and darken, severely reducing the visual contrast between the dial background and the pointer; furthermore, many densitometers have black pointers and dial substrates, resulting in similar grayscale values ​​in visible light images, creating low-contrast scenes that render traditional image processing algorithms based on grayscale thresholding, edge detection, or color space segmentation almost completely ineffective.

[0004] To address the aforementioned issues, for example, Chinese invention patent application CN109544628A discloses an accurate reading recognition system and method for pointer-type instruments. This system acquires binocular images of the instrument dial using a binocular camera, extracts the pixel coordinates of the main scale of the dial and the pointer tip using image preprocessing, calculates the three-dimensional information of the pointer tip and the three-dimensional information of the dial plane using a binocular detection algorithm, and finally calculates the instrument reading based on the vertical projection repositioning result of the pointer tip on the dial plane. However, the above-mentioned technical solutions rely on visible light binocular images. Under drastic lighting conditions such as strong outdoor light, backlight, and shadows, the image quality deteriorates significantly, resulting in a substantial decrease in the extraction accuracy of the scale and pointer tip. Furthermore, the above-mentioned technical solutions extract the pointer through HSV segmentation and Hough transform in image preprocessing. When the dial ages and yellows, and the pointer and dial colors become similar, the image contrast is extremely low, making it difficult for the above-mentioned image processing algorithms to reliably extract pointer features. In addition, the above-mentioned technical solutions only extract the single-point three-dimensional information of the pointer tip, which is insufficient for obtaining the overall posture information of the pointer. When the pointer is tilted or the camera viewing angle deviation is large, the accuracy of single-point projection correction is limited.

[0005] To address the aforementioned issues, there is an urgent need for a method for identifying instrument readings that does not rely on surface color and lighting conditions and can directly distinguish between pointers and dials based on their physical form. Summary of the Invention

[0006] To address the problems existing in the prior art, this invention proposes a pointer-type instrument reading recognition method and system based on structured light depth information.

[0007] The technical solution of the present invention is as follows: On one hand, the present invention provides a method for identifying pointer-type instrument readings based on structured light depth information, the method comprising: The structured light projector and camera were calibrated to obtain calibration parameters. A structured light pattern is projected onto the dial surface of the pointer instrument under test, and the structured light image modulated by the dial surface is acquired simultaneously. Structured light information is extracted from the structured light image and combined with calibration parameters to perform three-dimensional reconstruction, generating a three-dimensional point cloud of the dial area; Based on the 3D point cloud, the pointer region is segmented using the height difference between the pointer and the dial plane to obtain the pointer point cloud; Extract the pointer's main direction from the pointer point cloud, calculate the pointer angle by combining the dial center position and the zero-scale reference direction, and map the pointer angle into a reading according to the instrument's range.

[0008] Preferably, the system calibration steps include: The camera intrinsic parameters are calibrated using the Zhang Zhengyou calibration method, and the intrinsic parameter matrix and distortion coefficients of the camera are obtained by solving. The intrinsic parameter matrix and distortion coefficients are used for 3D reconstruction. The structured light projector is calibrated, where: For the line structured light scheme, laser stripe images are acquired and projected onto a planar target at different positions. The center points of the stripes in the laser stripe images are extracted, and the laser light plane equation is obtained by fitting the stripe center points. The light plane equation is used for triangulation to calculate three-dimensional coordinates. For the surface structured light scheme, the structured light projector and the camera are jointly calibrated, and the intrinsic parameters of the structured light projector and the extrinsic parameters relative to the camera are obtained. The intrinsic and extrinsic parameters are used for mapping the structured light projection coordinates to the three-dimensional spatial coordinates.

[0009] Preferably, a structured light pattern is projected onto the dial surface of the pointer-type instrument under test, and a structured light image modulated by the dial surface is acquired simultaneously, specifically as follows: When using a line structured light scheme, the projected structured light pattern is a line laser. The line laser is projected onto the surface of the dial being tested through a structured light projector. A stepper motor drives a reflector to make the light plane formed by the line laser sweep across the dial. A camera simultaneously captures each frame of laser stripe images. When using the surface structured light scheme, the projected structured light patterns are Gray code patterns and phase-shifted fringe patterns. The Gray code patterns and phase-shifted fringe patterns are sequentially projected onto the surface of the dial being tested through the structured light projector, and the images corresponding to each pattern are simultaneously acquired by the camera. A single projection covers the entire dial.

[0010] Preferably, structured light information is extracted from the structured light image, and 3D reconstruction is performed in combination with calibration parameters to generate a 3D point cloud of the dial area, specifically: In the line structured light scheme, the center line of each frame of laser stripe images acquired by the camera is extracted, and the three-dimensional coordinates of each light stripe point are calculated using the triangulation principle in combination with the calibration parameters. The calculation results of each frame are then integrated into a three-dimensional point cloud of the entire dial. In the structured light scheme, the absolute phase of each pixel is calculated from the images corresponding to the Gray code pattern and phase shift fringe pattern captured by the camera, or the absolute phase of each pixel is calculated from a single frame structured light image captured by the camera through a deep learning network. Combined with calibration parameters, the three-dimensional coordinates corresponding to each pixel are calculated to generate a three-dimensional point cloud.

[0011] Preferably, extracting the center line of each frame of laser stripe images acquired by the camera includes: extracting the center coordinates of the stripes in each frame of laser stripe images using the gray-scale centroid method, and using 3D modeling... The criteria are used to remove outliers from the center coordinates, resulting in a set of stripe center lines; The absolute phase of each pixel is calculated from the images corresponding to the Gray code pattern and the phase-shifted fringe pattern captured by the camera. This includes: using a four-step phase-shifting method combined with complementary Gray code to perform phase unrolling on the phase-shifted fringe image and Gray code image captured by the camera to obtain the absolute phase. The absolute phase of each pixel in a single frame structured light image acquired by a camera is calculated using a deep learning network. This process includes: inputting the single frame structured light image into a multi-task deep learning network model, which includes a shared encoder, a dual-branch decoder, and a channel-spatial hybrid attention module; the dual-branch decoder outputs a wrapping phase map and a stripe order map in parallel; and synthesizing the wrapping phase value with the corresponding stripe order to obtain the absolute phase map.

[0012] Preferably, the pointer area is divided using the height difference between the pointer and the dial plane, specifically as follows: The generated 3D point cloud is denoised, including statistical outlier filtering or radius filtering. After removing outliers, the denoised 3D point cloud is obtained. The RANSAC algorithm is used to fit the dial plane from the denoised 3D point cloud to obtain the dial plane equation; Calculate the signed distance from each point in the denoised 3D point cloud to the dial plane, and mark the points whose absolute value of the signed distance is greater than the preset height difference threshold as pointer candidate points; Euclidean clustering is performed on the candidate pointer points to obtain several connected regions. Connected regions with areas smaller than a preset value or aspect ratios that do not conform to pointer characteristics are filtered out. The connected region with the largest area and aspect ratio that conforms to pointer characteristics among the remaining connected regions after filtering is selected as the pointer point cloud.

[0013] Preferably, the pointer geometric features include: the number of points or area of ​​the connected domain, the length of the primary axis, the length of the secondary axis, the aspect ratio, and the positional relationship with the center of the dial, wherein the aspect ratio ranges from 5:1 to 15:1.

[0014] Preferably, the extraction of the main pointer direction from the pointer point cloud specifically involves: Principal component analysis is performed on the pointer point cloud, including constructing a covariance matrix, calculating the eigenvalues ​​and eigenvectors of the covariance matrix, obtaining the eigenvector corresponding to the largest eigenvalue as the principal direction vector of the pointer, and projecting the principal direction vector onto the dial plane. The direction of the pointer tip is determined by comparing the distances from both ends of the main direction vector to the center of the dial; specifically, the end with the larger distance is taken as the pointer tip.

[0015] Preferably, the method for determining the center position of the dial is as follows: The pointer point cloud is removed from the denoised 3D point cloud to obtain the dial background point cloud. The dial background point cloud is then fitted with a least-squares circle on the dial plane to obtain the center coordinates and radius of the dial circle. Using the center of the dial as the corner point, the preset zero-scale reference direction as the starting edge, and the projection direction of the pointer's main direction vector onto the dial plane as the ending edge, the angle between the starting edge and the ending edge is calculated and used as the pointer angle.

[0016] On the other hand, the present invention also provides a pointer-type instrument reading recognition system based on structured light depth information for implementing the method described in the present invention, the system comprising: The calibration module performs system calibration on the structured light projector and camera to obtain calibration parameters. The projection and acquisition module projects a structured light pattern onto the dial surface of the pointer instrument under test and simultaneously acquires the structured light image modulated by the dial surface. The 3D reconstruction module extracts structured light information from the structured light image and performs 3D reconstruction in combination with calibration parameters to generate a 3D point cloud of the dial area. The pointer segmentation module, based on 3D point clouds, uses the height difference between the pointer and the dial plane to segment the pointer region, thus obtaining the pointer point cloud; The reading calculation module extracts the pointer's main direction from the pointer point cloud, calculates the pointer angle by combining the dial center position and the zero scale reference direction, and maps the pointer angle into a reading according to the instrument's range.

[0017] The present invention has the following beneficial effects: 1. This invention provides a pointer instrument reading recognition method and system based on structured light depth information. By actively projecting a structured light pattern onto the surface of the dial being measured, a camera synchronously acquires the deformed pattern modulated by the dial surface. Based on the triangulation principle, a three-dimensional point cloud of the dial area is reconstructed. This three-dimensional point cloud is composed of the spatial coordinates of each sampling point on the dial surface and does not involve the color or grayscale information of the dial surface. Even under drastic lighting changes such as strong light, backlight, and shadow, as well as low-contrast scenes such as dial aging and yellowing, and pointers being the same color as the dial, the three-dimensional information of the dial surface can still be stably acquired, improving the environmental adaptability and scene versatility of the instrument reading recognition method. 2. This invention provides a pointer instrument reading recognition method and system based on structured light depth information. By calculating the signed distance from each point to the fitted dial plane in the three-dimensional point cloud, the point cloud with a height difference greater than a threshold is marked as pointer candidate points by using the physical height difference between the pointer and the dial plane. The segmentation is based on the spatial geometric position of the point cloud rather than the pixel gray value. Whether the pointer and the dial are distinguishable in terms of color or material does not affect the pointer segmentation result. The final reading result is also not affected by the gray-scale contrast in the visible light image, thus improving the robustness and reliability of pointer segmentation. 3. This invention provides a pointer instrument reading recognition method and system based on structured light depth information. By performing principal component analysis on the segmented complete pointer point cloud to extract the pointer principal direction, and combining the pointer angle with the center position of the dial, the complete attitude information of the pointer in three-dimensional space is obtained instead of the single point projection position in the two-dimensional image. This effectively eliminates the reading deviation caused by the pointer imaging position shift due to the tilt of the camera view, and improves the accuracy and stability of pointer angle calculation. Attached Figure Description

[0018] Figure 1 This is a flowchart of the method corresponding to an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] It should be understood that the step numbers used in the text are for ease of description only and are not intended to limit the order in which the steps are performed.

[0021] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0022] The terms “comprising” and “including” indicate the presence of the described feature, whole, step, operation, element and / or component, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0023] The term “and / or” refers to any combination of one or more of the associated listed items, as well as all possible combinations, and includes these combinations.

[0024] Example 1: This embodiment provides a pointer-type instrument reading recognition method based on structured light depth information, such as... Figure 1 As shown, the method is applicable to the automatic reading identification of various pointer-type instruments such as SF6 gas density meters, pressure gauges, and thermometers. The following explanation uses the pointer reading identification of an SF6 gas density meter as an example; the method includes: S1. Perform system calibration on the structured light projector and camera to obtain calibration parameters, including: S11. The camera's intrinsic parameters are calibrated using Zhang Zhengyou's calibration method. Specifically: Using a checkerboard calibration board, 15-20 images containing complete checkerboard corner points are captured under different postures. Sub-pixel level coordinates of the checkerboard corner points in each image are extracted. Since the three-dimensional coordinates of each corner point on the calibration board in the calibration board coordinate system are known (directly determined by the actual physical size of the checkerboard), a correspondence can be established between the known three-dimensional coordinates of the corner points in the world coordinate system and the pixel coordinates extracted from the image for each calibration board image. Based on this correspondence, a projection mapping equation is established according to the pinhole imaging principle, expressed as: ; In the formula, The scale factor in homogeneous coordinates; for The rotation matrix, for The translation vector, and These are extrinsic parameters that describe the rotation and translation transformations from the world coordinate system to the camera coordinate system. These are the pixel coordinates of the image; World coordinates; The intrinsic parameter matrix is ​​expressed by the formula: ; In the formula, and It is the scale factor of the focal length in the horizontal and vertical directions of the image, that is, the ratio of the physical length of the focal length to the pixel size. The two values ​​are close but not equal. The principal point coordinates, i.e., the intersection of the optical axis and the imaging plane, are the origin of the image coordinate system; Due to the distortion inherent in actual lenses, let the normalized ideal coordinates be... (The normalization refers to dividing the coordinates of a point in three-dimensional space in the camera coordinate system by the depth value of that point), to obtain the distance in front of the camera. The projected position on the plane, the unit of this coordinate is 1; while the distorted coordinate is... The distortion model can be expressed by the following formula: ; ; In the formula, , and The radial distortion coefficient describes the stretching or compression distortion of image points along the radial direction (from the image center to the edge); and The tangential distortion coefficient describes the tangential offset distortion caused by the lens not being perfectly parallel to the imaging plane; This represents the squared distance from the normalized coordinate point to the center point of the image. An overdetermined system of equations was constructed using the correspondence between the world coordinates and image coordinates of all corner points. The system was then solved using the least squares method to obtain the camera's intrinsic parameter matrix and distortion coefficients. S12. In this embodiment, different calibration methods are used depending on the type of structured light projector. Specifically: S121. For the line structured light scheme, the line laser beam is expanded to form a spatial light plane. It is necessary to determine the equation of this light plane in three-dimensional space. Specifically, during calibration, the line laser is projected onto planar targets placed at different spatial positions. The planar targets are marked with known positions, and their spatial pose can be determined by the image coordinates of the marks. An image containing laser stripes is acquired, the center point of the stripes is extracted, and the pixel coordinates of the center point of the stripes are converted into three-dimensional spatial coordinates using the pose parameters of the targets. The above operation was repeated at different target locations to obtain multiple sets of three-dimensional calibration points. The equation of the light plane was then fitted using the linear least squares method and expressed as follows: ; In the formula, These are the normal vector components of the light plane; These are the three-dimensional coordinates in the camera coordinate system. This is the distance parameter from the light plane to the camera origin; S122. For the surface structured light scheme, the structured light projector is considered as an inverse camera, and the system composed of the projector and the camera is jointly calibrated; a composite coded pattern is projected onto the calibration board, images are acquired synchronously, the correspondence between the pixel coordinates of the structured light projector and the three-dimensional spatial coordinates of the calibration board is established, and the intrinsic parameters of the structured light projector are obtained by solving. and the external parameters (rotation matrix) relative to the camera Translation vector ).

[0025] S2. Project a structured light pattern onto the dial surface of the pointer instrument under test, and simultaneously acquire the structured light image modulated by the dial surface. The method of structured light projection and acquisition also depends on the structured light scheme used. Specifically: S21. In the line structured light scheme adopted in this embodiment, a line laser is used in conjunction with a Powell prism to diffuse the laser into a thin light plane with a fan angle of 30°~60°. The line laser wavelength is selected as 850nm, 650nm or 450nm depending on the working conditions. A narrow-band filter is installed in front of the camera lens, with its center wavelength matching the selected laser wavelength, a bandwidth of ±5nm~±10nm, and a cutoff depth ≥OD4 (i.e., transmittance ≤0.01%), to suppress ambient light interference, so that this embodiment can work normally under strong sunlight of 100,000lx. A micro stepper motor drives a reflector to rotate at a constant angular velocity of approximately 0.5 rad / s, allowing the light plane to sweep across the entire dial surface. The camera synchronously acquires images at a frame rate of no less than 30fps. Each frame of the image records a contour line formed by the intersection of the light plane and the dial surface. The acquisition sequence of each frame of the image records the scanning position of the light plane on the dial. The scanning position information corresponds to each frame of the image and is used to determine the spatial stitching relationship of the reconstruction results of each frame. S22. In the structured light scheme adopted in this embodiment, a DLP projection module sequentially projects 6-8 Gray code patterns and 4 phase-shifting fringe patterns. The Gray code patterns provide coarse-grained regional encoding for the image plane of the structured light projector, while the phase-shifting fringe patterns provide fine-grained phase information. The combination of the two achieves absolute encoding for each pixel. A narrow-band filter is installed in front of the camera lens, with its center wavelength matching the projection wavelength of the DLP projection module (450nm in this embodiment). The camera synchronously captures images corresponding to each projected pattern, and a single projection can cover the entire dial area without the need for scanning motion.

[0026] S3. Extract structured light information from the structured light image and perform 3D reconstruction based on calibration parameters to generate a 3D point cloud of the dial area. Specifically: S31. When using a line structured light scheme: Extract the center line of the laser stripe from each frame of the laser stripe image. For a frame of the image, determine the spatial region of the laser stripe by the eight-neighborhood determination method. That is, based on the difference between the gray value of the pixel and the gray value of its eight neighboring pixels, the pixels belonging to the laser stripe are selected. The sub-pixel coordinates of the fringe center are calculated using the gray-level centroid method for the selected fringe region, including: On a cross-section perpendicular to the direction of stripe extension, the centroid position and center coordinates are calculated using grayscale values ​​as weights. The calculation formula is: ; ; In the formula, The row coordinates of the fringe center; The column coordinates of the fringe center; and These are the row and column indices of the pixel, respectively. The number of rows in the selected area; The number of columns in the selected area; For pixels The grayscale value at that location; After center extraction of the entire image, 3D model is used. The criterion for removing outliers is to calculate the mean and standard deviation of the coordinates of all center points and remove data points that deviate from the mean by more than 3 times the standard deviation. After obtaining the pixel coordinates of the fringe center line, the pixel coordinates are back-projected into a spatial ray originating from the camera's optical center using the camera's intrinsic parameters calibrated in step S11. Simultaneously, using the constraint that the pixel lies on the laser plane (i.e., the light plane equation calibrated in step S12), the ray equation and the light plane equation are solved simultaneously to find their intersection, thus obtaining the three-dimensional coordinates of the point. ; Perform the above calculation on all stripe center points of all frames, and stitch the calculation results of each frame together according to the scanning order recorded in step S2 to form a complete three-dimensional point cloud of the entire dial. S22. When using a surface structured light scheme: This embodiment employs a four-step phase-shifting method to calculate the wrapping phase from the acquired phase-shifted fringe images. The grayscale distribution of the four fringe images is expressed by the following formula: ; In the formula, the subscript Indicates the sequence number of the four images; These are the pixel coordinates of the image; For the first Image in pixels The grayscale value at that location; For the first The phase shift corresponding to the four phase shifts is taken sequentially in the four phase shift steps. , , , That is, each time increase ; This refers to the ambient light component, which is the background brightness in the image that does not change with the stripes; This refers to the stripe modulation amplitude, i.e., the contrast of the stripes; The wrap phase to be solved depends on the position of the pixel in the image plane of the structured light projector; The solution obtained by the four-step phase shift method It is called a wrapped phase because the arctangent function restricts its value to... Within the interval, there is periodicity. Jump, same phase value Possible correspondence The integer multiples of this value plus its different positions cannot be directly used to determine the unique position of each pixel in the structured light projector image plane. The formula for calculating the wrap phase is: ; In order to eliminate To address the ambiguity, phase unrolling is performed using the acquired Gray code pattern. The Gray code pattern provides a globally unique periodicity number for each pixel. This number is then multiplied by... Then, by adding the wrapped phase, a continuous absolute phase value can be obtained, eliminating the ambiguity caused by periodic jumps; after adaptive median filtering to correct the error, a smooth absolute phase without jumps is obtained. After obtaining the absolute phase, the phase value can be directly mapped to the corresponding pixel coordinates on the image plane of the structured light projector, thus determining the position of the structured light projector pixel corresponding to the camera pixel. For the camera pixel, its two-dimensional coordinates in the camera image (determined by the image position) are known, and its corresponding two-dimensional coordinates in the structured light projector image are also known through the absolute phase. Combining the camera intrinsic parameters, structured light projector intrinsic parameters, and extrinsic parameters between them calibrated in step S1, a triangulation equation for stereo vision is established to solve for the three-dimensional spatial coordinates corresponding to the pixel. Perform the above calculations on all pixels to generate a 3D point cloud.

[0027] S4. Based on the 3D point cloud, the pointer region is segmented using the height difference between the pointer and the dial plane to obtain the pointer point cloud. Specifically: S41. Preprocess the 3D point cloud using statistical outlier filtering or radius filtering, wherein: Statistical outlier filtering involves calculating the average distance from each point to all points in its neighborhood, which follows a Gaussian distribution. Points that exceed the mean by several times the standard deviation are identified as outliers and removed. Radius filtering includes identifying a point as an outlier if the number of points in its neighborhood within a specified radius is less than a threshold. Statistical outlier filtering or radius filtering effectively removes isolated noise points caused by glass reflection and environmental noise, resulting in a denoised 3D point cloud. S42. Because the 3D point cloud contains a large number of outliers that are not located on the dial plane, such as pointers and scales, directly using all points for least-squares fitting of the plane would produce serious bias. Therefore, this embodiment uses the RANSAC algorithm for robust estimation, including: From the denoised point cloud, three points are randomly selected to define a plane. The distances from all points to this plane are counted, and points with distances less than a preset threshold are counted as interior points. This process is repeated multiple times, retaining the plane model with the most interior points. Finally, a least-squares fine-fit is performed using all interior points of this model to obtain the dial plane equation. : ; In the formula, Let be the coordinates of any point in three-dimensional space; It is the unit normal vector of the plane, and its direction indicates the orientation of the plane; For the plane's bias term, its physical meaning is the signed distance from the origin of the coordinate system to the plane (when the normal vector is a unit vector); S43. Calculate the distance from each point in the denoised point cloud to the dial plane. The signed distance, with a positive or negative sign, indicates whether the point is above or below the dial plane. The formula for calculating the signed distance is: ; In the formula, For the first point cloud The signed distance from each point to the dial plane, where the sign indicates which side of the plane the point is on, and the absolute value indicates the perpendicular distance from the point to the plane; For the first point cloud The three-dimensional coordinates of a point in the camera coordinate system; Points located on the dial plane Pointers and other raised structures are located above the plane. And the values ​​are relatively large; a small number of noise points may be located below the plane. ; Will Points with a height difference greater than a preset threshold are marked as candidate pointer points. The value of the height difference threshold is adapted to the actual physical thickness of the pointer. In this embodiment, for an SF6 density meter with a pointer thickness of approximately 0.5mm to 2mm, the height difference threshold is set to 0.4mm. S44. Among the candidate points after height threshold segmentation, in addition to the pointer itself, there may also be other structures that protrude from the dial plane, such as scale lines, nameplates, and central axis, as well as a small number of residual discrete noise points. This embodiment further utilizes spatial continuity and geometric shape features for screening, including: Euclidean clustering is performed on the candidate pointer points, and points whose spatial distance is less than the cluster radius (about 1 mm) are grouped into the same connected domain. Point clouds belonging to the same physical structure are continuously clustered in space, and there are usually gaps between different structures. Therefore, clustering can separate different structures into independent connected domains. Calculate the geometric features of each connected component, including area (number of points), principal axis length (maximum dimension in the direction of extension of the connected component), secondary axis length (maximum dimension in the vertical direction), and aspect ratio (ratio of principal axis length to secondary axis length). The pointer's geometric features are a slender strip structure, with an aspect ratio typically between 5:1 and 15:1; the scale lines, while also slender, are significantly shorter than the pointer; discrete noise points are scattered, with an aspect ratio close to 1:1. Connected regions with aspect ratios outside the 5:1-15:1 range or excessively small areas are filtered out, and the largest area among the remaining connected regions is selected as the pointer point cloud.

[0028] S5. Extract the pointer's principal direction from the pointer point cloud, calculate the pointer angle by combining the dial center position and the zero-scale reference direction, and map the pointer angle to a reading according to the instrument's range. Specifically: S51. The center of the dial is the spatial reference point for angle calculation. The pointer point cloud is removed from the denoised 3D point cloud to obtain the dial background point cloud. This part of the point cloud represents a static structure on the dial plane. The dial background point cloud is then projected onto the dial plane. Above, the least squares method is used to fit the equation of the circle in the projection plane: ; In the formula, The two-dimensional coordinates of any point within the projection plane, these coordinates are derived from the three-dimensional coordinates. The result is obtained by projecting the image onto the dial plane. Let be the coordinates of the center of the circle to be determined, that is, the projection position of the pointer rotation axis on the dial plane; Let be the radius of the circle to be determined, which is the radius of the effective area of ​​the dial. Least squares fitting finds the optimal center of the circle. and radius This minimizes the sum of the squared distances from all projection points to the circumference, thus obtaining the coordinates of the circle's center. S52. Perform principal component analysis on the pointer point cloud, including: Calculate the centroid of the pointer point cloud Construct the covariance matrix: ; In the formula, for The covariance matrix has diagonal elements representing the variance of the point cloud along the three coordinate axes, and off-diagonal elements representing the covariance between the directions. The total number of points in the pointer point cloud; The first point cloud for pointers The three-dimensional coordinate vector of each point ; Let be the centroid coordinate vector of the pointer point cloud, i.e., the average position of all points; superscript Indicates matrix transpose; The three eigenvalues ​​of the covariance matrix correspond to the degree of dispersion of the point cloud in the three principal directions, and the corresponding eigenvectors correspond to the three principal directions of the point cloud distribution. The eigenvector corresponding to the largest eigenvalue is the direction in which the point cloud has the greatest degree of dispersion. For a slender pointer structure, this direction is the direction in which the pointer extends. Calculate the eigenvalues ​​and eigenvectors of the covariance matrix, and take the eigenvector corresponding to the largest eigenvalue as the principal direction vector of the pointer. Since the pointer may not be perfectly parallel to the dial plane, project this vector onto the dial plane. The component perpendicular to the dial plane is eliminated to obtain the projection direction of the pointer on the dial plane. At this point, the main axis direction is undirected. It is also necessary to determine which end is the pointer tip. In this embodiment, the distances from the two ends of the main axis to the center of the dial are compared. The main axis direction is extended to the boundary of the pointer point cloud. The distances from the boundary points at both ends to the center of the circle are calculated respectively. The end with the larger distance is the pointer tip (because the tip is located near the circumference of the dial, and the tail is close to the center of the circle). In this embodiment, distance criteria are preferred to determine the pointer tip. At the same time, in the scenario of continuous multi-frame monitoring, this embodiment further combines historical time sequence information to verify the tip direction: if the angle deviation between the tip direction determined in the current frame and the pointer direction in the previous frame exceeds a preset threshold (such as 30° / frame), the pointer movement trend in the historical frames is combined to make corrections to ensure the temporal continuity of the pointer tip direction and avoid misjudgment of the tip due to single-frame noise. S53. Taking the center of the dial as the vertex of the angle, the preset zero-scale reference direction (determined by the position of the 0-scale mark on the dial, pointing towards the 0-scale point on the circumference of the dial) as the starting edge, and the projection direction of the pointer's main direction vector onto the dial plane as the ending edge, calculate the angle between the two. ; In this embodiment, the zero-scale reference direction is set as follows: The pointer points to the vector. ,but: ; Based on the quadrant where the pointer is located Perform calibration to ensure the angle value is within the range of 0° to 360°; If the instrument's reading scale is linear, meaning the pointer rotation angle is proportional to the reading, then a linear mapping is performed based on the range. Let the angle corresponding to zero be... The angle corresponding to full scale is The corresponding reading range is arrive Then the current reading for: ; S6. Display the mapped readings on the local terminal and upload them to the monitoring center via serial port, Modbus, 4G, LoRa or Wi-Fi. Trigger an alarm when the reading exceeds the preset range. Preferably, the data output by the alarm includes one or more of the following: device number, timestamp, pointer angle, instrument reading, 3D reconstruction quality index (such as plane fitting residual), number of valid points and alarm status, so that the monitoring center can perform data recording and trend analysis.

[0029] Example 2: This implementation adds a multispectral fusion enhancement scheme to the first embodiment to further improve the robustness of the system under extreme lighting conditions. Specifically, in the structured light projection and acquisition in S2, a second near-infrared camera and a corresponding 850nm LED supplementary lighting module are added. The visible light channel camera (i.e., the main camera in the first embodiment) is used to capture the dial scale pattern and assist in center positioning; the infrared channel camera is used to enhance the signal-to-noise ratio of the structured light stripe image. The two cameras achieve simultaneous acquisition through hardware synchronous triggering. The two channel images are accurately registered through feature point matching. Then, the infrared-enhanced stripe image is input into the three-dimensional reconstruction module in S3. Since feature point matching is a conventional processing method in this field, it will not be described in detail here.

[0030] Example 3: This embodiment, based on the surface structured light scheme of Example 1, adopts a single-frame structured light reconstruction method based on a multi-task deep neural network as an alternative to the multi-frame acquisition mode in the surface structured light scheme. This embodiment uses the multi-task deep learning network model MTU-SEDD to achieve end-to-end computation from a single-frame deformed fringe image to the absolute phase. The MTU-SEDD includes a shared encoder, a dual-branch decoder, and a channel-spatial hybrid attention module, wherein: A1. The input of the shared encoder is a single-frame deformable stripe image, and the output is a multi-scale feature map. The shared encoder adopts a U-Net-like encoding structure, which is composed of multiple downsampling modules connected in series. Each downsampling module includes a multi-branch dilated convolution module and a convolutional downsampling layer with a stride of 2. The input image is passed through each downsampling module in sequence, gradually reducing the spatial resolution and increasing the number of channels, and finally outputting a multi-scale feature map containing local stripe deformation information and global stripe distribution information; The feature map output by the shared encoder is simultaneously provided to the two subsequent decoder branches, thus achieving feature sharing. Furthermore, the multi-branch dilated convolution module comprises four parallel branches, wherein: The first branch adopts The convolution kernel (dilation rate 1) is used to preserve the original spatial structure of the input features; the second branch employs... The convolution kernel (dilation rate 1) is used to extract local stripe deformation features; the third branch employs... A convolution kernel (dilation rate 2) is used to capture moderate-range fringe variations; the fourth branch employs... The convolution kernel (dilation rate 4) is used to capture the global stripe level distribution pattern; The outputs of the four branches are concatenated along the channel dimension, and then... Convolution is used to fuse features, resulting in the output feature map of the multi-branch dilated convolution module; A2. The dual-branch decoder receives the multi-scale feature map output by the shared encoder and outputs two prediction results in parallel, wherein: The first branch in the dual-branch system is the wrap phase regression branch, which outputs a wrap phase map. The wrap phase regression branch consists of multiple upsampling modules connected in series. Each upsampling module contains a transposed convolutional upsampling layer and a feature fusion layer. By skipping connections, the feature map of the corresponding layer of the encoder is concatenated with the upsampled feature map in the channel dimension and then convolved and fused to gradually restore the spatial resolution. Finally, the wrap phase prediction value with the same resolution as the input image is output. The second branch in the dual-branch system is the stripe level classification branch, which outputs the stripe level map. The structure of the stripe level classification branch is symmetrical to the wrapping phase regression branch, and it also includes an upsampling module and a skip connection. Finally, it outputs the stripe period number corresponding to each pixel. A3. Embed a channel-space hybrid attention module in the output of the shared encoder and in each upsampling module of the dual-branch decoder. The channel-space hybrid attention module includes a channel attention submodule and a spatial attention submodule, which are used to enhance the characteristic response of the phase transition region and suppress noise interference, wherein: The channel attention submodule performs global average pooling on the input feature map, compressing the spatial information of each channel into a scalar. This scalar is then passed through a fully connected layer and activation function to generate a channel weight vector, which is used to weight each channel of the original feature map. The spatial attention submodule performs average pooling on each channel of the input feature map and introduces an anisotropic Gaussian kernel to smooth the spatial attention map, enhancing the response in phase transition regions. The channel attention output and the spatial attention output are multiplied to obtain the final attention-enhanced feature map. A4. Preferably, the MTU-SEDD is trained using a multi-task joint loss function. The wrapping phase regression branch uses mean squared error loss to calculate the pixel-level difference between the predicted wrapping phase and the true wrapping phase. The stripe level classification branch uses cross-entropy loss to calculate the classification error between the predicted stripe level and the true stripe level. The total loss of MTU-SEDD is the weighted sum of the losses of the two branches. During training, a dynamic gradient balancing strategy is adopted, which adaptively adjusts the weight coefficients of each task by calculating the norm of the loss gradient of each task. During inference, a single frame of deformed stripe image is input into the trained MTU-SEDD network. After forward propagation, the network outputs both a wrapping phase map and a stripe order map. The wrapping phase value is multiplied by the corresponding stripe order and then added together to obtain the absolute phase map.

[0031] Example 4: This embodiment provides a pointer-type instrument reading recognition system based on structured light depth information, used to implement the method described in Embodiment 1. The system includes: The calibration module performs system calibration on the structured light projector and camera to obtain calibration parameters. The projection and acquisition module projects a structured light pattern onto the dial surface of the pointer instrument under test and simultaneously acquires the structured light image modulated by the dial surface. The 3D reconstruction module extracts structured light information from the structured light image and performs 3D reconstruction in combination with calibration parameters to generate a 3D point cloud of the dial area. The pointer segmentation module, based on 3D point clouds, uses the height difference between the pointer and the dial plane to segment the pointer region, thus obtaining the pointer point cloud; The reading calculation module extracts the pointer's main direction from the pointer point cloud, calculates the pointer angle by combining the dial center position and the zero scale reference direction, and maps the pointer angle into a reading according to the instrument's range.

[0032] It is worth noting that the system described in this invention belongs to the same inventive concept as the method described in Embodiment 1, 2 or 3, and will not be repeated here.

[0033] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, the simultaneous existence of A and B, or the existence of B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0034] Those skilled in the art will recognize that the units and algorithm steps described in the embodiments disclosed herein can be implemented using electronic hardware, computer software, or a combination of electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0035] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the aforementioned method embodiment one, and will not be repeated here.

[0036] In the several embodiments provided in this application, any function, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the part of the technical solution of this application that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0037] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention.

Claims

1. A pointer-type instrument reading recognition method based on structured light depth information, characterized in that, The method includes: The structured light projector and camera were calibrated to obtain calibration parameters. A structured light pattern is projected onto the dial surface of the pointer instrument under test, and the structured light image modulated by the dial surface is acquired simultaneously. Structured light information is extracted from the structured light image and combined with calibration parameters to perform three-dimensional reconstruction, generating a three-dimensional point cloud of the dial area; Based on the 3D point cloud, the pointer region is segmented using the height difference between the pointer and the dial plane to obtain the pointer point cloud; Extract the pointer's main direction from the pointer point cloud, calculate the pointer angle by combining the dial center position and the zero-scale reference direction, and map the pointer angle into a reading according to the instrument's range.

2. The method according to claim 1, characterized in that, The system calibration steps include: The camera intrinsic parameters are calibrated using the Zhang Zhengyou calibration method, and the intrinsic parameter matrix and distortion coefficients of the camera are obtained by solving. The intrinsic parameter matrix and distortion coefficients are used for 3D reconstruction. The structured light projector is calibrated, where: For the line structured light scheme, laser stripe images are acquired and projected onto a planar target at different positions. The center points of the stripes in the laser stripe images are extracted, and the laser light plane equation is obtained by fitting the stripe center points. The light plane equation is used for triangulation to calculate three-dimensional coordinates. For the surface structured light scheme, the structured light projector and the camera are jointly calibrated, and the intrinsic parameters of the structured light projector and the extrinsic parameters relative to the camera are obtained. The intrinsic and extrinsic parameters are used for mapping the structured light projection coordinates to the three-dimensional spatial coordinates.

3. The method according to claim 2, characterized in that, A structured light pattern is projected onto the dial surface of the pointer-type instrument under test, and the structured light image modulated by the dial surface is acquired simultaneously. Specifically: When using a line structured light scheme, the projected structured light pattern is a line laser. The line laser is projected onto the surface of the dial being tested through a structured light projector. A stepper motor drives a reflector to make the light plane formed by the line laser sweep across the dial. A camera simultaneously captures each frame of laser stripe images. When using the surface structured light scheme, the projected structured light patterns are Gray code patterns and phase-shifted fringe patterns. The Gray code patterns and phase-shifted fringe patterns are sequentially projected onto the surface of the dial being tested through the structured light projector, and the images corresponding to each pattern are simultaneously acquired by the camera. A single projection covers the entire dial.

4. The method according to claim 3, characterized in that, Structured light information is extracted from the structured light image and combined with calibration parameters for 3D reconstruction to generate a 3D point cloud of the dial area, specifically: In the line structured light scheme, the center line of each frame of laser stripe images acquired by the camera is extracted, and the three-dimensional coordinates of each light stripe point are calculated using the triangulation principle in combination with the calibration parameters. The calculation results of each frame are then integrated into a three-dimensional point cloud of the entire dial. In the structured light scheme, the absolute phase of each pixel is calculated from the images corresponding to the Gray code pattern and phase shift fringe pattern captured by the camera, or the absolute phase of each pixel is calculated from a single frame structured light image captured by the camera through a deep learning network. Combined with calibration parameters, the three-dimensional coordinates corresponding to each pixel are calculated to generate a three-dimensional point cloud.

5. The method according to claim 4, characterized in that, Extracting the center lines of the stripes in each frame of laser stripe images acquired by the camera includes: extracting the center coordinates of the stripes in each frame of laser stripe images using the gray-scale centroid method, and then using 3D modeling... The criteria are used to remove outliers from the center coordinates, resulting in a set of stripe center lines; The absolute phase of each pixel is calculated from the images corresponding to the Gray code pattern and the phase-shifted fringe pattern captured by the camera. This includes: using a four-step phase-shifting method combined with complementary Gray code to perform phase unrolling on the phase-shifted fringe image and Gray code image captured by the camera to obtain the absolute phase. The absolute phase of each pixel in a single frame structured light image acquired by a camera is calculated using a deep learning network. This process includes: inputting the single frame structured light image into a multi-task deep learning network model, which includes a shared encoder, a dual-branch decoder, and a channel-spatial hybrid attention module; the dual-branch decoder outputs a wrapping phase map and a stripe order map in parallel; and synthesizing the wrapping phase value with the corresponding stripe order to obtain the absolute phase map.

6. The method according to claim 1, characterized in that, The pointer area is divided using the height difference between the pointer and the dial plane, specifically as follows: The generated 3D point cloud is denoised, including statistical outlier filtering or radius filtering. After removing outliers, the denoised 3D point cloud is obtained. The RANSAC algorithm is used to fit the dial plane from the denoised 3D point cloud to obtain the dial plane equation; Calculate the signed distance from each point in the denoised 3D point cloud to the dial plane, and mark the points whose absolute value of the signed distance is greater than the preset height difference threshold as pointer candidate points; Euclidean clustering is performed on the candidate pointer points to obtain several connected regions. Connected regions with areas smaller than a preset value or aspect ratios that do not conform to pointer characteristics are filtered out. The connected region with the largest area and aspect ratio that conforms to pointer characteristics among the remaining connected regions after filtering is selected as the pointer point cloud.

7. The method according to claim 6, characterized in that, The pointer's geometric features include: the number of points or area of ​​the connected domain, the length of the primary axis, the length of the secondary axis, the aspect ratio, and its positional relationship with the center of the dial, wherein the aspect ratio ranges from 5:1 to 15:

1.

8. The method according to claim 1, characterized in that, Extracting the main pointer direction from the pointer point cloud specifically involves: Principal component analysis is performed on the pointer point cloud, including constructing a covariance matrix, calculating the eigenvalues ​​and eigenvectors of the covariance matrix, obtaining the eigenvector corresponding to the largest eigenvalue as the principal direction vector of the pointer, and projecting the principal direction vector onto the dial plane. The direction of the pointer tip is determined by comparing the distances from both ends of the main direction vector to the center of the dial; specifically, the end with the larger distance is taken as the pointer tip.

9. The method according to claim 6, characterized in that, The method for determining the center position of the dial is as follows: The pointer point cloud is removed from the denoised 3D point cloud to obtain the dial background point cloud. The dial background point cloud is then fitted with a least-squares circle on the dial plane to obtain the center coordinates and radius of the dial circle. Using the center of the dial as the corner point, the preset zero-scale reference direction as the starting edge, and the projection direction of the pointer's main direction vector onto the dial plane as the ending edge, the angle between the starting edge and the ending edge is calculated and used as the pointer angle.

10. A pointer-type instrument reading recognition system based on structured light depth information, used to implement the method described in any one of claims 1-9, characterized in that, The system includes: The calibration module performs system calibration on the structured light projector and camera to obtain calibration parameters. The projection and acquisition module projects a structured light pattern onto the dial surface of the pointer instrument under test and simultaneously acquires the structured light image modulated by the dial surface. The 3D reconstruction module extracts structured light information from the structured light image and performs 3D reconstruction in combination with calibration parameters to generate a 3D point cloud of the dial area. The pointer segmentation module, based on 3D point clouds, uses the height difference between the pointer and the dial plane to segment the pointer region, thus obtaining the pointer point cloud; The reading calculation module extracts the pointer's main direction from the pointer point cloud, calculates the pointer angle by combining the dial center position and the zero scale reference direction, and maps the pointer angle into a reading according to the instrument's range.

Images