Spring balance reading detection method based on imaging features
By using a deep neural network based on imaging features and a dial scale template matching method, the difficulty of reading detection of spring balances in intelligent experimental operation examination systems was solved, and high-precision reading estimation was achieved.
Patent Information
- Application Number
- CN202410227294.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-02-29
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2044-02-29
AI Technical Summary
In existing intelligent experimental operation examination systems, the automated reading detection of spring balances faces problems such as shaking, pointer oscillation, small pointer area, poor clarity of reading scale and inconsistencies, which lead to difficulties in detection and identification.
A method based on imaging features is adopted, which uses semantic segmentation of deep neural networks to obtain the spring balance region, performs singularity screening and geometric distortion correction in polygonal regions, and combines dial scale template matching to complete reading estimation. This includes instance segmentation, bilateral thresholding, and time-series image analysis to improve detection accuracy.
It achieves high-precision reading detection of spring balances, avoids the need for text recognition in the dial pointer area, adapts to the differences in different dial areas, and reduces detection errors.
Smart Images

Figure CN118097646B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and image recognition technology, specifically to a method for detecting the readings of a spring balance based on imaging features. Background Technology
[0002] A spring balance is a device used to measure force, primarily to determine the magnitude of the force applied to it. Its principle is based on the elastic properties of a spring; the degree of deformation of the spring determines the magnitude of the force applied. In secondary school, spring balances are commonly used in mechanics experiments and teaching activities. These experiments and activities help students understand and learn the principles of mechanics, including:
[0003] 1) By applying different magnitudes of force to stretch or compress the spring, and using a force gauge to display or record the corresponding deformation and values, students can understand the relationship between force and spring deformation.
[0004] 2) Using a spring balance to conduct Hooke's Law experiments, students can verify Hooke's Law based on the experimental data and understand the elastic restoring property of springs.
[0005] 3) Students can use a spring balance to conduct force balance experiments, compare the magnitude of the forces acting on different objects, and learn how to use a spring balance to measure the weight of an object or the force applied to it.
[0006] 4) Students can use spring balances to build simple suspension systems and study the effects of various forces on the system, such as tension, pressure, and friction.
[0007] 5) The use of a spring balance can help students understand the principles of mechanics, such as Newton's laws, pressure, tension, gravity and buoyancy, and verify these theories through experiments.
[0008] The above-mentioned teaching experiments and activities allow students to intuitively and vividly experience, observe and verify the principles of mechanics. By operating the spring balance and recording experimental data, they can deepen their understanding of mechanical concepts and also help cultivate their experimental skills, observation abilities and scientific thinking.
[0009] With the widespread application of intelligent experimental operation examination systems, the automated identification and estimation of readings from devices such as spring balances has become a primary challenge. Common reading detection methods for devices with relatively fixed positions (voltmeters, ammeters, balances, etc.) utilize motion detection and background modeling to determine the pointer's location, and then estimate the corresponding reading based on text recognition and device structure. Unlike these devices, spring balances exhibit numerous characteristics during measurement, including shaking, pointer oscillation, small pointer area, and poor and inconsistent scale clarity, making their detection and identification more difficult. Summary of the Invention
[0010] To address the challenges of detecting and identifying spring balance readings in intelligent experimental operation examination systems, which suffer from issues such as shaking, pointer oscillation, small pointer area, poor clarity of reading scales, and inconsistencies, this invention provides a spring balance reading detection method based on imaging features. This method obtains the pointer's positioning information and combines it with the spring balance's position information for reading detection.
[0011] This invention is achieved through the following technical solution: a spring balance reading detection method based on imaging features, which utilizes semantic segmentation of a deep neural network to obtain the spring balance region of a single frame image, and performs singularity point screening based on polygon regions to complete geometric distortion correction; it extracts imaging features from the semantically segmented region to obtain pointer position information, and combines this with dial scale template matching to complete reading estimation, specifically including the following steps:
[0012] S1: Obtain the spring balance region with shape features through instance segmentation, and obtain an accurate dictionary representation of the spring balance scale template based on preliminary manual identification; specifically:
[0013] S1.1: Extract multiple frames from a video file containing the dial markings of a spring balance; and randomly select the image training set, validation set, and test set for training the semantic segmentation neural network model.
[0014] S1.2: Use labelme, cvat and other annotation tools to manually mark the dial area of the spring balance with polygons, and try to make the marked area only include the dial area.
[0015] S1.3: Select a specific network model structure, including VGG and ResNet, and use instance segmentation frameworks such as YOLOv8 and segmentation-models-pytorch in conjunction with the dataset to train the network weights W with a certain degree of intersection-union ratio. s Used to segment the spring balance region in an image;
[0016] S1.4: Select a single frame image with good image quality from the labeled image of the spring balance dial area, and manually calibrate the position of the scale line on the edge of the dial pointer groove in the image ((x l y l ), (x r y r G), where G refers to the actual gravity value, forms the tick dictionary. M And use this image as the ruler template image I M ;
[0017] S1.5: A real-time single-frame image I is obtained by extracting frames from the video to be detected, and the trained network and its weights W are used. s The grayscale image of the spring balance dial region I is obtained by segmenting I and performing necessary mathematical morphological operations (dilation, erosion, opening and closing operations, etc.) and color space transformations to obtain a single connected region. s The connected region is far from the edge of image I, and its image size is h×w. The Laplacian operator is used to apply this to image I. s Calculate the sharpness index for the region; for regions with a sharpness index gs less than T... gs The judgment is that the spring balance is in motion and the pointer position is not detected.
[0018] S2: Based on the differences in image representation within the spring balance region, extract the pointer's image positioning information under different conditions using a bilateral thresholding operation; specifically:
[0019] S2.1: For I s Perform a two-sided threshold operation: Select the threshold Th max and Th min , where 0≤Th min <Th max ≤L, where L is the maximum gray value of the image, respectively for I s After performing binary thresholding and debinary thresholding, I is obtained. tp and I ts ;
[0020] S2.2: Calculate I tp The set of edge points of each connected region in and its corresponding minimum bounding rectangle feature set For any minimum bounding rectangle feature Use the form ((x) c ,y c ),(w c ,h c ),θ c ) indicates that (x c ,y c ) is the center of the rectangle at I s The coordinates, (w c ,g c θ represents the width and height of the rectangle. c The direction angle represents the minimum bounding rectangle, which is the angle between the top edge of the minimum bounding rectangle rotated clockwise around its top left vertex and the positive direction of the horizontal axis; when the center of the minimum bounding rectangle x c in I s The x-coordinate in the middle deviates from I s Vertical centerline x s Less than Th offsetAt that time, the minimum bounding rectangle is a potential pointer region, (x c ,y c This represents a potential pointer space coordinate; in the case of multiple potential pointer space coordinates, the connected region x is selected. c with I s The minimum absolute value of the difference between the x and y coordinates of the longitudinal centerline corresponds to (x) c ,y c ) as pointer space location loc tp When traversing R tp If no potential pointer space coordinates exist, then the threshold Th is considered not to have been passed. max The binary threshold operation detects the pointer position and continues with subsequent steps;
[0021] S2.3: Calculate I ts The set of edge points of each connected region and its corresponding minimum bounding rectangle feature set When the center of the smallest bounding rectangle is at I s The x-coordinate in the middle deviates from I s Vertical distance s from the central axis s Less than Th offset Furthermore, the horizontal width of the circumscribed rectangle is less than its vertical height and λw. s When the value is smaller than λ, it is considered as a potential region for a pointer slot, where λ is a constant coefficient less than 1.
[0022] S2.4: For all pointer slot potential regions corresponding to θ c For values whose absolute value is less than θ, the coordinates of the y-coordinate of the center point of the smallest bounding rectangle are used. c The potential region features of the pointer slots are arranged in ascending order of size, and the center point position of the rectangular side that is approximately perpendicular to the direction of the rectangular arrangement is obtained. and calculate Save as a potential pointer space coordinate loc ts When R ts When there exists only one minimum bounding rectangle in the array, ep U and ep D Save it as a potential pointer space coordinate.
[0023] S3: Reduce detection errors that may occur in single-frame images using temporal image analysis and clustering analysis methods; specifically: perform steps S1.5 and S2 on the subsequent N-1 consecutive frames, with a frame extraction time of Δt, and use the average value of c of the frames that are detected n times and whose position standard deviation is less than the mean square value of the area of all the bounding rectangles as the basis for the detection error. max The pointer position; otherwise, for all loc... tsPerform K-means clustering analysis, and use the cluster center with the largest number of clustered samples as the cluster based on Th... min The pointer position; denoted by loc tp or loc ts Let loc be the image position of the pointer; and select one frame as the reference image I based on the calculated pointer position and the relationship between the N frames. D .
[0024] S4: Combine the shape features obtained from instance segmentation to implement the standard spatial mapping of the spring balance dial, and estimate the pointer reading of the spring balance according to the custom reading dictionary structure. Specifically: for I D and I M Extracting the spring balance region to obtain I Ds and I Ms Contour detection is used to obtain the set of contour points C. Ds and C Ms By matching contour points, similar matching contour point pairs are found, and I is calculated based on the matching point pairs. D To I M The homography matrix H of the transformation converts loc to I. M coordinates in loc M Calculate loc M With Tick M The distance between the points representing the scale lines and their midpoints is loc. M It is classified as the scale line with the shortest distance.
[0025] Compared with the prior art, the present invention has the following beneficial effects: The spring balance reading detection method based on imaging features provided by the present invention obtains the dictionary structure of the spring balance dial scale with structural information by combining instance segmentation with manual labeling for reading estimation, avoiding the need for text recognition or scale recognition in the dial pointer area during the reading process; the bilateral threshold spring balance pointer positioning takes into account the differentiated representation of the dial area during the use of the spring balance, and provides corresponding pointer positioning strategies for different representation conditions. Attached Figure Description
[0026] Figure 1 This is a flowchart of the present invention. Case 1 in the figure is |x c -x s | <Th offset Case 2 is w c <min(h c ,λw s Case 3 is: θ c <θ. Detailed Implementation
[0027] The present invention will be further described below with reference to specific embodiments.
[0028] A method for detecting spring balance readings based on imaging features is proposed. This method utilizes semantic segmentation of a deep neural network to obtain the spring balance region in a single frame image, and then performs singularity point filtering based on polygonal regions to correct geometric distortion. Imaging features are extracted from the semantically segmented region to obtain pointer position information, which is then combined with dial scale template matching to estimate the reading. The process is as follows: Figure 1 As shown, the specific steps include the following:
[0029] S1: Obtain the spring balance region with shape features through instance segmentation, and obtain an accurate dictionary representation of the spring balance scale template based on preliminary manual identification; specifically:
[0030] S1.1: Extract multiple frames from a video file containing the dial markings of a spring balance; and randomly select the image training set, validation set, and test set for training the semantic segmentation neural network model.
[0031] S1.2: Use labelme, cvat and other annotation tools to manually mark the dial area of the spring balance with polygons, and try to make the marked area only include the dial area.
[0032] S1.3: Select a specific network model structure, including VGG and ResNet, and use instance segmentation frameworks such as YOLOv8 and segmentation-models-pytorch in conjunction with the dataset to train the network weights W with a certain degree of intersection-union ratio. s Used to segment the spring balance region in an image;
[0033] S1.4: Select a single frame image with good image quality from the labeled image of the spring balance dial area, and manually calibrate the position of the scale line on the edge of the dial pointer groove in the image ((x l ,y l ),(x r ,y r (),G), where G refers to the actual gravity value, forming the tick dictionary. M And use this image as the ruler template image I M ;
[0034] S1.5: A real-time single-frame image I is obtained by extracting frames from the video to be detected, and the trained network and its weights W are used. s The grayscale image of the spring balance dial region I is obtained by segmenting I and performing necessary mathematical morphological operations (dilation, erosion, opening and closing operations, etc.) and color space transformations to obtain a single connected region. sThe connected region is far from the edge of image I, and its image size is h×w. The Laplacian operator is used to apply this to image I. s Calculate the sharpness index for the region; for regions with a sharpness index gs less than T... gs The judgment is that the spring balance is in motion and the pointer position is not detected.
[0035] S2: Based on the differences in image representation within the spring balance region, extract the pointer's image positioning information under different conditions using a bilateral thresholding operation; specifically:
[0036] S2.1: For I s Perform a two-sided threshold operation: Select the threshold Th max and Th min , where 0≤Th min <Th max ≤L, where L is the maximum gray value of the image, respectively for I s After performing binary thresholding and debinary thresholding, I is obtained. tp and I ts ;
[0037] S2.2: Calculate I tp The set of edge points of each connected region in and its corresponding minimum bounding rectangle feature set For any minimum bounding rectangle feature Use the form ((x) c ,y c ),(w c ,h c ),θ c ) indicates that (x c ,y c ) is the center of the rectangle at I s The coordinates, (w c ,h c θ represents the width and height of the rectangle. c The direction angle represents the minimum bounding rectangle, which is the angle between the top edge of the minimum bounding rectangle rotated clockwise around its top left vertex and the positive direction of the horizontal axis; when the center of the minimum bounding rectangle x c in I s The x-coordinate in the middle deviates from I s Vertical centerline x s Less than Th offset At that time, the minimum bounding rectangle is a potential pointer region, (x c ,y c This represents a potential pointer space coordinate; in the case of multiple potential pointer space coordinates, the connected region x is selected. c with I s The minimum absolute value of the difference between the x and y coordinates of the longitudinal centerline corresponds to (x) c,y c ) as pointer space location loc tp When traversing R tp If no potential pointer space coordinates exist, then the threshold Th is considered not to have been passed. max The binary threshold operation detects the pointer position and continues with subsequent steps;
[0038] S2.3: Calculate I ts The set of edge points of each connected region and its corresponding minimum bounding rectangle feature set When the center of the smallest bounding rectangle is at I s The x-coordinate in the middle deviates from I s Vertical distance from the centerline x s Less than Th offset Furthermore, the horizontal width of the circumscribed rectangle is less than its vertical height and λw. s When the value is smaller than λ, it is considered as a potential region for a pointer slot, where λ is a constant coefficient less than 1.
[0039] S2.4: For all pointer slot potential regions corresponding to θ c For values whose absolute value is less than θ, the coordinates of the y-coordinate of the center point of the smallest bounding rectangle are used. c The potential region features of the pointer slots are arranged in ascending order of size, and the center point position of the rectangular side that is approximately perpendicular to the direction of the rectangular arrangement is obtained. and calculate Save as a potential pointer space coordinate loc ts When R ts When there exists only one minimum bounding rectangle in the array, ep U and ep D Save it as a potential pointer space coordinate.
[0040] S3: Reduce detection errors that may occur in single-frame images using temporal image analysis and clustering analysis methods; specifically: perform steps S1.5 and S2 on the subsequent N-1 consecutive frames, with a frame extraction time of Δt, and use the average value of c of the frames that are detected n times and whose position standard deviation is less than the mean square value of the area of all the bounding rectangles as the basis for the detection error. max The pointer position; otherwise, for all loc... ts Perform K-means clustering analysis, and use the cluster center with the largest number of clustered samples as the cluster based on Th... min The pointer position; denoted by loc tp or loc ts Let loc be the image position of the pointer; and select one frame as the reference image I based on the calculated pointer position and the relationship between the N frames. D .
[0041] S4: Combine the shape features obtained from instance segmentation to implement the standard spatial mapping of the spring balance dial, and estimate the pointer reading of the spring balance according to the custom reading dictionary structure. Specifically: for I D and I M Extracting the spring balance region to obtain I Ds and I Ms Contour detection is used to obtain the set of contour points C. Ds and C Ms By matching contour points, similar matching contour point pairs are found, and I is calculated based on the matching point pairs. D To I M The homography matrix H of the transformation converts loc to I. M coordinates in loc M Calculate loc M With Tick M The distance between the points representing the scale lines and their midpoints is loc. M It is classified as the scale line with the shortest distance.
[0042] The scope of protection claimed by this invention is not limited to the specific embodiments described above. Moreover, for those skilled in the art, this invention can have various modifications and alterations. Any modifications, improvements, and equivalent substitutions made within the concept and principles of this invention should be included within the scope of protection of this invention.
Claims
1. A method for detecting the reading of a spring balance based on imaging features, characterized in that: The spring balance region of a single frame image is obtained by semantic segmentation using a deep neural network, and singular points are screened based on polygon regions to correct geometric distortion. Imaging feature extraction is performed on the semantically segmented region to obtain pointer position information, and reading estimation is completed by combining it with dial scale template matching. The specific steps include the following: S1: Obtain the spring balance region with shape features through instance segmentation, and obtain an accurate dictionary representation of the spring balance scale template based on preliminary manual identification; details are as follows: S1.1: Extract multiple frames from the video file containing the dial markings of the spring balance to obtain multiple image files; and randomly select the image training set, validation set, and test set for training the semantic segmentation neural network model; S1.2: Use the annotation tool to manually mark the dial area of the spring balance with polygons. The marked area only includes the dial area. S1.3: Select a specific network model structure, use an instance segmentation framework combined with a dataset for training, and obtain network weights that satisfy a certain intersection-union ratio. The network weights Used to segment the spring balance region in an image; S1.4: Select a single frame image with good image quality from the labeled image of the spring balance dial area, and manually calibrate the position of the scale line on the edge of the dial pointer groove in the image. G refers to the actual value of gravity, forming a scale dictionary. And use this image as a ruler template image. ; S1.5: A real-time single-frame image obtained by extracting frames from the video to be detected. Using the trained network and its weights segmentation After necessary mathematical morphological operations and color space transformations, a grayscale image of the dial area of a spring balance, representing a single connected region, is obtained. The connected region is far from the image. The edge, its image size is Using the Laplace operator Calculate the sharpness index for the region, and then calculate the sharpness index. Less than The judgment is that the spring balance is in motion and the pointer position is not detected; S2: Based on the differences in image representation within the spring balance region, extract the pointer's image positioning information under different conditions using a bilateral thresholding operation; specifically as follows: S2.1: For Perform a two-sided threshold operation: Select the threshold and , of which 0 , For the maximum grayscale value of the image, respectively... After performing binary thresholding and debinary thresholding, the result is obtained. and ; S2.2: Calculation The set of edge points of each connected region in and its corresponding minimum bounding rectangle feature set For any minimum bounding rectangle feature use It means that among them Center of the rectangle coordinates Let the width and height be the rectangle. The direction angle represents the minimum bounding rectangle, which is the angle between the top edge of the minimum bounding rectangle rotated clockwise around its top left vertex and the positive direction of the horizontal axis; when the center of the minimum bounding rectangle... exist The x-coordinate deviation Vertical centerline Less than At that time, the minimum bounding rectangle is a potential pointer region. That is, a potential pointer space coordinate; in the case of multiple potential pointer space coordinates, a connected region is selected. and The minimum absolute value of the difference between the x-coordinates of the longitudinal centerline corresponds to As a pointer space location When traversing If no potential pointer space coordinates exist, the threshold is considered not met. The binary threshold operation detects the pointer position and continues with subsequent steps; S2.3: Calculation The set of edge points of each connected region and its corresponding minimum bounding rectangle feature set When the center of the smallest bounding rectangle is at The x-coordinate deviation Vertical distance of the centerline Less than Furthermore, the horizontal width of the circumscribed rectangle is less than its vertical height and width. When the value is smaller than the specified value, it is considered a potential region for a pointer slot. It is a constant coefficient less than 1; S2.4: Corresponding to all pointer slot potential areas The absolute value is less than According to the ordinate of the center point of the smallest bounding rectangle The potential region features of the pointer slots are arranged in ascending order of size, and the center point position of the rectangular side that is approximately perpendicular to the direction of the rectangular arrangement is obtained. and ,calculate Save as a potential pointer space coordinate ;when When there is only one minimum bounding rectangle in the array, and Save it as a potential pointer space coordinate; S3: Reduce the probabilistic detection error in single-frame images using temporal image analysis and clustering analysis methods; specifically as follows: For the subsequent continuum The frame image is processed through steps S1.5 and S2, with a frame extraction duration of [duration missing]. , for those detected Secondly, and the standard deviation of the location is less than the mean square value of the area of all circumscribed rectangles. The average value as based on The pointer position; otherwise, for all Perform K-means clustering analysis, and use the cluster center with the largest number of cluster samples as the basis for clustering. The pointer position; record or Image position of pointer ; Based on the calculated relationship between the pointer position and the N frames, one frame is selected as the reference image. ; S4: Combine the shape features obtained from instance segmentation to implement standard spatial mapping of the spring balance dial, and estimate the pointer reading of the spring balance based on a custom reading dictionary structure; details are as follows: right and Extraction of the spring balance region and Contour detection is used to obtain a set of contour points. and By matching contour points, similar matching contour point pairs are found, and the results are calculated based on the matching point pairs. Towards homography matrix of transformation ,Will Transform into coordinates in ,calculate and The distance between the point representing the scale line and the midpoint will It is classified as the scale line with the shortest distance.
2. The method for detecting the reading of a spring balance based on imaging features according to claim 1, characterized in that: In step S1.2, the annotation tool is LabelMe or CVAT; in S1.3, the specific network model structure is VGG or ResNet, and the instance segmentation framework is either YOLOv8 or segmentation-models-pytorch; in S1.5, the mathematical morphology operation is either dilation, erosion, or opening / closing operations.
