A pointer-type electrical instrument reading method and device for grading physical, chemical, and biological experiments

Abstract

The present invention relates to the field of image processing technology, specifically a method and device for reading a pointer-type electrical meter for scoring physical, chemical, and biological experiments. The method utilizes an imaging scaling factor and a key point mapping matrix to map the scale information and pointer rotation center information of a pointer-type electrical meter template onto a top-down perspective image, and performs readings based on the meter type, main scale value, and division value. The method can accurately locate the key points and pointer needle tip position of the pointer-type electrical meter, thereby achieving rapid scale positioning based on the electrical meter template mapping, and then performing readings based on the angles between the pointer and each main scale. On the one hand, the method alleviates the impact of poor dial visibility on scale positioning through a template mapping positioning method, and on the other hand, alleviates the impact of poor dial visibility on reading analysis by analyzing the angles between the pointer and each scale, thereby improving the usability and accuracy of the electrical meter reading method.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a pointer-type electrical instrument reading method and device for scoring physical, chemical, and biological experiments. Background Art

[0002] Currently, manual reading of analog electrical instruments is often used in the teaching and assessment of physics, chemistry, and biology experiments, both domestically and internationally. Automating the reading of analog electrical instruments could improve teaching and assessment efficiency and reduce the influence of subjective factors. Current research focuses on further improving the usability and accuracy of automated reading of analog electrical instruments.

[0003] Currently, there are problems with scale positioning and reading when the dial area is not visible in the automated reading of pointer-type electrical instruments. The following are some of the problems:

[0004] 1. Scale positioning problem when the dial area has poor visibility:

[0005] Among existing methods for reading analog electrical meters, mainstream scale location methods include threshold segmentation, image segmentation, and semantic segmentation. These methods all place high demands on the visual field of the instrument's dial area. In actual experimental operations, images of experimental conditions are often captured from a bird's-eye view. In such cases, the dial area of ​​an analog electrical meter may experience partially blurred or obscured scales, resulting in poor visibility. Under these conditions, existing methods for locating the scale of an analog electrical meter often fail to accurately locate the scale. This limitation may affect the applicability of automated reading methods for electrical meters.

[0006] 2. Reading problems when the dial area has poor visibility:

[0007] Among existing methods for reading pointer-type electrical instruments, the mainstream approach is to record the endpoints of a sector-shaped scale and calculate the angle or distance between the pointer and the endpoints to obtain the reading. If the endpoints of the sector-shaped scale are blurred or obscured, these methods often fail to accurately locate the reading range. This limitation can affect the accuracy of automated reading methods for electrical instruments.

[0008] Existing solutions fail to fully account for the poor visibility of analog electrical meter dials during physics, chemistry, and biology experiments. In reality, students use analog electrical meters in more complex and varied scenarios. Therefore, a more adaptable automated reading method for analog electrical meter readings is needed that can adapt to the diverse operational scenarios encountered during physics, chemistry, and biology experiment teaching and assessment, while improving reading accuracy. Summary of the Invention

[0009] In response to the problems of scale positioning and reading problems encountered in the automated reading of pointer-type electrical instruments in the prior art when the dial area has poor visibility, the present invention provides a method and device for reading pointer-type electrical instruments for use in grading physical, chemical, and biological experiments. The method can accurately locate the key points and pointer needle tip positions of the pointer-type electrical instrument, thereby achieving rapid scale positioning based on the electrical instrument template mapping, and then reading based on the angle between the pointer and each major scale. On the one hand, this method alleviates the impact of poor dial visibility on scale positioning through template mapping positioning methods. On the other hand, it alleviates the impact of poor dial visibility on reading analysis by analyzing the angle between the pointer and each scale, thereby improving the usability and accuracy of the electrical instrument reading method.

[0010] To achieve the above objectives, the present invention provides the following technical solutions:

[0011] In a first aspect, the present invention provides a method for reading a pointer-type electrical instrument, comprising:

[0012] Generate a pointer type electrical instrument template, wherein the pointer type electrical instrument template includes key point information, scale information and pointer rotation center information of the pointer type electrical instrument template;

[0013] Obtaining a top-view image of the pointer-type electrical instrument in the electrical operation experiment from the video stream, and obtaining key point information of the top-view image using a preset target detection algorithm;

[0014] Obtaining the meter type, main scale values, and division values ​​of the pointer-type electrical meter based on key point information of the top-view image;

[0015] Calculating an imaging scaling factor and a key point mapping matrix according to the key point information of the pointer-type electrical instrument template and the key point information of the top-view image;

[0016] The scale information and pointer rotation center information of the pointer-type electrical meter template are mapped onto the top-view image using the imaging zoom factor and key point mapping matrix, and readings are taken in combination with the meter type, main scale value and division value.

[0017] In a second aspect, the present invention provides a pointer-type electrical meter reading device, comprising:

[0018] An electrical instrument template generation module, which is used to generate a pointer-type electrical instrument template, wherein the pointer-type electrical instrument template includes key point information, scale information, and pointer rotation center information of the pointer-type electrical instrument template;

[0019] A key point information acquisition module is used to acquire a top-down perspective image of the pointer-type electrical instrument in the electrical operation experiment from a video stream, and obtain key point information of the top-down perspective image through a preset target detection algorithm;

[0020] an electrical meter scale generation module, configured to obtain the meter type, main scale values, and division values ​​of the pointer-type electrical meter based on key point information of the top-view image;

[0021] A mapping module, configured to calculate an imaging scaling factor and a key point mapping matrix based on key point information of the pointer-type electrical instrument template and key point information of the top-view image;

[0022] A reading module is used to map the scale information and pointer rotation center information of the pointer-type electrical meter template onto the top-view image using the imaging zoom factor and key point mapping matrix, and to perform readings in combination with the meter type, main scale value and division value.

[0023] In a third aspect, an embodiment of the present invention further provides an electronic device, including a processor and a memory;

[0024] The memory is used to store programs;

[0025] The processor executes the program to implement the method described above.

[0026] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the method described above.

[0027] In a fifth aspect, embodiments of the present invention further provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned method.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] The present invention can efficiently and accurately read the pointer-type electrical instruments in physical, chemical and biological experiments, thereby performing intelligent scoring on related experiments. Specifically,

[0030] 1. The present invention extracts the bounding box and category information of the pointer-type electrical instrument and its components through target detection, thereby extracting the information of the pointer-type electrical instrument from the video image;

[0031] 2. The present invention generates a pointer-type electrical instrument template and performs template mapping and positioning based on the correspondence between the key points of the pointer-type electrical instrument template and the key points in the video image. This obtains the main scale coordinates of the pointer-type electrical instrument dial and the coordinates of the pointer rotation center point in the video image, thus achieving the acquisition of the main scale direction.

[0032] 3. The present invention extracts the coordinates of the needle tip of the pointer in the dial of the pointer-type electrical instrument through key point detection, and obtains the pointer direction by combining it with the coordinates of the pointer rotation center point; further, the present invention improves the accuracy of analysis by comparing the pointer direction with the minimum angle determination, the second minimum angle determination, the angle ratio determination of the main scale direction, and calculating the reading of the pointer-type electrical instrument through experimental phenomena on the screen in many aspects.

[0033] It can be seen that the present invention has broad application prospects in the field of physics, chemistry and biology experimental teaching and evaluation. It improves the applicability and accuracy of the pointer-type electrical instrument reading method, reduces the dependence on experimental image acquisition equipment, and is expected to promote the innovation and development of physics, chemistry and biology experimental teaching and evaluation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0035] Figure 1 Flowchart of a method for reading a pointer-type electrical meter in an embodiment of the present invention;

[0036] Figure 2 Schematic diagram of a pointer-type electrical instrument template in an embodiment of the present invention;

[0037] Figure 3 Schematic diagram of reading in an embodiment of the present invention;

[0038] Figure 4 This is another flow chart of the method for reading a pointer-type electrical meter in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0040] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0041] Example 1:

[0042] See also Figures 1 to 3 The embodiment of the present invention provides a method for reading a pointer-type electrical instrument, which may specifically include the following steps:

[0043] Step 100: Generate a pointer-type electrical instrument template, wherein the pointer-type electrical instrument template includes key point information, scale information, and pointer rotation center information of the pointer-type electrical instrument template.

[0044] In this step, the main task is to start the processing flow. Then, the template of the pointer-type electrical meter and its key features are defined, including the coordinates and relative position relationships of key elements such as the negative terminal, positive terminal, dial letters, scales, and the pointer rotation center of the meter. Next, the scale set and scale value of the electrical meter are defined. Depending on the meter range, the scale value and scale set will change accordingly. Finally, the coordinates of the pointer rotation center and its position in the template are set.

[0045] The key point information of the pointer-type electrical instrument template specifically includes:

[0046] The negative terminal coordinate of the pointer-type electrical instrument is C0, and the two positive terminal coordinates are C1 and C2;

[0047] The letter coordinate A on the dial of a pointer-type electrical instrument;

[0048] The template reference line L1 with C0 and C2 as the starting and ending points respectively and its template length l1, C1 is the midpoint of L1;

[0049] The template reference line L2 with C1 and A as the starting point and end point respectively and its template length l2.

[0050] The scale information of the pointer-type electrical instrument template specifically includes:

[0051] There are n main scales in the pointer type electrical instrument, corresponding to the scale values ​​E = {e1, e2, ..., e n}, under different ranges, the value of E is different;

[0052] Set the main scale set P in the pointer electrical instrument template, P = {p1, p2, ..., p n}, where p i =[x i ,y i , d i ] T ,(x i ,y i ) is the scale p i The coordinates of the projection position on the L1 reference line relative to C0, d i It is p i The distance from the reference line L1.

[0053] The pointer rotation center information of the pointer-type electrical instrument template specifically includes:

[0054] Set the pointer rotation center S in the pointer type electrical instrument template = [x s ,y s , d s ] T , where (x s ,y s ) is the coordinate of the projection position of the pointer rotation center S on the reference line L1 relative to C0, d s is the distance between S and the template reference line L1.

[0055] Step 200: Obtain a top-view image of the pointer-type electrical instrument in the electrical operation experiment from a video stream, and obtain key point information of the top-view image using a preset target detection algorithm.

[0056] In this step, the main task is to obtain a top-down image of the electrical operation experiment from the video stream and establish a coordinate system with the upper left corner of the image as the origin, the width direction as the positive direction of the x-axis, the height direction as the positive direction of the y-axis, and the normal direction of the image plane as the z-axis. Then, the target detection algorithm is used to process the image to identify and obtain the bounding boxes of the electrical instrument, the dial letter bounding box, the negative terminal, and the two positive terminals that appear in the image.

[0057] The key point information of the top-view image is obtained by using a preset target detection algorithm, specifically including:

[0058] Establish a coordinate system with the upper left corner of the top-down image as the origin, the width direction as the positive direction of the x-axis, the height direction as the positive direction of the y-axis, and the plane normal direction of the top-down image as the z-axis;

[0059] The target detection algorithm is used to detect the target in the top-view image, and the pointer electrical meter bounding box R1, the dial letter bounding box R2, the negative terminal bounding box R3, the positive terminal bounding box R4 and the positive terminal bounding box R5 appearing in the top-view image are obtained.

[0060] Step 300: Obtain the meter type, main scale values, and division values ​​of the pointer-type electrical meter based on the key point information of the top-view image.

[0061] In this step, the main task is to determine the type of meter based on the letter category on the meter dial, and determine the meter range based on the wiring condition of the positive terminal, and accordingly obtain the main scale values ​​and division values ​​of the current electrical meter.

[0062] The method of obtaining the meter type, main scale value E, and division value D of the pointer-type electrical meter based on the key point information of the top-view image specifically includes:

[0063] Determine the type of the electric meter based on the dial letter boundary box R2 of the pointer electric meter;

[0064] Determine the meter range based on the negative terminal boundary box R3, the positive terminal boundary box R4, and the positive terminal boundary box R5.

[0065] According to the type and range of the electric meter, the main scale value E and the division value D of the pointer-type electrical meter are obtained.

[0066] Step 400: Calculate an imaging scaling factor and a key point mapping matrix according to the key point information of the pointer-type electrical instrument template and the key point information of the top-view image.

[0067] In this step, the imaging zoom factor, rotation angle, and pitch distortion coefficient are obtained by calculating the relationship between the actual position of the key points and the corresponding positions in the template. In addition, the key point mapping matrix is ​​calculated, which is used to map the scale set and pointer rotation center in the template to the actual image.

[0068] The step of calculating the imaging scaling factor based on the key point information of the pointer-type electrical instrument template and the key point information of the top-view image specifically includes:

[0069] The center positions of the negative terminal bounding box R3, the positive terminal bounding box R4, and the positive terminal bounding box R5 are respectively used as key points C0′, C1′, and C2′;

[0070] C0′ and C2′ serve as the starting point and end point respectively to form a reference line segment L1′=[(x0, y0), (x2, y2)], where (x0, y0), (x2, y2) are the coordinates of C0′ and C2′ respectively, and the length of L1′ is

[0071] The imaging scaling coefficient α of the pointer-type electrical instrument is calculated according to L1 ′ and the template reference line L1 , wherein the calculation formula of the imaging scaling coefficient α is α=l1 ′ / l1 .

[0072] The step of calculating the key point mapping matrix according to the key point information of the pointer-type electrical instrument template and the key point information of the top-view image specifically includes:

[0073] Assume that the rotation angle between L1′ and the positive direction of the x-axis is ρ, then the sine value is sinρ=(y2-y0) / l1′, and the cosine value is cosρ=(x2-x0) / l1′;

[0074] The center position of the dial letter bounding box R2 is used as the letter key point A′ of the pointer electrical instrument, C1′ and A′ are used as the starting point and end point respectively, and the reference line segment L2′=[(x1, y1), (x a ,y a )], where (x1, y1), (x a ,y a ) are the coordinates of C1′ and A′, and the length of L2′ Assume that the rotation angle between L2′ and the positive direction of the x-axis is θ, then the sine value is sinθ=(y a -y1) / l2′, the cosine value is cosθ=(x a -x1) / l2′;

[0075] The pitch angle distortion coefficient β of the pointer type electrical instrument is calculated based on the length ratio of L2′ and L1′ and the length ratio of the pointer type electrical instrument template reference lines L2 and L1. The calculation formula of the pitch angle distortion coefficient β is β = (l2′ / l1′) / (l2 / l1);

[0076] The key point mapping matrix M is obtained according to the rotation angle ρ, the rotation angle θ and the pitch angle distortion coefficient β according to the following preset formula, where δ is the distortion adjustment coefficient:

[0077]

[0078] Step 500: Map the scale information and pointer rotation center information of the pointer-type electrical meter template onto the top-view image using the imaging zoom factor and key point mapping matrix, and perform readings based on the meter type, main scale values, and division values.

[0079] In this step, the main purpose is to use the imaging scaling factor and the key point mapping matrix to scale, rotate and translate the main scale set in the electrical instrument template to obtain a global coordinate set in the image. Then, similarly, the pointer rotation center is mapped to obtain its global coordinates in the image. Next, a local image of the electrical instrument bounding box area is selected, and the needle tip key point detection model is input to obtain the key point coordinates with the highest confidence, and mapped to the global image coordinate system as the coordinates of the pointer tip. Then, for each scale in the mapped main scale coordinate set, the angle between it and the pointer line segment is calculated. By finding the scale with the smallest angle and the second smallest angle, the angle relationship and scale value are used to calculate the pointer reading. Finally, based on the angle between the pointer and the two scales, combined with the scale value, the final reading result is obtained through the set rules.

[0080] The step of mapping the scale information and the pointer rotation center information of the pointer-type electrical instrument template onto the top-view image using the imaging scaling factor and the key point mapping matrix specifically includes:

[0081] According to the imaging scaling coefficient α and the key point mapping matrix M, the scales in the main scale set P in the electrical instrument template are scaled and rotated, and the key point C0′ is translated to obtain the main scale coordinate set P′.

[0082] P′={p1′, p2′,...,p n ′}, where p i ′=[x i ′,y i ′] T It is the global image coordinate of a major scale, which is determined by the scale p in the pointer type electrical instrument template. i =[x i ,y i , d i ] T The mapping is obtained and the calculation process is:

[0083]

[0084] According to the imaging scaling factor α and the key point mapping matrix M, the pointer rotation center S in the pointer type electrical instrument template is calculated as follows: s ,y s , d s ] T Mapping is performed to obtain the global image coordinates of the pointer rotation center S′=[x s ′,y s ′] T , the calculation process is:

[0085]

[0086] The method of reading the meter type, the main scale value E and the division value D specifically includes:

[0087] Select the local image of the electrical instrument bounding box R1 area, input the preset needle tip key point detection model, and output the key point coordinates (x t ,y t ), and maps the coordinates to the global graph coordinates (x t ′,y t ′) as the pointer tip coordinate T=(x t ′,y t ′), use the pointer rotation center coordinate S′ as the starting point and the pointer tip coordinate T as the end point to obtain the pointer line segment L z =[(x s ′,y s ′),(x t ′,y t ′)];

[0088] For each scale p in the main scale coordinate set P′ i ′, taking S′ as the starting point, p i ' as the end point to obtain the scale segment L i =[(x s ′,y s ′),(x i ′,y i ′)], calculate L z The angle between each scale segment is calculated as follows:

[0089]

[0090] According to the previous step, we can obtain the minimum angle σ i and the second smallest angle σ j The main scale line segments and the corresponding scale values ​​e i and e j , read the angle between the pointer segment and the minimum angle scale segment, and the angle between the pointer and the second minimum angle scale segment. Let the reading result be result. The reading process is:

[0091]

[0092] When the pointer tip is to the left of the smallest major scale mark, that is, x tWhen x'<x0', the pointer reading is less than the lower limit of the instrument measurement range. At this time, the reading of the part exceeding the range is estimated by calculating the ratio of the angle between the pointer and the smallest main scale to the angle between the smallest main scale and the second smallest main scale, and the final reading result is obtained by subtracting the estimated reading from the smallest angle scale reading; when the pointer tip is between the smallest main scale and the largest main scale, that is, x0'≤x t '≤x n ', the pointer reading is within the range of the instrument. At this time, the reading is estimated by calculating the ratio of the angle between the pointer and the smallest angle scale to the angle between the smallest angle scale and the second smallest angle scale, and the final reading result is obtained by adding the result to the smallest angle scale reading; when the pointer tip is on the right side of the largest main scale, that is, x n '<x t ', the pointer reading is greater than the upper limit of the instrument range. At this time, the reading of the part exceeding the range is estimated by calculating the ratio of the angle between the pointer and the largest main scale to the angle between the largest main scale and the second largest main scale, and the final reading result is obtained by adding the result to the smallest angle scale reading.

[0093] Embodiment 2

[0094] See Figures 1 to 4 , a method for reading a pointer-type electrical instrument provided by an embodiment of the present invention may specifically further include steps 1-20:

[0095] Step 1: Start.

[0096] Step 2: Define the template of the pointer-type electrical instrument and the range of the electrical meter. As Figure 2 shown, the electrical instrument template includes the negative terminal connection coordinate C0 of the electrical instrument, the two positive terminal connection coordinates C1 and C2 of the electrical instrument, the electrical instrument dial letter coordinate A, the template reference line L1 with C0 and C2 as the starting point and the end point respectively and its template length l1, C1 is the midpoint of L1, the template reference line L2 with C1 and A as the starting point and the end point respectively and its template length l2.

[0097] Step 3: Define that there are n main scales in the electrical instrument, corresponding to scale values E = {e1, e2,..., e n}, and the values of E are different in different range cases.

[0098] Step 4: Define the main scale set P in the electrical instrument template, P = {p1, p2,..., p n}, where p i = [x i , y i , d i T , (x i , y i ) is the scale pi The coordinates of the projection position on the L1 reference line relative to C0, d i It is p i The distance from the reference line L1.

[0099] Step 5: Define the pointer rotation center S in the electrical instrument template = [x s ,y s , d s ] T , where (x s ,y s ) is the coordinate of the projection position of the pointer rotation center S on the reference line L1 relative to C0, d s is the distance between S and the template reference line L1.

[0100] Step 6: Obtain a top-down image of the electrical manipulation experiment from the video stream. Establish a coordinate system with the upper left corner of the image as the origin, the width direction as the positive x-axis, the height direction as the positive y-axis, and the normal direction of the image plane as the z-axis.

[0101] Step 7: Use the target detection algorithm to perform target detection on the image and obtain the electrical instrument bounding box R1, the electrical instrument dial letter bounding box R2, the negative terminal bounding box R3, and the two positive terminal bounding boxes R4 and R5 that appear in the image.

[0102] Step 8: Determine the meter type based on the letter category on the meter dial, determine the meter range based on the positive terminal connection, and obtain the main scale values ​​E and division values ​​D of the current meter.

[0103] Step 9: Use the center positions of R3, R4 and R5 as the key points C0′, C1′ and C2′ respectively, and C0′ and C2′ as the starting point and end point respectively to form the reference line segment L1′=[(x0, y0), (x2, y2)], where (x0, y0), (x2, y2) are the coordinates of C0′ and C2′ respectively, and the length of L1′ is

[0104] Step 10: Calculate the imaging scaling factor α of the electrical instrument based on L1′ and the template reference line L1. The calculation formula of α is α=l1′ / l1.

[0105] Step 11: Assume that the rotation angle between L1′ and the positive direction of the x-axis is ρ, then the sine value is sinρ=(y2-y0) / l1′, and the cosine value is cosρ=(x2-x0) / l1′.

[0106] Step 12: Use the center position of R2 as the key point A' of the electrical instrument letter, C1' and A' as the starting point and end point respectively, and obtain the reference line segment L2'=[(x1, y1), (x a,y a )], where (x1, y1), (x a ,y a ) are the coordinates of C1′ and A′, and the length of L2′ Assume that the rotation angle between L2′ and the positive direction of the x-axis is θ, then the sine value is sinθ=(y a -y1) / l2′, the cosine value is cosθ=(x a -x1) / l2′.

[0107] Step 13: Calculate the pitch angle distortion coefficient β of the electrical instrument based on the length ratio of L2′ to L1′ and the length ratio of the template reference line L2 to L1. The calculation formula of β is β=(l2′ / l1′) / (l2 / l1).

[0108] Step 14: Obtain the key point mapping matrix M according to the rotation angle ρ, the rotation angle θ and the pitch angle distortion coefficient β, where δ is the distortion adjustment coefficient:

[0109]

[0110] Step 15: Scale and rotate the scales in the main scale set P in the electrical instrument template according to the imaging scaling coefficient α and the key point mapping matrix M, and perform translation transformation according to the key point C0′ to obtain the main scale coordinate set P′, P′={p1′, p2′, .., p n ′), where p i ′=[x i ′,y i ′] T is the global image coordinate of a major scale, which is determined by the scale p in the template i =[x i ,y i , d i ] T The mapping is obtained and the calculation process is:

[0111]

[0112] Step 16: As above, the pointer rotation center S = [x s ,y s , d s ] T Mapping is performed to obtain the global image coordinates of the pointer rotation center S′=[x s ′,y s ′] T , the calculation process is:

[0113]

[0114] Step 17: Select a local image of the boundary box R1 of the electrical instrument, input it into the needle tip key point detection model, and output the key point coordinates (x t , y t ) with the highest confidence, and map the coordinates to the global map coordinates (x t ′, y t ′) as the pointer tip coordinates T = (x t ′, y t '). Use the pointer rotation center coordinate S′ as the starting point and the pointer tip coordinate T as the ending point to obtain the pointer line segment L z = [(x s ′, y s ′), (x t ′, y t ′)].

[0115] Step 18: For each scale p i ′ in the main scale coordinate set P′, use S′ as the starting point and p i ′ as the ending point to obtain the scale line segment L i = [(x s ′, y s ′), (x i ′, y i ′)], and calculate the angle between L z and each scale line segment respectively. The calculation formula for the angle is:

[0116]

[0117] Step 19: According to the main scale line segments with the minimum angle σ i and the second minimum angle σ j obtained from the previous step and the corresponding scale values e i and e j , read the pointer through the angle between the pointer line segment and the minimum angle scale line segment and the angle between the pointer line segment and the second minimum angle scale line segment. Let the reading result be result, and the reading process is:

[0118]

[0119] When the pointer tip is on the left side of the smallest main scale, that is, x t ′ < x0′, the pointer reading is less than the lower limit of the instrument measurement. At this time, estimate the reading of the part exceeding the range by calculating the ratio of the angle between the pointer and the smallest main scale to the angle between the smallest main scale and the second smallest main scale, and use the smallest angle scale reading minus the estimated reading as the final reading result; when the pointer tip is between the smallest main scale and the largest main scale, that is, x0′ ≤ x t ′ ≤ xn ′, the pointer reading is within the range of the instrument. At this time, the reading is estimated by calculating the ratio of the angle between the pointer and the minimum angle scale and the angle between the minimum angle scale and the second smallest angle scale, and the sum is added to the reading of the minimum angle scale as the final reading result; when the pointer tip is to the right of the largest main scale, that is, x n ' <x t ', the pointer reading is greater than the upper limit of the instrument's range. At this time, the reading exceeding the range is estimated by calculating the ratio of the angle between the pointer and the largest main scale to the angle between the largest main scale and the second largest main scale, and the sum of the reading at the minimum angle is taken as the final reading.

[0120] Step 20: End.

[0121] Example 3

[0122] Based on the same inventive concept, an embodiment of the present invention further provides a pointer-type electrical instrument reading device, comprising:

[0123] An electrical instrument template generation module, which is used to generate a pointer-type electrical instrument template, wherein the pointer-type electrical instrument template includes key point information, scale information, and pointer rotation center information of the pointer-type electrical instrument template;

[0124] A key point information acquisition module is used to acquire a top-down perspective image of the pointer-type electrical instrument in the electrical operation experiment from a video stream, and obtain key point information of the top-down perspective image through a preset target detection algorithm;

[0125] an electrical meter scale generation module, configured to obtain the meter type, main scale values, and division values ​​of the pointer-type electrical meter based on key point information of the top-view image;

[0126] A mapping module, configured to calculate an imaging scaling factor and a key point mapping matrix based on key point information of the pointer-type electrical instrument template and key point information of the top-view image;

[0127] A reading module is used to map the scale information and pointer rotation center information of the pointer-type electrical meter template onto the top-view image using the imaging zoom factor and key point mapping matrix, and to perform readings in combination with the meter type, main scale value and division value.

[0128] Since the device is a device corresponding to the pointer-type electrical meter reading method of the embodiment of the present invention, and the principle of solving the problem of the device is similar to that of the method, the implementation of the device can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0129] Example 4

[0130] Based on the same inventive concept, an embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the above-described method for reading a pointer-type electrical meter.

[0131] It is understood that the memory may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory may be used to store instructions, programs, codes, code sets, or instruction sets. The memory may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, instructions for implementing the various method embodiments described above, etc.; the data storage area may store data created based on the use of the server, etc.

[0132] The processor may include one or more processing cores. The processor utilizes various interfaces and circuits to connect various components within the server. It executes various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory, as well as accessing data stored in memory. Optionally, the processor may be implemented using at least one of the following hardware forms: digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor may integrate one or a combination of a central processing unit (CPU) and a modem. The CPU primarily processes the operating system and application programs, while the modem handles wireless communications. It is understood that the modem may not be integrated into the processor and may be implemented separately via a single chip.

[0133] Since the electronic device is the electronic device corresponding to the pointer-type electrical meter reading method of the embodiment of the present invention, and the principle of solving the problem by the electronic device is similar to that of the method, the implementation of the electronic device can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0134] Example 5

[0135] Based on the same inventive concept, an embodiment of the present invention also provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, at least one program, a code set, or an instruction set. The at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the pointer-type electrical meter reading method as described above.

[0136] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program. The program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0137] Since the storage medium is the storage medium corresponding to the pointer-type electrical meter reading method of the embodiment of the present invention, and the principle of solving the problem by the storage medium is similar to that of the method, the implementation of the storage medium can refer to the implementation process of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0138] Example 6

[0139] In some possible implementations, various aspects of the methods of the embodiments of the present invention may also be implemented in the form of a program product, which includes program code. When the program product is executed on a computer device, the program code is used to cause the computer device to perform the steps of the method for reading a pointer-type electrical meter according to various exemplary embodiments of the present application as described above in this specification. The executable computer program code or "code" used to perform the various embodiments may be written in a high-level programming language such as C, C++, C#, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or various other programming languages.

[0140] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0141] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0142] The above embodiments are intended only to illustrate the technical concepts and features of the present invention. Their purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. They are not intended to limit the scope of protection of the present invention. Any equivalent changes or modifications made based on the essence of the present invention are intended to be covered by the scope of protection of the present invention.

Claims

1. A method for reading a pointer-type electrical instrument, characterized in that: include: Generate a pointer type electrical instrument template, wherein the pointer type electrical instrument template includes key point information, scale information and pointer rotation center information of the pointer type electrical instrument template; The key point information of the pointer electrical instrument template specifically includes: the coordinates C0 of the negative terminal of the pointer electrical instrument, the coordinates C1 and C2 of the two positive terminal of the pointer electrical instrument; the letter coordinate A of the dial of the pointer electrical instrument; the template reference line L1 with C0 and C2 as the starting point and end point, and its template length l1, where C1 is the midpoint of L1; the template reference line L2 with C1 and A as the starting point and end point, and its template length l2; Obtaining a top-view image of the pointer-type electrical instrument described in the electrical operation experiment from a video stream, and obtaining key point information of the top-view image using a preset target detection algorithm; the key point information of the pointer-type electrical instrument template specifically includes: establishing a coordinate system with the upper left corner of the top-view image as the origin, the width direction as the positive direction of the x-axis, the height direction as the positive direction of the y-axis, and the plane normal direction of the top-view image as the z-axis; performing target detection on the top-view image using the target detection algorithm, and obtaining a dial letter bounding box R2, a negative terminal bounding box R3, a positive terminal bounding box R4, and a positive terminal bounding box R5 of the pointer-type electrical instrument appearing in the top-view image; Obtaining the meter type, main scale values, and division values ​​of the pointer-type electrical meter according to the key point information of the top-view image; The imaging scaling factor and the key point mapping matrix are calculated based on the key point information of the pointer electrical instrument template and the key point information of the top-view image; wherein, the center positions of the negative terminal boundary box R3, the positive terminal boundary box R4 and the positive terminal boundary box R5 are respectively used as key points C0', C1' and C2'; C0' and C2' are respectively used as the starting point and the end point to form the reference line segment L1', the length of L1' is l1', and the imaging scaling factor α of the pointer electrical instrument is calculated based on L1' and the template reference line L1, and L1' is set to be at the positive and negative ends of the x-axis. The rotation angle between the directions is ρ, and the center position of the dial letter bounding box R2 is used as the letter key point A' of the pointer electrical instrument. C1' and A' are used as the starting point and end point respectively. The reference line segment L2' and the length l2' of L2' are obtained. The rotation angle between L2' and the positive direction of the x-axis is set to θ. The pitch angle distortion coefficient β of the pointer electrical instrument is calculated based on the length ratio of L2' and L1' and the length ratio of the pointer electrical instrument template reference line L2 and L1. The key point mapping matrix M is obtained based on the rotation angle ρ, the rotation angle θ and the pitch angle distortion coefficient β. The scale information and pointer rotation center information of the pointer-type electrical meter template are mapped onto the top-view image using the imaging zoom factor and key point mapping matrix, and readings are taken in combination with the meter type, main scale value and division value.

2. The method for reading a pointer-type electrical meter according to claim 1, wherein: The scale information of the pointer-type electrical instrument template specifically includes: There are n main scales in the pointer type electrical instrument, corresponding to the scale values ​​E={e1,e2,...,e n }, under different ranges, the value of E is different; Set the main scale set P in the pointer electrical instrument template, P = {p1, p2, ..., p n }, where p i =[x i ,y i ,d i ] T ,(x i ,y i ) is the scale p i The coordinates of the projection position on the L1 reference line relative to C0, d i It is p i The distance from the reference line L1; The pointer rotation center information of the pointer type electrical instrument template specifically includes: Set the pointer rotation center S in the pointer type electrical instrument template = [x s ,y s ,d s ] T , where (x s ,y s ) is the coordinate of the projection position of the pointer rotation center S on the reference line L1 relative to C0, d s is the distance between S and the template reference line L1.

3. The method for reading a pointer type electrical meter according to claim 2, wherein: The method of obtaining key point information of the top-view image by using a preset target detection algorithm specifically includes: The target detection algorithm is used to perform target detection on the top-view image to obtain a bounding box R1 of the pointer-type electrical instrument appearing in the top-view image.

4. The method for reading a pointer-type electrical meter according to claim 3, wherein: The method of obtaining the meter type, main scale value E, and division value D of the pointer-type electrical meter based on the key point information of the top-view image specifically includes: Determine the type of the electric meter based on the dial letter boundary box R2 of the pointer electric meter; Determine the meter range based on the negative terminal boundary box R3, the positive terminal boundary box R4, and the positive terminal boundary box R5. According to the type and range of the electric meter, the main scale value E and the division value D of the pointer-type electrical meter are obtained.

5. The method for reading a pointer type electrical meter according to claim 4, characterized in that: The calculating of the imaging scaling factor according to the key point information of the pointer-type electrical instrument template and the key point information of the top-view image specifically includes: Reference line segment L1' = [(x0, y0), (x2, y2)], where (x0, y0) and (x2, y2) are the coordinates of C0' and C2' respectively, and the length of L1' The calculation formula of the imaging scaling factor α is α=l1' / l1.

6. The method for reading a pointer-type electrical meter according to claim 5, characterized in that: The calculating of the key point mapping matrix according to the key point information of the pointer-type electrical instrument template and the key point information of the top-view image specifically includes: For the rotation angle ρ, the sine value is sinρ=(y2-y0) / l1', and the cosine value is cosρ=(x2-x0) / l1'; Reference line segment L2'=[(x1,y1),(x a ,y a )], where (x1,y1), (x a ,y a ) are the coordinates of C1' and A', and the length of L2' For the rotation angle θ, the sine value is sinθ=(y a -y1) / l2', the cosine value is cosθ=(x a -x1) / l2'; The calculation formula of the pitch angle distortion coefficient β is β = (l2' / l1') / (l2 / l1); The key point mapping matrix M is obtained according to the rotation angle ρ, the rotation angle θ and the pitch angle distortion coefficient β according to the following preset formula, where δ is the distortion adjustment coefficient:

7. The method for reading a pointer type electrical meter according to claim 6, characterized in that: Mapping the scale information and pointer rotation center information of the pointer-type electrical instrument template onto the top-view image using the imaging scaling factor and the key point mapping matrix specifically includes: According to the imaging scaling coefficient α and the key point mapping matrix M, the scales in the main scale set P in the electrical instrument template are scaled and rotated, and the key point C0' is translated to obtain the main scale coordinate set P'. P'={p1',p2',...,p n '}, where p i '=[x i ',y i '] T It is the global image coordinate of a major scale, which is determined by the scale p in the pointer type electrical instrument template. i =[x i ,y i ,d i ] T The mapping is obtained and the calculation process is: According to the imaging scaling factor α and the key point mapping matrix M, the pointer rotation center S in the pointer type electrical instrument template is calculated as follows: s ,y s ,d s ] T Mapping is performed to obtain the global image coordinates of the pointer rotation center S'=[y s ',y s '] T , the calculation process is:

8. The method for reading a pointer-type electrical meter according to claim 7, characterized in that: The method of taking readings based on the meter type, the main scale value E and the division value D specifically includes: Select the local image of the electrical instrument bounding box R1 area, input the preset needle tip key point detection model, and output the key point coordinates (x t ,y t ), and maps the coordinates to the global graph coordinates (x t ',y t ') as the pointer tip coordinate T=(x t ',y t '), use the pointer rotation center coordinate S' as the starting point and the pointer tip coordinate T as the end point to obtain the pointer line segment L z =[(x s ',y s '),(x t ',y t ')]; For each scale p in the main scale coordinate set P' i ', respectively taking S' as the starting point, p i 'Get the scale segment L as the end point i =[(x s ',y s '),(x i ',y i ')], calculate L z The angle between each scale segment is calculated as follows: According to the previous step, the minimum angle σ is obtained i and the second smallest angle σ j The main scale line segments and the corresponding scale values ​​e i and e j , read the angle between the pointer segment and the minimum angle scale segment, and the angle between the pointer segment and the second minimum angle scale segment. Let the reading result be result. The reading process is:

9. A pointer-type electrical instrument reading device, characterized in that: include: An electrical instrument template generation module, which is used to generate a pointer-type electrical instrument template, wherein the pointer-type electrical instrument template includes key point information, scale information, and pointer rotation center information of the pointer-type electrical instrument template; The key point information of the pointer electrical instrument template specifically includes: the coordinates C0 of the negative terminal of the pointer electrical instrument, the coordinates C1 and C2 of the two positive terminal of the pointer electrical instrument; the letter coordinate A of the dial of the pointer electrical instrument; the template reference line L1 with C0 and C2 as the starting point and end point, and its template length l1, where C1 is the midpoint of L1; the template reference line L2 with C1 and A as the starting point and end point, and its template length l2; A key point information acquisition module is configured to acquire a top-view image of the pointer-type electrical instrument in the electrical operation experiment from a video stream, and obtain key point information of the top-view image using a preset target detection algorithm. The key point information of the pointer-type electrical instrument template specifically includes: establishing a coordinate system with the upper left corner of the top-view image as the origin, the width direction as the positive x-axis, the height direction as the positive y-axis, and the plane normal direction of the top-view image as the z-axis; performing target detection on the top-view image using a target detection algorithm to obtain a dial letter bounding box R2, a negative terminal bounding box R3, a positive terminal bounding box R4, and a positive terminal bounding box R5 of the pointer-type electrical instrument appearing in the top-view image; an electrical meter scale generation module, configured to obtain the meter type, main scale values, and division values ​​of the pointer-type electrical meter based on key point information of the top-view image; A mapping module is used to calculate the imaging scaling factor and the key point mapping matrix based on the key point information of the pointer electrical instrument template and the key point information of the top-view image; wherein the center positions of the negative terminal bounding box R3, the positive terminal bounding box R4, and the positive terminal bounding box R5 are respectively used as key points C0', C1', and C2'; C0' and C2' are respectively used as the starting point and end point to form a reference line segment L1', the length of L1' is l1', and the imaging scaling factor α of the pointer electrical instrument is calculated based on L1' and the template reference line L1, and L1' is set. The rotation angle between the dial and the positive direction of the x-axis is ρ, and the center position of the dial letter bounding box R2 is used as the letter key point A' of the pointer electrical instrument. C1' and A' are used as the starting point and end point respectively. The reference line segment L2' and the length l2' of L2' are obtained. The rotation angle between L2' and the positive direction of the x-axis is set to θ. The pitch angle distortion coefficient β of the pointer electrical instrument is calculated based on the length ratio of L2' and L1' and the length ratio of the pointer electrical instrument template reference line L2 and L1. The key point mapping matrix M is obtained based on the rotation angle ρ, the rotation angle θ and the pitch angle distortion coefficient β. A reading module is used to map the scale information and pointer rotation center information of the pointer-type electrical meter template onto the top-view image using the imaging zoom factor and key point mapping matrix, and to perform readings in combination with the meter type, main scale value and division value.

Images