Reading method and device of pointer type instrument, electronic equipment and storage medium

Abstract

The invention relates to a reading method and device of a pointer type instrument, electronic equipment and a storage medium, and the method comprises the steps: collecting an instrument image, reflected in a plane mirror, of the pointer type instrument through a camera; determining key points of the pointer type instrument according to the instrument image; and determining the reading of the pointer type instrument according to the key point and the measuring range of the pointer type instrument. According to the invention, the problem of mistakes and omissions caused by manual reading can be reduced, and the instrument reading can be accurately and rapidly identified.

Description

Technical Field

[0001] The present application relates to the technical field of pointer instruments, and in particular to a reading method, device, electronic device and storage medium for a pointer instrument. Background Art

[0002] Pointer meters are widely used in various industrial and energy scenarios such as water power, electricity, mining, etc., and are used to monitor the status of various instruments, meters, and equipment. In order to monitor the operating status of equipment, the data of pointer meters is currently mainly recorded by manual meter reading, which not only increases labor costs, but also easily results in reading errors. Summary of the invention

[0003] (1) Technical issues to be solved

[0004] The present application provides a reading method, device, electronic device and storage medium for a pointer-type meter, which solves the problem that current manual meter reading not only increases labor costs but also easily results in reading errors and omissions.

[0005] (2) Technical solution

[0006] In a first aspect, the present application provides a method for reading a pointer instrument, comprising:

[0007] The image of the pointer-type instrument reflected in the plane mirror is collected by a camera;

[0008] Determining key points of the pointer instrument according to the instrument image;

[0009] The reading of the pointer instrument is determined according to the key point and the measuring range of the pointer instrument.

[0010] Furthermore, the key points include a minimum scale key point, a maximum scale key point, a pointer distal end key point, and a pointer axis key point.

[0011] Further, determining the key points of the pointer instrument according to the instrument image includes:

[0012] According to the instrument image, determining the reading area of ​​the pointer instrument through an image recognition model;

[0013] According to the reading area, the key points of the pointer instrument are determined by a key point detection model.

[0014] Furthermore, after determining the key points of the pointer instrument through a key point detection model according to the reading area, the method further includes:

[0015] The reading area and the key points are corrected according to the instrument standard image to obtain the corrected reading area and key points.

[0016] Furthermore, the step of correcting the reading area and the key points according to the instrument standard image to obtain the corrected reading area and key points includes:

[0017] Performing affine transformation on the key points and corresponding points on the instrument standard image to obtain an affine transformation matrix;

[0018] The reading area is affine transformed using the affine transformation matrix to obtain a corrected reading area and key points.

[0019] Furthermore, before determining the reading of the pointer instrument according to the key point and the measuring range of the pointer instrument, the method further includes:

[0020] According to the maximum scale key point, the measuring range of the pointer instrument is determined by a text recognition model.

[0021] Further, determining the reading of the pointer instrument according to the key point and the measuring range of the pointer instrument comprises:

[0022] Determine the pointer rotation angle according to the minimum scale key point, the maximum scale key point, the pointer far end key point, and the pointer axis key point;

[0023] The reading of the pointer instrument is determined according to the pointer rotation angle and the measuring range of the pointer instrument.

[0024] In a second aspect, the present application provides a reading device for a pointer-type instrument, comprising:

[0025] An image acquisition module, used for acquiring an instrument image reflected by a pointer-type instrument in a plane mirror through a camera;

[0026] A key determination module, used to determine the key points of the pointer instrument according to the instrument image;

[0027] The reading determination module is used to determine the reading of the pointer instrument according to the key point and the measuring range of the pointer instrument.

[0028] In a third aspect, the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for reading a pointer instrument as described above when executing the computer program.

[0029] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method for reading a pointer instrument as described above is implemented.

[0030] (3) Beneficial effects

[0031] The above technical solution of the present application has the following advantages:

[0032] The first aspect of the present application provides a method for reading pointer-type instruments, which uses machine vision to read pointer-type instruments, which can reduce the problem of errors and omissions in manual readings, reduce the labor costs of enterprises, and report reading results more promptly than manual inspections. It solves the pain points of data collection, access, analysis, and management of pointer-type instruments such as electricity, water, oil, gas, and heat, improves the digitalization and intelligence of enterprises, can accurately and quickly identify instrument readings, and improves monitoring efficiency and accuracy.

[0033] It can be understood that the beneficial effects of the second, third and fourth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 A schematic diagram of a method for reading a pointer-type instrument provided in this application;

[0036] Figure 2 A schematic diagram of the structure of the reading device of the pointer instrument provided in this application;

[0037] Figure 3 A schematic diagram of the structure of the electronic device provided in this application. DETAILED DESCRIPTION

[0038] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0039] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0040] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0041] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways. "Multiple" means "two or more".

[0042] The specific implementation methods of the present application are further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present application but are not intended to limit the scope of the present application.

[0043] like Figure 1 As shown, the reading method of the pointer instrument provided in this embodiment specifically includes the following steps:

[0044] S100. Collecting, by means of a camera, an instrument image of a pointer-type instrument reflected in a plane mirror.

[0045] A short object distance plane mirror imaging device is designed, which is mainly composed of a camera and a plane mirror, wherein the camera collects the image in the plane mirror. The camera is placed on the surface of the pointer instrument dial and can be directly pasted on the instrument dial. The plane mirror is placed above the pointer instrument dial, and the camera collects the instrument image reflected by the plane mirror. The short object distance plane mirror imaging device reduces the installation size of the device and the working distance of the camera collecting the pointer instrument through the plane mirror reflection imaging principle, thus greatly shortening the object distance and improving the imaging quality.

[0046] S200: Determine key points of the pointer instrument according to the instrument image.

[0047] In some embodiments, the key points include a minimum scale key point, a maximum scale key point, a pointer distal end key point, and a pointer axis key point.

[0048] In some embodiments, determining the key points of the pointer instrument based on the instrument image includes: determining a reading area of ​​the pointer instrument through an image recognition model based on the instrument image; and determining the key points of the pointer instrument through a key point detection model based on the reading area.

[0049] In some embodiments, after determining the key points of the pointer instrument through a key point detection model based on the reading area, it also includes: correcting the reading area and the key points according to the instrument standard image to obtain corrected reading area and key points.

[0050] In some embodiments, the correction of the reading area and the key points according to the instrument standard image to obtain the corrected reading area and key points includes: performing an affine transformation on the key points and the corresponding points on the instrument standard image to obtain an affine transformation matrix; performing an affine transformation on the reading area using the affine transformation matrix to obtain the corrected reading area and key points.

[0051] The collected image is input into the industrial analog instrument recognition system for image analysis to obtain image recognition results. The reading area of ​​the positioning pointer instrument is detected in the image through the image recognition model, and the extracted instrument reading area is image corrected. The key points of the reading area are detected, including the maximum and minimum scales, the key points at the far end of the pointer, and the key points of the pointer axis. The angles of the maximum and minimum scales and the angle of the pointer are calculated. The reading area is image corrected, and the detected key points are corrected with the instrument standard image to obtain the corrected reading area and key point coordinates.

[0052] Locating the pointer instrument reading area from the instrument panel image includes the following steps: collecting a certain amount of instrument images as a training data set, and annotating the data, annotating the instrument reading area and the instrument category. It is used to train a detection model for locating the instrument reading area from the initial collected image. Combined with the detection model after training optimization, the instrument reading area is detected and located from the collected image to reduce the interference of invalid areas on subsequent image analysis. The detection model adopts a convolutional neural network model, for example, the target detection model is implemented using algorithms such as YOLO.

[0053] The key point detection of the reading area includes the following steps: Annotation is performed on the collected pointer instrument image dataset, and the annotation includes the maximum scale, the minimum scale, and the endpoints of the pointer. The image is preprocessed first, including image resizing, normalization, and other operations. Based on the key point detection model, the prepared dataset is used for training. The trained model is used to infer the newly collected images to obtain the location of the key points. For each image, the model will return the coordinates of the key points, which are the maximum scale, the minimum scale, and the endpoints of the pointer. The key point detection model can use traditional key point detection methods, such as the SIFT algorithm in the image processing library OpenCV, or the Loftr algorithm based on deep learning.

[0054] The detected key points are corrected with the instrument standard image to obtain the corrected reading area and key point coordinates, which includes the following steps: the four detected key points and the corresponding points on the standard image are affine transformed to obtain the affine transformation matrix M. The affine matrix M is used to perform affine transformation on the subsequent detected images to obtain the corrected reading area and key point coordinates.

[0055] S300: Determine the reading of the pointer instrument according to the key point and the measuring range of the pointer instrument.

[0056] In some embodiments, before determining the reading of the pointer instrument based on the key points and the range of the pointer instrument, the method further includes: determining the range of the pointer instrument through a text recognition model based on the maximum scale key points.

[0057] In some embodiments, determining the reading of the pointer instrument based on the key points and the range of the pointer instrument includes: determining the pointer rotation angle based on the minimum scale key point, the maximum scale key point, the pointer far end key point, and the pointer axis key point; determining the reading of the pointer instrument based on the pointer rotation angle and the range of the pointer instrument.

[0058] The instrument range is obtained by performing text recognition in the maximum scale key point area or by setting the configuration file in advance. The current pointer instrument reading is calculated based on the maximum and minimum scale angles, the pointer angle and the instrument range. The instrument reading results recognized by the analysis system are transmitted to the front-end monitoring page for real-time reading monitoring.

[0059] Performing text recognition in the maximum scale key point area or obtaining the instrument range V by setting the configuration file in advance includes the following steps: near the detected maximum scale position, using text recognition technology to extract and recognize numbers, converting the recognized numbers into the corresponding range, and performing necessary unit conversion and correction according to the characteristics of the instrument. When there are many numbers near the key point and the background interference is large, the configuration file can be used to set the instrument range in advance, which is simple and efficient.

[0060] After the key points of the pointer instrument are detected, they are post-processed, including the following steps: Post-process the detected key points, including sorting the key points according to the actual position distribution to ensure their correct order. Ensure that the minimum scale key points, maximum scale key points, pointer far end key points, and pointer center key points can be distinguished. And the angle α between the instrument pointer and the minimum scale is calculated through the coordinates of the key points at both ends of the pointer and the coordinates of the minimum scale key points. The angle β between the minimum and maximum scales is calculated through the coordinates of the minimum key point and the coordinates of the maximum scale key point.

[0061] By obtaining the angle α and the angle β, and combining the instrument range V, the current pointer reading is calculated, including the following steps: the instrument reading result R = α / β*V. The identification result is transmitted to the front-end monitoring system for real-time monitoring.

[0062] The following is an explanation through specific embodiments.

[0063] Image recognition and analysis of the collected images are mainly divided into four steps.

[0064] Step 1 uses an object detection algorithm to detect the analog instrument area in the image. The implementation method is as follows: collect and annotate an image dataset containing various analog instruments, use the annotated dataset to train the YOLOv5 model, adjust the hyperparameters to optimize performance, apply the trained model to the input image, and output the bounding box of the analog instrument.

[0065] Step 2 uses key point detection technology to detect the four key points of the analog instrument: minimum scale, maximum scale, pointer end, and pointer axis. The implementation method is as follows: annotate the key points (minimum scale, maximum scale, pointer end, and pointer axis) for the image containing the analog instrument, and use the annotated dataset to fine-tune the OpenPose model. Perform key point detection on the instrument area output by the target detection to obtain the positions of the four key points.

[0066] Step 3: Identify the range information of the instrument through character recognition technology. Tesseract is an open source OCR engine that supports character recognition in multiple languages. The implementation method is as follows: Collect instrument images containing range information and mark their corresponding ranges. If necessary, train Tesseract to improve the recognition accuracy of specific fonts or formats. Perform character recognition on the detected instrument area to extract the range information.

[0067] Step 4: Calculate the rotation angle of the pointer through the above information, and further calculate the meter reading. The implementation method is as follows: First, calculate the rotation angle of the pointer through the coordinates of the key points at both ends of the pointer. Assume (x center ,y center ) is the coordinate of the key point of the pointer axis (also the center point of the dial), (x pointrr ,y pointer ) is the coordinate of the key point at the end of the pointer, (x min ,y min ) is the coordinate of the minimum scale key point, (x max ,y max ) is the coordinate of the maximum scale key point. Pointer angle θpointer=arctan2(y pointer -y center ,x pointer -x center The angle between the minimum scale and the key point of the pointer axis is θmin = arctan2(y min -y center ,x min -x center ).

[0068] The angle between the maximum scale and the key point of the pointer axis is θmax = arctan2(y max -y center ,x max -x center ). The angle between the pointer and the minimum scale θ=θpointer-θmin. Finally, the reading is calculated. Through linear mapping, the angle of the pointer is linearly mapped to the range of the instrument. According to the range and angle, the specific value indicated by the pointer is calculated. Among them, R min and R max are the minimum and maximum values ​​of the measuring range, θmin and θmax are the angles of the pointer when it is at the minimum and maximum scales.

[0069] The method for reading a pointer instrument provided in an embodiment of the present application collects an image of a pointer analog instrument through a camera, and locates the reading area covering the pointer instrument in the collected image through an image recognition model. Key point detection is performed on the reading area to locate the minimum and maximum scale key points of the dial and the key points of the far end and the axis end of the pointer. The text recognition model is used to perform text recognition on the located maximum scale key point area to obtain the maximum scale value of the dial. When the instrument range is not accurately recognized, the instrument range can be obtained by means of a configuration file. Finally, the instrument reading is performed based on the angle relationship between the pointer and the maximum and minimum scales and the scale value.

[0070] This application uses machine vision to read pointer-type instruments, which can reduce the error and omission problems that are prone to occur in manual readings, reduce the company's employment costs, and report reading results more promptly than manual inspections. It solves the pain points of data collection, access, analysis, and management of pointer-type instruments such as electricity, water, oil, gas, and heat, improves the digitalization and intelligence of enterprises, can accurately and quickly identify instrument readings, and improves monitoring efficiency and accuracy.

[0071] Corresponding to the reading method of the pointer instrument described in the above embodiment, as Figure 2 As shown, this embodiment provides a reading device for a pointer instrument, and the reading device 200 for the pointer instrument includes:

[0072] The image acquisition module 201 is used to acquire the instrument image reflected by the pointer instrument in the plane mirror through the camera;

[0073] A key determination module 202, configured to determine key points of the pointer instrument according to the instrument image;

[0074] The reading determination module 203 is used to determine the reading of the pointer instrument according to the key point and the measuring range of the pointer instrument.

[0075] It should be noted that the information interaction, execution process, etc. between the above-mentioned modules / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0076] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0077] The present application embodiment also provides an electronic device 300, such as Figure 3 As shown, it includes a memory 301, a processor 302, and a computer program 303 stored in the memory 301 and executable on the processor 302. When the processor 302 executes the computer program 303, the steps of the pointer instrument reading method provided in the first aspect are implemented.

[0078] In applications, the electronic device may include, but is not limited to, a processor and a memory. Figure 3 This is only an example of an electronic device and does not constitute a limitation on the electronic device. It may include more or fewer components than shown in the figure, or combine certain components, or different components, such as input and output devices, network access devices, etc. Input and output devices may include cameras, audio acquisition / playback devices, display screens, etc. Network access devices may include a network module for wirelessly communicating with external devices.

[0079] In applications, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0080] In application, the memory may be an internal storage unit of an electronic device in some embodiments, such as a hard disk or memory of the electronic device. The memory may also be an external storage device of the electronic device in other embodiments, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. The memory may also include both an internal storage unit and an external storage device of the electronic device. The memory is used to store operating systems, applications, boot loaders (BootLoader), data, and other programs, such as program codes of computer programs, etc. The memory may also be used to temporarily store data that has been output or is to be output.

[0081] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0082] The present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to an electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0083] Those of ordinary skill in the art will appreciate that the devices and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0084] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of devices, which can be electrical, mechanical or other forms.

[0085] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for reading a pointer instrument, characterized in that: include: The image of the pointer-type instrument reflected in the plane mirror is collected by a camera; Determining key points of the pointer instrument according to the instrument image; The reading of the pointer instrument is determined according to the key point and the measuring range of the pointer instrument.

2. The method for reading a pointer instrument as claimed in claim 1, characterized in that: The key points include the minimum scale key point, the maximum scale key point, the pointer far end key point, and the pointer axis key point.

3. The method for reading a pointer instrument as claimed in claim 1, characterized in that: Determining the key points of the pointer instrument according to the instrument image includes: According to the instrument image, determining the reading area of ​​the pointer instrument through an image recognition model; According to the reading area, the key points of the pointer instrument are determined by a key point detection model.

4. The method for reading a pointer instrument as claimed in claim 3, characterized in that: After determining the key points of the pointer instrument by a key point detection model according to the reading area, the method further includes: The reading area and the key points are corrected according to the instrument standard image to obtain the corrected reading area and key points.

5. The method for reading a pointer instrument as claimed in claim 4, characterized in that: The step of correcting the reading area and the key points according to the instrument standard image to obtain the corrected reading area and key points includes: Performing affine transformation on the key points and corresponding points on the instrument standard image to obtain an affine transformation matrix; The reading area is affine transformed using the affine transformation matrix to obtain a corrected reading area and key points.

6. The method for reading a pointer instrument as claimed in claim 2, characterized in that: Before determining the reading of the pointer instrument according to the key point and the measuring range of the pointer instrument, the method further comprises: According to the maximum scale key point, the measuring range of the pointer instrument is determined by a text recognition model.

7. The method for reading a pointer instrument as claimed in claim 2, characterized in that: Determining the reading of the pointer instrument according to the key point and the measuring range of the pointer instrument comprises: Determine the pointer rotation angle according to the minimum scale key point, the maximum scale key point, the pointer far end key point, and the pointer axis key point; The reading of the pointer instrument is determined according to the pointer rotation angle and the measuring range of the pointer instrument.

8. A reading device for a pointer instrument, characterized in that: include: An image acquisition module, used for acquiring an instrument image reflected by a pointer-type instrument in a plane mirror through a camera; A key determination module, used to determine the key points of the pointer instrument according to the instrument image; The reading determination module is used to determine the reading of the pointer instrument according to the key point and the measuring range of the pointer instrument.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the pointer instrument reading method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for reading a pointer instrument as claimed in any one of claims 1 to 7 is implemented.

Images