Data reading method and device of pointer instrument, electronic equipment and storage medium
Through the method of combining image processing with neural networks, the pointer area is automatically extracted and the reading is calculated, which solves the problems of low efficiency and low accuracy of pointer instrument data reading, and achieves efficient and accurate automatic data reading.
Patent Information
- Application Number
- CN202411969754.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-30
- Publication Date
- 2025-05-02
AI Technical Summary
In the prior art, the data reading efficiency of pointer instruments is low and the accuracy is not high. Especially in industrial field environments with high temperature, high noise, high voltage and high radiation, it is difficult to manually read data.
Through the method of combining image processing with neural network, the target image of the pointer instrument collected in real time is processed, the pointer area is extracted, and the reading result of the pointer instrument is automatically determined based on the angle between the target line on the pointer area and the scale value information and the scale value information.
It realizes automated data reading, improves data reading efficiency and accuracy, reduces sensitivity to light changes, and is suitable for various types of pointer instruments.
Smart Images
Figure CN119919920A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure generally relates to the fields of image processing and machine vision. More specifically, the present disclosure relates to a data reading method, device, electronic device and storage medium of a pointer instrument. Background Art
[0002] Pointer meters are widely used in industrial measurement fields such as electricity, finance, chemical industry, machinery and electronics due to their advantages of anti-electromagnetic interference, anti-vibration, oil resistance, anti-freeze and waterproof, and low price. In industrial sites, in order to determine whether industrial equipment is operating correctly, it is necessary to frequently read the data of various pointer meters on the industrial site.
[0003] At present, the data of conventional pointer instruments is read manually by operators, but industrial sites generally have problems such as high temperature, high noise, high pressure and high radiation. It is not easy for operators to frequently go to industrial sites to read data, and manual reading of data has low data reading efficiency. At the same time, manual reading of data will also be affected by human subjective factors, resulting in low data reading accuracy.
[0004] In view of this, there is an urgent need to provide a data reading method, device, electronic device and storage medium for a pointer instrument to realize automated data reading and improve data reading efficiency and accuracy. Summary of the invention
[0005] In order to at least solve one or more of the technical problems mentioned above, the present disclosure proposes a data reading method, device, electronic device and storage medium for a pointer instrument in multiple aspects.
[0006] In a first aspect, the present disclosure provides a method for reading data of a pointer instrument, the method comprising: acquiring a pointer instrument target image obtained by performing image acquisition on a target pointer instrument; matching an acquired pointer instrument template image with the pointer instrument target image to obtain an image to be identified; performing an interest region extraction operation on the image to be identified to obtain a first interest region, and inputting the first interest region into a trained semantic segmentation model to perform pointer extraction to obtain a pointer region; the first interest region is a pointer instrument region; determining a reading result of the target pointer instrument based on a target angle value of an angle between a target straight line on the pointer region and a specified direction and the acquired scale value information; the target straight line is a straight line connecting the farthest pixel point on the pointer region and a pointer rotation axis point; the scale value information comprises: one-to-one corresponding scale values and angle values.
[0007] In some embodiments, the acquired pointer instrument template image is matched with the pointer instrument target image to obtain the image to be identified, including: performing image enhancement processing on the pointer instrument target image and the pointer instrument template image respectively to obtain a target enhanced image and a template enhanced image; performing a region of interest extraction operation on the template enhanced image to obtain a second region of interest, the second region of interest at least including: a pointer instrument region; performing image matching on the second region of interest and the target enhanced image to obtain a mapping matrix; the mapping matrix is used to map the pointer instrument target image to the second region of interest; based on the mapping matrix, the pointer instrument target image is mapped to the second region of interest to obtain the image to be identified.
[0008] In some embodiments, the farthest pixel point on the pointer area is determined by calculating the distance between each pixel point in the pointer area and the pointer rotation axis point, and taking the pixel point corresponding to the maximum distance as the farthest pixel point.
[0009] In some embodiments, the scale value information is determined by the following steps: for the pointer instrument template image, obtaining the coordinates of each scale value on the pointer instrument template image; for each scale value, calculating the angle value between the straight line connecting the scale value and the pointer pivot point and the specified direction based on the coordinates of the scale value and the coordinates of the pointer pivot point; and using the angle value corresponding to each scale value and the scale value as the scale value information.
[0010] In some embodiments, the target angle value based on the angle between the target straight line on the pointer area and the specified direction and the acquired scale value information determines the reading result of the target pointer instrument, including: if the target angle value matches any angle value in the scale value information, then the scale value corresponding to the matched angle value is used as the reading result of the pointer instrument; if the target angle value falls between the second angle value and the first angle value in the scale value information, then the reading result of the target pointer instrument is determined based on the first angle value, the second angle value, the target angle value, the first scale value corresponding to the first angle value, and the second scale value corresponding to the second angle value.
[0011] In some embodiments, the first scale value is less than the second scale value; the determining the reading result of the target pointer instrument based on the first angle value, the second angle value, the target angle value, the first scale value corresponding to the first angle value, and the scale value corresponding to the second angle value includes: Reading=value1+Per*|value2-value1|, wherein Reading is the reading result of the target pointer instrument; value1 is the first scale value; value2 is the second scale value; Per is an intermediate parameter, Angle1 is the first angle value; Angle2 is the second angle value; α1 is the target angle value.
[0012] In some embodiments, the target pointer instrument is a single pointer instrument or a multi-pointer instrument.
[0013] In a second aspect, the present disclosure provides a data reading device for a pointer instrument, the device comprising: a pointer instrument target image acquisition module, for acquiring a pointer instrument target image obtained by performing image acquisition on a target pointer instrument; a matching module, for matching an acquired pointer instrument template image with the pointer instrument target image to obtain an image to be identified; an extraction module, for performing an interest region extraction operation on the image to be identified to obtain a first interest region, and inputting the first interest region into a trained semantic segmentation model for pointer extraction to obtain a pointer region; the first interest region is a pointer instrument region; a data reading module, for determining a reading result of the target pointer instrument based on a target angle value of an angle between a target straight line on the pointer region and a specified direction and the acquired scale value information; the target straight line is a straight line connecting the farthest pixel point on the pointer region and a pointer rotation axis point; the scale value information comprises: one-to-one corresponding scale values and angle values.
[0014] In a third aspect, the present disclosure provides an electronic device comprising: a processor configured to execute program instructions; and a memory configured to store the program instructions, wherein when the program instructions are loaded and executed by the processor, the processor executes the data reading method of a pointer instrument according to the first aspect or any embodiment of the first aspect.
[0015] In a fourth aspect, the present disclosure provides a computer-readable storage medium having program instructions stored therein. When the program instructions are loaded and executed by a processor, the processor executes the data reading method of a pointer instrument according to the first aspect or any embodiment of the first aspect.
[0016] Through the data reading method, device, electronic device and storage medium of the pointer instrument provided above, the disclosed embodiment processes the real-time collected pointer instrument target image by combining image processing with neural network, and then obtains the reading result of the target pointer instrument, which improves the data reading efficiency and accuracy compared with the manual reading method. Furthermore, when extracting the pointer area, it is extracted through a neural network (i.e., a semantic segmentation model), which is not affected by light on the collected pointer instrument target image, improves the generalization to different light environments, and improves the accuracy of pointer area extraction. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] By reading the detailed description below with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:
[0018] Figure 1 An exemplary flow chart showing a method for reading data of a pointer instrument according to some embodiments of the present disclosure;
[0019] Figure 2A A schematic diagram showing a single-pointer circular instrument image according to some embodiments of the present disclosure;
[0020] Figure 2B A schematic diagram showing a single-pointer square instrument image according to some embodiments of the present disclosure;
[0021] Figure 2C A schematic diagram showing a multi-pointer instrument image according to some embodiments of the present disclosure;
[0022] Figure 2D A schematic diagram showing a target image of a pointer instrument according to some embodiments of the present disclosure is shown;
[0023] Figure 2E A schematic diagram showing a pointer instrument template image according to some embodiments of the present disclosure;
[0024] Figure 2F A schematic diagram showing an image to be recognized according to some embodiments of the present disclosure;
[0025] Figure 2G A schematic diagram showing a first region of interest of some embodiments of the present disclosure;
[0026] Figure 3 A schematic diagram showing a designated coordinate system of some embodiments of the present disclosure is shown;
[0027] Figure 4A A schematic diagram showing a pointer area of some embodiments of the present disclosure;
[0028] Figure 4B A schematic diagram showing the farthest pixel point on the pointer area and the pointer rotation axis point in some embodiments of the present disclosure is shown;
[0029] Figure 5A Some embodiments of the present disclosure are shown Figure 2A The diagram of the annotation result of the single-pointer circular instrument image shown;
[0030] Figure 5B Some embodiments of the present disclosure are shown Figure 2B The schematic diagram of the annotation result of the single-pointer square instrument image shown;
[0031] Figure 5C A curve diagram showing the change of the average intersection-over-union (MIOU) ratio in each round during the model training process of some embodiments of the present disclosure is shown;
[0032] Fig. 6A A schematic diagram showing a template-enhanced image according to some embodiments of the present disclosure;
[0033] Figure 6B A schematic diagram showing a target enhanced image according to some embodiments of the present disclosure;
[0034] Figure 6C A schematic diagram showing a second region of interest according to some embodiments of the present disclosure;
[0035] Fig.6D A schematic diagram showing matching images of some embodiments of the present disclosure is shown;
[0036] Figure 7 An overall flow chart showing a method for reading data of a pointer-type instrument according to some embodiments of the present disclosure is shown;
[0037] Figure 8 An exemplary structural block diagram of a data reading device for a pointer-type instrument according to some embodiments of the present disclosure is shown;
[0038] Fig. 9 An exemplary structural block diagram of an electronic device according to some embodiments of the present disclosure is shown. DETAILED DESCRIPTION
[0039] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.
[0040] It should be understood that the terms "include" and "comprising" used in the specification and claims of the present disclosure indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.
[0041] It should also be understood that the terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the disclosure. As used in this disclosure and claims, the singular forms of "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in this disclosure and claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations.
[0042] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.
[0043] The specific implementation of the present disclosure is described in detail below with reference to the accompanying drawings.
[0044] As described in the background technology, in order to determine whether industrial equipment is operating correctly, operators need to frequently go to the industrial site to read data, which has low data reading efficiency. In addition, manual data reading is also affected by human subjective factors, resulting in low data reading accuracy. At the same time, if it is necessary to determine whether industrial equipment is operating correctly more frequently, it is necessary to increase the number of operators to conduct more frequent inspections, which also greatly increases the labor cost.
[0045] Therefore, the disclosed embodiment provides a method for reading data of a pointer instrument, which processes the pointer instrument target image collected in real time by combining image processing and neural network, and then obtains the reading result of the pointer instrument, thereby realizing automated data reading. Compared with the manual data reading method, it improves the data reading efficiency and accuracy.
[0046] In the disclosed embodiment, multiple high-definition visible light cameras can be set up at the industrial site to collect target images of each pointer instrument and then read the data of each pointer instrument; or a mobile robot can be set up at the industrial site for inspection. During the inspection process of the mobile robot, the high-definition visible light camera set on the mobile robot collects the target image of each pointer instrument and then reads the data of each pointer instrument. The disclosed embodiment does not specifically limit the method of real-time collection of the target image of the pointer instrument, which can be determined according to actual conditions.
[0047] Reference Figure 1 , Figure 1 The method 100 is a flowchart of the steps of a pointer instrument data reading method according to some embodiments of the present disclosure. The method is applied to an electronic device, as an embodiment, the electronic device here is, for example, a terminal, a server or the above-mentioned mobile robot, etc., and the embodiments of the present disclosure are not specifically limited.
[0048] like Figure 1 As shown, first, in step S110, a pointer instrument target image obtained by image acquisition for a target pointer instrument is obtained. Then, in step S120, the acquired pointer instrument template image is matched with the pointer instrument target image to obtain an image to be identified. Then, in step S130, an area of interest extraction operation is performed on the image to be identified to obtain a first area of interest, and the first area of interest is input into a trained semantic segmentation model for pointer extraction to obtain a pointer area; the first area of interest is a pointer instrument area; then, in step S140, the reading result of the target pointer instrument is determined based on the target angle value of the angle between the target straight line on the pointer area and the specified direction and the acquired scale value information; the target straight line is the straight line where the line connecting the farthest pixel point on the pointer area and the pointer rotation axis point is located; the scale value information includes: one-to-one corresponding scale values and angle values.
[0049] For example, in the disclosed embodiment, the target pointer instrument in step S110 refers to a pointer instrument that needs to read data. The target pointer instrument can be of many types, for example, a single-pointer circular instrument (such as Figure 2A As shown), single pointer square instrument (as Figure 2B as shown) and multi-pointer instruments (such as Figure 2C As shown), etc., the disclosed embodiments do not specifically limit this.
[0050] It should be noted that Figure 2C The multi-pointer instrument shown includes three pointers, namely a reading pointer for reading data and two threshold limit pointers, which can be respectively located at the minimum scale and the maximum scale of the dial, or at other scales of the dial, which is not specifically limited in the disclosed embodiment.
[0051] In the disclosed embodiment, the pointer instrument target image (such as Figure 2D The target image of the pointer instrument may include: the pointer instrument area and other object areas. It can be acquired by a high-definition camera installed on a mobile robot or a high-definition camera fixedly installed above the pointer instrument. The disclosed embodiment does not specifically limit the camera for acquiring the target image of the pointer instrument.
[0052] For example, in the disclosed embodiment, the pointer instrument template image in step S120 is a clear image captured in advance by a mobile robot or a fixedly installed camera when the light is good. The pointer instrument template image may only include the target pointer instrument area. Of course, in order to facilitate subsequent image processing, the pointer instrument template image may also include the target pointer instrument area and other object areas (such as Figure 2E The disclosed embodiments are only Figure 2E The pointer instrument template image shown is used as an example for description.
[0053] Based on the above description, if this method is applied to a terminal or server, a mobile robot or a fixedly installed camera collects the pointer instrument template image and the pointer instrument target image and uploads them to the terminal or server for subsequent image processing; if this method is applied to a mobile robot, then the mobile robot collects the pointer instrument template image and saves it locally. After collecting the pointer instrument target image, subsequent image processing is performed on the pointer instrument template image and the pointer instrument target image.
[0054] In the embodiment of the present disclosure, in step S120, the image to be identified is an image obtained by mapping the pointer instrument target image to the pointer instrument template image. The image to be identified may include the pointer instrument area and other object areas (such as Figure 2F shown).
[0055] In the disclosed embodiment, the acquired pointer instrument template image and the pointer instrument target image are matched specifically as follows: the pointer instrument template image and the pointer instrument target image are matched according to a specified matching algorithm (for example, Surf algorithm, SIFT algorithm) to obtain multiple groups of matching point pairs, and a mapping matrix is obtained based on the multiple groups of matching point pairs (to distinguish it from the mapping matrix below, the mapping matrix can be recorded as a first mapping matrix), and then the pointer instrument target image is mapped to the pointer instrument template image according to the first mapping matrix to obtain the image to be identified.
[0056] Here, the Surf algorithm and the SIFT algorithm are both conventional algorithms, and the method of obtaining a mapping matrix based on multiple sets of matching point pairs is also a conventional algorithm. Please refer to the description of relevant documents, which will not be repeated here.
[0057] In the disclosed embodiment, in order to reduce the influence of light on the image during the image acquisition process, the acquired pointer instrument template image and the pointer instrument target image can also be matched by first performing image enhancement processing on the pointer instrument template image and the pointer instrument target image, and then matching to obtain the image to be identified. The specific implementation method can be found in the description of the following embodiment, which will not be repeated here.
[0058] Exemplarily, in step S130, Figure 2G As shown, the first region of interest is a pointer instrument region. The first region of interest is obtained by performing a region of interest extraction operation on the image to be identified. Specifically, in response to a selection operation performed on the image to be identified, the region of interest is extracted from the image to be identified to obtain the first region of interest.
[0059] As an optional implementation of the present disclosure, the coordinates of each vertex of the first region of interest and the pointer rotation axis point ( Figure 2G The coordinates R(x) of the circle position in Center ,y Center ) to save it for later use. The coordinates mentioned above are all coordinates in a specified coordinate system. Here, the specified coordinate system can be an image coordinate system or a custom coordinate system. For example, Figure 3 As shown, a coordinate system is established with the center of the image as the coordinate origin O, and straight lines parallel to the two sides of the image as the first coordinate axis (X axis) and the second coordinate axis (Y axis) of the coordinate system. The disclosed embodiment does not specifically limit the specified coordinate system.
[0060] In the disclosed embodiment, the semantic segmentation model in step S130 is a pre-trained model for pointer extraction, such as Figure 4A As shown, the first region of interest is input into the trained semantic segmentation model for pointer extraction, and a black background area and a white pointer area are output, thereby realizing pointer extraction.
[0061] Exemplarily, in step S140, Figure 3 As shown, the specified direction can be any direction of the positive direction of the X axis (0°), the negative direction of the X axis (180°), the positive direction of the Y axis (90°), and the negative direction of the Y axis (270°) of the above-mentioned specified coordinate system, or other directions, which are not specifically limited in the present disclosure. Figure 4A The description is made by taking 0° in FIG.
[0062] In the disclosed embodiment, the target straight line is the straight line connecting the farthest pixel point on the pointer area and the pointer rotation axis point, such as Figure 4B As shown, the circle on the left in the figure is the farthest pixel point on the pointer area, and the circle on the right is the pointer rotation axis point. Based on the description of the above specified direction, the target angle value of the angle between the target straight line and the specified direction can be obtained.
[0063] For example, the coordinates of the farthest pixel are (x Farthest_End ,y Farthest_End ), the coordinate of the pointer rotation axis point is R(x Center ,y Center ), calculate the slope of the line connecting the farthest pixel point and the pointer pivot point Then the target angle value of the angle between the target straight line and the specified direction is α1 = arctan(k).
[0064] In the disclosed embodiment, the scale value information includes: one-to-one corresponding scale values and angle values. The scales of pointer instruments in actual industrial sites may be uniform or uneven; at the same time, many pointer instruments are placed at an angle, which may block part of the instrument scale lines and scale values, resulting in failure to read data. Therefore, in order to ensure that the correct pointer reading can be read after the pointer is extracted, after the pointer instrument template image is acquired, the angle value corresponding to each scale value within the pointer scale range can be obtained based on the pointer instrument template image.
[0065] Specifically, for the pointer instrument template image, the coordinates of each scale value on the pointer instrument template image are obtained; for each scale value, the angle value between the straight line connecting the scale value and the pointer pivot point and the specified direction is calculated based on the coordinates at the scale value and the coordinates of the pointer pivot point; the angle value corresponding to each scale value and the scale value is used as scale value information.
[0066] For example, the coordinates (x n ,y n ), where n is the number of scales corresponding to the pointer scale range. Then, for each scale value, the angle between the straight line connecting the scale value and the pointer pivot point and the specified direction is calculated based on the coordinates of the scale value and the coordinates of the pointer pivot point. The specific calculation method is the same as the method for calculating the target angle value mentioned above, so it will not be repeated here. In this way, a one-to-one correspondence between scale values and angle values is obtained, namely (Angle1, value1), (Angle2, value2), (Angle3, value3)...(Angle n ,value n ), where Anglen Indicates the nth angle value, value n Indicates the nth tick value.
[0067] In the disclosed embodiment, in step S140, after obtaining the target angle value, the reading result of the target pointer instrument is determined based on the target angle value and the acquired scale value information. Specifically, if the target angle value matches any angle value in the scale value information, the scale value corresponding to the matched angle value is used as the reading result of the target pointer instrument; if the target angle value falls between the second angle value and the first angle value in the scale value information, the reading result of the target pointer instrument is determined based on the first angle value, the second angle value, the target angle value, the first scale value corresponding to the first angle value, and the second scale value corresponding to the second angle value.
[0068] In the disclosed embodiment, the reading result can be calculated by the following formula:
[0069] Reading=value1+Per*|value2-value1|Formula 1
[0070] Among them, Reading is the reading result of the target pointer instrument; value1 is the first scale value; value2 is the second scale value; Per is the intermediate parameter, Angle1 is the first angle value; Angle2 is the second angle value; α1 is the target angle value; the first scale value is smaller than the second scale value.
[0071] The disclosed embodiment processes the real-time collected pointer instrument target image by combining image processing with a neural network, and then obtains the reading result of the pointer instrument, which improves the data reading efficiency and accuracy compared to the manual reading method. Furthermore, when extracting the pointer area, it is extracted through a neural network (i.e., a semantic segmentation model), which is not affected by the light (for example, different time periods, different weather, etc.) on the collected pointer instrument target image, thereby improving the generalization to different light environments and the accuracy of pointer area extraction. Furthermore, the use of a neural network method to extract the pointer area can be applicable to a variety of pointer instruments, including single-pointer circular instruments, single-pointer square instruments, and multi-pointer instruments, with high universality and strong generalization.
[0072] The following describes the training process of the above semantic segmentation model, as follows:
[0073] (1) Training sample preparation: There are 528 sample images of circular single-pointer instruments and 784 sample images of square single-pointer instruments.
[0074] (2) Training sample annotation: The 528 sample images of circular single-pointer instruments and the 784 sample images of square single-pointer instruments can be annotated using the image annotation software labelme or manually. The sample images of circular single-pointer instruments can be annotated as follows: Figure 2A As shown, the annotation result for the circular single pointer instrument is Figure 5A The image shown, Figure 5A In the figure, the red part is the pointer; the sample image of the square single pointer instrument can be as follows Figure 2B As shown, the marking result for the square single pointer instrument is Figure 5B The image shown, Figure 5B The red part is the pointer.
[0075] (3) Model training: Divide the training samples, for example, into training data set, validation data set and test data set in a ratio of 7:2:1 to ensure that the model has good generalization ability on unseen data. In terms of model selection, the DeepLabv3+ model can be used as the initial model for semantic segmentation, and the backbone network Backbone of the DeepLabv3+ model can use the ResNet50_vd network. In the pre-training stage of the model, the existing data set CITYSCAPES can be used for training; in the training stage of the model, the model is trained on the training data set and adjusted on the validation data set. After the model training is completed, the test data set is used to finally evaluate the performance of the model.
[0076] During the model training process, the size of the image input to the model is 512*512; the number of iterations (Epoch) of the model training is set to a specified number (for example, 500 times); the learning rate (Learning Rate) of the model is set to 0.015; the batch size (Batch Size) is set to 3, and the graphics card of the device for training the model is NVIDIAGeForce RTX2080SUPER; the entire training time is about 8 hours.
[0077] Combine the following Figure 5C , that is, the average intersection-over-union (MIOU) change curve of each round in the model training process is used to evaluate the model, such as Figure 5C As shown in the figure, the horizontal axis of the curve is the number of model iterations (Epoch), and the vertical axis is the average intersection-over-union ratio (miou), which is Figure 5C It can be seen that at the 180th Epoch, the miou (average intersection-over-union ratio) value of the model is optimal, which is 0.9324. At this time, the average accuracy of image segmentation (mAcc) is: 0.9972 (not shown in the figure).
[0078] In the disclosed embodiment, before model training, in order to provide more training data, test data and verification data, data enhancement may be performed on the training samples to obtain more training samples. Here, data enhancement may be performed by random horizontal flipping and the like, which is not specifically limited in the disclosed embodiment.
[0079] As an optional implementation of the disclosed embodiment, in the above step S120, the acquired pointer instrument template image is matched with the pointer instrument target image to obtain an image to be identified, which can be specifically: performing image enhancement processing on the pointer instrument target image and the pointer instrument template image respectively to obtain a target enhanced image and a template enhanced image; performing a region of interest extraction operation on the template enhanced image to obtain a second region of interest, and the second region of interest at least includes: a pointer instrument region; performing image matching on the second region of interest and the target enhanced image to obtain a mapping matrix; the mapping matrix is used to map the pointer instrument target image to the second region of interest; based on the mapping matrix, the pointer instrument target image is mapped to the second region of interest to obtain the image to be identified.
[0080] Exemplarily, there may be many image enhancement methods used to perform image enhancement processing on the pointer instrument target image and the pointer instrument template image, such as contrast enhancement, brightness adjustment, color enhancement, image enhancement processing based on human visual perception characteristics (Retinex), etc. The present disclosed embodiment does not make specific limitations on this. The present disclosed embodiment only describes the image enhancement processing as Retinex image enhancement as an example.
[0081] The target enhanced image obtained by performing Retinex image enhancement on the pointer instrument target image is as follows: Figure 6B As shown in FIG. 1 , the template enhanced image obtained by performing Retinex image enhancement on the pointer instrument template image is as follows: Fig. 6A shown.
[0082] The disclosed embodiment can reduce the impact of different lighting on the collected images during the actual inspection process of the mobile robot by performing Retinex image enhancement processing on the collected pointer instrument target image and pointer instrument template image, and is also beneficial for feature extraction during subsequent image matching.
[0083] In the disclosed embodiment, the second region of interest may only include the pointer instrument region. In order to have richer features in subsequent image matching, the second region of interest may also include the pointer instrument region and other object regions (such as Figure 6C As shown), the second region of interest is not specifically limited in the present disclosure embodiment, as long as the second region of interest at least includes the pointer instrument area.
[0084] In the disclosed embodiment, the second region of interest is obtained by performing a region of interest extraction operation on the template enhanced image. Specifically, in response to a selection operation performed on the template enhanced image, the region of interest is extracted from the template enhanced image to obtain the second region of interest.
[0085] Based on this, in the embodiment of the present disclosure, the coordinates of the vertices of the second region of interest may also be stored so as to be directly called when needed later.
[0086] In the disclosed embodiment, there are many ways to perform image matching between the second region of interest and the target enhanced image, such as Surf algorithm, SIFT algorithm, etc. The disclosed embodiment does not specifically limit the image matching method, which can be determined according to actual conditions.
[0087] The disclosed embodiment is described by taking the Surf algorithm as an example of a matching method. The Surf algorithm is used to perform image matching on the second region of interest and the target enhanced image to obtain a matching image such as Fig.6D As shown, the matching image includes multiple groups of feature point pairs, one of the feature points in each group of feature point pairs comes from the second region of interest, and the other feature point comes from the target enhanced image. The two feature points in each group of feature point pairs indicate the same feature on the second region of interest and the target enhanced image. For example, the pointer pivot point on the second region of interest and the pointer pivot point on the target enhanced image are a group of feature point pairs. A 2*3 mapping matrix (referred to as the second mapping matrix) is calculated based on the multiple groups of feature point pairs. Then, based on the 2*3 mapping matrix, the pointer instrument target image is mapped to the second region of interest to obtain the image to be identified.
[0088] It should be noted that the above-mentioned method of calculating the mapping matrix and mapping one image to another image based on the mapping matrix are conventional methods. The specific implementation methods can be found in relevant materials and will not be described in detail in the present disclosed embodiment.
[0089] As an optional implementation of the disclosed embodiment, the farthest pixel point on the above-mentioned pointer area is determined in the following manner: for each pixel point in the pointer area, the distance between the pixel point and the pointer rotation axis point is calculated, and the pixel point corresponding to the maximum distance is taken as the farthest pixel point.
[0090] Exemplarily, for the pointer area, each pixel point in the pointer area is traversed, and the distance between each pixel point and the pointer pivot point is calculated. The distance here can be Euclidean distance, Manhattan distance, etc., which is not specifically limited in the disclosed embodiment.
[0091] After calculating the distance between each pixel point in the pointer area and the pointer rotation axis point, the pixel point corresponding to the maximum distance is taken as the farthest pixel point.
[0092] Combine the following Figure 7 The data reading method of the pointer instrument of the present disclosure is described as follows: Figure 7 As shown, for the pointer-type instrument that needs to be processed, a template image (i.e., the pointer-type instrument template image in the above embodiment) and a real-time image (i.e., the pointer-type instrument target image in the above embodiment) are collected, and Retinex image enhancement processing is performed on the template image and the real-time image, respectively. Then, the region of interest is extracted for the enhanced template image to obtain a second region of interest. Based on the second region of interest and the enhanced real-time image, a feature matrix (i.e., the second mapping matrix in the above embodiment) is calculated through the Surf image, and then the real-time image is mapped to the second region of interest through an affine transformation to obtain an image to be identified. For the image to be identified, an area of interest on the dial is extracted to obtain a first region of interest, and the angle of the pointer is extracted and calculated based on deep learning semantic segmentation, and the final reading result is calculated based on the percentage of the calculated pointer angle to the dial range.
[0093] Figure 8 A schematic diagram of a data reading device 800 of a pointer-type instrument according to some embodiments of the present disclosure is shown.
[0094] like Figure 8 As shown, the data reading device 800 of the pointer instrument includes: a pointer instrument target image acquisition module 810, which is used to acquire the pointer instrument target image obtained by performing image acquisition on the target pointer instrument; a matching module 820, which is used to match the acquired pointer instrument template image with the pointer instrument target image to obtain the image to be identified; an extraction module 830, which is used to perform an interest region extraction operation on the image to be identified to obtain a first interest region, and input the first interest region into a trained semantic segmentation model to perform pointer extraction to obtain a pointer region; the first interest region is the pointer instrument region; a data reading module 840, which is used to determine the reading result of the target pointer instrument based on the target angle value of the angle between the target straight line on the pointer region and the specified direction and the acquired scale value information; the target straight line is the straight line where the line connecting the farthest pixel point on the pointer region and the pointer rotation axis point is located; the scale value information includes: one-to-one corresponding scale values and angle values.
[0095] As an optional implementation of the disclosed embodiment, the above-mentioned matching module 820 is specifically used to: perform image enhancement processing on the pointer instrument target image and the pointer instrument template image respectively to obtain a target enhanced image and a template enhanced image; perform a region of interest extraction operation on the template enhanced image to obtain a second region of interest, and the second region of interest at least includes: a pointer instrument region; perform image matching on the second region of interest and the target enhanced image to obtain a mapping matrix; the mapping matrix is used to map the pointer instrument target image to the second region of interest; based on the mapping matrix, the pointer instrument target image is mapped to the second region of interest to obtain an image to be identified.
[0096] As an optional implementation of the disclosed embodiment, the farthest pixel point on the pointer area is determined in the following manner: for each pixel point in the pointer area, the distance between the pixel point and the pointer rotation axis point is calculated, and the pixel point corresponding to the maximum distance is taken as the farthest pixel point.
[0097] As an optional implementation of the disclosed embodiment, the scale value information is determined by the following modules: a coordinate acquisition module, used to obtain the coordinates of each scale value on the pointer-type instrument template image; a calculation module, used to calculate, for each scale value, the angle value between the straight line connecting the scale value and the pointer pivot point and the specified direction based on the coordinates at the scale value and the coordinates of the pointer pivot point; a scale value information acquisition module, used to use the angle value corresponding to each scale value and the scale value as scale value information.
[0098] As an optional implementation of the disclosed embodiment, the above-mentioned data reading module 840 is specifically used for: if the target angle value matches any angle value in the scale value information, then the scale value corresponding to the matched angle value is used as the reading result of the target pointer instrument; if the target angle value falls between the second angle value and the first angle value in the scale value information, then the reading result of the pointer instrument is determined based on the first angle value, the second angle value, the target angle value, the first scale value corresponding to the first angle value, and the second scale value corresponding to the second angle value.
[0099] As an optional implementation of the disclosed embodiment, the first scale value is less than the second scale value; the data reading module 840 determines the reading result of the pointer instrument based on the first angle value, the second angle value, the target angle value, the first scale value corresponding to the first angle value, and the scale value corresponding to the second angle value: Reading = value1 + Per* | value2 - value1 |, wherein Reading is the reading result of the target pointer instrument; value1 is the first scale value; value2 is the second scale value; Per is an intermediate parameter, Angle1 is the first angle value; Angle2 is the second angle value; α1 is the target angle value.
[0100] As an optional implementation of the disclosed embodiment, the pointer instrument is a single-pointer instrument or a multi-pointer instrument.
[0101] Correspondingly, the disclosed embodiment also provides Figure 8 The hardware structure diagram of the device shown is an exemplary structural block diagram of the electronic device 900, as shown in FIG. Fig. 9 As shown, the electronic device 900 may be a device for implementing the above method 100. Fig. 9 As shown, the electronic device 900 includes: a processor 910 and a memory 920. The memory 920 is configured to store program instructions; the processor 910 is configured to load and execute the program instructions stored in the memory 920 to implement the corresponding pointer instrument data reading method embodiment shown above.
[0102] As an embodiment, the memory 920 may be any electronic, magnetic, optical or other physical storage device, and may contain or store information, such as program instructions, data, etc. For example, the memory 920 may be: a volatile memory, a non-volatile memory or a similar storage medium. Specifically, the memory 920 may be a RAM (Radom Access Memory), a flash memory, a storage drive (such as a hard disk drive), a solid state drive, any type of storage disk (such as a CD, a DVD, etc.), or a similar storage medium, or a combination thereof.
[0103] Although multiple embodiments of the present disclosure have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art may think of many changes, modifications, and alternatives without departing from the thought and spirit of the present disclosure. It should be understood that in the process of practicing the present disclosure, various alternatives to the embodiments of the present disclosure described herein may be adopted. The attached claims are intended to define the scope of protection of the present disclosure, and therefore cover equivalents or alternatives within the scope of these claims.
Claims
1. A method for reading data of a pointer instrument, characterized in that: The method comprises: Acquire a pointer instrument target image obtained by performing image acquisition on the target pointer instrument; Matching the acquired pointer instrument template image with the pointer instrument target image to obtain an image to be recognized; Performing an interest region extraction operation on the image to be identified to obtain a first interest region, and inputting the first interest region into a trained semantic segmentation model to perform pointer extraction to obtain a pointer region; the first interest region is a pointer instrument region; The reading result of the target pointer instrument is determined based on the target angle value of the angle between the target straight line on the pointer area and the specified direction and the acquired scale value information; the target straight line is the straight line where the line connecting the farthest pixel point and the pointer rotation axis point on the pointer area is located; the scale value information includes: one-to-one corresponding scale values and angle values.
2. The method according to claim 1, characterized in that The step of matching the acquired pointer instrument template image with the pointer instrument target image to obtain the image to be recognized includes: Performing image enhancement processing on the pointer instrument target image and the pointer instrument template image respectively to obtain a target enhanced image and a template enhanced image; Performing an area of interest extraction operation on the template enhanced image to obtain a second area of interest, wherein the second area of interest at least includes: a pointer instrument area; Performing image matching on the second region of interest and the target enhanced image to obtain a mapping matrix; the mapping matrix is used to map the pointer instrument target image to the second region of interest; The pointer instrument target image is mapped to the second region of interest based on the mapping matrix to obtain the image to be recognized.
3. The method according to claim 1, characterized in that The farthest pixel point on the pointer area is determined by: For each pixel point in the pointer area, the distance between the pixel point and the pointer rotation axis point is calculated, and the pixel point corresponding to the maximum distance is taken as the farthest pixel point.
4. The method according to claim 1, characterized in that: The scale value information is determined by the following steps: For the pointer-type instrument template image, obtaining the coordinates of each scale value on the pointer-type instrument template image; For each scale value, the angle value between the specified direction and the straight line connecting the scale value and the pointer pivot point is calculated according to the coordinates of the scale value and the coordinates of the pointer pivot point; Each scale value and the angle value corresponding to the scale value are used as the scale value information.
5. The method according to claim 1, characterized in that The method of determining the reading result of the target pointer instrument based on the target angle value of the angle between the target straight line on the pointer area and the specified direction and the acquired scale value information includes: If the target angle value matches any angle value in the scale value information, the scale value corresponding to the matched angle value is used as the reading result of the target pointer instrument; If the target angle value falls between the second angle value and the first angle value in the scale value information, the reading result of the pointer instrument is determined based on the first angle value, the second angle value, the target angle value, the first scale value corresponding to the first angle value, and the second scale value corresponding to the second angle value.
6. The method according to claim 5, characterized in that The first scale value is less than the second scale value; and determining the reading result of the target pointer instrument based on the first angle value, the second angle value, the target angle value, the first scale value corresponding to the first angle value, and the scale value corresponding to the second angle value includes: Reading=value1+Per*|value2-value1| Wherein, Reading is the reading result of the target pointer instrument; value1 is the first scale value; value2 is the second scale value; Per is an intermediate parameter, Angle1 is the first angle value; Angle2 is the second angle value; α1 is the target angle value.
7. The method according to claim 1, characterized in that The target pointer instrument is a single pointer instrument or a multi-pointer instrument.
8. A data reading device for a pointer instrument, characterized in that: The device comprises: A pointer instrument target image acquisition module is used to acquire a pointer instrument target image obtained by performing image acquisition on a target pointer instrument; A matching module, used for matching the acquired pointer instrument template image with the pointer instrument target image to obtain an image to be identified; An extraction module, used for performing an interest region extraction operation on the image to be identified to obtain a first interest region, and inputting the first interest region into a trained semantic segmentation model to perform pointer extraction to obtain a pointer region; the first interest region is a pointer instrument region; A data reading module is used to determine the reading result of the target pointer instrument based on the target angle value of the angle between the target straight line on the pointer area and the specified direction and the acquired scale value information; the target straight line is the straight line where the line connecting the farthest pixel point on the pointer area and the pointer rotation axis point is located; the scale value information includes: one-to-one corresponding scale values and angle values.
9. An electronic device, characterized in that: include: a processor configured to execute program instructions; as well as A memory configured to store the program instructions, when the program instructions are loaded and executed by the processor, enables the processor to execute the data reading method of the pointer instrument according to any one of claims 1-7.
10. A computer-readable storage medium having program instructions stored therein, characterized in that: When the program instructions are loaded and executed by the processor, the processor executes the data reading method of the pointer instrument according to any one of claims 1 to 7.