Instrument image digital recognition system and method

Abstract

The present invention discloses a meter image digital recognition system and method, the system comprises: an image acquisition terminal, a cloud recognition platform and a client; wherein the image acquisition terminal and the client are respectively connected to the cloud recognition platform in communication; the image acquisition terminal is arranged on the meter to be recognized, and is used to acquire the meter image with the meter reading, and transmit the acquired meter image to the cloud recognition platform through the network; the cloud recognition platform is used to process the meter image using a preset image processing algorithm, recognize the digital information in the meter image, and transmit the recognition result to the client through the network; the client is used to display the digital recognition result output by the cloud recognition platform. The present invention reduces the labor cost of meter reading, improves the accuracy, real-time and reliability of meter reading tasks, and is convenient for users to query.

Description

Technical Field

[0001] The present invention relates to the field of deep learning and computer vision technology, and in particular to an instrument image digital recognition system and method. Background Art

[0002] Generally speaking, the meter readings are composed of 5-6 digit digital wheels. When the usage of the corresponding statistical quantity of the meter increases, the meter will increase the reading value by rotating the wheel. By identifying these values, the purpose of measuring user usage can be achieved.

[0003] Therefore, accurately identifying the meter reading is very important for both users and businesses. The current meter reading method is generally for the meter reader to go to the meter installation site and read the meter manually. However, some special situations such as extreme geographical conditions and bad weather conditions will cause great difficulties to the meter reading task. Even under normal conditions, manual meter reading will undoubtedly bring a certain burden to the staff and the company. At the same time, for some meters that change frequently, the accuracy of manual meter reading is also questionable. If the time factor is taken into account, manual meter reading will inevitably reduce the timeliness of the reading, resulting in inaccurate data. Summary of the invention

[0004] The present invention provides an instrument image digital recognition system and method to solve the technical problems of low recognition accuracy and numerical precision in the existing instrument image digital recognition methods.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] In one aspect, the present invention provides an instrument image digital recognition system, which comprises: an image acquisition terminal, a cloud recognition platform and a client; wherein:

[0007] The image acquisition terminal and the client are respectively connected to the cloud recognition platform for communication;

[0008] The image acquisition terminal is arranged on the instrument to be identified, and is used to acquire instrument images with instrument readings within a fixed time period, and transmit the acquired instrument images to the cloud identification platform via a network;

[0009] The cloud recognition platform is used to process the instrument image using a preset image processing algorithm, recognize digital information in the instrument image, and transmit the recognition result to the client through the network;

[0010] The client is used to display the digital recognition results output by the cloud recognition platform.

[0011] Furthermore, the image acquisition terminal includes a camera, NB-IoT, Sim-Card and MCU; wherein the NB-IoT and Sim-Card are used to realize the communication between the image acquisition terminal and the cloud recognition platform under the control of the MCU;

[0012] The camera is used to collect instrument images with instrument readings under the control of the MCU.

[0013] Furthermore, the cloud recognition platform includes a first connection module, a data processing module, a digital recognition module and a data storage module; wherein,

[0014] The first connection module is used to establish a connection between the image acquisition terminal and the cloud recognition platform, and a connection between the cloud recognition platform and the client;

[0015] The data processing module is used to pre-process the instrument image uploaded by the image acquisition terminal;

[0016] The digital recognition module is used to recognize digital information in the instrument image preprocessed by the data processing module based on a preset deep learning model;

[0017] The data storage module is used to store the instrument image and the corresponding digital recognition result in a database.

[0018] Furthermore, the process of the data processing module preprocessing the instrument image uploaded by the image acquisition terminal includes:

[0019] The instrument image is processed in sequence by digital area capture, image grayscale and image normalization.

[0020] Furthermore, the deep learning model adopted by the digital recognition module is a sliding window joint recognition algorithm model.

[0021] Furthermore, the database used by the data storage module is a MySQL database.

[0022] Furthermore, the client includes a second connection module and a front-end display module; wherein,

[0023] The second connection module is used to realize the connection between the client and the cloud recognition platform;

[0024] The front-end display module is used to display the digital recognition results.

[0025] Furthermore, the sliding window joint recognition algorithm model includes a convolutional network part, a feature segmentation part based on a sliding window and a joint recognition part; wherein the convolutional network part is used to extract a feature map containing the features of the entire instrument digital sequence; the feature segmentation part based on a sliding window uses a sliding window to reasonably divide the features of the entire digital sequence, and outputs multiple independent features, each independent feature contains the digital features of adjacent digits and their dependencies, which are used for digital classification and recognition of adjacent digits, and outputs the classification results of each sliding window; the joint recognition part is used to combine the classification results of multiple independent sliding windows to obtain a digital recognition result.

[0026] Furthermore, the joint recognition part combines the classification results of multiple independent sliding windows, including:

[0027] When decoding the common digits identified by multiple windows, the probability distribution of different results of the common digits on two adjacent branches is statistically analyzed respectively, and the result with a larger probability is taken as the decoding result of the common digit.

[0028] On the other hand, the present invention also provides an instrument image digital recognition method implemented by using the above instrument image digital recognition system, the instrument image digital recognition method comprising:

[0029] The image acquisition terminal acquires instrument images with instrument readings;

[0030] The image acquisition terminal transmits the collected instrument images to the cloud recognition platform through the network;

[0031] The cloud recognition platform receives the instrument image; and processes the received instrument image using a preset image processing algorithm to recognize the digital information therein. After the recognition is completed, the instrument image and the corresponding digital information recognition result are saved, and the corresponding digital information recognition result is transmitted to the client through the network;

[0032] The client receives the digital information recognition result and displays the digital information recognition result.

[0033] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0034] The present invention adopts the method of hardware collection + cloud recognition + client display, which can solve the dangers brought to meter readers by complex environments and reduce the labor cost of meter reading. At the same time, the sliding window joint recognition algorithm adopted in the cloud can effectively identify the most difficult digital carry situation in the meter reading task, ensuring the recognition accuracy and precision of the meter. A large number of historical readings are saved in the cloud database for user query. Using the client for display, it is convenient to provide users and meter readers with services anytime and anywhere. The various parts of the whole system are combined with each other to improve the accuracy, real-time and reliability of the meter reading task. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 is a structural block diagram of an instrument image digital recognition system provided by an embodiment of the present invention;

[0037] Figure 2 is a schematic diagram of the execution flow of the instrument image recognition method provided by an embodiment of the present invention;

[0038] Figure 3 It is a schematic diagram of the execution flow of the sliding window joint recognition algorithm provided by an embodiment of the present invention;

[0039] Figure 4 It is a schematic diagram of a sliding window joint recognition algorithm provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0041] In view of the problems of low recognition accuracy and numerical precision in the existing instrument image digital recognition methods, this embodiment provides a high-precision instrument image recognition system based on a sliding window joint recognition algorithm. The structure of the instrument image recognition system is as follows: Figure 1 As shown, it includes an image acquisition terminal, a cloud recognition platform and a client; wherein the image acquisition terminal needs to be installed on the instrument panel to be recognized to facilitate image acquisition. The image acquisition terminal and the client are respectively connected to the cloud recognition platform for data transmission.

[0042] The image acquisition terminal includes a camera, NB-IoT, Sim-Card and MCU; the image acquisition terminal is used to collect digital images of instruments through the camera within a preset fixed time period under the control of the MCU. Each digital image of the instrument collected is transmitted to the cloud recognition platform through NB-IoT.

[0043] The cloud recognition platform is used to process instrument images using a preset image processing algorithm, identify digital information in the instrument images, and transmit the recognition results to the client via the network.

[0044] Specifically, the cloud recognition platform includes a first connection module, a data processing module, a digital recognition module and a data storage module; wherein the first connection module is used to establish a connection between the image acquisition terminal and the cloud recognition platform, and a connection between the cloud recognition platform and the client; the data processing module is used to pre-process the instrument image uploaded by the image acquisition terminal for subsequent recognition; the digital recognition module is used to recognize the digital information in the instrument image pre-processed by the data processing module based on a preset deep learning model; the data storage module is used to store the instrument image and the corresponding digital recognition results in a database. Furthermore, in this embodiment, the first connection module adopts an XML-RPC connection method; the database used is a MySQL database; and the deep learning model used is a sliding window joint recognition algorithm model.

[0045] The client is used to display the digital recognition results output by the cloud recognition platform. Specifically, the client includes a second connection module and a front-end display module; wherein the second connection module is used to realize the connection between the client and the cloud recognition platform; and the front-end display module is used to display the results using Jupyter Notebook.

[0046] Based on the above, the cloud recognition platform of this embodiment is connected to the image acquisition terminal via the first connection module, and the received image is pre-processed, including digital area interception, image graying, and image normalization operations. The processed image is transmitted to the digital recognition module, and the sliding window joint recognition algorithm model is used for intelligent recognition to obtain the digital recognition result. These images and digital recognition results will be saved in the cloud database through the data storage module. After that, a connection is established between the cloud recognition platform and the client through the first connection module, and the instrument image and recognition result queried by the user are displayed on the front-end page.

[0047] Among them, this embodiment adopts the sliding window joint recognition algorithm, which can effectively recognize the instrument numbers in complex situations, such as Figure 2As shown in the figure, in the sliding window joint recognition algorithm process, the image acquisition terminal will first collect an instrument image at the current moment, and after transmitting it to the cloud recognition platform through NB-IoT, it will first undergo image preprocessing to extract the reading area, and at the same time, the label corresponding to the image will be converted into the form of multiple digital fragments required by the algorithm, and then the preprocessed image will be input into the sliding window joint recognition algorithm model.

[0048] like Figure 3 As shown in the figure, the sliding window joint recognition algorithm model is based on deep learning and is mainly composed of a convolutional network part, a feature segmentation part based on a sliding window, and a joint recognition part. The convolutional network performs a large amount of calculations on the image to extract feature maps, which contain the features of the entire instrument digital sequence. These feature maps are then passed to the feature segmentation network part based on the sliding window. This part of the network uses a sliding window to reasonably divide the features of the entire digital sequence and output multiple independent features. Each independent feature contains the digital features of adjacent digits and their dependencies, which can be used for digital classification and recognition of adjacent digits, and outputs the classification results of each sliding window. Since there are common digits between these classification results, and the recognition values ​​of the common digits between the classification results of different sliding windows may be different, it is necessary to use a joint recognition method to combine the classification results of multiple independent sliding windows.

[0049] like Figure 4 As shown in the figure, the common digits identified by multiple windows will be further decoded. During the decoding process, the probability distribution of different results of the common digits on two adjacent branches is statistically analyzed, and then the result with the larger probability is taken as the decoding result of the common digit. Assuming that the sliding window is set to a two-digit length, that is, each branch corresponds to a 100 classification task, the classification results output by the branch are 0 to 99, which can be regarded as a two-digit sequence {00,01,02,…,99}. represents the kth digit in the mth branch, v represents the value of the common digit in the two branches, and t represents any value from 0 to 9. Then the probability of two digits in the digital sequence output by each branch is calculated as k can be 1 or 2. When k=1, it represents the higher digit of a two-digit number, and when k=2, it represents the lower digit. Therefore, the common digits between branches can be used and Indicates that the low bit of the previous branch and the high bit of the next branch. com When , the calculation of the larger probability of the two branches is In the figure, the classification is performed according to the output probability. The classification result of the fourth branch is 35, and the classification result of the fifth branch is 60. In terms of the common digit, the fourth branch tends to identify 5, while the fifth branch is more inclined to identify 6. At this time, the maximum probability decoding algorithm will count the probability that the common digit on the fourth branch is 5, that is, the sum of all probabilities that the classification result of the fourth branch belongs to the set {05,15,25,…,95}, and use this as the probability that the common digit is 5. At the same time, the sum of the probabilities that the classification result on the fifth branch belongs to the set {60,61,62,…,69} will also be counted, and this will be used as the probability that the common digit is 6. Finally, the two probability sums are compared, and the result with a larger probability is used as the recognition result of the common digit.

[0050] The process of performing instrument image recognition by the instrument image recognition system of this embodiment is as follows:

[0051] 1) The image acquisition terminal acquires digital images of instruments;

[0052] 2) The image acquisition terminal transmits the instrument digital image to the cloud recognition platform through NB-IoT;

[0053] 3) The cloud recognition platform receives the image through the first connection module;

[0054] 4) The cloud recognition platform processes the image using the data processing module;

[0055] 5) The cloud recognition platform uses the digital recognition module to recognize the numbers in the image;

[0056] 6) The cloud recognition platform uses the data storage module to save images and recognition results;

[0057] 7) The cloud recognition platform transmits the recognition result to the client using the first connection module;

[0058] 8) The client receives the recognition result using the second connection module;

[0059] 9) The client uses the front-end page to display the recognition results.

[0060] In summary, this embodiment adopts the method of hardware collection + cloud recognition + client display, which can solve the dangers brought to meter readers by complex environments and reduce the labor cost of meter reading. At the same time, the sliding window joint recognition algorithm used in the cloud can effectively identify the most difficult digital carry situation in the meter reading task, ensuring the recognition accuracy and precision of the meter. A large number of historical readings are saved in the cloud database for user query. Using the client for display can conveniently provide users and meter readers with services anytime and anywhere. The various parts of the entire system are combined with each other to improve the accuracy, real-time and reliability of meter reading tasks.

[0061] In addition, it should be noted that the present invention can be provided as a method, an apparatus or a computer program product. Therefore, the embodiments of the present invention can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0062] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0063] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0064] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0065] Finally, it should be noted that the above is a preferred embodiment of the present invention. It should be pointed out that although the preferred embodiment of the present invention has been described, for those skilled in the art, once the basic creative concept of the present invention is known, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should also be regarded as the protection scope of the present invention. Therefore, the attached claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. An instrument image digital recognition system, characterized in that: The instrument image digital recognition system includes: an image acquisition terminal, a cloud recognition platform and a client; wherein, The image acquisition terminal and the client are respectively connected to the cloud recognition platform for communication; The image acquisition terminal is arranged on the instrument to be identified, and is used to acquire instrument images with instrument readings within a fixed time period, and transmit the acquired instrument images to the cloud identification platform via a network; The cloud recognition platform is used to process the instrument image using a preset image processing algorithm, recognize digital information in the instrument image, and transmit the recognition result to the client through the network; The client is used to display the digital recognition results output by the cloud recognition platform; The cloud recognition platform includes a first connection module, a data processing module, a digital recognition module and a data storage module; wherein, The first connection module is used to establish a connection between the image acquisition terminal and the cloud recognition platform, and a connection between the cloud recognition platform and the client; The data processing module is used to pre-process the instrument image uploaded by the image acquisition terminal; The digital recognition module is used to recognize digital information in the instrument image preprocessed by the data processing module based on a preset deep learning model; The data storage module is used to store the instrument image and the corresponding digital recognition result in the database; The process of the data processing module preprocessing the instrument image uploaded by the image acquisition terminal includes: The instrument image is processed in sequence by digital area interception, image grayscale conversion and image normalization; The deep learning model adopted by the digital recognition module is a sliding window joint recognition algorithm model; The sliding window joint recognition algorithm model includes a convolutional network part, a feature segmentation part based on a sliding window and a joint recognition part; wherein the convolutional network part is used to extract a feature graph containing the features of the entire instrument digital sequence; the feature segmentation part based on a sliding window uses a sliding window to reasonably divide the features of the entire digital sequence, and outputs multiple independent features, each of which contains the digital features of adjacent digits and their dependencies, and is used for digital classification and recognition of adjacent digits, and outputs the classification results of each sliding window; the joint recognition part is used to combine the classification results of multiple independent sliding windows to obtain a digital recognition result; The joint recognition part combines the classification results of multiple independent sliding windows, including: When decoding the common digits identified by multiple windows, the probability distribution of different results of the common digits on two adjacent branches is statistically analyzed respectively, and the result with a larger probability is taken as the decoding result of the common digit.

2. The instrument image digital recognition system according to claim 1, characterized in that: The image acquisition terminal includes a camera, NB-IoT, Sim-Card and MCU; wherein the NB-IoT and Sim-Card are used to realize communication between the image acquisition terminal and the cloud recognition platform under the control of the MCU; The camera is used to collect instrument images with instrument readings under the control of the MCU.

3. The instrument image digital recognition system according to claim 1, characterized in that: The database used by the data storage module is MySQL database.

4. The instrument image digital recognition system according to claim 1, characterized in that: The client includes a second connection module and a front-end display module; wherein, The second connection module is used to realize the connection between the client and the cloud recognition platform; The front-end display module is used to display the digital recognition results.

5. A method for digitally recognizing an instrument image using the instrument image digital recognition system according to any one of claims 1 to 4, characterized in that: The instrument image digital recognition method comprises: The image acquisition terminal acquires instrument images with instrument readings; The image acquisition terminal transmits the collected instrument images to the cloud recognition platform through the network; The cloud recognition platform receives the instrument image; and processes the received instrument image using a preset image processing algorithm to recognize the digital information therein. After the recognition is completed, the instrument image and the corresponding digital information recognition result are saved, and the corresponding digital information recognition result is transmitted to the client through the network; The client receives the digital information recognition result and displays the digital information recognition result.

Images