Instrument monitoring system and method

Abstract

The invention discloses an instrument monitoring system and method. The system comprises a camera, a communication unit, an image recognition module and a display module, the camera is used for obtaining picture information containing instrument information, and the picture information is sent to the image recognition module through the communication unit; the image recognition module comprises an image division unit, an identification unit and an extraction unit, and the image division unit is used for dividing picture information into a first area and a second area; the identification unit identifies the first area and / or the second area, and the extraction unit is used for extracting predetermined image information in the first area and / or the second area to form identification information; and the display module is used for displaying the identification information. According to the instrument monitoring system and method provided by the invention, the old instrument can be intelligently monitored under the conditions that the instrument is not replaced and the work is not stopped.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent factories, and in particular relates to a system and a monitoring method for old instruments based on cameras. Background Art

[0002] The various basic pipe networks (water, electricity, gas, etc.) in the old workshops are relatively old, and the old instruments used are generally mechanical instruments, so various data are collected and summarized by traditional manual meter reading. This method is inefficient, prone to errors, and cannot monitor various data in real time. In addition, the cost of replacing instruments is huge; in addition, if production is not allowed to stop, such instruments cannot be replaced or improved. Therefore, it is necessary to provide a solution to realize intelligent monitoring of old instruments without replacing instruments and without stopping production. Summary of the invention

[0003] In order to overcome the shortcomings of the prior art, the present invention provides an instrument monitoring system and an instrument monitoring method. The instrument monitoring system is based on the Internet of Things. Without modifying the old instruments, the instruments are photographed in real time by a camera, an image recognition module is added to the server in the IT computer room, and the data (water, electricity, gas and other numerical values) captured by the camera are captured, and the above data are displayed on a display screen.

[0004] The present invention provides an instrument monitoring system, comprising: a camera, a communication unit, an image recognition module and a display module;

[0005] The camera is used to obtain picture information containing instrument information, and send the picture information to the image recognition module using the communication unit;

[0006] The image recognition module includes an image division unit, an identification unit and an extraction unit, wherein the image division unit is used to divide the image information into a first area and a second area; the identification unit identifies the first area and / or the second unit, and the extraction unit is used to extract predetermined image information in the first area and / or the second area to form recognition information;

[0007] The display module is used to display the identification information.

[0008] Preferably, the first area of ​​the picture information includes instrument information.

[0009] Preferably, the extraction unit extracts numerical information representing meter readings in the first area.

[0010] Preferably, the image division unit includes a slicing component and a classification component; the slicing component divides the image information into a number of sub-images according to coordinate pairs; the classification component divides the sub-images into a number of groups according to the coordinate positions and the feature information; the identification unit marks the instrument information with the same type of identification.

[0011] Preferably, the camera is at least one of a single-sensing visible light camera, a thermal imaging dual-sensing camera, or a visible light imaging dual-sensing camera.

[0012] The present invention also provides an instrument monitoring method, which can be applied to the instrument monitoring system described in any of the above embodiments, wherein the method comprises the following steps:

[0013] Acquire image information containing instrument information through the camera, and send the image information to the image recognition module using the communication unit;

[0014] The image information is divided into a first area and a second area by image recognition; the identification unit identifies the first area and / or the second area, extracts predetermined image information in the first area and / or the second area, and forms identification information;

[0015] The display module is used to display the identification information.

[0016] Preferably, the process of dividing the picture information into the first area and the second area by image recognition includes the following steps:

[0017] Convert image information into grayscale image;

[0018] Divide the grayscale image into several units and obtain the grayscale value of each unit;

[0019] The image information is divided into a first area and a second area according to the gray value pair.

[0020] Preferably, the time for the camera to obtain image information is set.

[0021] Preferably, the process of extracting the predetermined image information in the first area and / or the second area to form the identification information includes the following sub-steps:

[0022] Acquire a feature vector in the predetermined image information, and calculate text information representing the text according to the feature vector;

[0023] The text information in the changed area of ​​the text information is obtained as identification information.

[0024] Preferably, after extracting the predetermined image information in the first area and / or the second area to form the identification information, the following steps are also included:

[0025] Get the identification information obtained from the last monitoring;

[0026] Calculate the currently acquired identification information with the identification information obtained in the previous monitoring to obtain the actual usage value information;

[0027] The actual usage value information is displayed, and the previously acquired identification information is marked as the identification information of the last monitoring.

[0028] Compared with the prior art, the beneficial effect of the present invention is that the present invention captures videos or images through mechanical instruments in the camera area. All installed cameras are connected to the image recognition module through the communication module. The image recognition module can identify and capture photos or photo sets taken by the camera, and obtain corresponding text information (including identification information). The text information can be displayed through the display screen to achieve a visual display of data such as the current degree and actual usage degree of the above-mentioned mechanical instrument. Thereby, a method for intelligent monitoring of old instruments is implemented without replacing the old instruments and without stopping the work. The image division unit is used to divide the picture information into the first area and the second area, and only the image of the specific area needs to be deeply analyzed, thereby increasing the speed of image recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic diagram of a practical application scenario of an instrument monitoring system in a preferred embodiment of the present invention;

[0030] Figure 2 A schematic diagram of a module of an instrument monitoring system in a preferred embodiment of the present invention;

[0031] Figure 3 The present invention is a flow chart of an instrument monitoring method in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0032] The present invention is further described below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that, under the premise of no conflict, the various embodiments or technical features described below can be arbitrarily combined to form a new embodiment.

[0033] This preferred embodiment provides an instrument monitoring system, including: a camera, a communication unit, an image recognition module and a display module. The camera can be installed at a location where there is an instrument to be monitored. The communication unit can be a wired or wireless network for transmitting information. The communication unit can be connected to the camera, the image recognition module and the display module respectively. The display module can be an electronic display screen or an electronic display screen. The camera is used to obtain picture information containing instrument information, and the communication unit is used to send the picture information to the image recognition module; the image recognition module includes an image division unit, an identification unit and an extraction unit, and the image division unit is used to divide the picture information into a first area and a second area; the identification unit identifies the first area and / or the second unit, and the extraction unit is used to extract predetermined image information in the first area and / or the second area to form identification information; the display module is used to display the identification information.

[0034] The present invention installs a camera near a mechanical instrument. The camera angle and field of view are adjusted so that the mechanical instruments in the area can be filmed. All installed cameras are connected to the network and integrated into the IT room. An image recognition module is installed on the server in the IT room to recognize and capture photos taken by the camera or data in a photo collection; a central processing module is installed in the server and connected to the central control display screen to visualize the above data. This makes it possible to implement a solution for intelligent monitoring of old instruments without replacing the old instruments and without stopping the work. The image division unit is used to divide the image information into a first area and a second area, and only the image of a specific area needs to be deeply analyzed, thereby increasing the speed of image recognition.

[0035] As a preferred embodiment, the first area of ​​the image information includes instrument information. Therefore, only the image information in the first area needs to be advanced. In addition, the first area may also include several sub-areas. The image information of at least one sub-area includes text information with variable values. The text information with variable values ​​is identification information, which may be the current degree of the mechanical instrument. The image information of at least one sub-area includes text information with fixed values, and the text information with fixed values ​​may be other description information of the mechanical instrument, such as description information such as number, brand, and manufacturer. The data identification module may also include a calculation unit, which may subtract the degree monitored last time from the current degree to obtain the degree actually used. The current degree and the degree actually used may be displayed on the display screen respectively. After display, the current degree is redefined as the degree monitored last time. In addition, the data identification module may also include a warning unit. When the degree actually used is less than or equal to zero, it indicates that there is a problem with the mechanical instrument or the camera. At this time, the warning module may generate a warning instruction and display it on the display screen.

[0036] As a preferred embodiment, the extraction unit extracts the numerical information representing the meter reading in the first area. The text information with a variable value is extracted as the numerical information representing the meter reading. The text information with a fixed value can be used as auxiliary information to determine that the meter captured by the video has not changed. If the text information with a fixed value in the picture information representing the same mechanical meter changes, the warning unit can generate a warning instruction and display it on the display screen.

[0037] The image division unit includes a slicing component and a classification component; the slicing component divides the image information into several sub-images according to the coordinate pairs; the classification component can convert each sub-image into a grayscale image. Since the image or video obtained by the camera is at the same position and the appearance of the mechanical instrument is almost unchanged, after each sub-image is converted into a grayscale image, the characteristic information representing the mechanical instrument (the grayscale value after conversion into the grayscale image) can be almost maintained within a predetermined range. Since the position is unchanged, it can be basically determined that the position of the predetermined area in the image information is the corresponding mechanical instrument information. At the same time, the outline of the mechanical instrument can be determined by combining the grayscale value, so that the classification component can classify the sub-image. For example, a sub-image with a grayscale value of 200-255 can be the outline of the mechanical instrument. Grayscale refers to an image that contains only brightness information but not color information. The brightness of the grayscale image changes continuously from dark to bright. The description of the grayscale image still reflects the distribution and characteristics of the overall and local chromaticity and brightness levels of the entire image, just like a color image. All pixels with a grayscale greater than or equal to the threshold are judged to belong to a specific object.

[0038] The classification component divides the sub-images into several groups according to the feature information and coordinate positions; the feature information and coordinate positions can make the classification component more accurate in classification, and the use of the feature information of the grayscale image can greatly increase the speed of the classification component in classifying the sub-images. The identification unit marks the same type of identification on the sub-image group containing mechanical instrument information.

[0039] The extraction unit can use the U-Net network for semantic segmentation to extract the feature vector of the specific sub-image (including color, texture, shape and other object-related information), and then identify and extract the text information of the sub-image according to the feature vector through the grouped convolution (convolutional neural network, CNN) and dual attention (grouped convolution dual attention, GCDA) mechanism. The text information may include information such as Chinese characters, English letters, numerical values ​​or symbols. Preferably, the extraction unit only extracts the feature vector of the sub-image containing mechanical instrument information, thereby reducing the number of feature vectors to be extracted, and can greatly improve the accuracy and efficiency of the extracted feature vector. The U-Net network is composed of a series of convolution and pooling operations, and a symmetrical upsampling path is added to the network, which can efficiently extract features from the input image and generate pixel-level segmentation results. The core structure of the U-Net network is an encoder-decoder model, in which the encoder part is composed of convolution and pooling layers, which can extract low-level feature information of the input image, and the decoder part is composed of deconvolution and upsampling layers, which can restore the feature information extracted from the encoder to the original image size and generate pixel-level segmentation results. In addition, the U-Net network also uses a skip connection structure to connect the feature information in the encoder and decoder, so that the network can use both low-level and high-level feature information, thereby improving the accuracy of segmentation. Grouped convolution and dual attention mechanisms are used to extract text information from images. Grouped convolution is a convolution operation in a convolutional neural network. It divides the input feature vector into several groups and performs convolution operations on each group separately. Finally, the feature vectors of each group with similar or similar features are spliced ​​together as output, thereby extracting text. The extracted text information includes recognition information.

[0040] The camera is at least one of a single-sensing visible light camera, a thermal imaging dual-sensing camera, or a visible light imaging dual-sensing camera. The camera can obtain high-definition video or photo images in different climate environments, preferably video. Since the reading of the degree of the mechanical meter requires a certain interval after which the actual degree used during the interval is calculated. Therefore, the degree of the mechanical meter does not need to be followed up in real time, and a predetermined time period can be set for acquisition, thereby reducing the task of video processing.

[0041] The instrument monitoring system described in the above embodiments captures video or images through mechanical instruments in the camera area. All installed cameras are connected to the image recognition module through the communication module. The image recognition module can recognize and capture photos or photo sets taken by the camera, and obtain corresponding text information (including identification information). The text information can be displayed on the display screen to achieve a visual display of data such as the current degree and actual usage degree of the above-mentioned mechanical instrument. Thereby, a method for intelligent monitoring of old instruments is implemented without replacing the old instruments and without stopping the work. The image division unit is used to divide the picture information into the first area and the second area, and only the image of the specific area needs to be deeply analyzed, thereby increasing the speed of image recognition.

[0042] Preferably, the present invention further provides an instrument monitoring method, which can be applied to the instrument monitoring system described in any of the above embodiments, and the method comprises the following steps:

[0043] Step S1, obtaining picture information containing instrument information through the camera, and sending the picture information to the image recognition module using the communication unit;

[0044] Step S2, dividing the picture information into a first area and a second area through image recognition; the identification unit identifies the first area and / or the second area, extracts predetermined image information in the first area and / or the second area, and forms identification information.

[0045] Step S3: displaying the identification information via a display module.

[0046] Since the reading of the mechanical watch requires a certain interval and the calculation of the actual usage degree during the interval, in step S1, the time for the camera to obtain image information can be set, for example, taking a picture once at 12 o'clock every morning, or taking a picture once at 8 o'clock on Sunday morning, or taking a picture once at 20 o'clock on the last day of each month. The specific time can be set according to the actual situation.

[0047] In the step S2, the process of dividing the image information into the first area and the second area by image recognition includes the following steps: converting the image information into a grayscale image; dividing the grayscale image into a plurality of units, obtaining the grayscale value of each unit; and dividing the image information into the first area and the second area according to the grayscale value pair. Preferably, before converting the image information into a grayscale image, each image information can be divided into a plurality of sub-images according to the coordinates. When dividing the image information into the first area and the second area according to the grayscale value pair, the grayscale value can also be used together with the coordinates for division, which can improve the accuracy of the division. Since the image or video obtained by the camera is at the same position and the appearance of the mechanical instrument is almost unchanged, after each sub-image is converted into a grayscale image, the characteristic information representing the mechanical instrument (the grayscale value after conversion into the grayscale image) can be almost maintained within the predetermined range. Since the position is unchanged, it can be basically determined that the position of the predetermined area in the image information is the corresponding mechanical instrument information, and the outline of the mechanical instrument can be determined by combining the grayscale value, so as to facilitate the classification component to classify the sub-image. The first area and the second area are divided according to the classification structure.

[0048] Grayscale refers to the process of converting a color image into a grayscale image. Grayscale refers to an image that contains only brightness information but no color information. The brightness of a grayscale image changes continuously from dark to bright. The description of a grayscale image, like a color image, still reflects the distribution and characteristics of the overall and local chromaticity and brightness levels of the entire image. Binarization refers to setting the grayscale value of a pixel on an image to 0 or 255, that is, making the entire image appear in obvious black and white. A binary image that can still reflect the overall and local features of the image is obtained by selecting a grayscale image with 256 brightness levels through an appropriate threshold. All pixels with a grayscale greater than or equal to the threshold are judged to belong to a specific object, and their grayscale value is 255. Otherwise, these pixels are excluded from the object area, and the grayscale value is 0, indicating the background or exceptional object area. In digital image processing, binary images occupy a very important position. First of all, the binarization of an image is conducive to further processing of the image, making the image simpler, and reducing the amount of data, which can highlight the outline of the target of interest. Secondly, to process and analyze binary images, we must first binarize the grayscale image to obtain a binary image.

[0049] The process of extracting predetermined image information in the first area and / or the second area to form identification information includes the following sub-steps: obtaining a feature vector in the predetermined image information, and calculating text information representing the text based on the feature vector; obtaining text information in a changing area in the text information as identification information.

[0050] The feature vector of a specific sub-image (including color, texture, shape and other object-related information) can be extracted by using a U-Net network for semantic segmentation in the extraction unit, and the text information of the sub-image can be identified and extracted according to the feature vector through group convolution and dual attention mechanism. The text information can include information such as Chinese characters, English letters, numerical values ​​or symbols. The specific process of obtaining the text information can refer to the description in the above instrument monitoring system embodiment, which will not be repeated here.

[0051] After the image information is divided, the first area is defined as a sub-image group including instrument information. Therefore, only the image information of the sub-images in the first area needs to be advanced. In addition, the first area may also include several sub-areas. The image information of at least one sub-area contains text information with variable values. The text information with variable values ​​is identification information, which may be the current degree of the mechanical instrument. The image information of at least one sub-area contains text information with fixed values, and the text information with fixed values ​​may be other description information of the mechanical instrument, such as description information such as number, brand, and manufacturer. The actual degree used may be obtained by subtracting the degree monitored last time from the current degree. The current degree and the actual degree used may be displayed on the display screen respectively. After display, the current degree is redefined as the degree monitored last time, so that the actual degree used may be calculated after the next monitoring. In addition, when the actual degree used is less than or equal to zero, it indicates that there is a problem with the mechanical instrument or the camera. At this time, the warning module may generate a warning instruction and display it on the display screen.

[0052] As a preferred embodiment, text information with variable values ​​is extracted as identification information representing the meter reading. Text information with fixed values ​​can be used as auxiliary information to determine that the meter captured by the video has not changed. All mechanical meters can be classified and managed through auxiliary information and identification information. If the text information with fixed values ​​in the picture information representing the same mechanical meter changes, a warning instruction can be generated and displayed on the display screen.

[0053] As a preferred embodiment, after extracting the predetermined image information in the first area and / or the second area and forming the identification information, the following steps are also included:

[0054] Obtain identification information obtained from the last monitoring (e.g., the degree currently monitored); calculate the identification information currently obtained with the identification information obtained from the last monitoring (e.g., the degree obtained from the last monitoring) to obtain actual usage value information (e.g., the actual usage degree); display the actual usage value information, and mark the previously obtained identification information as the identification information of the last monitoring.

[0055] The present invention installs a camera near a mechanical instrument. The camera angle and field of view are adjusted so that it can take video or image shots of the mechanical instruments in the area. All installed cameras are connected to the network and integrated into the IT room. An image recognition module is installed on the server in the IT room to recognize and capture photos taken by the camera or data in a photo collection; a central processing module is installed in the server and connected to the central control display screen to realize a visual display of the above data. This makes it possible to implement a method for intelligent monitoring of old instruments without replacing the old instruments and without stopping the work. The image division unit is used to divide the picture information into a first area and a second area, and only the image of a specific area needs to be deeply analyzed, thereby increasing the speed of image recognition.

[0056] The instrument monitoring method described in the above embodiments obtains the picture information containing the instrument information through the camera, and sends the picture information to the image recognition module by using the communication unit; divides the picture information into the first area and the second area by image recognition; the identification unit identifies the first area and / or the second unit, extracts the predetermined image information in the first area and / or the second area, and forms the identification information. The display module is used to display the identification information. The data such as the current degree and the actual use degree of the above-mentioned mechanical instrument are visualized. Thus, a method for intelligent monitoring of old instruments is realized without replacing the old instruments and without stopping the work. The image division unit is used to divide the picture information into the first area and the second area, and only the image of the specific area needs to be deeply analyzed, thereby increasing the speed of image recognition.

[0057] The above-mentioned embodiments are only preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantial changes and substitutions made by technicians in this field on the basis of the present invention shall fall within the scope of protection required by the present invention.

Claims

1. An instrument monitoring system, It is characterized in that include: Camera, communication unit, image recognition module and display module; The camera is used to obtain picture information containing instrument information, and send the picture information to the image recognition module using the communication unit; The image recognition module includes an image division unit, an identification unit and an extraction unit, wherein the image division unit is used to divide the image information into a first area and a second area; the identification unit identifies the first area and / or the second unit, and the extraction unit is used to extract predetermined image information in the first area and / or the second area to form recognition information; The display module is used to display the identification information.

2. The instrument monitoring system according to claim 1, Features: The first area of ​​the picture information includes instrument information.

3. The instrument monitoring system according to claim 2, Features: The extracting unit extracts numerical information representing the meter reading in the first area.

4. The instrument monitoring system according to any one of claims 1 to 3, Features: The image division unit includes a slicing component and a classification component; the slicing component divides the image information into several sub-images according to the coordinate pairs; the classification component divides the sub-images into several groups according to the coordinate positions and the characteristic information; the identification unit marks the instrument information with the same type of identification.

5. The instrument monitoring system according to any one of claims 1 to 3, Features: The camera is at least one of a single-sensing visible light camera, a thermal imaging dual-sensing camera, or a visible light imaging dual-sensing camera.

6. An instrument monitoring method, It is characterized in that The steps include: Acquire image information containing instrument information through the camera, and send the image information to the image recognition module using the communication unit; The image information is divided into a first area and a second area by image recognition; the identification unit identifies the first area and / or the second area, extracts predetermined image information in the first area and / or the second area, and forms identification information; The display module is used to display the identification information.

7. The instrument monitoring method according to claim 6, It is characterized in that The process of dividing the image information into the first area and the second area by image recognition includes the following steps: Convert image information into grayscale image; Divide the grayscale image into several units and obtain the grayscale value of each unit; The image information is divided into a first area and a second area according to the gray value pair.

8. The instrument monitoring method according to claim 6 or 7, It is characterized in that Set the time for the camera to obtain image information.

9. The instrument monitoring method according to claim 6 or 7, It is characterized in that The process of extracting predetermined image information in the first area and / or the second area to form identification information includes the following sub-steps: obtaining a feature vector in the predetermined image information, and calculating text information representing the text based on the feature vector; obtaining text information in a changing area in the text information as identification information.

10. The instrument monitoring method according to claim 6 or 7, It is characterized in that After extracting the predetermined image information in the first area and / or the second area and forming the identification information, the following steps are also included: obtaining the identification information obtained from the last monitoring; Calculate the currently acquired identification information with the identification information obtained in the previous monitoring to obtain the actual usage value information; The actual usage value information is displayed, and the previously acquired identification information is marked as the identification information of the last monitoring.

Images