Method and system for realizing pointer type industrial instrument reading based on deep learning
Through deep learning technology, the YOLOv5 and DeepLabv3 algorithms are used to automatically identify and segment the images of pointer-type industrial instruments, which solves the problems of inefficient and insufficient safety of traditional manual readings, and realizes efficient and accurate instrument readings, improving the safety of industrial production Sex and reliability.
Patent Information
- Application Number
- CN202410973702.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-19
- Publication Date
- 2024-11-08
AI Technical Summary
Traditional manual reading pointer-type industrial instruments are inefficient and error-prone, especially in dangerous environments, and cannot meet the needs of modern industrial automation and intelligence.
Using a deep learning-based method, image preprocessing is performed through the YOLOv5 algorithm, combined with OpenCV and OCR technology to detect the circular instrument panel, the image is segmented using the DeepLabv3 algorithm, the instrument reading is extracted, and the pointer position is calculated using polar coordinates and matrix transformation to obtain the instrument reading.
Significantly reduce errors and delays in manual reading process, improve reading accuracy and consistency, improve work and production safety and reliability, and is suitable for different types of pointer-type industrial instruments without additional hardware investment. High cost performance.
Smart Images

Figure CN118918303A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of industrial instrument reading, and in particular to a method and system for realizing pointer-type industrial instrument reading based on deep learning. Background Art
[0002] Pointer-type industrial instruments are widely used in industrial production to monitor and record physical parameters such as pressure, temperature, and current. These instruments display readings through mechanical pointers and have the advantages of simple structure and high reliability. However, manual reading is inefficient and error-prone. At the same time, such instruments often appear in places that may be harmful to personnel, which is not conducive to the safety of the reading personnel. Therefore, the manual reading method can no longer meet the needs of modern industrial automation and intelligence. Therefore, using artificial intelligence to identify instrument readings instead of traditional manual readings has great application significance. Summary of the invention
[0003] In view of the above problems existing in the prior art, the present invention is proposed.
[0004] Therefore, the present invention provides a method for realizing pointer-type industrial instrument reading based on deep learning, which can solve the problem that the traditional manual reading method is inefficient and prone to errors, especially in dangerous or inconvenient environments for manual operation, and safety is also threatened. The method of the present invention not only significantly reduces the errors and delays that may occur in the manual reading process, improves the accuracy and consistency of the reading, but also improves the safety and reliability of work and production.
[0005] In order to solve the above technical problems, the present invention provides the following technical solutions, a method for realizing pointer-type industrial instrument reading based on deep learning, comprising: collecting instrument images, and preprocessing the images using the YOLOv5 algorithm; detecting the circular instrument panel by combining OpenCV and OCR technology, and segmenting the images using the DeepLabv3 algorithm; performing coordinate transformation and matrix transformation on the segmented instrument image results, extracting the horizontal coordinate array, and calculating the instrument reading by the ratio of the pointer position in the scale line.
[0006] As a preferred solution of the method for realizing pointer-type industrial instrument reading based on deep learning described in the present invention, the preprocessing includes taking an image of a pointer-type instrument to be detected, using surveillance cameras of the factory and the park to transmit the surveillance screen in real time, and intercepting an image containing the instrument from the surveillance video stream through an image processing system to ensure that the image is clear and contains complete instrument information;
[0007] The YOLOv5 algorithm is used to preprocess the collected images, detect the area where the instrument is located in the image, crop the square image containing the instrument, and adjust the image size. The formula is expressed as:
[0008] I resized =resize(I,(S,S))
[0009] Among them, I resized is the resized image, I is the input image, resize is the resize function, and S is the target size;
[0010] The image pixel values are normalized, and the normalization formula is expressed as:
[0011]
[0012] Among them, I normalized is the normalized image, dpi is the pixel value;
[0013] The image pixel values are normalized and the normalization formula is expressed as:
[0014]
[0015] Among them, I standardized is the standardized image, μ and σ are the mean and standard deviation of the pixel values calculated during the preprocessing process;
[0016] The YOLOv5 algorithm includes three modules: Backbone feature extraction backbone network module, Neck feature fusion module and Head detection head, each module includes convolution and pooling modules respectively;
[0017] Replace the standard convolutional module in the YOLOv5 Backbone feature extraction network with the depthwise separable convolutional module of MobileNet as the backbone network of YOLOv5;
[0018] Add the CBAM module to the Neck part of YOLOv5, consider the spatial and channel attention mechanisms, and improve the feature fusion effect;
[0019] Use the SPPELAN module to replace the traditional SPP spatial pyramid pooling structure;
[0020] Keep the Head detection structure of YOLOv5;
[0021] Using the depthwise separable convolution module, the computational complexity of the convolution kernel is reduced from O(K 2 ·D·N) is reduced to O(K 2 ·D+D·N), where K is the convolution kernel size, D is the input feature map depth, and N is the output feature map depth.
[0022] As a preferred solution of the method for realizing pointer-type industrial instrument reading based on deep learning described in the present invention, wherein: the detection circular instrument panel combines OpenCV technology and ORC technology to calculate the center coordinates (x0, y0) and the center radius r0, and identify the maximum value V marked on the instrument max and minimum value V min , get the instrument range information L = V max -V min ;
[0023] Use the scale line model and pointer model of the pointer instrument trained by the DeepLabv3 algorithm to identify the positions of the scale lines and pointers in the cropped image, perform target segmentation, and generate an image containing scale line and pointer element markers;
[0024] In the training process of the scale model and the pointer model, ResNet-50 is used as the backbone network of DeepLabv3. In the last three blocks of the backbone network, dilated convolution is used to expand the receptive field. The dilation rates are set to 2, 4, and 8 respectively, and the high-dimensional feature map output by the backbone network is passed to the ASPP module.
[0025] In the ASPP module, a dilated convolution layer with dilation rates of 1, 6, 12, and 18 and a global average pooling layer are used to fuse multi-scale features, so that the model can perceive images at different scales and improve the detection and segmentation capabilities of targets in complex scenes.
[0026] As a preferred solution of the method for realizing pointer-type industrial instrument reading based on deep learning described in the present invention, the coordinate transformation includes performing polar coordinate transformation on the segmented instrument image result, transforming each pixel point coordinate (x, y) into linear coordinate (θ, r), and obtaining a rectangular mark image. The coordinate transformation formula is expressed as:
[0027]
[0028] Among them, θ is the angle of the pixel point in the polar coordinate system, arctan is the inverse tangent function, and r is the radius of the pixel point in the polar coordinate system.
[0029] As a preferred solution of the method for realizing pointer-type industrial instrument reading based on deep learning described in the present invention, the matrix conversion includes performing matrix conversion on pixel points, converting the image into matrices of two pixel point horizontal and vertical coordinates (x, y), respectively, which are recorded as scale line matrices M S With the pointer matrix M p , the value of each point in the matrix is 0 or 1, 0 means that the point does not belong to the target pixel, 1 means that the point belongs to the target pixel, and the matrix structure is expressed as,
[0030]
[0031] Among them, (x k ,y l ) is a pixel point, k and l are different pixel index values, k∈(0,End), l∈(0,End), (x k )∈{0,1},(y l )∈{0,1}.
[0032] As a preferred solution of the method for realizing pointer-type industrial instrument reading based on deep learning described in the present invention, wherein: the extraction of the horizontal coordinate array includes processing the matrix, filtering the y-axis information, and obtaining an array [x0, x1, ..., x] containing only the horizontal coordinate x. End ];
[0033] The pixel matrix is represented as A, A[i,j] is the pixel in the i-th row and j-th column, and each column j of the two-dimensional array A is saliently processed;
[0034] If there is at least one pixel in the column whose x value is equal to 1, the j-th element value of the converted one-dimensional array B is 1;
[0035] If the x values of all pixels in the column are 0, the j-th element value of the converted one-dimensional array B is 0;
[0036] The calculation method is optimized to find the maximum value of all values in each column j of the two-dimensional array A;
[0037] If the result is 1, the j-th element value of the converted one-dimensional array B is 1;
[0038] If the result is 0, the j-th element value of the converted one-dimensional array B is 0;
[0039] The mathematical formula for significant transformation is:
[0040] B[j]=max i A[i,j]
[0041] Among them, max i is the function that finds the maximum value of the i-th column;
[0042] Extract the center point of the scale, take the segment with continuous x value of 1, and calculate the midpoint subscript x of the segment through the starting subscript and ending subscript of the segment si , merge all midpoint subscripts into a new array x s0 ,x s1 ,...,x sEnd ];
[0043] In the pointer matrix, consider that the pointer mark image contains only one, identify the unique pointer mark, and determine the pointer horizontal coordinate x p .
[0044] As a preferred solution of the method for realizing pointer-type industrial instrument reading based on deep learning described in the present invention, wherein: the calculating instrument reading includes calculating the nth value x of the pointer subscript in the scale subscript array sn and the n+1th value x s(n+1) The proportion of time is expressed as follows:
[0045]
[0046] Where P is the position ratio of the pointer;
[0047] The final reading of the meter is V,
[0048]
[0049] Among them, sEnd is the number of elements in the scale line horizontal coordinate x array.
[0050] Another object of the present invention is to provide a system for realizing pointer-type industrial instrument reading based on deep learning, by using the YOLOv5 algorithm to preprocess the image, combining OpenCV and OCR technology to detect the circular instrument panel, and using the DeepLabv3 algorithm to segment the image, efficient and accurate instrument reading can be achieved. The method not only significantly reduces the errors and delays that may occur in the manual reading process, improves the accuracy and consistency of the reading, but also improves the safety and reliability of work and production. The method of the present invention is not only applicable to a single type of instrument, but can also be extended to different pointer-type industrial instruments, with wide applicability and flexibility. Through existing monitoring cameras and computing equipment, no additional hardware investment is required, and the function can be realized by using image processing services, which has a high cost performance. The present invention provides an efficient, reliable and economical industrial instrument reading solution.
[0051] As a preferred solution of the system for realizing pointer-type industrial instrument reading based on deep learning described in the present invention, it includes: a preprocessing module, an image detection and segmentation module and a reading calculation module;
[0052] The preprocessing module transmits images in real time through the surveillance camera, and uses the YOLOv5 algorithm to crop, resize, normalize and standardize the images to provide basic data for subsequent feature extraction and analysis;
[0053] The image detection and segmentation module combines OpenCV and OCR technology to detect the circular instrument panel, calculate the center coordinates and radius, obtain the range information, and use the DeepLabv3 algorithm to segment the scale lines and pointers;
[0054] The reading calculation module converts the segmented instrument image result into polar coordinates and matrix, extracts the horizontal coordinate array of the scale lines and the pointer, and obtains the instrument reading by calculating the position ratio of the pointer in the scale lines.
[0055] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and is characterized in that when the processor executes the computer program, the steps of any one of the methods for realizing pointer-type industrial instrument reading based on deep learning are implemented.
[0056] A computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of any one of the methods for realizing pointer-type industrial instrument reading based on deep learning are implemented.
[0057] Beneficial effects of the present invention: The method of the present invention reduces errors and delays in the manual reading process through automated reading and real-time monitoring, thereby improving the safety and reliability of work and production. The mature and stable YOLOv5 target detection algorithm and the deep learning image segmentation model DeepLabv3 are adopted to accurately identify and segment the scale lines and pointers in the instrument image to ensure the accuracy and consistency of the reading. At the same time, the method of the present invention is not only applicable to a single type of instrument, but can also be extended to different pointer-type industrial instruments, with wide applicability and flexibility. Based on existing surveillance cameras and computing equipment, no additional hardware investment is required, and the functions can be realized by using image processing services, which has a high cost-effectiveness. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.
[0059] Figure 1 A flowchart of a method for realizing pointer-type industrial instrument reading based on deep learning is provided for one embodiment of the present invention.
[0060] Figure 2 A diagram showing target detection effects of a method for realizing pointer-type industrial instrument reading based on deep learning provided by an embodiment of the present invention.
[0061] Figure 3 An image segmentation marking scale line and pointer effect diagram of a method for realizing pointer-type industrial instrument reading based on deep learning provided by an embodiment of the present invention.
[0062] Figure 4 A diagram showing the effect of scale lines and pointer markings after linearization processing of a method for realizing pointer-type industrial instrument reading based on deep learning provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0063] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.
[0064] Example 1
[0065] Reference Figure 1-Figure 4 , which is the first embodiment of the present invention, provides a method for realizing pointer-type industrial instrument reading based on deep learning, comprising:
[0066] S1: Collect instrument images and preprocess the images using the YOLOv5 algorithm.
[0067] like Figure 2 As shown, it should be noted that the preprocessing includes taking the image of the pointer instrument to be detected, using the surveillance cameras of the factory and the park to transmit the surveillance screen in real time, and intercepting the image containing the instrument from the surveillance video stream through the image processing system to ensure that the image is clear and contains complete instrument information;
[0068] The YOLOv5 algorithm is used to preprocess the collected images, detect the area where the instrument is located in the image, crop the square image containing the instrument, and adjust the image size. The formula is expressed as:
[0069] I resized =resize(I,(S,S))
[0070] Among them, I resized is the resized image, I is the input image, resize is the resize function, and S is the target size;
[0071] The image pixel values are normalized, and the normalization formula is expressed as:
[0072]
[0073] Among them, Inormalized is the normalized image, dpi is the pixel value;
[0074] The image pixel values are normalized and the normalization formula is expressed as:
[0075]
[0076] Among them, I standardized is the standardized image, μ and σ are the mean and standard deviation of the pixel values calculated during the preprocessing process;
[0077] The YOLOv5 algorithm includes three modules: Backbone feature extraction backbone network module, Neck feature fusion module and Head detection head, each module includes convolution and pooling modules respectively;
[0078] Replace the standard convolutional module in the YOLOv5 Backbone feature extraction network with the depthwise separable convolutional module of MobileNet as the backbone network of YOLOv5;
[0079] Add the CBAM module to the Neck part of YOLOv5, consider the spatial and channel attention mechanisms, and improve the feature fusion effect;
[0080] Use the SPPELAN module to replace the traditional SPP spatial pyramid pooling structure;
[0081] Keep the Head detection structure of YOLOv5;
[0082] Using the depthwise separable convolution module, the computational complexity of the convolution kernel is reduced from O(K 2 ·D·N) is reduced to O(K 2 ·D+D·N), where K is the convolution kernel size, D is the input feature map depth, and N is the output feature map depth.
[0083] The depthwise separable convolution process consists of two steps, depthwise convolution and pointwise convolution.
[0084] The depth convolution, for each input channel C and output position (h, w), is expressed mathematically as follows:
[0085]
[0086] Among them, D is the depth of the convolution, K is the size of the convolution, and α and β are different index values;
[0087] The output feature map size is H×W×C in ;
[0088] Among them, H and W are the image height and width respectively, C inis the number of input channels,
[0089] The point-by-point convolution, for each output channel m and output position (h, w), is expressed mathematically as follows:
[0090]
[0091] Among them, O m is the output feature map of point-by-point convolution, P is the point-by-point convolution kernel, c is the index value of the input channel, and m is the index value of the output channel;
[0092] The output feature map size is H×W×C out ;
[0093] Among them, C out is the number of output channels;
[0094] Assume the standard convolution kernel size is K×K×C in ×C out ;
[0095] The computational complexity of a standard convolution is H×W×K×K×C in ×C out ;
[0096] The computational complexity of depthwise separable convolution is H×W×K×K×C in +H×W×C in ×C out ;
[0097] By comparison, we can see that using depthwise separable convolution can effectively reduce the amount of computation, thereby improving computational efficiency. By reducing the amount of computation, depthwise separable convolution makes the model more lightweight and suitable for resource-constrained devices, thereby expanding the application scope of deep learning models.
[0098] S2: Combine OpenCV and OCR technology to detect the circular dashboard and use the DeepLabv3 algorithm to segment the image.
[0099] It should be noted that the circular instrument panel detection combines OpenCV technology, uses Hough circle transform to detect circular structures in images, uses cv2.HoughCircles() function to perform Hough circle transform, sets parameters to detect circular objects in images, and the function returns the center coordinates (a, b) and radius r0 of the detected circle.
[0100] Use ORC technology to identify the maximum value V marked on the meter max and minimum value V min , get the instrument range information L = V max -V min ;
[0101] like Figure 3 and Figure 4 As shown, the scale line model and pointer model of the pointer instrument trained by the DeepLabv3 algorithm are used to identify the positions of the scale lines and pointers in the cropped image, perform target segmentation, and generate an image containing scale line and pointer element marks;
[0102] Furthermore, during the training of the scale model and the pointer model, ResNet-50 was used as the backbone network of DeepLabv3. In the last three blocks of the backbone network, dilated convolution was used to expand the receptive field. The dilation rates were set to 2, 4, and 8 respectively, and the high-dimensional feature maps output by the backbone network were passed to the ASPP module.
[0103] In the ASPP module, a dilated convolution layer with dilation rates of 1, 6, 12, and 18 and a global average pooling layer are used to fuse multi-scale features, so that the model can perceive images at different scales and improve the detection and segmentation capabilities of targets in complex scenes.
[0104] S3: Perform coordinate transformation and matrix transformation on the segmented instrument image results, extract the horizontal coordinate array, and calculate the instrument reading through the ratio of the pointer position in the scale line.
[0105] It should be noted that the matrix conversion includes performing matrix conversion on the pixel points, converting the image into matrices of two pixel point horizontal and vertical coordinates (x, y), which are respectively recorded as scale line matrices M and S With the pointer matrix M p , the value of each point in the matrix is 0 or 1, 0 means that the point does not belong to the target pixel, 1 means that the point belongs to the target pixel, and the matrix structure is expressed as,
[0106]
[0107] Among them, (x k ,y l ) is a pixel point, k and l are different pixel index values, k∈(0,End), l∈(0,End), (x k )∈{0,1},(y l )∈{0,1}.
[0108] Furthermore, the extraction of the horizontal coordinate array includes processing the matrix, filtering the y-axis information, and obtaining an array [x0, x1, ..., x End ];
[0109] The pixel matrix is represented as A, A[i,j] is the pixel in the i-th row and j-th column, and each column j of the two-dimensional array A is saliently processed;
[0110] If there is at least one pixel in the column whose x value is equal to 1, the j-th element value of the converted one-dimensional array B is 1;
[0111] If the x values of all pixels in the column are 0, the j-th element value of the converted one-dimensional array B is 0;
[0112] The calculation method is optimized to find the maximum value of all values in each column j of the two-dimensional array A;
[0113] If the result is 1, the j-th element value of the converted one-dimensional array B is 1;
[0114] If the result is 0, the j-th element value of the converted one-dimensional array B is 0;
[0115] The mathematical formula for significant transformation is:
[0116] B[j]=max i A[i,j]
[0117] Among them, max i is the function that finds the maximum value of the i-th column;
[0118] Extract the center point of the scale, take the segment with continuous x value of 1, and calculate the midpoint subscript x of the segment through the starting subscript and ending subscript of the segment si , merge all midpoint subscripts into a new array x s0 ,x s1 ,...,x sEnd ];
[0119] In the pointer matrix, consider that the pointer mark image contains only one, identify the unique pointer mark, and determine the pointer horizontal coordinate x p .
[0120] Further, the calculating of the meter reading includes calculating the nth value x of the pointer subscript in the scale subscript array. sn and the n+1th value x s(n+1) The proportion of time is expressed as,
[0121]
[0122] Where P is the position ratio of the pointer;
[0123] The final reading of the meter is V,
[0124]
[0125] Among them, sEnd is the number of elements in the scale line horizontal coordinate x array.
[0126] Example 2
[0127] The second embodiment of the present invention provides a method for realizing pointer-type industrial instrument reading based on deep learning. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through experiments.
[0128] In the same scenario, data testing was performed using our technical solution, manual readings, and traditional technical solutions, as shown in Tables 1 and 2.
[0129] Table 1. Data comparison table a
[0130] Performance Indicators Our technical solution Manual reading Traditional technical solutions Identify the number of instruments 100 100 100 Average reading error rate 0.002493% 1.44% 0.003901% Average time required to identify a single instrument 489 ms 5 seconds 549 ms
[0131] Table 2. Data comparison table b
[0132] Performance Indicators Our technical solution Manual reading Traditional technical solutions Identify the number of instruments 200 200 200 Average reading error rate 0.002715% 1.78% 0.004801% Average time for identifying a single instrument 504 ms 5 seconds 572 ms
[0133] In terms of the average error rate of readings, our technical solution is significantly better than manual readings and traditional technical solutions in terms of reading accuracy, with a lower error rate, indicating that it has higher reliability.
[0134] In terms of the average time consumption for single instrument identification, the identification time of our technical solution in Table 1 is 489 milliseconds, which is lower than the 549 milliseconds of the traditional technical solution and much lower than the 5 seconds of manual reading. The identification time of our technical solution in Table 2 is 504 milliseconds, which is still lower than the 572 milliseconds of the traditional technical solution and much lower than the 5 seconds of manual reading.
[0135] Our technical solution performs well in terms of accuracy and efficiency, with an error rate significantly lower than manual reading and traditional technical solutions, and the recognition time is kept within a reasonable range.
[0136] By comparing the data in Table 1 and Table 2, it can be concluded that our technical solution can provide higher reading accuracy and higher recognition efficiency under various test conditions. It is an advanced technology that is superior to manual reading and traditional technical solutions.
[0137] Example 3
[0138] The third embodiment of the present invention is different from the first two embodiments in that:
[0139] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.
[0140] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.
[0141] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.
[0142] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0143] Example 4
[0144] A fourth embodiment of the present invention provides a system for realizing pointer-type industrial instrument reading based on deep learning, characterized in that it includes a preprocessing module, an image detection and segmentation module, and a reading calculation module;
[0145] The preprocessing module transmits images in real time through the surveillance camera, and uses the YOLOv5 algorithm to crop, resize, normalize and standardize the images to provide basic data for subsequent feature extraction and analysis;
[0146] The image detection and segmentation module combines OpenCV and OCR technology to detect the circular instrument panel, calculate the center coordinates and radius, obtain the range information, and use the DeepLabv3 algorithm to segment the scale lines and pointers;
[0147] The reading calculation module converts the segmented instrument image result into polar coordinates and matrix, extracts the horizontal coordinate array of the scale lines and the pointer, and obtains the instrument reading by calculating the position ratio of the pointer in the scale lines.
[0148] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.
Claims
1. A method for realizing pointer-type industrial instrument reading based on deep learning, characterized in that: include, Collect instrument images and use the YOLOv5 algorithm to preprocess the images; Combine OpenCV and OCR technology to detect circular dashboards, and use DeepLabv3 algorithm to segment images; The segmented instrument image results are subjected to coordinate transformation and matrix transformation, the horizontal coordinate array is extracted, and the instrument reading is calculated through the ratio of the pointer position in the scale line.
2. A method for realizing pointer-type industrial instrument reading based on deep learning as claimed in claim 1, characterized in that: The preprocessing includes taking the image of the pointer instrument to be detected, using the monitoring cameras of the factory and the park to transmit the monitoring screen in real time, and intercepting the image containing the instrument from the monitoring video stream through the image processing system to ensure that the image is clear and contains complete instrument information; The YOLOv5 algorithm is used to preprocess the collected images, detect the area where the instrument is located in the image, crop the square image containing the instrument, and adjust the image size. The formula is expressed as: lresized = resize(I,(S,,S)) Among them, Iresize d is the resized image, I is the input image, resize is the resize function, and S is the target size; The image pixel values are normalized, and the normalization formula is expressed as: Among them, Inormalize d is the normalized image, dpi is the pixel value; The image pixel values are normalized and the normalization formula is expressed as: Among them, Istandardize d is the standardized image, μ and σ are the mean and standard deviation of the pixel values calculated during the preprocessing process; The YOLOv5 algorithm includes three modules: Backbone feature extraction backbone network module, Neck feature fusion module and Head detection head, each module includes convolution and pooling modules respectively; Replace the standard convolutional module in the YOLOv5 Backbone feature extraction network with the depthwise separable convolutional module of MobileNet as the backbone network of YOLOv5; Add the CBAM module to the Neck part of YOLOv5, consider the spatial and channel attention mechanism, and improve the feature fusion effect; Use the SPPELAN module to replace the traditional SPP spatial pyramid pooling structure; Keep the Head detection structure of YOLOv5; Using the depthwise separable convolution module, the computational complexity of the convolution kernel is reduced from O(K 2 ·D·N) is reduced to O(K 2 ·D+D·N); Among them, K is the convolution kernel size, D is the input feature map depth, and N is the output feature map depth.
3. A method for realizing pointer-type industrial instrument reading based on deep learning as claimed in claim 2, characterized in that: The detection of circular instrument panel combines OpenCV technology and ORC technology to calculate the center coordinates (a, b) and the center radius r0, and identify the maximum value V marked on the instrument max and minimum value V min , get the instrument range information L = V max -V min ; Use the scale line model and pointer model of the pointer instrument trained by the DeepLabv3 algorithm to identify the positions of the scale lines and pointers in the cropped image, perform target segmentation, and generate an image containing scale line and pointer element markers; In the training process of the scale model and the pointer model, ResNet-50 is used as the backbone network of DeepLabv3. In the last three blocks of the backbone network, dilated convolution is used to expand the receptive field. The dilation rates are set to 2, 4, and 8 respectively, and the high-dimensional feature map output by the backbone network is passed to the ASPP module. In the ASPP module, a dilated convolutional layer with dilation rates of 1, 6, 12, and 18 and a global average pooling layer are used to fuse multi-scale features. The model can perceive images at different scales and improve the detection and segmentation capabilities of targets in complex scenes.
4. A method for realizing pointer-type industrial instrument reading based on deep learning as claimed in claim 3, characterized in that: The coordinate conversion includes performing polar coordinate conversion on the segmented instrument image result, converting the coordinates (x, y) of each pixel point into linear coordinates (θ, r) to obtain a rectangular mark image. The coordinate conversion formula is expressed as: Among them, θ is the angle of the pixel point in the polar coordinate system, arctan is the inverse tangent function, and r is the radius of the pixel point in the polar coordinate system.
5. A method for realizing pointer-type industrial instrument reading based on deep learning as claimed in claim 4, characterized in that: The matrix conversion includes performing matrix conversion on the pixel points, converting the image into matrices of two pixel point horizontal and vertical coordinates (x, y), respectively, which are recorded as scale line matrices M S With the pointer matrix Mp, the value of each point in the matrix is 0 or 1, 0 means that the point does not belong to the target pixel, 1 means that the point belongs to the target pixel, and the matrix structure is expressed as, Among them, (x k ,y l ) is a pixel point, k and l are different pixel index values, k∈(0,End), l∈(0,End), (x k )∈{0,1},(y l )∈{0,1}.
6. A method for realizing pointer-type industrial instrument reading based on deep learning as claimed in claim 5, characterized in that: The method of extracting the horizontal coordinate array includes processing the matrix, filtering the y-axis information, and obtaining an array [x0, x1, ..., x] containing only the horizontal coordinate x. End ]; The pixel matrix is represented as A, A[i,j] is the pixel in the i-th row and j-th column, and each column j of the two-dimensional array A is saliency processed; If there is at least one pixel in the column whose x value is equal to 1, the j-th element value of the converted one-dimensional array B is 1; If the x value of all pixels in the column is 0, the j-th element value of the converted one-dimensional array B is 0; The calculation method is optimized to find the maximum value of all values in each column j of the two-dimensional array A; If the result is 1, the j-th element value of the converted one-dimensional array B is 1; If the result is 0, the j-th element value of the converted one-dimensional array B is 0; The mathematical formula for significant transformation is: B[j]max;A[i,j] Among them, max i is the function that finds the maximum value of the i-th column; Extract the center point of the scale, take the segment with continuous x value of 1, and calculate the midpoint subscript xs of the segment through the starting subscript and ending subscript of the segment i , merge all midpoint subscripts into a new array [x s0 ,x s1 ,...,xs End ]; In the pointer matrix, consider that the pointer mark image contains only one, identify the unique pointer mark, and determine the pointer horizontal coordinate x p .
7. A method for realizing pointer-type industrial instrument reading based on deep learning as claimed in claim 6, characterized in that: The calculation of the instrument reading includes calculating the nth value x of the pointer subscript in the scale subscript array sn and the n+1th value x s(n+1) The proportion of time is expressed as follows: Where P is the position ratio of the pointer; Finally, the meter reading V is obtained. Among them, sEnd is the number of elements in the scale line horizontal coordinate x array.
8. A system for realizing the method of reading pointer-type industrial instruments based on deep learning according to any one of claims 1 to 7, characterized in that: It includes a preprocessing module, an image detection and segmentation module, and a reading calculation module; The preprocessing module transmits images in real time through the surveillance camera, and uses the YOLOv5 algorithm to crop, resize, normalize and standardize the images to provide basic data for subsequent feature extraction and analysis; The image detection and segmentation module combines OpenCV and OCR technology to detect the circular instrument panel, calculate the center coordinates and radius, obtain the range information, and use the DeepLabv3 algorithm to segment the scale lines and pointers; The reading calculation module converts the segmented instrument image result into polar coordinates and matrix, extracts the horizontal coordinate array of the scale lines and the pointer, and obtains the instrument reading by calculating the position ratio of the pointer in the scale lines.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.
Citation Information
Patent Citations
Pointer type instrument reading identification method and identification device
CN115049916A
Pointer type instrument reading identification method and system based on machine learning
CN116403223A
Deep learning-based pointer instrument detection method and system
CN117037162A
Cited By
Pointer type pressure gauge reading method based on hierarchical parallel network
CN119762780A
Industrial instrument reading identification method and device based on edge-end cooperation, and medium
CN120107947A
Edge-end cooperation-based industrial instrument reading recognition method, device and medium
CN120107947B
